Sliding window determination method based on integer programming and information entropy

Abstract

The invention discloses a sliding window determination method based on integer programming and information entropy, and belongs to the technical field of data segmentation in machine learning. The method aims at solving the problem that an existing sliding window length and step length selection method cannot dynamically adapt to the change characteristics of data, and the data analysis result in machine learning is affected. Comprising the following steps: removing outliers from initial time sequence data to obtain a preprocessed time sequence data total set; setting the length and the step length of a sliding window, and slicing the time sequence data total set to obtain data subsets; in each iteration process, calculating the relative mutual information entropy of each data subset and the time sequence data set and the adjacent mutual information entropy of the adjacent data subset; an optimized information entropy function is constructed and solved, so that the information entropy function is minimum on the basis that the sum of all adjacent mutual information entropies is minimum and the relative mutual information entropy is maximum; meanwhile, the length of the sliding window is larger than the step length and is an integer, and the optimal sliding window length and the optimal step length are obtained. According to the invention, effective segmentation of data can be realized.

Description

Technical Field

[0001] The invention relates to a sliding window determination method based on integer programming and information entropy, and belongs to the technical field of data segmentation in machine learning. Background Art

[0002] Sliding window technology is the most commonly used data segmentation method for time series in machine learning. However, the length and step size of the sliding window directly affect the performance of machine learning training and the accuracy of the results.

[0003] Sliding window technology plays an extremely critical role in time series analysis, signal processing, and various data-based prediction and analysis tasks. The reasonable selection of sliding window length and step size is crucial to the accuracy of analysis results. However, traditional methods for determining sliding window length and step size mainly rely on empirical judgment or trial and error, lack theoretical basis, and are difficult to adapt to the characteristics of different data.

[0004] The empirical method is that experts choose a seemingly appropriate sliding window length and step size for experimentation based on their in-depth understanding of the specific problem domain and past experience. Its advantages are simplicity, speed, and no need for complex calculations, but its disadvantages are that it is highly dependent on personal experience, lacks universality and theoretical basis, is prone to deviations, and cannot ensure optimal performance; the fixed length and step size rule is to pre-set a fixed sliding window length and step size, and then apply it to the entire data set. The fixed length and step size are usually based on past implementation experience or recommended values ​​in the literature. Its advantages are simple implementation and small amount of calculation, but its disadvantages are that the fixed window length and step size may not be applicable to all data sets, and cannot dynamically adapt to the changing characteristics of the data, resulting in large differences in effects on different data sets.

[0005] In machine learning, the most important factor limiting the training speed of the algorithm is the size of the data set. A data set that is too large cannot be input into the neural network at one time, so it is often necessary to split the data set before input, that is, to use the sliding window method. When choosing the sliding window length and step size, multiple factors need to be weighed. If the sliding window length is too long, it may not be able to effectively reduce the original data set, but will lead to excessive smoothing; if it is too short, it may lose important features in the original data set. Similarly, if the step size is too small, the data set will be too similar, so that the neural network loses its generalization ability; if the step size is too large, key information may be missed and subtle changes in the data may not be captured. Therefore, the sliding window length should be appropriately larger than the step size to find a balance between analysis accuracy, efficiency and data utilization. Summary of the invention

[0006] In view of the problem that the existing sliding window length and step size selection method cannot dynamically adapt to the changing characteristics of data, which affects the data analysis results in machine learning, the present invention provides a sliding window determination method based on integer programming and information entropy.

[0007] A sliding window determination method based on integer programming and information entropy of the present invention comprises:

[0008] Preprocess the initial time series data to remove outliers and obtain the total set of preprocessed time series data X = {x1, x2, ..., x n};

[0009] Set the sliding window length L i and step length S i , slice the total set of time series data X to obtain multiple data subsets after slicing

[0010] Where i represents the number of iterations, represents the Nth data subset of the i-th iteration; k∈(1,n), k=(N-1)×S i +1;

[0011] In each iteration, each data subset is calculated The relative mutual information entropy with the total set of time series data X Simultaneously calculate the neighbor mutual information entropy of adjacent data subsets

[0012] Based on all relative mutual information entropy Neighborhood mutual information entropy Construct an optimized information entropy function H and solve the optimized information entropy function H so that the mutual information entropy and minimum relative mutual information entropy On the basis of maximum, the information entropy function H is minimized; at the same time, combined with the integer programming model, the sliding window length L i and step length S i are all integers, and the sliding window length is L i Greater than step size S i ; The final sliding window length L i and step length S i As the optimal sliding window length L * and the optimal step size S * .

[0013] According to the sliding window determination method based on integer programming and information entropy of the present invention, the method for removing outliers is:

[0014] The mean μ and standard deviation σ of the initial time series data are calculated, and the data points outside the range of μ±3σ are removed as outliers to obtain the total set X of preprocessed time series data.

[0015] According to the sliding window determination method based on integer programming and information entropy of the present invention, the relative mutual information entropy The calculation method is:

[0016]

[0017] In the formula for The joint probability distribution of and X is, for The marginal distribution of , p(X) is the marginal distribution of X.

[0018] According to the sliding window determination method based on integer programming and information entropy of the present invention, the neighbor mutual information entropy of adjacent data subsets The calculation method is:

[0019]

[0020] In the formula for and The joint probability distribution of .

[0021] According to the sliding window determination method based on integer programming and information entropy of the present invention, the optimized information entropy function H is:

[0022]

[0023] The Monte Carlo method is used to solve the optimal information entropy function H to obtain the optimal sliding window length L * and the optimal step size S * .

[0024] According to the sliding window determination method based on integer programming and information entropy of the present invention, the method of solving the optimization information entropy function H using the Monte Carlo method is:

[0025] Randomly generate sliding window length L i and step length S i , and make L i ∈(L min ,L max ), S i ∈(S min ,S max ), where L min is the minimum sliding window length, L max is the maximum length of the sliding window, S min is the minimum step size, S max is the maximum step length;

[0026] L i ←L min +rand()×(L max -L min ),

[0027] S i ←S min +rand()×(S max -S min ),

[0028] Where rand() is a function of random numbers in the interval (0,1);

[0029] In L i Greater than S i , calculate the current information entropy function H(L i ,S i ); the current information entropy function H(L i ,S i ) and the current optimal information entropy function H obtained by iteration min Compare:

[0030] If H(L i ,S i )<H min , update the current optimal information entropy function H min , optimal sliding window length L * and the optimal step size S * :

[0031]

[0032] Until the maximum number of iterations is reached, the optimal sliding window length L is obtained * and the optimal step size S * .

[0033] Beneficial effects of the present invention: The method of the present invention constructs an optimized information entropy function based on information entropy, and automatically determines the optimal sliding window length and step size in combination with integer programming. It does not rely on manual experience or trial and error, and is of great significance to improving the analysis accuracy of data in the field of machine learning.

[0034] The method of the present invention is efficient and highly automated, can adapt to different types of time series data, has high applicability and promotion value in multiple application fields, and helps to improve the accuracy, stability and efficiency of data analysis and processing. Information entropy is used as an evaluation indicator, which has strong theoretical support. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a flow chart of the sliding window determination method based on integer programming and information entropy according to the present invention;

[0036] Figure 2 It is a schematic diagram of data selection for sliding windows;

[0037] Figure 3It is a comparison diagram of optimized information entropy function values ​​obtained by the method of the present invention and the existing method. DETAILED DESCRIPTION

[0038] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0039] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0040] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, but they are not intended to limit the present invention.

[0041] Specific implementation method 1. Combination Figure 1 and Figure 2 As shown, the present invention provides a sliding window determination method based on integer programming and information entropy, comprising:

[0042] Preprocess the initial time series data to remove outliers and obtain the total set of preprocessed time series data X = {x1, x2, ..., x n};

[0043] Set the sliding window length L i and step length S i , slice the total set of time series data X to obtain multiple data subsets after slicing

[0044] Where i represents the number of iterations, represents the Nth data subset of the i-th iteration; k∈(1,n), k=(N-1)×S i +1;

[0045] In each iteration, each data subset is calculated The relative mutual information entropy with the total set of time series data X Simultaneously calculate the neighbor mutual information entropy of adjacent data subsets

[0046] Based on all relative mutual information entropy Neighborhood mutual information entropy Construct an optimized information entropy function H and solve the optimized information entropy function H so that the mutual information entropy and minimum relative mutual information entropy On the basis of maximum, the information entropy function H is minimized; at the same time, combined with the integer programming model, the sliding window length L i and step length S i are all integers, and the sliding window length is L i Greater than step size S i ; The final sliding window length L i and step length S i As the optimal sliding window length L * and the optimal step size S * .

[0047] As an example, the method to remove outliers is:

[0048] The 3σ method is used to remove outliers; specifically, it includes calculating the mean μ and standard deviation σ of the initial time series data, removing data points outside the range of μ±3σ as outliers, and obtaining the total set X of preprocessed time series data.

[0049] Furthermore, the relative mutual information entropy The calculation method is:

[0050]

[0051] In the formula for The joint probability distribution of and X is, for The marginal distribution of , p(X) is the marginal distribution of X.

[0052] Neighborhood mutual information entropy of adjacent data subsets The calculation method is:

[0053]

[0054] In the formula for and The joint probability distribution of .

[0055] Going further, the optimized information entropy function H is:

[0056]

[0057] The Monte Carlo method is used to solve the optimal information entropy function H to obtain the optimal sliding window length L * and the optimal step size S * .

[0058] The construction of the information entropy function H enables the current data subset to fully reflect the state of the total data set, while ensuring sufficient inconsistency between adjacent data subsets, so that the constructed data subset can fully retain information while compressing the size of the data set, and can efficiently train the neural network. Information entropy, as an indicator of the amount of information and uncertainty, can reflect the impact of window length on data partitioning.

[0059] In this embodiment, when calculating the information entropy function H, a constraint condition is set so that the optimal sliding window length L * and the optimal step size S * The values ​​are all integers and are within a reasonable range.

[0060] The Monte Carlo method is used to solve the information entropy function H, and different sliding window lengths L are set. i and step length S i Find the length and step size that minimize the objective function, and use the length and step size that meet the conditions as the optimal sliding window conditions.

[0061] The method of using the Monte Carlo method to solve the optimization information entropy function H is:

[0062] First, initialize the optimal information entropy function value H min , optimal sliding window length L * and the optimal step size S * :

[0063] H min ←∞,(L * ,S * )←(0,0);

[0064] Randomly generate sliding window length L i and step length S i , and make L i ∈(L min ,L max ), S i ∈(S min ,S max ), where L min is the minimum sliding window length, L max is the maximum length of the sliding window, S min is the minimum step size, S max is the maximum step length;

[0065] L i ←L min +rand()×(L max -L min ),

[0066] S i ←Smin +rand()×(S max -S min ),

[0067] Where rand() is a function of random numbers in the interval (0,1);

[0068] In L i Greater than S i , calculate the current information entropy function H(L i ,S i ); the current information entropy function H(L i ,S i ) and the current optimal information entropy function H obtained by iteration min Compare:

[0069] If H(L i ,S i )<H min , update the current optimal information entropy function H min , optimal sliding window length L * and the optimal step size S * :

[0070]

[0071] Until the maximum number of iterations is reached, the optimal sliding window length L is obtained * and the optimal step size S * .

[0072] Finally, the optimal sliding window condition and the corresponding objective function value (L * ,S * ,H min ).

[0073] The effect of the present invention is verified by experiments below:

[0074] Taking the satellite momentum wheel speed data as an example, the data length is 10000. The optimization function information entropy value obtained by the optimization method of the present invention is compared with the optimization information entropy value obtained by the empirical method. Figure 3 As shown in the figure, the empirical method selects commonly used lengths of 100, 200, 500, and 1000, and the step length is set to 10% of the data length, 20% of the data length, and 50% of the data length.

[0075] The minimum index function value obtained by the empirical method is 1.32, its window length is 1000, and the step length is 200. The best window length optimized by the method of the present invention is 7592, the step length is 759, and the index function value is -0.219. It can be seen that after the optimization by the method of the present invention, the objective function value has been greatly reduced, indicating that the data set divided according to the current window length and step length can not only fully reflect the state of the total data set, but also ensure sufficient inconsistency between adjacent data subsets, so that the constructed data subset can compress the size of the data set while fully retaining the information.

[0076] Although the present invention is described herein with reference to specific embodiments, it should be understood that these embodiments are merely examples of the principles and applications of the present invention. It should therefore be understood that many modifications may be made to the exemplary embodiments and that other arrangements may be devised without departing from the spirit and scope of the present invention as defined by the appended claims. It should be understood that the various dependent claims and features described herein may be combined in a manner different from that described in the original claims. It should also be understood that the features described in conjunction with a single embodiment may be used in other described embodiments.

Claims

1. A sliding window determination method based on integer programming and information entropy, characterized in that: include: Preprocess the initial time series data to remove outliers and obtain the total set of preprocessed time series data X = {x1, x2, ..., x n }; Set the sliding window length L i and step length S i , slice the total set of time series data X to obtain multiple data subsets after slicing Where i represents the number of iterations, represents the Nth data subset of the i-th iteration; k∈(1,n), k=(N-1)×S i +1; In each iteration, each data subset is calculated The relative mutual information entropy with the total set of time series data X Simultaneously calculate the neighbor mutual information entropy of adjacent data subsets Based on all relative mutual information entropy Neighborhood mutual information entropy Construct an optimized information entropy function H and solve the optimized information entropy function H so that the mutual information entropy and minimum relative mutual information entropy On the basis of maximum, the information entropy function H is minimized; at the same time, combined with the integer programming model, the sliding window length L i and step length S i are all integers, and the sliding window length is L i Greater than step size S i ; The final sliding window length L i and step length S i As the optimal sliding window length L * and the optimal step size S * .

2. The method for determining a sliding window based on integer programming and information entropy according to claim 1, characterized in that: The method to remove outliers is: The mean μ and standard deviation σ of the initial time series data are calculated, and the data points outside the range of μ±3σ are removed as outliers to obtain the total set X of preprocessed time series data.

3. The method for determining a sliding window based on integer programming and information entropy according to claim 2, characterized in that: Relative mutual information entropy The calculation method is: In the formula for The joint probability distribution of and X is, for The marginal distribution of , p(X) is the marginal distribution of X.

4. The method for determining a sliding window based on integer programming and information entropy according to claim 3, characterized in that: Neighborhood mutual information entropy of adjacent data subsets The calculation method is: In the formula for and The joint probability distribution of .

5. The method for determining a sliding window based on integer programming and information entropy according to claim 4, characterized in that: The optimized information entropy function H is: The Monte Carlo method is used to solve the optimal information entropy function H to obtain the optimal sliding window length L * and the optimal step size S * .

6. The method for determining a sliding window based on integer programming and information entropy according to claim 5, characterized in that: The method of using the Monte Carlo method to solve the optimization information entropy function H is: Randomly generate sliding window length L i and step length S i , and make L i ∈(L min ,L max ), S i ∈(S min ,S max ), where L min is the minimum sliding window length, L max is the maximum length of the sliding window, S min is the minimum step size, S max is the maximum step length; L i ←L min +rand()×(L max -L min ), S i ←S min +rand()×(S max -S min ), Where rand() is a function of random numbers in the interval (0,1); In L i Greater than S i , calculate the current information entropy function H(L i ,S i ); the current information entropy function H(L i ,S i ) and the current optimal information entropy function H obtained by iteration min Compare: If H(L i ,S i )<H min , update the current optimal information entropy function H min , optimal sliding window length L * and the optimal step size S * : Until the maximum number of iterations is reached, the optimal sliding window length L is obtained * and the optimal step size S * .

Images