Smooth processing method for surge data value

Abstract

The invention relates to a smoothing processing method for a surge data value. The smoothing processing method is suitable for the situation that an abnormal surge data value occurs in the process that equal-interval time sequence data changes along with time. The method comprises the following steps of: firstly, calculating an average value (Mean, # imgabs0 #) and a standard deviation (Standard Deviation, sigma) of to-be-processed equal-interval time sequence data values, setting a judgment interval capable of judging a sudden increase data value, requiring that the to-be-processed data does not have a value lower than the lower limit of the judgment interval, and marking a data value higher than # imgabs1 # as the sudden increase data value; secondly, accumulating the parts of which the surge data values are higher than # imgabs2; and finally, the accumulated value is redistributed according to the trend proportion of the original data, and smoothing processing of the data is completed. According to the method, the surge data value can be smoothly adjusted, and the accuracy of subsequent data analysis can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a smoothing method for surge data values. Background Art

[0002] Data values ​​may experience atypical surges due to a variety of factors, such as holidays, specific social events, or sudden changes in environmental conditions. This anomaly may cause the results to deviate from expectations during data analysis. Therefore, it is necessary to effectively preprocess such data. This processing method can smoothly adjust data values ​​that show abnormal surges. Through this processing, it can ensure that the data is more accurate in subsequent analysis and prediction. Summary of the invention

[0003] 1. Technical issues to be solved

[0004] In view of the deficiencies of existing research, the present invention provides a method for smoothing surge data values, which solves the problems raised in the background technology.

[0005] (II) Technical solution

[0006] To achieve the above objectives, the present invention is implemented by the following technical scheme: a method for smoothing a surge data value, the steps comprising:

[0007] S1: Calculate the average value of the equally spaced time series data to be processed (Mean, ) and standard deviation (Standarddeviation, σ);

[0008] S2: According to The judgment interval that can judge the surge data value is set by σ, requiring that the data to be processed must not have a value lower than the lower limit of the judgment interval, and the data value higher than the upper limit of the judgment interval is marked as a surge data value;

[0009] S3: Increase the surge data value above The parts are accumulated;

[0010] S4: redistribute the data according to the trend ratio of the original data using the accumulated value calculated in S3 to complete the data smoothing process;

[0011] S5: Whether the data exceeds the normal data value after processing Determine whether cyclic smoothing is required for the data value.

[0012] Preferably, the specific calculation method of the mean value and standard deviation of the equally spaced time series data values ​​to be processed in step S1 is as follows:

[0013]

[0014]

[0015] x i : The equally spaced time series data values ​​to be processed;

[0016] N: The number of equally spaced time series data values ​​to be processed.

[0017] Preferably, the sudden data value determination interval in step S2 is specifically:

[0018]

[0019] The data to be processed must not be less than And the data to be processed falling within the judgment interval are assigned to the set F = {a f |a f is a normal data value}, where the subscript f is the sequence position of the normal data value in the original data to be processed, and the set formed by f is recorded as F Δ ; The data to be processed is higher than The values ​​of are assigned to the set J = {b j |b j is the surge data value}, where the subscript j is the sequence position of the surge data value in the original data to be processed, and the set composed of j is recorded as J Δ .

[0020] Preferably, the specific formula calculated in step S3 is as follows:

[0021]

[0022] C: The surge data value is higher than The cumulative value of the part.

[0023] Preferably, the smoothing method in step S4 is specifically: calculating the original data trend ratio, that is, the ratio of each normal data value in the data to be processed to the sum of all normal data values, the formula is as follows:

[0024]

[0025] h f : Original data trend ratio.

[0026] By increasing the surge data value above The accumulated value of the part is smoothed according to the trend ratio of the original data to obtain the processed data. The formula is as follows:

[0027]

[0028]

[0029] The data value after the surge data value processing;

[0030] Normal data value Processed data value.

[0031] Preferably, the processing method in step S5 is specifically:

[0032] If the processed data value does not exceed the upper limit, the smoothing result is output; if the processed data value exceeds the upper limit, the smoothing result is output. Cyclic smoothing is required.

[0033] The cyclic smoothing method is to accumulate the part of the data that exceeds the upper limit and smooth it according to the original proportion of the surge data value. The formula is as follows:

[0034]

[0035]

[0036]

[0037]

[0038] D: Normal data value after processing data is higher than The cumulative value of the part;

[0039] g: China Super League The sequence position of the processed data value in the original data to be processed, the set composed of g is recorded as G Δ ;

[0040] Normal data values ​​that exceed the upper limit after smoothing;

[0041] h j : original scale;

[0042] The surge data value is the data value after the cyclic processing;

[0043] Normal data values ​​exceeding the upper limit are processed in a loop.

[0044] (III) Beneficial effects

[0045] The present invention provides a smoothing method for surge data values. The present invention has the following beneficial effects: The present invention provides a data smoothing method, which can effectively process a group of equally spaced time series data with surge data values. The present invention can weaken certain factors affecting the data, thereby revealing more accurate information and improving the accuracy and effect of data analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A step diagram of a method for smoothing a surge data value according to the present invention;

[0047] Figure 2 This is a data value judgment chart for the surge in confirmed disease information in 2020;

[0048] Figure 3 This is the smoothing result chart of the 2020 data values;

[0049] Figure 4 This is a judgment chart for whether to perform cyclic smoothing on the processed data values ​​in 2020. DETAILED DESCRIPTION

[0050] In order to enable those skilled in the art to better understand the technical solution of the present invention, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0051] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the present invention is further explained below in conjunction with specific implementation methods.

[0052] The method for smoothing the surge data value includes the following steps:

[0053] S1: Calculate the average value of the equally spaced time series data to be processed (Mean, ) and standard deviation (Standarddeviation, σ);

[0054] S2: According to The judgment interval that can judge the surge data value is set by σ, requiring that the data to be processed must not have a value lower than the lower limit of the judgment interval, and the data value higher than the upper limit of the judgment interval is marked as a surge data value;

[0055] S3: Increase the surge data value above The parts are accumulated;

[0056] S4: redistribute the data according to the trend ratio of the original data using the accumulated value calculated in S3 to complete the data smoothing process;

[0057] S5: Whether the data exceeds the normal data value after processing Determine whether cyclic smoothing is required for the data value.

[0058] Preferably, the specific calculation method of the mean value and standard deviation of the equally spaced time series data values ​​to be processed in step S1 is as follows:

[0059]

[0060]

[0061] x i : The equally spaced time series data values ​​to be processed;

[0062] N: The number of equally spaced time series data values ​​to be processed.

[0063] Preferably, the sudden data value determination interval in step S2 is specifically:

[0064]

[0065] The data to be processed must not be less than And the data to be processed falling within the judgment interval are assigned to the set F = {a f |a f is a normal data value}, where the subscript f is the sequence position of the normal data value in the original data to be processed, and the set formed by f is recorded as F Δ ; The data to be processed is higher than The values ​​of are assigned to the set J = {b j |b j is the surge data value}, where the subscript j is the sequence position of the surge data value in the original data to be processed, and the set composed of j is recorded as J Δ .

[0066] Preferably, the specific formula calculated in step S3 is as follows:

[0067]

[0068] C: The surge data value is higher than The cumulative value of the part.

[0069] Preferably, the smoothing method in step S4 is specifically: calculating the original data trend ratio, that is, the ratio of each normal data value in the data to be processed to the sum of all normal data values, the formula is as follows:

[0070]

[0071] h f : Original data trend ratio.

[0072] By increasing the surge data value above The accumulated value of the part is smoothed according to the trend ratio of the original data to obtain the processed data. The formula is as follows:

[0073]

[0074]

[0075] The data value after the surge data value processing;

[0076] Normal data value Processed data value.

[0077] Preferably, the processing method in step S5 is specifically:

[0078] If the processed data value does not exceed the upper limit, the smoothing result is output; if the processed data value exceeds the upper limit, the smoothing result is output. Cyclic smoothing is required.

[0079] The cyclic smoothing method is to accumulate the part of the data that exceeds the upper limit and smooth it according to the original proportion of the surge data value. The formula is as follows:

[0080]

[0081]

[0082]

[0083]

[0084] D: Normal data value after processing data is higher than The cumulative value of the part;

[0085] g: China Super League The sequence position of the processed data value in the original data to be processed, the set composed of g is recorded as G Δ ;

[0086] Normal data values ​​that exceed the upper limit after smoothing;

[0087] h j : original scale;

[0088] The surge data value is the data value after the cyclic processing;

[0089] Normal data values ​​exceeding the upper limit are processed in a loop.

[0090] Implementation examples:

[0091] Taking the weekly confirmed information data of a certain disease collected in a certain place in 2020 as an example, the specific implementation steps are as follows:

[0092] S1: Calculate the mean and standard deviation of the weekly confirmed information data of a disease in 2020:

[0093]

[0094]

[0095] S2: Set the surge data value judgment interval: (110, 645). The number of people diagnosed with the disease in the first week of 2020 is 689, which is marked as a surge data value. The number of people diagnosed with the disease in the thirty-third week of 2020 is 378, which is marked as a normal data value. The specific judgment results are as follows: Figure 2 shown.

[0096] S3: Calculate the surge data value higher than The cumulative value of the part:

[0097] C = 1356-378-267 = 711

[0098] S4: Calculate the trend ratio of the original data of the number of confirmed cases of the disease in the 33rd week of 2020:

[0099]

[0100] The number of confirmed cases of the disease in the first week and the thirty-third week of 2020 was smoothed to obtain the processed data values:

[0101]

[0102]

[0103] The specific calculation results are as follows Figure 3 shown.

[0104] S5: Determine whether the processed data value of the normal data value exceeds the upper limit of the judgment interval of the surge data value (645). There is no data value exceeding the upper limit in the processed data value of the number of confirmed diseases in 2020, and output the smoothing processing result. The specific judgment result is as follows Figure 4 shown.

[0105] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and the present invention can be implemented in other specific forms without departing from the spirit or basic features of the present invention. Therefore, no matter from which point of view, the embodiments should be regarded as exemplary and non-restrictive. The scope of the present invention is defined by the attached claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any figure mark in the claims should not be regarded as limiting the claims involved.

[0106] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

Claims

1. A method for smoothing surge data values, characterized in that: The steps include: S1: Calculate the average value of the equally spaced time series data to be processed (Mean, ) and standard deviation (Standarddeviation, σ); S2: According to The judgment interval that can judge the surge data value is set by σ, requiring that the data to be processed must not have a value lower than the lower limit of the judgment interval, and the data value higher than the upper limit of the judgment interval is marked as a surge data value; S3: Increase the surge data value above The parts are accumulated; S4: redistribute the data according to the trend ratio of the original data using the accumulated value calculated in S3 to complete the data smoothing process; S5: Whether the data exceeds the normal data value after processing Determine whether cyclic smoothing is required for the data value.

2. The method for smoothing surge data values ​​according to claim 1, characterized in that: The specific calculation method of the mean value and standard deviation of the equally spaced time series data values ​​to be processed in step S1 is as follows: x i : The equally spaced time series data values ​​to be processed; N: The number of equally spaced time series data values ​​to be processed.

3. The method for smoothing surge data values ​​according to claim 1, characterized in that: The specific interval for determining the surge data value in step S2 is: The data to be processed must not be less than And the data to be processed falling within the judgment interval are assigned to the set F = {a f |a f is a normal data value}, where the subscript f is the sequence position of the normal data value in the original data to be processed, and the set formed by f is recorded as F Δ ; The data to be processed is higher than The values ​​of are assigned to the set J = {b j |b j is the surge data value}, where the subscript j is the sequence position of the surge data value in the original data to be processed, and the set composed of j is recorded as J Δ .

4. The method for smoothing surge data values ​​according to claim 1, characterized in that: The specific formula calculated in step S3 is as follows: C: The surge data value is higher than The cumulative value of the part.

5. The method for smoothing surge data values ​​according to claim 1, characterized in that: The smoothing method in step S4 is specifically: calculating the original data trend ratio, that is, the ratio of each normal data value in the data to be processed to the sum of all normal data values, the formula is as follows: h f : Original data trend ratio. By increasing the surge data value above The accumulated value of the part is smoothed according to the trend ratio of the original data to obtain the processed data. The formula is as follows: The data value after the surge data value processing; Normal data value Processed data value.

6. The method for smoothing surge data values ​​according to claim 1, characterized in that: The processing method in step S5 is specifically: If the processed data value does not exceed the upper limit, the smoothing result is output; if the processed data value exceeds the upper limit, the smoothing result is output. Cyclic smoothing is required. The cyclic smoothing method is to accumulate the part of the data that exceeds the upper limit and smooth it according to the original proportion of the surge data value. The formula is as follows: D: Normal data value after processing data is higher than The cumulative value of the part; g: China Super League The sequence position of the processed data value in the original data to be processed, the set composed of g is recorded as G Δ ; Normal data values ​​that exceed the upper limit after smoothing; h j : original scale; The surge data value is the data value after the cyclic processing; Normal data values ​​exceeding the upper limit are processed in a loop.

Images