A random denoising statistical method and system based on accumulated semaphore

Abstract

The application discloses a random denoising statistical method and system based on accumulated semaphore, which quickly traverses a historical sample set in a statistical period, and collects sample features; in combination with the sample features, noise data is processed, and the historical sample set and the sample features are synchronously updated; according to the updated sample features, accumulated results are collected from the updated historical sample set; according to the sample features, the updated historical sample set and the sample features, and the collected accumulated results, an accumulated semaphore measuring point statistical algorithm device is designed, sample features are generated by traversal, noise data is processed according to noise types, the sample features and the historical sample set are updated, and correct accumulated results are newly collected. According to user request parameters, the application automatically identifies noise data, processes noise data according to noise types, collects and statistically correct accumulated results, reduces human intervention cost, and improves statistical efficiency and accuracy.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of measuring point data statistics, and particularly relates to a random denoising statistical method and system based on cumulative signal quantity. BACKGROUND

[0002] Industrial manufacturing enterprises use sensors to collect various data from equipment, among which there is a signal, the value of which is continuously accumulated and constantly increasing until the upper limit of the instrument or is manually cleared. This signal is a cumulative signal quantity. Cumulative signal quantity exists widely in industrial manufacturing enterprises. For centralized control and management, cumulative signal quantity is collected and stored into a real-time database through a communication network. The upper application obtains historical data of the cumulative signal quantity measuring point from the real-time database, and summarizes and statistics the cumulative quantity in any period.

[0003] Cumulative signal quantity from sensors, through a communication network and a collection program to a real-time database, sometimes data mutation, misplacement and other abnormalities occur at a certain link, causing noise data in the archived history of the cumulative signal quantity measuring point. Noise data will interfere with the summary and statistical analysis of the cumulative quantity, and if not properly handled, it will cause inaccurate statistical results.

[0004] Noise data is inevitable, and current noise data processing is mainly on the application side. When the application side summarizes and statistics the cumulative signal quantity measuring point, if the result is abnormal, the noise data is manually filtered, deleted or replaced, and then re-summarized. However, a large amount of cumulative signal quantity summary and statistics will inevitably consume a large amount of human cost. Therefore, how to automatically identify and filter noise to reduce labor costs and improve accuracy has become a problem that needs to be solved in the statistical process of the cumulative signal quantity measuring point. SUMMARY

[0005] The technical problem to be solved by the present application is to provide a random denoising statistical method and system based on cumulative signal quantity to solve the statistical problem of the cumulative signal quantity measuring point of industrial manufacturing enterprises caused by noise data, reduce the cost of manual data verification, and improve the accuracy of the cumulative quantity measuring point summary data.

[0006] The present application adopts the following technical solutions:

[0007] A random denoising statistical method based on cumulative signal quantity, comprising the following steps:

[0008] S1, quickly traversing the historical sample set of the statistical period, and summarizing the sample features;

[0009] S2, processing noise data in combination with the sample features obtained in step S1, and synchronously updating the historical sample set and the sample features;

[0010] S3, according to the sample features updated in step S2, summarizing the cumulative result from the updated historical sample set;

[0011] S4. Based on the sample features in step S1, the historical sample set and sample features updated in step S2, and the cumulative results summarized in step S3, design a cumulative signal measurement point statistical algorithm, traverse to generate sample features, process noise data according to noise type, update sample features and historical sample set, and re-summarize to generate correct cumulative results.

[0012] Specifically, in step S1, the summarized sample features are as follows:

[0013] Feature = f(x0, x len-1 ,x max ,x min ,turnDownSet)

[0014] Where, x0, x len-1 x max x min These are the first, last, maximum, and minimum values ​​of the historical sample set X, respectively. len is the length of the historical sample set X, and turnDownSet is the set of suspected noisy data found by traversing the historical sample set X.

[0015] Furthermore, if the data is determined to be suspected noise, then i-1 is added to the TurnDownSet collection. The rule for determining suspected noise data is as follows:

[0016] x i-1 >x i Then x i-1 This data is suspected to be noisy.

[0017] Specifically, in step S2, processing the noisy data involves:

[0018] Analyze the features of the historical sample set X, select a denoising strategy, sequentially traverse the set of suspected noise locations in the TurnDownSet, identify the noise type at each suspected noise location i and process it, and then generate a new historical sample set X. new .

[0019] Furthermore, the denoising strategy is as follows:

[0020] If Len(TurnDownSet) = 0, there is no noise; if Len(TurnDownSet) > 0, the sample set contains suspected noisy data.

[0021] Furthermore, the specific steps for identifying and processing the noise type at each suspected noise location i are as follows:

[0022] S201. Obtain noise data x from the TurnDownSet collection.i At the serial number i of the sample set X;

[0023] S202, obtaining a random number r according to the formula;

[0024] S203, dividing three sample areas according to i obtained in step S201 and the random number r obtained in step S202;

[0025] S204, counting the standard deviation of the three sample areas divided in step S203;

[0026] S205, according to the standard deviation divided in step S204, accumulating the essence of the signal quantity change is the change of the production business, determining the noise type;

[0027] S206, processing noise data according to the noise type determined in step S205, if misplacement, swapping positions from the order i forward until the front order is adjusted; if mutation, eliminating x i ; if normal, clearing the table and keeping;

[0028] S207, repeating steps S201 to S206 to process each suspected noise data until the processing is completed, generating a new historical sample set X new .

[0029] Further, in step S205, the normal change of the cumulative signal quantity is within the three standard deviations divided in step S204, as follows:

[0030] If σ1 / σ0<=3&&σ1 / σ2<=3&&x i+1 / x i-1 >=0.8, x i belongs to misplacement;

[0031] If σ1 / σ0<=3&&σ1 / σ2<=3&&x i+1 / x i-1 <0.8, x i belongs to normal clearing the table;

[0032] If σ1 / σ0>3&&σ1 / σ2>3, x i belongs to mutation;

[0033] The three sample areas are as follows:

[0034] X (i-r,i-1) =[i-r,i-1]

[0035] X (i-r,i) =[i-r,i]

[0036] X (i+1,i+r) =[i+1,i+r]

[0037] The standard deviations of the three sample intervals are as follows:

[0038] σ0 = StdDev(X (i-r,i-1) )

[0039] σ1 = StdDev(X (i-r,i) )

[0040] σ2 = StdDev(X (i+1,i+r) ).

[0041] Specifically, in step S3, the updated historical sample set X new is quickly traversed to summarize a new sample feature Feature new The statistical method is selected:

[0042] If Len(Feature new .TurnDownSet) == 0 is established, the cumulative amount is calculated as follows:

[0043] Cumulative amount = Feature new .x len-1 - Feature new .x0

[0044] If Len(Feature new .TurnDownSet) > 0 is established, the cumulative amount is calculated as follows:

[0045] Cumulative amount = Feature new .x max + Feature new .x len-1 - Feature new .x0

[0046] Where Feature new .x len-1 is the last sample data of the new historical sample set after processing the noise data, Feature new .x0 is the first data of the new historical sample set after processing the noise data, and Feature new .x max is the maximum sample data of the new historical sample set after processing the noise data.

[0047] Specifically, in step S4, when the cumulative signal measurement point statistical algorithm device is started, an access link is established through a real-time database connection pool, if the connection fails, log information that the real-time database cannot be connected is output, and the starting is stopped and directly exited; if the connection is successful, the loading is continued, and a statistical request forwarded by a user interface is waited for; a real-time connection is obtained from the real-time database connection pool, historical sample data sets are read through the real-time connection in combination with a request measurement point and a request period; the historical sample sets are traversed, sample features are generated; noise positions collected based on the sample features are combined with a preset algorithm to identify noise types one by one, noise data is processed according to the noise types, new historical sample sets and sample features are generated; and correct cumulative results are summarized according to the new historical sample sets and sample features.

[0048] Another technical solution of the application is a random denoising statistical system based on cumulative signal quantity, comprising:

[0049] A traversal module is used for quickly traversing historical sample sets in a statistical period and summarizing sample features.

[0050] An updating module is used for processing noise data in combination with the sample features obtained by the traversal module, and synchronously updating the historical sample sets and the sample features.

[0051] A cumulative module is used for summarizing cumulative results from the updated historical sample sets according to the sample features updated by the updating module.

[0052] A statistical module is used for designing a cumulative signal measurement point statistical algorithm device according to the sample features, the updated historical sample sets and the sample features, and the cumulative results summarized by the cumulative module, generating sample features, processing noise data according to noise types, updating the sample features and the historical sample sets, and re-summarizing to generate correct cumulative results.

[0053] Compared with the prior art, the application has at least the following beneficial effects:

[0054] The random denoising statistical method based on cumulative signal quantity analyzes historical sample sets in a cumulative quantity signal measurement point statistical period, obtains sample features, adopts a random binning method according to the sample features, statistically processes standard deviations of three random bins before, in and after noise data, confirms noise according to a preset rule, removes noise, adjusts to generate new historical sample sets, and automatically identifies and removes noise data through the new historical sample sets to summarize cumulative quantities in a statistical period, thereby reducing the cost of manual data verification and improving the statistical accuracy of cumulative quantity measurement point data.

[0055] Further, the historical sample set is quickly traversed to collect data such as suspected noise positions, maximum values, minimum values, first and last values in the sample set, to determine whether the original sample set has noise data, and if there is no noise data, the sample characteristics can be directly summarized to accumulate the results, which is fast and efficient. The cumulative amount continues to grow, and if mutation, misplacement or turnover occurs, the situation where the latter number is less than the former number will occur, the position where the situation occurs is collected, and the suspected noise data position is determined, so as to prepare data for further analysis and processing of noise data.

[0056] Further, the judgment and identification of suspected noise data are based on whether the out-of-range change caused by noise data is in a reasonable interval. The historical sample interval collected during the continuous growth of the cumulative signal amount is generally several seconds to several minutes, and after the occurrence of suspected noise, the data growth in a very short time is generally within 3 standard deviations, and if it is greater than 3 standard deviations, it can be considered as data mutation; if it is within 3 standard deviations, and the latter value grows to 80% of the former value within 10 seconds, it can be judged as misplacement, because if it is normal turnover, it is equivalent to starting from zero, and it is impossible to grow to 80% of the former value in a short time. The data presets of 3 standard deviations and 80% are mainly derived from experience accumulation and induction, and practice has proved that they can indeed play a preset role.

[0057] Further, after the suspected noise data is disposed, a new historical sample set is generated, new sample characteristics are generated by re-traversing, and the new sample characteristics are used to summarize the cumulative results. According to the length of the statistical period, there may be multiple turnover situations in the sample set, and the new sample characteristics are used to identify normal turnover situations. If there is no turnover, the cumulative amount continues to grow, and the cumulative results can be obtained by subtracting the first value from the last value; if there is turnover, the cumulative results of each segment are summarized as the total result according to the sample characteristics.

[0058] Further, a cumulative amount algorithm device is designed to automatically calculate the cumulative signal amount results according to user requests, automatically identify noise data, process noise data, and combine sample characteristics to summarize correct cumulative results during the calculation process. Reducing manual intervention can improve work efficiency and ensure the correctness of the cumulative signal amount statistical results.

[0059] In summary, the present application is derived from the practice summary of application, and there are a large number of cumulative signal amounts in power generation enterprises, such as power generation meter code, desulfurization equipment consumable meter code, etc. From the device sensor to the real-time database, there are inevitably some data mutations and misplacements caused by communication, network, faults, etc., which are manually intervened and processed each time, the sample data collected in the real-time database is modified, which is unsafe and easy to misoperate, low in efficiency and causes waste of manpower. In order to solve such problems, the present method is generated through continuous research and summary, which realizes automatic identification of noise and filtering of noise, reduces labor cost and improves accuracy.

[0060] The technical solutions of the present application will be further described in detail below with reference to the drawings and examples. BRIEF DESCRIPTION OF DRAWINGS

[0061] Figure 1 A schematic diagram of the present application is shown in the figure.

[0062] Figure 2 A schematic diagram of the method for removing noise data for automatic identification of standard deviation of randomly binned samples is shown in the figure.

[0063] Figure 3 A schematic diagram of the cumulative quantity statistic is shown in the figure. DETAILED DESCRIPTION

[0064] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0065] In the description of the present application, it should be understood that the terms "comprise" and "include" indicate the existence of described features, integers, steps, operations, elements and / or components, but do not exclude one or more other features, integers, steps, operations, elements, components and / or sets thereof.

[0066] It should also be understood that the terms used in the specification of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification and the appended claims of the present application, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include the plural forms.

[0067] It should be further understood that the term "and / or" used in the specification and the appended claims of the present application means any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0068] Various structural schematic diagrams according to the disclosed embodiments of the present application are shown in the drawings. These diagrams are not drawn to scale, in which certain details are exaggerated for the purpose of clear expression, and certain details can be omitted. The shapes of various regions, layers and their relative sizes and positional relationships shown in the figures are only exemplary, and in actuality, there can be deviations due to manufacturing tolerances or technical limitations, and regions / layers with different shapes, sizes and relative positions can be additionally designed by those skilled in the art according to actual needs.

[0069] Please refer to Figure 1The application is a random denoising statistical method based on cumulative semaphore, comprising the following steps:

[0070] S1, quickly traversing the historical sample set of the statistical period, and collecting sample features;

[0071] reading a historical sample set X={x0, x1, x2,...x n} of a time cumulative semaphore Tag from a real-time database;

[0072] quickly traversing the historical sample set X, and collecting sample features Feature:

[0073] Feature=f(x0, x len-1 , x max , x min , turnDownSet)

[0074] wherein x0, x len-1 , x max , x min are the first value, the last value, the maximum value and the minimum value of the historical sample set X respectively, len is the length of the historical sample set X, and turnDownSet is a suspected noise data set found by traversing the sample set X.

[0075] TurnDownSet={i1, i2, i3,....i n}

[0076] wherein i is the serial number of the noise data x i in the sample set X, and according to the continuous growth characteristics of the cumulative semaphore, the suspected noise data judgment rule is:

[0077] x i-1 >x i , x i-1 is suspected noise data;

[0078] if it is judged that suspected noise is generated, i-1 is recorded in the TurnDownSet set.

[0079] S2, combining the sample features, processing the noise data, and synchronously updating the historical sample set and the sample features;

[0080] analyzing the sample features Feature of the historical sample set X, and selecting a denoising strategy:

[0081] if Len(TurnDownSet)==0 is established, there is no noise and no processing is needed;

[0082] The TurnDownSet suspected noise position set is sequentially traversed, and the following steps are used to identify the noise type of each suspected noise position i:

[0083] S201, obtaining noise data x from the TurnDownSet set i In the sequence number i of the sample set X;

[0084] S202, obtaining a random number r according to the formula, as follows:

[0085] r = Rand (1, 100)

[0086] S203, dividing three sample areas according to i obtained in step S201 and the random number r obtained in step S202 as follows:

[0087] X (i-r,i-1) = X [i-r, i-1]

[0088] X (i-r,i) = X [i-r, i]

[0089] X (i+1,i+r) = X [i+1, i+r]

[0090] S204, calculating the standard deviations of the three sample intervals divided in step S203:

[0091] σ0 = StdDev (X (i-r,i-1) )

[0092] σ1 = StdDev (X (i-r,i) )

[0093] σ2 = StdDev (X (i+1,i+r) )

[0094] S205, according to the standard deviations divided in step S204, accumulating the essence of signal quantity change is the change of production business, determining the noise type;

[0095] Normal change is within the three standard deviations divided in step S204, as follows:

[0096] If σ1 / σ0 <= 3 && σ1 / σ2 <= 3 && x i+1 / x i-1 >= 0.8, x i belongs to misplacement;

[0097] If σ1 / σ0 <= 3 && σ1 / σ2 <= 3 && x i+1 / x i-1 < 0.8, x i belongs to normal zero translation table;

[0098] If σ1 / σ0>3 && σ1 / σ2>3, then x i belongs to mutation;

[0099] S206, according to the noise type determined in step S205, respectively processing noise data;

[0100] If misalignment, from the order i forward to adjust the position until the front order are adjusted;

[0101] If mutation, then eliminate x i ;

[0102] If normal clear table, then keep;

[0103] S207, loop step S201 to step S206, processing each suspected noise data until the processing is completed, generating a new historical sample set X new .

[0104] Identify noise data type, process noise position i, eliminate noise position from sample features, and update sample features synchronously.

[0105] S3, according to the new sample features, from the historical sample set X new updated in step S2, summarize the cumulative results;

[0106] Fast traversal of the updated historical sample set X new , summarize to generate new sample features Feature new Select statistical method:

[0107] If Len(Feature new .TurnDownSet)==0 is established, the cumulative amount calculation formula is:

[0108] Cumulative amount = Feature new .x len-1 -Feature new .x0

[0109] If Len(Feature new .TurnDownSet)>0 is established, the cumulative amount calculation formula is:

[0110] Cumulative amount = Feature new .x max +Feature new .x len-1 -Feature new .x0

[0111] S4, according to the traversal, identification, denoising and summarizing process of steps S1 to S3, the algorithm model is constructed, the cumulative signal measurement point summation algorithm is designed based on the algorithm model, the user request is received, the historical sample set is read from the real-time library according to the request measurement point and the request period, the historical sample set is quickly traversed, and the sample features are generated; if there is no suspected noise data in the sample features, the cumulative result is directly obtained by subtracting the first value from the last value according to the cumulative signal quantity characteristics; if there is suspected noise data, the suspected noise position is traversed one by one, the noise type is identified and judged one by one, the noise data is processed according to the preset judgment rule of the noise type set, and the sample features and the historical sample set are updated synchronously; according to the updated sample features and the historical sample set, if there are still multiple normal turnover tables, the cumulative results are summarized and merged in sections, and the cumulative results of the whole are merged and summarized; if there is no normal turnover table, the cumulative result is obtained by subtracting the first value from the last value.

[0112] The cumulative signal measurement point summation algorithm, referred to as a "cumulative quantity statistic", comprises a real-time database access connection pool, a cumulative quantity algorithm and a user request interface module.

[0113] Among them, the cumulative quantity algorithm is an algorithm implementation module designed according to the sample features of step S1, the random denoising process of S2, and the updated sample features and historical sample set of S3.

[0114] Please refer to Figure 2 , the suspected noise data is Xi, a random number r is obtained, and three boxes are established.

[0115] Box 1: X[i-r, i-1] is a normal data set before the suspected noise data position, and the standard deviation σ1 of box 1 is calculated and obtained;

[0116] Box 2: X[i-r, i] is a data set containing suspected noise data, and the standard deviation σ2 of box 2 is calculated and obtained;

[0117] Box 3: X[i+1, i+r] is a data set after the suspected noise data, and the standard deviation σ3 of box 3 is calculated and obtained;

[0118] According to the standard deviations of box 1, box 2 and box 3, the noise type is identified according to the preset rule, and the processing mode is determined.

[0119] Please refer to Figure 3When the cumulative amount statisticator starts, firstly, an access link is established through a real-time database connection pool, if the connection fails, log information that the real-time database cannot be connected is outputted, and the starting is stopped and directly exited; if the connection succeeds, the cumulative amount algorithmic module is continued to be loaded, and is ready for use, and waits for a statistical request forwarded by a user interface; the user request interface is published based on multiple protocols, so that the client can access in multiple ways, and in addition, the user request interface is designed and encapsulated based on user demand, and the summary demand of the user is tried to meet.

[0120] In still another embodiment of the present application, a random denoising statistical system based on cumulative signal quantity is provided, which can be used to realize the random denoising statistical method based on cumulative signal quantity.

[0121] The traversal module quickly traverses the historical sample set in a statistical period, and aggregates sample features;

[0122] The updating module processes noise data in combination with the sample features obtained by the traversal module, and synchronously updates the historical sample set and the sample features;

[0123] The accumulation module aggregates cumulative results from the updated historical sample set according to the sample features updated by the updating module;

[0124] The statistical module designs a cumulative signal quantity measurement point statistical algorithmic module according to the sample features, the updated historical sample set and the sample features, and the cumulative results aggregated by the accumulation module, traverses to generate sample features, processes noise data according to noise types, updates the sample features and the historical sample set, and re-aggregates to generate correct cumulative results.

[0125] In still another embodiment of the present application, a terminal device is provided, which comprises a processor and a memory, the memory being configured to store a computer program, the computer program comprising program instructions, and the processor being configured to execute the program instructions stored in the computer storage medium. The processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., which are the computing core and control core of the terminal, and are suitable for implementing one or more instructions, and are particularly suitable for loading and executing one or more instructions to implement a corresponding method flow or a corresponding function; the processor in the embodiments of the present application can be used for the operation of the random denoising statistical method based on the cumulative signal quantity, comprising:

[0126] The statistical time period history sample set is quickly traversed to summarize sample features; the noise data is processed in combination with the sample features, and the history sample set and the sample features are synchronously updated; the cumulative result is summarized from the updated history sample set according to the updated sample features; the cumulative signal quantity measuring point statistical algorithm is designed according to the sample features, the updated history sample set and the sample features, and the summarized cumulative result; the sample features are generated by traversal, the noise data is processed according to the noise type, the sample features and the history sample set are updated, and the correct cumulative result is re-summarized.

[0127] In still another embodiment of the present application, the present application further provides a storage medium, specifically a computer readable storage medium (Memory), which is a memory device in the terminal device, and is used for storing programs and data. It can be understood that the computer readable storage medium herein can include the built-in storage medium in the terminal device, and of course can also include the expansion storage medium supported by the terminal device. The computer readable storage medium provides a storage space, and the storage space stores the operating system of the terminal. Furthermore, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space, and the instructions can be one or more computer programs (including program codes). It should be noted that the computer readable storage medium herein can be a high-speed RAM memory, or a non-volatile memory such as at least one disk memory.

[0128] The one or more instructions stored in the computer readable storage medium can be loaded and executed by the processor to implement the corresponding steps of the random denoising statistical method based on the cumulative signal amount in the above embodiments; the one or more instructions stored in the computer readable storage medium are loaded and executed by the processor to implement the following steps:

[0129] The statistical period history sample set is quickly traversed to summarize sample characteristics; the noise data is processed in combination with the sample characteristics, and the history sample set and the sample characteristics are synchronously updated; the cumulative result is summarized from the updated history sample set according to the updated sample characteristics; the cumulative signal amount measuring point statistical algorithm is designed according to the sample characteristics, the updated history sample set and the sample characteristics, and the summarized cumulative result; the sample characteristics are generated by traversal, the noise data is processed according to the noise type, the sample characteristics and the history sample set are updated, and the correct cumulative result is re-summarized.

[0130] To make the objects, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some embodiments of the present application but not all the embodiments. The components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. All other embodiments obtained by a person of ordinary skill in the art without creative work on the basis of the embodiments in the present application belong to the scope of protection of the present application.

[0131] In a certain power plant of China Huaneng, the measuring point GKDL2.DL0402 is #3 generator (reference)_positive active (total), and data mutation, dislocation and the like may occur occasionally.

[0132] Mutation data examples:

[0133] 1.26460[2021-01-21 01:15:30]GOOD

[0134] 2.26460[2021-01-21 01:15:39]GOOD

[0135] 3.26460[2021-01-21 01:15:49]GOOD

[0136] 4.26960[2021-01-21 01:16:33]GOOD

[0137] 5.26960[2021-01-21 01:16:53]GOOD

[0138] 6 1636087998989983[2021-01-21 01:17:10]GOOD data mutation position

[0139] 7 26960[2021-01-21 01:01:25]GOOD

[0140] 8 27460[2021-01-21 01:01:34]GOOD

[0141] 9 27460[2021-01-21 01:01:44]GOOD

[0142] 10 27460[2021-01-21 01:01:53]GOOD

[0143] 11 27960[2021-01-21 01:02:04]GOOD

[0144] Assuming the random number is 5, the box 1 interval is sample 1 to sample 5, and the standard deviation is calculated:

[0145] σ1=StdDev(26460,26460,2640,26960,26960)=244.9489743

[0146] The box 2 interval is sample 1 to sample 6, and the standard deviation is calculated:

[0147] σ2=StdDev(26460,26460,2640,26960,26960,1636087998989983)=6.09734E+14

[0148] The box 3 interval is sample 7 to sample 11, and the standard deviation is calculated:

[0149] σ3=StdDev(26960,27460,27460,27460,27960)=316.227766

[0150] As can be seen, σ2 / σ1>3&&σ2 / σ3>3 is true, confirming that sample 6 belongs to data mutation, and directly excluding the sample data.

[0151] Misplacement data example:

[0152] 1.15860[2021-10-01 00:01:30]GOOD

[0153] 2.16360[2021-10-01 00:01:39]GOOD

[0154] 3.16360[2021-10-01 00:01:49]GOOD

[0155] 4.16360[2021-10-01 00:01:58]GOOD

[0156] 5.16360[2021-10-01 00:02:07]GOOD

[0157] 6.16360[2021-10-01 00:02:16]GOOD

[0158] 7.16360[2021-10-01 00:02:25]GOOD

[0159] 8.16860[2021-10-01 00:02:34]GOOD data misalignment position

[0160] 9.16360[2021-10-01 00:02:44]GOOD

[0161] 10.16360[2021-10-01 00:02:53]GOOD

[0162] 11.16360[2021-10-01 00:03:02]GOOD

[0163] 12.16360[2021-10-01 00:03:11]GOOD

[0164] 13.16360[2021-10-01 00:03:20]GOOD

[0165] 14.16360[2021-10-01 00:03:29]GOOD

[0166] 15.16860[2021-10-01 00:03:39]GOOD

[0167] 16.16860[2021-10-01 00:03:48]GOOD

[0168] 17.16860[2021-10-01 00:03:57]GOOD

[0169] 18.16860[2021-10-01 00:04:06]GOOD

[0170] 19.16860[2021-10-01 00:04:15]GOOD

[0171] Assume the random number is 7,

[0172] The box 1 interval is sample 1 to sample 7, and the standard deviation is calculated:

[0173] σ1=StdDev(15860,16360,16360,16360,16360,16360,16360)=174.9635531

[0174] The box 2 interval is sample 1 to sample 8, and the standard deviation is calculated:

[0175] σ2=StdDev(15860,16360,16360,16360,16360,16360,16360,16860)=250

[0176] The box 3 interval is sample 9 to sample 15, and the standard deviation is calculated:

[0177] σ3=StdDev(16360,16360,16360,16360,16360,16360,16860)=174.9635531

[0178] Therefore, σ2 / σ1<3&&σ2 / σ3<3&&sample 9 / sample 8>=0.8 are established, and it is confirmed that sample 8 belongs to data misplacement, and the moving processing is moved to the correct position.

[0179] In summary, the random denoising statistical method and system based on cumulative signal quantity provided by the application utilize the designed cumulative quantity statistic device, automatically identify noise data according to user request parameters, process noise data according to noise types, and collect and statistically correct cumulative results, so as to reduce human intervention cost and improve statistical efficiency and accuracy.

[0180] Those skilled in the art should understand that the embodiments of the application can be provided as a method, a system, or a computer program product. Therefore, the application can adopt a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the application can adopt a computer program product in the form of one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program codes.

[0181] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flow or blocks. Figure 1 one or more flow or blocks.

[0182] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks. Figure 1 one or more flow or blocks. Figure 1 one or more flow or blocks.

[0183] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flow or blocks. Figure 1 one or more flow or blocks.

[0184] The above merely provides the technical idea of the present application, and cannot be used to limit the protection scope of the present application. Any modification made according to the technical idea of the present application, on the basis of the technical solutions, falls within the protection scope of the claims of the present application.

Claims

1. A statistical method for random denoising based on accumulated semaphores, characterized in that, Includes the following steps: S1. Quickly traverse the historical sample set for the statistical period and summarize the sample characteristics; S2. Combining the sample features obtained in step S1, process the noisy data and synchronously update the historical sample set and sample features. The specific process for processing the noisy data is as follows: Analysis of historical sample sets Sample features Select a denoising strategy and iterate sequentially. Set of suspected noise locations, identify each suspected noise location. The noise type is identified and processed, and a new historical sample set is generated after processing. ; Identify each suspected noise location The noise type is identified and processed as follows: S201, from Obtaining noise data from the collection In the sample set Serial number ; S202. Obtain random numbers according to the formula. ; S203, Based on the information obtained in step S201 And the random number obtained in step S202 Divide the sample into three regions; S204, Standard deviation of the three sample intervals divided in statistical step S203; S205. Based on the standard deviation defined in step S204, the cumulative signal quantity change is essentially a change in production operations, and the noise type is determined. S206. Process the noise data according to the noise type determined in step S205. If there is a misalignment, start from the correct order. Rearrange the items forward until the previous order has been adjusted; if there is a sudden change, remove the affected item. ; If the table is reset and flipped normally, retain the result; S207. Repeat steps S201 to S206, processing each suspected noisy data point until processing is complete, generating a new historical sample set. ; S3. Based on the sample features updated in step S2, summarize the cumulative results from the updated historical sample set; S4. Based on the sample features in step S1, the historical sample set and sample features updated in step S2, and the cumulative results summarized in step S3, design a cumulative signal measurement point statistical algorithm, traverse to generate sample features, process noise data according to noise type, update sample features and historical sample set, and re-summarize to generate correct cumulative results.

2. The random denoising statistical method based on accumulated semaphore according to claim 1, characterized in that, In step S1, the sample features are summarized. Specifically: in, , , , These are historical sample sets. The first value, last value, maximum value, and minimum value of . It is a historical sample set Length, It is to traverse the historical sample set The discovered dataset is suspected to be noisy.

3. The random denoising statistical method based on accumulated semaphore according to claim 2, characterized in that, If it is determined to be suspected noise generation, then Record The rule for judging suspected noisy data in the set is as follows:

4. The random denoising statistical method based on accumulated semaphore according to claim 1, characterized in that, The specific noise reduction strategy is as follows: like Established, noise-free; like This is valid; the sample set contains data that appears to be noisy.

5. The random denoising statistical method based on accumulated semaphore according to claim 1, characterized in that, In step S205, the cumulative signal quantity changes normally within the three standard deviations defined in step S204, as follows: The three sample regions are as follows: The standard deviations of the three sample intervals are as follows:

6. The random denoising statistical method based on accumulated semaphore according to claim 1, characterized in that, In step S3, the updated historical sample set is quickly traversed. This generates new sample features. Select statistical method: like Valid, the cumulative amount is calculated as follows: like Valid, the cumulative amount is calculated as follows: in, This refers to the last sample data in the new historical sample set after processing the noisy data. This is the first data point in the new historical sample set after processing the noisy data. This represents the maximum sample data in the new historical sample set after processing the noisy data.

7. The random denoising statistical method based on accumulated semaphore according to claim 1, characterized in that, In step S4, when the cumulative signal measurement point statistical algorithm starts, it establishes an access link through the real-time database connection pool. If the connection fails, it outputs a log message indicating that it cannot connect to the real-time database, stops the startup, and exits directly. If the connection is successful, it continues loading and waits for the statistical request forwarded by the user interface. It obtains a real-time connection from the real-time database connection pool, and reads the historical sample dataset through the real-time connection, combined with the requested measurement point and the requested time period. It traverses the historical sample set to generate sample features. Based on the noise locations collected by the sample features, it identifies the noise type one by one using a preset algorithm, processes the noise data according to the noise type, and generates a new historical sample set and sample features. Based on the new historical sample set and sample features, it summarizes the correct cumulative results.

8. A random denoising statistical system based on accumulated semaphores, characterized in that, include: The traversal module quickly traverses the historical sample set for the statistical period and summarizes the sample features. The update module, combining the sample features obtained from the traversal module, processes noisy data and synchronously updates the historical sample set and sample features. Specifically, the processing of noisy data involves: Analysis of historical sample sets Sample features Select a denoising strategy and iterate sequentially. Set of suspected noise locations, identify each suspected noise location. The noise type is identified and processed, and a new historical sample set is generated after processing. ; Identify each suspected noise location The noise type is identified and processed as follows: from Obtaining noise data from the collection In the sample set Serial number ; Obtain random numbers according to the formula ; based on the steps obtained and random numbers Divide the data into three sample regions; calculate the standard deviation of each of the three sample regions; based on the standard deviation, the cumulative signal change is essentially a change in production operations, thus determining the noise type; process the noise data according to the determined noise type, and if there is a misalignment, start from the previous sequence. Rearrange the items forward until the previous order has been adjusted; if there is a sudden change, remove the affected item. ; If the table is cleared and flipped normally, retain it; repeat the above steps to process each suspected noisy data until all data is processed and a new historical sample set is generated. ; The cumulative module summarizes the cumulative results from the updated historical sample set based on the sample features updated by the update module. The statistics module designs a cumulative signal measurement point statistical algorithm based on sample characteristics, updated historical sample sets and sample characteristics, and the cumulative results summarized by the cumulative module. It iterates through and generates sample characteristics, processes noise data according to noise type, updates sample characteristics and historical sample sets, and re-summarizes to generate the correct cumulative results.

Images