Data detection method, calculation module and computer readable storage medium

Abstract

The invention provides a data detection method, a calculation module and a computer readable storage medium, and the data detection method comprises the steps: continuously obtaining and recording a plurality of data sets according to a preset sequence, and each data set comprises one or more pieces of data; numerical values of data in the data sets monotonically change along with a preset sequence, and each data set corresponds to one position in the preset sequence. And detecting a previous data set of the current data set in the preset sequence. And judging whether the previous data set is credible or not, and when the previous data set does not exist or is not credible, continuing to trace forwards according to a preset sequence until a valid data set is obtained. And calculating the data quality of the current data set in combination with the current data set, the effective data set and a preset sequence. According to the embodiment of the invention, the method can be used as a supplement of a delay waiting process, and preliminary evaluation is carried out under the condition that a preposed data set is not received, so that the timeliness and detection efficiency of a calculation process are improved, and the number of undetectable data sets in a data detection system is reduced.

Description

Technical Field

[0001] The present invention generally relates to the technical field of data acquisition and processing, and in particular to a data detection method, a computing module and a computer-readable storage medium. Background Art

[0002] Data collection and analysis systems in industrial applications rely on sensors or instruments installed at different locations in the industrial system to obtain data and conduct summary analysis to reflect the actual production situation and guide the subsequent production process. However, in the actual application of the Internet of Things, sensors distributed in different locations, network equipment, transmission paths, etc. are affected by the working environment. The actual production process is harsh, such as corrosive gas or liquid environment, electromagnetic radiation environment, mechanical vibration environment, etc., and it is often impossible to obtain accurate data or data cannot be transmitted in time. Therefore, before processing the data, the credibility of the data is usually judged, the data is tested, the abnormal data is marked, and the subsequent processing process is carried out after discarding or optimizing.

[0003] The currently used data detection method usually uses delayed waiting when processing missing or disordered data. For example, when intermediate data is missing, the calculation and analysis process using the missing data is suspended and waits for a preset time. If the missing data is still not obtained after the preset time, the missing data is skipped, and the subsequent data based on the missing data is also marked as abnormal, for example, the missing data is a calculation condition or a verification physical examination. This method can solve the problem of disordered data in some cases. After receiving the data during the delayed waiting process, the subsequent calculation or verification process can continue, but it cannot solve the problem of data loss. In addition, in the case of data loss, an error can only be reported after the waiting timeout. This data detection method has low timeliness, low system operation efficiency, and the lost data will also affect subsequent data detection.

[0004] The contents of the background technology section are merely the technologies known to the inventors and do not necessarily represent the prior art in the field. Summary of the invention

[0005] In view of one or more defects in the prior art, the present invention provides a data detection method, comprising:

[0006] Continuously acquiring and recording multiple data sets according to a preset sequence, wherein the data sets include one or more data; the values ​​of the data in the multiple data sets change monotonically with the preset sequence, and each data set corresponds to a position in the preset sequence;

[0007] After acquiring the current data set, detecting, by a processor, a previous data set of the current data set in the preset order;

[0008] Determine whether the previous data set is credible through the processor;

[0009] When the previous data set does not exist or is unreliable, the processor continues to trace back in the preset order until a valid data set is obtained;

[0010] The data quality of the current data set is calculated by a processor in combination with the current data set, the valid data set and the preset order.

[0011] According to one aspect of the present invention, when the previous data set of the current data set is a valid data set within a preset sequence, the data quality of the current data set is calculated according to the current data set, the previous data set and the preset sequence.

[0012] According to one aspect of the present invention, the data detection method further includes:

[0013] Collect valid data sets to establish a collection as historical data;

[0014] Performing threshold calculation on the historical data in combination with the preset sequence to obtain a permitted threshold, wherein the threshold calculation includes calculating one or more of a difference, a ratio, a variable average, a weighted average, a variable median, and a variable range;

[0015] The step of calculating the data quality of the current data set includes:

[0016] Performing the same calculation as the threshold calculation on the current data set and the valid data set;

[0017] The calculation result is compared with the permission threshold to calculate the data quality of the current data set.

[0018] According to one aspect of the present invention, the step of performing threshold calculation to obtain the permitted threshold further comprises:

[0019] Extracting a plurality of detection sets from the historical data according to a preset rule, each of the detection sets including a plurality of valid data sets and corresponding positions of the valid data sets in the preset sequence;

[0020] Performing threshold calculations on the plurality of detection sets respectively to obtain a plurality of permitted thresholds;

[0021] The step of performing the same calculation as the threshold calculation on the current data set and the valid data set comprises:

[0022] Merging the current data set into the multiple detection sets respectively, and performing the same calculation as the threshold calculation in the multiple detection sets respectively;

[0023] The step of comparing the calculation result with the permission threshold comprises:

[0024] The plurality of calculation results are compared with the corresponding permission thresholds to calculate the data quality of the current data set.

[0025] According to one aspect of the present invention, the data set includes multiple data, and the data types and quantities in different data sets correspond to each other; the step of calculating the data quality of the current data set includes: calculating the data quality of multiple data in the current data set one by one.

[0026] According to one aspect of the present invention, the multiple data in each of the data sets are ranked, and the threshold calculation also includes: the year-on-year comparison of the data in the same data set and / or the year-on-year comparison of the data with the same ranking in multiple valid data sets in the preset order.

[0027] According to one aspect of the present invention, the step of obtaining a valid data set comprises:

[0028] Tracing back multiple data sets within a valid range in a preset order;

[0029] Keep all valid data sets within the valid range;

[0030] The step of calculating the data quality of the current data set includes:

[0031] The data quality of the current data set is calculated by combining the current data set, all valid data sets and the preset order.

[0032] According to one aspect of the present invention, the step of calculating the data quality of the current data set comprises:

[0033] Using all valid data sets respectively to calculate the data quality of the current data set, and obtain one or more process results;

[0034] The data quality of the current data set is calculated according to the one or more process results.

[0035] According to one aspect of the present invention, the step of calculating the data quality of the current data set according to one or more process results comprises:

[0036] Weight one or more data quality results. If there are multiple data quality results, perform weighted averaging to calculate the data quality of the current data set.

[0037] The weight of the process result is related to the position of the corresponding valid data set in the preset order in the calculation of the process result. As the distance between the valid data set and the current data set in the preset order gets closer, the weight of the process result corresponding to the valid data set gradually increases.

[0038] According to one aspect of the present invention, when there is no valid data set within the valid range, the data detection method is stopped and an error is reported.

[0039] According to one aspect of the present invention, the preset sequence is a time sequence, the data sets are acquired in a time sequence according to a fixed step size or an arbitrary time interval, and the data sets correspond to the time when the data sets are acquired.

[0040] According to one aspect of the present invention, the data in the data set has a limit value in the value that changes with the preset order within a unit time; the step of calculating the data quality of the current data set includes: when the change between the current data set and the valid data set exceeds the limit value of the value change within the time interval between the current data set and the valid data set, judging that the data quality of the current data set is unqualified.

[0041] According to one aspect of the present invention, the step of calculating the data quality of the current data set comprises:

[0042] Calculate the difference between the current data set and the valid data set;

[0043] Determine whether the difference between the current data set and the valid data set exceeds the limit value of the data set value change between the time corresponding to the current data set and the time corresponding to the valid data set;

[0044] When the difference between the current data set and the valid data set exceeds the limit value, the data quality of the current data set is judged to be unqualified.

[0045] According to one aspect of the present invention, the data detection method further includes:

[0046] When the data quality of the current data set is qualified, the data set located between the current data set and the valid data set in the preset order is validated and / or corrected according to the current data set and the valid data set.

[0047] According to one aspect of the present invention, the present invention further includes a computing module, the computing module comprising:

[0048] A processor, wherein the processor is configured to execute the data detection method as described above; and

[0049] A data acquisition module is in communication with the processor and is configured to output a data set to the processor.

[0050] According to one aspect of the present invention, the computing module further includes:

[0051] A database is in communication with the processor and is configured to store the data sets and the data qualities corresponding to the data sets in a preset order.

[0052] According to one aspect of the present invention, the present invention also includes a computer-readable storage medium, wherein the computer-readable storage medium includes computer-executable commands stored thereon, and the executable commands implement the data detection method as described above when executed by a processor.

[0053] Compared with the prior art, the embodiments of the present invention provide a data detection method, which, when detecting the current data set, traces back according to a preset order, skips the data set that does not exist or has not been obtained, and uses the valid data set and the preset sequence conditions to calculate the data quality of the current data set, which can be used as a supplement to the delayed waiting process, and perform a preliminary evaluation without receiving the previous data set, thereby improving the timeliness and detection efficiency of the calculation process and reducing the number of undetectable data sets in the data detection system. The present invention also includes an embodiment of a computing module and a computer-readable storage medium, which can apply the data detection method in the aforementioned embodiment to detect the data set. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0055] Figure 1 is a flow chart of a data detection method in one embodiment of the present invention;

[0056] Figure 2 is a flow chart of a data detection method including a threshold value calculation process in one embodiment of the present invention;

[0057] Figure 3 is a flow chart of a data detection method including a process of establishing a detection set in one embodiment of the present invention;

[0058] Figure 4 is a flow chart of a data detection method including a process of calculating data quality using multiple data sets in one embodiment of the present invention;

[0059] Figure 5 It is a structural block diagram of a computing module in one embodiment of the present invention. DETAILED DESCRIPTION

[0060] In the following, only some exemplary embodiments are briefly described. As those skilled in the art will appreciate, the described embodiments may be modified in various ways without departing from the spirit or scope of the present invention. Therefore, the drawings and descriptions are considered to be exemplary and non-restrictive in nature.

[0061] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, which are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first" and "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0062] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, an electrical connection, or can communicate with each other; it can be directly connected, or indirectly connected through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0063] In the present invention, unless otherwise clearly specified and limited, a first feature being "above" or "below" a second feature may include that the first and second features are in direct contact, or may include that the first and second features are not in direct contact but are in contact through another feature between them. Moreover, a first feature being "above", "above" and "above" a second feature includes that the first feature is directly above and obliquely above the second feature, or simply means that the first feature is higher in level than the second feature. A first feature being "below", "below" and "below" a second feature includes that the first feature is directly below and obliquely below the second feature, or simply means that the first feature is lower in level than the second feature.

[0064] The disclosure below provides many different embodiments or examples to realize different structures of the present invention. In order to simplify the disclosure of the present invention, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present invention. In addition, the present invention can repeat reference numbers and / or reference letters in different examples, and this repetition is for the purpose of simplicity and clarity, which itself does not indicate the relationship between the various embodiments and / or settings discussed. In addition, the present invention provides various specific examples of processes and materials, but those of ordinary skill in the art can be aware of the application of other processes and / or the use of other materials.

[0065] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0066] Figure 1 The specific process of the data detection method 100 according to an embodiment of the present invention is shown, which is used to evaluate the data quality of the currently acquired data set. Figure 1 The data detection method 100 in this embodiment is described.

[0067] like Figure 1 As shown, in step S101, multiple data sets are continuously acquired and recorded according to a preset order, where the preset order can be a time sequence or multiple data sets acquired at the same time, and are sorted or numbered according to the preset order. For example, post-instrument sensors such as ammeters and power meters arranged at different positions in a power supply line can be sorted or numbered according to the current direction.

[0068] One of the data sets includes one or more data. The data types in the same data set can be the same, such as power values ​​at different positions in a line, or different, such as multiple parameter values ​​at the same position in the line. In this embodiment, the values ​​of the data in multiple data sets preferably change monotonically with the preset order. For example, the data set includes the power consumption of the same equipment in the line, which gradually increases with time sequence, or the data set includes the flow in the branch pipeline, and the flow gradually increases or decreases with the extension direction of the main pipeline. Of course, the values ​​of the data in the data set change monotonically with the preset order as a theoretical change. When a fault or reading error occurs, the values ​​of the data in the data set may be abnormal. Each data set corresponds to a position in the preset order. The reasonable range of the data in the data set can be determined according to the position of the data set in the preset order and the trend of the monotonous change of the data set along the preset order.

[0069] In step S102, after acquiring the current data set, the previous data set of the current data set in the preset order is detected, for example, by a processor, wherein the previous data set represents the data set that is closest to the current data set in the preset order. For example, when the preset order is in chronological order, the current data set may represent the data set acquired at the current moment, and the previous data set may represent the data set acquired at the previous preset data acquisition moment. When other standards are used as the preset order, the current data set may represent the data set that needs to be evaluated for data quality.

[0070] In step S103, it is determined whether the previous data set is credible, which can also be executed by the processor, wherein the credibility is used to indicate the accuracy of the data in the previous data set, for example, whether it is credible is determined based on the calculated data quality, specifically, the data quality can be quantified and a threshold can be set, or the initial credibility can be set according to the acquisition process of different data sets in a preset order, for example, the credibility is higher for data acquired through direct detection or data acquired by sensors with a stable working environment, while the credibility is lower for data acquired through multiple calculations or data acquired by sensors with a harsh working environment. Whether it is credible can be adjusted according to the specific application scenario and accuracy requirements, preferably, the data quality of the data set is calculated in sequence according to the preset order, and whether the previous data set is credible is determined based on the data quality.

[0071] The previous data set is unreliable and also includes the situation that the previous data set does not exist, for example, the previous data set does not obtain data, or the data is not transmitted to the processor that executes the data detection method 100 in this embodiment. Specifically, the situation that the previous data set does not exist can be defined as the data quality of the previous data set is 0. In this embodiment, the previous data set does not exist means that the previous data set is not obtained when calculating the data quality of the current data set. In this case, no delay waiting is performed, and the subsequent step S104 is directly performed. After the subsequent step is completed, or during the subsequent step, the previous data set is obtained, that is, the previous data set is delayed in delivery, resulting in a disordered data set (the time of obtaining the previous data set is later than the current data set). The steps of the data detection method 100 in this embodiment are not suspended. Preferably, a conventional data detection method can be executed at the same time. The data quality of the current data set calculated in this embodiment can be used as a supplement to the data quality obtained by the conventional data detection method to further improve the accuracy of the data quality of the current data set.

[0072] When there is no previous data set or the previous data set is unreliable, in step S104, the processor continues to trace back in a preset order until a valid data set is obtained. The valid data set refers to a data set with qualified data quality. For example, when recording the previous data set, the same data detection method 100 is used to calculate the data quality and record it with the data set. When tracing back in the preset order, the data quality corresponding to the data set can be directly called to determine whether the data set is valid. The valid data set can be one data set or multiple valid data sets, which will be described in detail when the calculation process is described later.

[0073] In step S105, the data quality of the current data set is calculated by the processor in combination with the current data set, the valid data set and the preset sequence. The data set in this embodiment changes monotonically with the preset sequence. The data change between the current data set and the valid data set, combined with the corresponding preset sequence, can be used to evaluate the accuracy of the current data set. For example, the limit, mean, and change rate of the change amount of the current data set relative to the valid data set can be selected according to the specific application scenario and the data type in the data set. One or more parameters are used to evaluate the data quality of the data set.

[0074] The data detection method 100 provided in this embodiment is used for the case where the data set is missing or disordered. The data quality of the current data set can be directly calculated using the previous valid data set to obtain a preliminary evaluation result, for example, the data set with obvious errors can be excluded. If only the delayed waiting method is used, the disorder and missing cannot be accurately judged before the time threshold is reached, which will lead to the accumulation of the number of data sets whose data quality cannot be confirmed, occupying computing resources and storage space. The larger the Internet of Things system, the more data sets are accumulated. In practical applications, the data quality of the data set in a smoothly running system (i.e., there is no missing or disordered data set) is often higher and more reliable, but when the data set is missing or disordered, the data quality of the data set will also decrease, and the number of data sets that can be clearly excluded is larger. The data detection method 100 in this embodiment is used to perform preliminary screening on the data set, which can reduce the pressure of the data processing system, improve the timeliness and efficiency of data detection, and the data detection method in this embodiment has a basis for continued optimization, which can further improve the accuracy of calculating data quality.

[0075] According to a preferred embodiment of the present invention, Figure 1As shown, in the data detection method 100, when the previous data set of the current data set is credible, in step S106, the data quality of the current data set is calculated according to the current data set, the previous data set and the preset order. Specifically, in this embodiment, the previous data set is credible, that is, the previous data set can be regarded as a valid data set, and the data quality of the current data set is calculated using a similar method in step S105, which is not expanded here.

[0076] Figure 2 The specific flow of a data detection method 200 according to a preferred embodiment of the present invention is shown, which includes the process of calculating the threshold value. In the data detection method 200, steps S201, S204, S205 and S206 are substantially the same as steps S101, S102, S103 and S104 of the data detection method 100 in the aforementioned embodiment, and are not described in detail here.

[0077] The data detection method 200 specifically includes the process of establishing historical data and performing threshold calculation on the historical data. In step S202, valid data sets are collected to establish a set as historical data. In this embodiment, the data quality of the data set can be calculated after the data set is obtained. Specifically, the data detection method 100 in the aforementioned embodiment can be used, or other evaluation criteria can be used to obtain a valid data set. A valid data set means that the data in the data set is credible and can be used for subsequent calculations or judgment processes. One valid data set can exist in multiple different detection sets at the same time, and the use of the valid data set in subsequent calculations is not restricted at this time.

[0078] In this step, a set of valid data sets is established as historical data, for example, stored separately in a database to facilitate data analysis and call. This step does not mean that untrusted data sets are not saved, but untrusted data sets are excluded from the historical data in this embodiment to avoid interference with subsequent calculation results.

[0079] In step S203, threshold calculation is performed on the historical data in combination with a preset sequence to obtain a permitted threshold, wherein the permitted threshold is used as a standard for evaluating the data quality of the current data set, and the threshold calculation represents the calculation process of the permitted threshold, and the threshold calculation includes calculating one or more of a difference, a ratio, a variable average, a weighted average, a variable median, and a variable range. Furthermore, when the data set includes multiple types of data, different threshold calculation methods can also be used to calculate the corresponding permitted thresholds between different data.

[0080] Specifically, in this embodiment, the data set includes data that increases or decreases with a preset order, and takes the time positive sequence as an example of the preset order, such as power consumption, flow rate, temperature under continuous operation, ion amount in the solution, material amount in the storage space, etc., which can be expressed according to the difference or ratio per unit time. Changes that have occurred before can be expressed. Further, the average, weighted average, median, etc. of the changes in adjacent valid data sets can be calculated, which can be used to express the changes and trends of the data set with the preset order within a period of time, wherein the weighted average can be set so that the closer the valid data set is to the current data set in the preset order, the greater its weight, indicating that the new data has a greater weight than the old data, and the specific weight distribution can be set according to the actual situation. Among them, the average and median are used to characterize variables within a certain range, preferably used in a small range, or within the range where the variable changes show a uniform change trend, to improve accuracy.

[0081] The variable extreme value indicates the maximum or minimum value of the data variable (such as increment or decrement) in the valid data set within a period of time, and corresponds to the preset order. Taking the time positive order as an example, for example, within an hour, the maximum increment value in the valid data set is 10, and the minimum increment value is 7. Then the variable within an hour is within the range of 7-10, which can be regarded as the normal range, and its credibility increases. Although it does not necessarily mean that it is unreliable data, it can reduce its credibility evaluation accordingly. Preferably, the permission threshold can be obtained by combining the aforementioned multiple threshold calculation methods, and the permission threshold can be used to evaluate the data quality of the current data set.

[0082] The above is only an example of using the time sequence as the preset sequence. According to different application scenarios, other preset sequences can also be selected, such as the sampling sequence distributed along a specific direction, the light intensity value collection that varies with distance, the power data collection of the equipment being started or shut down in sequence, etc. It should be noted that the sequence of step S202 to step S203 is not limited to Figure 2 The order shown in can update the historical data after a data set is determined to be a valid data set.

[0083] In step S207, the current data set and the valid data set are calculated in the same way as the threshold value calculation. Specifically, for example, the permitted threshold value is obtained by difference calculation. The same calculation method is used to calculate the difference between the current data set and the valid data set to obtain the calculation result. The other threshold value calculation processes are similar. The valid data set can be any one or more valid data sets in the historical data. Preferably, the valid data set is a valid data set closest to the current data set in the preset sequence, or multiple valid data sets within a certain range in the preset sequence, and are also combined with the corresponding positions in the preset sequence during the calculation process.

[0084] In step S208, the calculation result is compared with the permitted threshold to calculate the data quality of the current data set. Specifically, the degree of closeness to the permitted threshold can be set as the standard of the data quality of the current data set, or the current data set corresponding to the calculation result exceeding the permitted threshold can be regarded as an unqualified data set. The permitted threshold is set according to the calculation process and the specific data type.

[0085] The data detection method 200 in this embodiment can further improve the accuracy of the data quality of the current data set, set an appropriate threshold calculation process, apply the permitted threshold to evaluate the data quality of the current data set, and the permitted threshold can be regarded as setting the value range or credibility scale of the current data set. The actual data obtained in the current data set (after calculation with the valid data set) is compared with the permitted threshold, which can be used to judge the credibility of the current data set. This embodiment makes full use of the valid data sets before the preset order, reduces the errors or errors caused by calculating the data quality of the current data set using only one or several valid data sets, and thus improves the accuracy of judging the credibility of the current data set.

[0086] Figure 3 The specific process of the data detection method 300 according to a preferred embodiment of the present invention is shown, which includes the process of establishing a detection set. In this embodiment, steps S301, S302, S306 and S307 in the data detection method 300 are basically the same as steps S201, S202, S205 and S206 in the data detection method 200 in the aforementioned embodiment, and are not repeated here.

[0087] In step S303, multiple detection sets are extracted from the historical data according to a preset rule, wherein the preset rule is, for example, extracting valid data sets at fixed intervals according to a preset order, or sorting or evenly distributing the extraction according to the data quality quantified by the valid data sets, or randomly extracting from the historical data. The number of valid data sets in each detection set may be the same or different. Preferably, each detection set is configured to include multiple valid data sets, and each valid data set also has a position corresponding to the valid data set in the preset order.

[0088] In step S304, threshold calculations are performed on multiple detection sets respectively to obtain multiple permitted thresholds. The specific threshold calculation process can refer to the threshold calculation process in the aforementioned embodiment. In different detection sets, the threshold calculation processes can be the same or different, or each detection set can also select multiple threshold calculation processes to obtain multiple permitted thresholds. The multiple permitted thresholds correspond to multiple detection sets. When a detection set obtains multiple permitted thresholds through multiple threshold calculation processes, the obtained multiple permitted thresholds all correspond to the detection set. In this step, each detection set corresponds to at least one permitted threshold.

[0089] In step S308, the current data set is merged into multiple detection sets respectively, and the same calculation as the threshold calculation is performed in the multiple detection sets. Specifically, for example, if three detection sets are extracted from the historical data in step S303, the current data set is merged into the three detection sets respectively, and the current data set is independently calculated in the three detection sets as the corresponding threshold calculation, that is, the calculation process performed in each detection set is the same as the threshold calculation process of the detection set to obtain the permitted threshold, for example, the threshold calculation is to calculate the difference, and accordingly, the current data set is limited to the detection set, and the difference is also calculated using the valid data set in the detection set. For the detection sets corresponding to multiple permitted thresholds, the current data set is also calculated multiple times to obtain multiple calculation results. In different detection sets, the current data set and the multiple valid data sets in the detection set and the positions in the preset order (such as time series or spatial position, etc.) are calculated to obtain multiple calculation results, wherein the number of calculation results is equal to the number of multiple permitted thresholds obtained in step S304, and corresponds one to one according to the detection set and the threshold calculation method.

[0090] In step S309, the multiple calculation results are compared with the corresponding permission threshold value to calculate the data quality of the current data set. Specifically, the data quality intermediate value of the current data set obtained by calculating each calculation result and the corresponding permission threshold value can be obtained by using step S208 in the data detection method 200 in the above embodiment, and the multiple calculation results are calculated respectively to obtain multiple data quality intermediate values. Different data quality intermediate values ​​can be added, averaged or weighted averaged to finally obtain the data quality of the current data set.

[0091] This embodiment further improves the accuracy of the data quality evaluation of the current data set. For the valid data set in the historical data, although the valid data set is recognized as a data set that meets the data quality requirements, it does not mean that the valid data set represents the real reading, and errors may still exist. According to the data detection method in this embodiment, the distribution of valid data sets in different detection sets is different, and different permitted thresholds are obtained through multiple threshold calculation processes, which can offset the errors in part of the valid data set itself. Therefore, the solution of this embodiment can further improve the accuracy of the data quality evaluation of the current data set. Furthermore, after calculating and obtaining multiple permitted thresholds, some extreme values ​​can be discarded and not used in subsequent calculations and comparisons.

[0092] According to a specific embodiment of the present invention, there are multiple data in a data set, for example, a period can be 24 hours or other time intervals, and the data in a data set includes multiple data collected within 24 hours, and the previous data set is used to record the same type of data in the previous period. The data in a data set can also be data collected from different locations in a device, or different parameter data of a device, such as temperature, power consumption, etc. Preferably, the multiple data in a data set are related. The data types and quantities in different data sets correspond to each other. Of course, in actual applications, there may be incomplete data, such as some data in a data set is missing or out of order, which can be temporarily recorded and occupied.

[0093] In the case where there are multiple data in a data set, when calculating the data quality of the current data set, the data quality of the multiple data in the current data set is calculated one by one. Preferably, there is a sorting between the multiple data in one data set, and the sorting can be a time series or other pre-set sorting, and the sorting method of the data in the multiple data sets is the same. When calculating the threshold value for the valid data set, for different data in the data set, the threshold calculation can also include the year-on-year comparison of the data in the same data set and / or the year-on-year comparison of the data with the same sorting in multiple valid data sets in a preset order.

[0094] Among them, the year-on-year comparison of data in the same data set, for example, a data set includes multiple data of the same type within a time period, the threshold calculation process can calculate the change amount of different data in a valid data set, such as the change amount of the data ranked second in the valid data set compared to the data ranked first, as the year-on-year value. The data in the time period of one data set shows periodic changes, and the time period is divided on this basis. For example, a production line has a production cycle of one week, in which the production tasks of each day are different, and the corresponding power consumption, raw material consumption and other values ​​will also be different. Based on this, the year-on-year comparison between data of the same ranking in multiple valid data sets in a preset order is similar. The data at the same ranking position in multiple data sets are subjected to threshold calculation, for example, refer to the threshold calculation process in the aforementioned embodiment.

[0095] Figure 4 The specific process of the data detection method 400 according to a preferred embodiment of the present invention is shown, which includes the process of calculating the data quality of the current data set using multiple valid data sets. Specifically, steps S401, S402 and S403 of the data detection method 400 in this embodiment are basically the same as steps S101, S102 and S103 in the data detection method 100 in the aforementioned embodiment, and are not repeated here.

[0096] In step S404, multiple data sets within the effective range are traced back in a preset order, where the effective range is a range pre-set in the preset order, for example, 10 data sets are traced back in the preset order. Preferably, the effective range is set with reference to specific application scenarios and actual conditions. For example, after an update such as production line optimization or equipment replacement, the reference significance of the historical data before the update decreases, and the effective range can be defined until the update is completed, and the data before the update is discarded.

[0097] In step S405, all valid data sets within the valid range are retained. In this embodiment, the number of valid data sets can be one or more, and the specific number is not limited and is selected according to the actual acquisition situation. Further, if there is no valid data set within the valid range, the data detection method 400 is stopped and an error is reported.

[0098] In step S406, the data quality of the current data set is calculated according to the current data set, all valid data sets and the preset order, wherein the data quality of the current data set can be calculated according to the calculation method in the aforementioned embodiment. When there are multiple valid data sets in the valid range, each valid data set can be used to perform the calculation separately. According to a preferred embodiment of the present invention, when calculating the data quality of the current data set, all valid data sets are used to calculate the data quality of the current data set to obtain one or more process results, wherein one valid data set corresponds to one process result. In this embodiment, the process result does not necessarily represent the data quality of the current data set. When there is only one valid data set in the valid range, a process result obtained by calculation can be regarded as the data quality of the current data set, or the data quality can be obtained by continuing to calculate, which is related to the quantification algorithm of the data quality, for example, a weight is assigned to the only process result to calculate the data quality of the current data set. When there are multiple valid data sets in the valid range, the multiple process results obtained by calculation can be calculated by averaging or weighted averaging to obtain the data quality of the current data set.

[0099] Preferably, the weight of the process result is related to the position of the corresponding valid data set in the preset order in the calculation of the process result. As the distance between the valid data set and the current data set in the preset order is close, the weight of the process result corresponding to the valid data set gradually increases. For example, there are 5 valid data sets in the effective range, each of which corresponds to a process result. The positions of the 5 valid data sets and the current data set in the preset order are different and arranged in sequence. The process result corresponding to each valid data set is assigned different weights according to the distance between the valid data set and the current data set in the preset order. The closer the sorting distance to the current data set in the preset order, the greater the weight of the process result. For example, the weights of the corresponding process structures of the 5 valid data sets according to the sorting distance to the current data set are 1, 0.9, 0.8, 0.7, and 0.6 respectively. In the preferred embodiment of the present invention, the preset order is a time positive order, wherein the closer the distance to the current data set in the preset order is, the closer the valid data set is to the current data set in terms of acquisition time, that is, the higher the effectiveness is, so a higher weight can be assigned and the weight is reduced in sequence in the direction of tracing back in the preset order. Furthermore, the sorting of valid data sets in the preset order includes the placeholders of unreliable or missing data sets in the preset order, rather than re-sorting the valid data sets, which can further reflect the time interval with the current data set in the positive time sequence.

[0100] Preferably, the preset order is a time sequence, and the data set can be acquired according to a fixed step size. For example, the data set includes the power consumption of an electrical device. The data set is acquired continuously in a time sequence with a fixed step size of 15 minutes. The data set can also be acquired in a time sequence at any time interval, wherein the data set corresponds to the moment when the data set is acquired. The preset order is a time sequence, and the data set changes monotonically with the preset order, which conforms to the actual application situation, such as changes in reserves and consumption, changes in temperature of heat accumulation and heat dissipation, etc. Of course, in other embodiments of the present invention, the preset order can also be a spatial order, such as changes in flow in a line, etc. This embodiment is illustrated using the time sequence as an example.

[0101] Further, in the preferred solution, the data in the data set has a limit value in the unit time according to the preset sequence. For example, the data in the data set includes the power consumption of the equipment, the raw material consumption, the product output, etc., and when the equipment is at full load, the above data all have maximum values, which can be used as the limit value in this embodiment. Specifically, the limit value can also be a part of the specific value of the equipment at full load, such as 80%, which is determined according to the actual production process. The limit value represents the theoretical change value of the data in the data set in the unit time. When the change between the current data set and the valid data set exceeds the limit value of the value change in the time interval between the current data set and the valid data set, the data quality of the current data set is judged to be unqualified. For example, the maximum power consumption of the equipment in the unit time is taken as the limit value. This limit value is multiplied by the time interval between the current data set and the valid data set to calculate the maximum value of the power consumption increment of the current data set relative to the valid data set. When the change of the current data set exceeds the maximum value compared with the valid data set, it can be judged that there is an error in the current data set. The above is only a simplified introduction to the embodiment of the present invention, in which the value of the limit value may also include the lower limit value. Taking the power consumption as an example, the power consumption of the device in standby mode also has a theoretical value. If the change between the current data set and the valid data set is less than the lower limit value, it can also be determined that there is a theoretical error in the current data set. The technical solution in this embodiment can quickly exclude obviously unqualified data sets and reduce the number of unverified data sets in the system. For further judgment of the data quality of the current data set, reference can be made to the data detection method in the aforementioned embodiment.

[0102] Specifically, the difference between the current data set and the valid data set can be calculated to determine whether the difference between the current data set and the valid data set exceeds the limit value of the data set value change between the time corresponding to the current data set and the time corresponding to the valid data set. When the difference between the current data set and the valid data set exceeds the limit value, it is determined that the data quality of the current data set is unqualified. According to different actual situations, the difference can also be a ratio or other calculation process.

[0103] According to a preferred embodiment of the present invention, the data detection method specifically further includes: when the data quality of the current data set is qualified, according to the current data set and the valid data set, the data set located between the current data set and the valid data set in the preset order is validated and / or corrected. The data detection method in this embodiment can evaluate the data quality of the current data set through the valid data set. When the data quality of the current data set is qualified, that is, the current data set is also a valid data set, in the preset order, the missing data set between the valid data sets, or the data set with unqualified data quality can be validated and / or corrected by the valid data sets on both sides using the clamping method. Wherein the validity verification can mean the evaluation of the data quality of the data set existing in the middle, for example, the unqualified data set existing between the valid data set and the current data set (which has been confirmed as a valid data set) is evaluated for its data quality to cooperate with the subsequent calculation process, maintenance process, etc., and the data set with a small image on the system or a data set with a small error can also be corrected, and the original data set with unqualified data quality can be replaced with the corrected data set. For the missing data set, the above method can also be used to supplement it.

[0104] Furthermore, the data set corrected or supplemented by the method in this embodiment is not measured data, but secondary data obtained through calculation. Therefore, when applied to subsequent calculation processes or data detection methods of subsequent data sets, the above-mentioned corrected or supplemented data set can be standardized, or its credibility can be reduced. For example, when applied in a data detection method, the above-mentioned data set can be used as a valid data set, but its weight can be set to be reduced.

[0105] like Figure 5 As shown, the present invention also includes an embodiment of a computing module 1, wherein the computing module 1 includes a processor 10 and a data acquisition module 20, wherein the processor 10 is configured to be able to execute the data detection method described in the aforementioned embodiment, the data acquisition module 20 is connected to the processor 10 by signal, and can output a data set to the processor 10. Specifically, for example, the data acquisition module 20 is a sensor set at a working position, or a data summary transmission module that can receive data acquired by the sensor and transmit it to the processor 10. Preferably, the data acquisition module 20 sequentially transmits the data set to the processor according to the preset order in the aforementioned data detection method.

[0106] According to a preferred embodiment of the present invention, the computing module 1 also includes a database 30, which communicates with the processor 10 and is configured to save a data set and a data quality corresponding to the data set in a preset order. Specifically, for example, the processor 10 executes the aforementioned data detection method, obtains the quantified data quality of the current data set, and transmits the data set and the data quality of the data set to the database 30. Preferably, the data can 30 can also be divided into separate areas to store a collection of all valid data sets as historical data, and the processor 10 can retrieve the valid data set from the database 30 when executing the data detection method in the aforementioned embodiment.

[0107] The present invention also includes a computer-readable storage medium, including computer executable commands stored thereon, and when the executable commands are executed by a processor, the data detection method described in the aforementioned embodiment is implemented. The computer-readable storage medium may specifically be, for example, a hard disk, a magnetic disk, a magnetic tape, an optical disk, etc.

[0108] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A data detection method, comprising: Continuously acquiring and recording a plurality of data sets according to a preset sequence, wherein the data sets include one or more data; The values ​​of the data in the plurality of data sets vary monotonically with the preset sequence, and each data set corresponds to a position in the preset sequence; After acquiring the current data set, detecting, by a processor, a previous data set of the current data set in the preset order; Determine whether the previous data set is credible through the processor; When the previous data set does not exist or is unreliable, the processor continues to trace back in the preset order until a valid data set is obtained; The data quality of the current data set is calculated by a processor in combination with the current data set, the valid data set and the preset order.

2. According to the data detection method of claim 1, when the previous data set of the current data set is a valid data set within a preset sequence, the data quality of the current data set is calculated based on the current data set, the previous data set and the preset sequence.

3. The data detection method according to claim 1, further comprising: Collect valid data sets to establish a collection as historical data; Performing threshold calculation on the historical data in combination with the preset sequence to obtain a permitted threshold, wherein the threshold calculation includes calculating one or more of a difference, a ratio, a variable average, a weighted average, a variable median, and a variable range; The step of calculating the data quality of the current data set includes: Performing the same calculation as the threshold calculation on the current data set and the valid data set; The calculation result is compared with the permission threshold to calculate the data quality of the current data set.

4. The data detection method according to claim 3, wherein the step of performing threshold calculation to obtain the permitted threshold further comprises: Extracting a plurality of detection sets from the historical data according to a preset rule, each of the detection sets including a plurality of valid data sets and corresponding positions of the valid data sets in the preset sequence; Performing threshold calculations on the plurality of detection sets respectively to obtain a plurality of permitted thresholds; The step of performing the same calculation as the threshold calculation on the current data set and the valid data set comprises: Merging the current data set into the multiple detection sets respectively, and performing the same calculation as the threshold calculation in the multiple detection sets respectively; The step of comparing the calculation result with the permission threshold comprises: The plurality of calculation results are compared with the corresponding permission thresholds to calculate the data quality of the current data set.

5. The data detection method according to claim 3, wherein the data set includes multiple data, and the data types and quantities in different data sets correspond to each other; the step of calculating the data quality of the current data set includes: Calculate the data quality of multiple data in the current data set one by one.

6. The data detection method according to claim 5, wherein the plurality of data in each of the data sets are sorted, and the threshold calculation further comprises: The year-on-year comparison of data in the same data set and / or the year-on-year comparison of data in the same order in multiple valid data sets in the preset order.

7. The data detection method according to claim 1, wherein the step of obtaining a valid data set comprises: Tracing back multiple data sets within a valid range in a preset order; Keep all valid data sets within the valid range; The step of calculating the data quality of the current data set includes: The data quality of the current data set is calculated by combining the current data set, all valid data sets and the preset order.

8. The data detection method according to claim 7, wherein the step of calculating the data quality of the current data set comprises: Using all valid data sets respectively to calculate the data quality of the current data set, and obtain one or more process results; The data quality of the current data set is calculated based on the one or more process results.

9. The data detection method according to claim 8, wherein the step of calculating the data quality of the current data set according to one or more process results comprises: Weight one or more data quality results. If there are multiple data quality results, perform weighted averaging to calculate the data quality of the current data set. The weight of the process result is related to the position of the corresponding valid data set in the preset order in the calculation of the process result. As the distance between the valid data set and the current data set in the preset order gets closer, the weight of the process result corresponding to the valid data set gradually increases.

10. The data detection method according to claim 7, when there is no valid data set within the valid range, the data detection method is stopped and an error is reported.

11. The data detection method according to claim 1, wherein the preset order is a time sequence, the data set is acquired in a time sequence according to a fixed step size or an arbitrary time interval, and the data set corresponds to the time when the data set is acquired.

12. The data detection method according to claim 11, wherein the value of the data in the data set that changes in the preset order within a unit time has a limit value; and the step of calculating the data quality of the current data set comprises: When the change between the current data set and the valid data set exceeds the limit value of the value change within the time interval between the current data set and the valid data set, it is determined that the data quality of the current data set is unqualified.

13. The data detection method according to claim 12, wherein the step of calculating the data quality of the current data set comprises: Calculate the difference between the current data set and the valid data set; Determine whether the difference between the current data set and the valid data set exceeds the limit value of the data set value change between the time corresponding to the current data set and the time corresponding to the valid data set; When the difference between the current data set and the valid data set exceeds the limit value, the data quality of the current data set is judged to be unqualified.

14. The data detection method according to claim 1, further comprising: When the data quality of the current data set is qualified, the data set located between the current data set and the valid data set in the preset order is validated and / or corrected according to the current data set and the valid data set.

15. A computing module, comprising: A processor, wherein the processor is configured to execute the data detection method according to any one of claims 1 to 14; and A data acquisition module is in communication with the processor and is configured to output a data set to the processor.

16. The computing module according to claim 15, further comprising: A database is in communication with the processor and is configured to store the data sets and the data qualities corresponding to the data sets in a preset order. 17 . A computer-readable storage medium, comprising computer-executable commands stored thereon, wherein the executable commands implement the data detection method according to any one of claims 1 to 14 when executed by a processor.

Images