Industrial process data filling method and device
By classifying and addressing data loss types and stability in industrial processes, the method effectively fills data gaps in industrial processes, improving accuracy and supporting subsequent modeling efforts.
Patent Information
- Application Number
- CN202311252975.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-09-26
- Publication Date
- 2025-03-28
AI Technical Summary
Current data filling methods for industrial process data are inadequate, as they often rely on a one-dimensional approach, leading to low accuracy and inability to support subsequent modeling due to high data loss and instability in industrial environments.
The method involves determining whether data loss is instantaneous or long-term, assessing data stability, and using either current or related dimensions for data filling based on these assessments, employing interpolation and machine learning for precise data prediction.
This approach enhances data filling accuracy, improves data quality, and provides a robust foundation for subsequent data mining and modeling by tailoring filling strategies to specific data loss types and stability conditions.
Smart Images
Figure CN119719636A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present invention relate to the field of data processing technology, and specifically to an industrial process data filling method and device. Background Art
[0002] Industrial process data refers to the data reflecting the status of equipment, raw materials, and products during the production process when industrial equipment such as oil refining and chemical processing is in operation, such as equipment operation status data (DCS), equipment material experimental analysis data (LIMS), product status data, etc. These industrial process data are characterized by high dimensionality, large volume, and high value. However, the industrial process data obtained from industrial production sites often have low data quality and problems such as missing data.
[0003] The lack of original data often seriously affects the accuracy and effect of subsequent model building and training. Therefore, data filling through effective data cleaning methods is a key factor in improving model calculation accuracy and modeling efficiency. At present, traditional data filling methods often make calculations based on the data before and after the required supplementary data in a single dimension. The traditional data filling method has a low filling accuracy and cannot provide data support for subsequent modeling. Summary of the invention
[0004] In view of the defects in the prior art, an embodiment of the present invention provides an industrial process data filling method and device.
[0005] An embodiment of the present invention provides an industrial process data filling method, including: obtaining industrial process data to be filled, and determining whether the data missing type of the industrial process data is instantaneous data missing or long-term data missing; in response to the data missing type being instantaneous data missing, further determining the stability of the industrial process data, if the industrial process data is stable data, using the industrial process data of the current dimension to fill the missing data to be filled, if the industrial process data is unstable data, filling the missing data to be filled according to the industrial process data of the current dimension and related dimensions; in response to the data missing type being long-term data missing, filling the missing data to be filled according to the industrial process data of the current dimension and related dimensions.
[0006] According to an industrial process data filling method provided by an embodiment of the present invention, the determination of whether the data missing type of the industrial process data is instantaneous data missing or long-term data missing includes: obtaining a data missing time span based on the industrial process data; in response to the data missing time span being greater than a data sampling time interval and less than or equal to a preset time span threshold, the data missing type is instantaneous data missing; in response to the data missing time span being greater than the preset time span threshold, the data missing type is long-term data missing.
[0007] According to an industrial process data filling method provided by an embodiment of the present invention, the stability of the industrial process data is judged, including: obtaining data of the industrial process data in a preset time unit; based on the data of the industrial process data in the preset time unit, obtaining a change trend of existing data within a preset distance before and after a data missing period of the industrial process data; in response to knowing that the data changes monotonically and slowly according to the change trend, determining that the industrial process data is stable data, otherwise it is unstable data.
[0008] According to an industrial process data filling method provided by an embodiment of the present invention, the missing data to be filled is filled using the industrial process data of the current dimension, including: obtaining a first data missing time period in which data missing occurs according to the industrial process data; obtaining the preceding and following existing data closest to the first data missing time period in the industrial process data, and obtaining first data to be filled for the first data missing time period by interpolation calculation according to the preceding and following existing data; and filling the first data missing time period according to the first data to be filled.
[0009] According to an embodiment of the present invention, a method for filling in industrial process data is provided, wherein data filling is performed on missing data to be filled based on industrial process data of a current dimension and related dimensions, comprising: obtaining a second data missing period where data missing occurs based on the industrial process data; wherein the second data missing period does not include a period of equipment downtime and maintenance; obtaining a fitting curve through machine learning based on the industrial process data of the current dimension and related dimensions; obtaining the previous existing data and the subsequent existing data closest to the second data missing period of the industrial process data; obtaining a first fitting value of the previous existing data, a second fitting value of the subsequent existing data, and a third fitting value of the industrial process data at the time of data missing to be filled based on the fitting curve; calculating a first interpolation result of the first fitting value and the second fitting value at the time of data missing, and calculating a second interpolation result of the previous existing data and the subsequent existing data at the time of data missing; obtaining the second data to be filled at the time of data missing based on the first interpolation result, the second interpolation result, and the third fitting value; and filling in data at the time of data missing using the second data to be filled.
[0010] According to an industrial process data filling method provided by an embodiment of the present invention, the second data to be filled is represented as:
[0011]
[0012] Among them, M T Indicates the second data to be filled, M″ T represents the first interpolation result, represents the second interpolation result, M′ T represents the third fitting value.
[0013] An embodiment of the present invention also provides an industrial process data filling device, including: a data missing type judgment module, used to: obtain the industrial process data to be filled, and judge whether the data missing type of the industrial process data is instantaneous data missing or long-term data missing; an instantaneous data missing filling module, used to: in response to the data missing type being instantaneous data missing, further judge the stability of the industrial process data; if the industrial process data is stable data, use the industrial process data of the current dimension to fill the missing data to be filled; if the industrial process data is unstable data, fill the missing data to be filled according to the industrial process data of the current dimension and related dimensions; a long-term data missing filling module, used to: in response to the data missing type being long-term data missing, fill the missing data to be filled according to the industrial process data of the current dimension and related dimensions.
[0014] An embodiment of the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of any of the above-mentioned industrial process data filling methods are implemented.
[0015] An embodiment of the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of any of the above-mentioned industrial process data filling methods are implemented.
[0016] An embodiment of the present invention further provides a computer program product, including a computer program, which implements the steps of any of the above-mentioned industrial process data filling methods when executed by a processor.
[0017] The industrial process data filling method and device provided by the embodiment of the present invention obtain the industrial process data to be filled, determine whether the data missing type of the industrial process data is instantaneous data missing or long-term data missing, and in response to the data missing type being instantaneous data missing, further determine the stability of the industrial process data. If the industrial process data is stable data, the industrial process data of the current dimension is used to fill the missing data to be filled. If the industrial process data is unstable data, the missing data to be filled is filled according to the industrial process data of the current dimension and related dimensions. In response to the data missing type being long-term data missing, the missing data to be filled is filled according to the industrial process data of the current dimension and related dimensions. The scheme of using single-dimensional data to fill the missing data is abandoned, and instead a data filling scheme for missing data is obtained based on the data missing type, data stability, etc., which can effectively realize the accurate prediction of missing data, improve the accuracy of missing data filling, thereby improving the quality of industrial process data, and providing a good data foundation for subsequent data mining and modeling. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] In order to more clearly illustrate the technical solution of the present invention, the following is a brief introduction to the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0019] Figure 1 is a flowchart of an industrial process data filling method provided by an embodiment of the present invention;
[0020] Figure 2 is a structural schematic diagram of an industrial process data filling device provided by an embodiment of the present invention;
[0021] Figure 3It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0022] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0023] Figure 1 FIG. 1 is a flow chart of the industrial process data filling method provided by an embodiment of the present invention. Figure 1 As shown, the method includes:
[0024] Step S1, obtaining industrial process data to be filled, and determining whether the data missing type of the industrial process data is instantaneous data missing or long-term data missing;
[0025] The data of industrial process data at a certain moment is not only related to the data in the current dimension, but also greatly affected by the data in other dimensions at the current moment. The embodiment of the present invention uses single-dimensional data for data filling or multi-dimensional data for data filling according to the type of data missing or the comprehensive judgment combined with the data stability.
[0026] The industrial process data to be filled is industrial process data with missing data that needs to be filled. After obtaining the industrial process data to be filled, the data missing type of the industrial process data is first determined, where the data missing type includes instantaneous data missing and long-term data missing. Among them, the data missing time span of instantaneous data missing is shorter, and the data missing time span of long-term data missing is longer. The length of the data missing time span can be determined according to a preset rule.
[0027] Step S2, in response to the data missing type being instantaneous data missing, further determine the stability of the industrial process data; if the industrial process data is stable data, use the industrial process data of the current dimension to fill in the missing data to be filled; if the industrial process data is unstable data, fill in the missing data to be filled based on the industrial process data of the current dimension and related dimensions.
[0028] If the industrial process data is missing instantaneously, the stability of the industrial process data is further judged, and different data filling schemes are adopted according to the judgment result of whether the industrial process data is stable.
[0029] If the industrial process data is stable data, then the single-dimensional supplementary data can be used, so that the missing data to be filled can be filled using the industrial process data of the current dimension. The current dimension is the dimension to which the industrial process data belongs.
[0030] If the industrial process data is unstable data, when it is in an unstable state, because the actual working condition data has high dimensionality, multiple observation points are often used to measure a certain dimension. Therefore, the unstable data is affected by the data of related dimensions, and it is necessary to combine the data of related dimensions to predict and fill in the missing data, so as to fill in the missing data to be filled according to the industrial process data of the current dimension and related dimensions.
[0031] Step S3: In response to the data missing type being long-term data missing, the missing data to be filled is filled according to the industrial process data of the current dimension and related dimensions.
[0032] If the industrial process data is missing for a long time, the long-term data missing is also affected by the data of related dimensions. It is necessary to combine the data of related dimensions to predict and fill the missing data, so as to fill the missing data based on the industrial process data of the current dimension and related dimensions.
[0033] Among them, the relevant dimensions can be obtained based on experience or through dimension correlation analysis.
[0034] The industrial process data filling method provided by the embodiment of the present invention obtains the industrial process data to be filled, determines whether the data missing type of the industrial process data is instantaneous data missing or long-term data missing, and in response to the data missing type being instantaneous data missing, further determines the stability of the industrial process data. If the industrial process data is stable data, the industrial process data of the current dimension is used to fill the missing data to be filled. If the industrial process data is unstable data, the missing data to be filled is filled according to the industrial process data of the current dimension and related dimensions. In response to the data missing type being long-term data missing, the missing data to be filled is filled according to the industrial process data of the current dimension and related dimensions. The method abandons the one-size-fits-all solution of filling the missing data with single-dimensional data, but instead obtains a data filling solution for missing data based on the data missing type, data stability, etc., which can effectively realize the accurate prediction of missing data, improve the accuracy of missing data filling, thereby improving the quality of industrial process data, and providing a good data foundation for subsequent data mining and modeling.
[0035] According to an industrial process data filling method provided by an embodiment of the present invention, the determination of whether the data missing type of the industrial process data is instantaneous data missing or long-term data missing includes: obtaining a data missing time span based on the industrial process data; in response to the data missing time span being greater than a data sampling time interval and less than or equal to a preset time span threshold, the data missing type is instantaneous data missing; in response to the data missing time span being greater than the preset time span threshold, the data missing type is long-term data missing.
[0036] When determining whether the data missing type of industrial process data is instantaneous data missing or long-term data missing, first obtain the data missing time span based on the industrial process data. The data missing time span refers to the time of continuous data missing. If multiple segments of data missing occur in the industrial process data, the time span of each segment of data missing can be obtained, and the stability can be calculated and data filling can be performed separately.
[0037] A preset time span threshold can be set as a boundary for judging instantaneous data missing and long-term data missing. If the data missing time span is greater than the preset time span threshold, the data missing type is long-term data missing. If the data missing time span is less than or equal to (the abbreviation of less than or equal to) the preset time span threshold, since the data missing time span is less than or equal to the data sampling time interval, it is normal, so it is necessary to further judge whether the data missing time span is greater than the data sampling time interval. If the data missing time span is greater than the data sampling time interval and less than or equal to the preset time span threshold, the data missing type is instantaneous data missing.
[0038] For example, an industrial process data is represented as L = {L1, L2, L3, L i ,......,L h ,L n}, where {L i ,......,L h} segment is the missing data period, and the corresponding data missing time span is Assume the original data sampling interval is T min , design the preset time span threshold T for instantaneous data missing and long-term data missing judgment based on the experience value max , then according to ΔT and T min and T max The relative size of the missing data is used to determine the type of missing data.
[0039] If T min <ΔT≤T max , then the instantaneous data is missing; if T max <ΔT, it means long-term data missing.
[0040] The industrial process data filling method provided by an embodiment of the present invention obtains the data missing time span based on the industrial process data. In response to the data missing time span being greater than the data sampling time interval and less than or equal to a preset time span threshold, the data missing type is instantaneous data missing. In response to the data missing time span being greater than the preset time span threshold, the data missing type is long-term data missing, thereby achieving accurate judgment of the data missing type.
[0041] According to an industrial process data filling method provided by an embodiment of the present invention, the stability of the industrial process data is judged, including: obtaining data of the industrial process data in a preset time unit; based on the data of the industrial process data in the preset time unit, obtaining a change trend of existing data within a preset distance before and after a data missing period of the industrial process data; in response to knowing that the data changes monotonically and slowly according to the change trend, determining that the industrial process data is stable data, otherwise it is unstable data.
[0042] Stability needs to reflect integrity. Therefore, when judging the stability of industrial process data, the industrial process data is first represented as data in a preset time unit. For example, industrial process data in days is obtained, and stability is judged based on the industrial process data in days.
[0043] The time when the data is missing can be known based on the original industrial process data. Therefore, after converting it to days, it is also possible to know which day the data is missing. Therefore, the data missing period of the industrial process data in the preset time unit can also be determined. Based on the data of the industrial process data in the preset time unit, the change trend of the existing data within the preset distance before and after the data missing period of the industrial process data is obtained (the existing data is the original collected data). If the data is monotonously and slowly changing according to the change trend, the industrial process data is determined to be stable data, otherwise it is unstable data.
[0044] When obtaining the change trend of the existing data within the preset distance before and after the data missing period of the industrial process data, the stability can be judged by calculating the slope representing the data change based on the data in the first preset time unit and the data in the second preset time unit before the data missing period. The stability can also be judged by calculating the slope representing the data change based on the data in the first two preset time units and the data in the second preset time unit before the data missing period. If the slope does not change much (based on the preset threshold) and is monotonic, it means that the industrial process data is stable data, otherwise it is unstable data.
[0045] For example, on a certain day (denoted as t i )'s industrial process data are listed as Missing data is First, scale the time period to days, then t i The day before i The data of -1 is t i The next day i +1 data is t i Two days later i +2 data is So:
[0046]
[0047]
[0048]
[0049]
[0050] correspond The average values are:
[0051]
[0052]
[0053]
[0054] based on and The slope K1 representing the change in data is obtained based on and The slope K2 representing the data change is obtained, which can be expressed as:
[0055]
[0056]
[0057] If, according to the set threshold, K1 and K2 are close (indicating that the data does not change much) and have the same sign (indicating monotonic change), it means that the industrial process data is stable data, otherwise it is unstable data.
[0058] The industrial process data filling method provided by the embodiment of the present invention obtains the data of the industrial process data in a preset time unit, and based on the data of the industrial process data in the preset time unit, obtains the change trend of the existing data within a preset distance before and after the data missing period of the industrial process data. In response to knowing that the data changes monotonously and slowly according to the change trend, the industrial process data is determined to be stable data, otherwise it is unstable data, thereby improving the accuracy of the judgment of the stability of the industrial process data.
[0059] According to an industrial process data filling method provided by an embodiment of the present invention, the missing data to be filled is filled using the industrial process data of the current dimension, including: obtaining a first data missing time period in which data missing occurs according to the industrial process data; obtaining the preceding and following existing data closest to the first data missing time period in the industrial process data, and obtaining first data to be filled for the first data missing time period by interpolation calculation according to the preceding and following existing data; and filling the first data missing time period according to the first data to be filled.
[0060] When the data missing type of the industrial process data is instantaneous data missing and the industrial process data is stable, the missing data to be filled is filled using the industrial process data of the current dimension. When filling the missing data to be filled using the industrial process data of the current dimension, the first data missing period where data missing occurs is obtained according to the industrial process data, the previous and next existing data closest to the first data missing period in the industrial process data are obtained, the first data to be filled in the first data missing period is obtained by interpolation calculation based on the previous and next existing data, and the first data missing period is filled according to the first data to be filled. The first data missing period includes at least one data missing moment.
[0061] For example, the closest existing data to a data missing period is L3, and the time corresponding to L3 is T3. The closest existing data after is L5, and the time corresponding to L5 is T5. Then the data L of the data missing period is t The predicted value (the first data to be filled) is expressed as:
[0062]
[0063] The industrial process data filling method provided by the embodiment of the present invention obtains the first data missing time period where data missing occurs based on the industrial process data, obtains the existing data before and after the first data missing time period of the industrial process data, obtains the first data to be filled in the first data missing time period by interpolation calculation based on the existing data before and after, and fills the first data missing time period with data based on the first data to be filled, thereby achieving fast and easy data filling under the condition of instantaneous data missing and stable data, and avoiding unnecessary complex calculations.
[0064] According to an embodiment of the present invention, a method for filling in industrial process data is provided, wherein data filling is performed on missing data to be filled based on industrial process data of a current dimension and related dimensions, comprising: obtaining a second data missing period where data missing occurs based on the industrial process data; wherein the second data missing period does not include a period of equipment downtime and maintenance; obtaining a fitting curve through machine learning based on the industrial process data of the current dimension and related dimensions; obtaining previous existing data and subsequent existing data closest to the second data missing period of the industrial process data; obtaining a first fitting value of the previous existing data, a second fitting value of the subsequent existing data, and a third fitting value of the data missing moment of the industrial process data to be filled based on the fitting curve; calculating a first interpolation result of the first fitting value and the second fitting value, and calculating a second interpolation result of the previous existing data and the subsequent existing data; obtaining second data to be filled at the data missing moment based on the first interpolation result, the second interpolation result, and the third fitting value; and filling in data at the data missing moment using the second data to be filled.
[0065] When the data missing type is instantaneous data missing, but the industrial process data is non-stable data, the missing data to be filled is filled according to the industrial process data of the current dimension and related dimensions. Also, when the data missing type is long-term data missing, the missing data to be filled is filled according to the industrial process data of the current dimension and related dimensions.
[0066] It is understandable that long-term data loss may include data loss caused by equipment downtime and maintenance. Since data filling only needs to fill in the data loss when the equipment is in operation, the second data loss period does not include the equipment downtime and maintenance period. For example, industrial process data is Among them, T-1, T, and T+1 are the time periods for missing data. According to the actual working conditions of the device, T-1 is the shutdown period for maintenance, so the actual data time period that needs to be supplemented includes T and T+1.
[0067] Among them, one implementation method of filling the missing data to be filled according to the industrial process data of the current dimension and related dimensions is:
[0068] A second data missing period where data missing occurs is obtained based on the industrial process data; a fitting curve is obtained through machine learning based on the industrial process data of the current dimension and related dimensions; the previous existing data and the subsequent existing data closest to the second data missing period of the industrial process data are obtained; according to the fitting curve, a first fitting value of the previous existing data, a second fitting value of the subsequent existing data, and a third fitting value of the industrial process data at the time when the data is missing to be filled are obtained; a first interpolation result of the first fitting value and the second fitting value at the time when the data is missing is calculated, and a second interpolation result of the previous existing data and the subsequent existing data at the time when the data is missing is calculated; second data to be filled at the time when the data is missing is obtained based on the first interpolation result, the second interpolation result, and the third fitting value; and data filling is performed at the time when the data is missing using the second data to be filled.
[0069] The idea of the above method is as follows: a fitting curve is obtained according to the industrial process data of the current dimension and related dimensions, and a first fitting value of the previous existing data closest to the second data missing period of the industrial process data, a second fitting value of the subsequent existing data closest to the second data missing period of the industrial process data, and a third fitting value of the data missing moment of the industrial process data to be filled are fitted according to the fitting curve. The first interpolation result at the data missing moment is obtained by interpolation calculation based on the first fitting value and the second fitting value, and the difference between the first interpolation result and the third fitting value at the data missing moment is calculated as the first data fluctuation; the second interpolation result of the previous existing data and the subsequent existing data at the data missing moment is calculated, and the difference between the second interpolation result and the second data to be filled at the data missing moment is calculated as the second data fluctuation. Among them, the ratio of the first data fluctuation to the third fitting value is equal to the ratio of the second data fluctuation to the second data to be filled, so that the second data to be filled at the data missing moment can be obtained according to the first interpolation result, the second interpolation result and the third fitting value, so as to fill the data missing moment with the second data to be filled.
[0070] The industrial process data filling method provided by the embodiment of the present invention obtains a second data missing period where data missing occurs according to the industrial process data, wherein the second data missing period does not include an equipment shutdown and maintenance period, obtains a fitting curve based on the industrial process data of the current dimension and related dimensions through machine learning, obtains the previous existing data and the subsequent existing data closest to the second data missing period of the industrial process data, obtains a first fitting value of the previous existing data, a second fitting value of the subsequent existing data, and a third fitting value of the industrial process data at the data missing moment to be filled according to the fitting curve, calculates a first interpolation result of the first fitting value and the second fitting value at the data missing moment, and calculates a second interpolation result of the previous existing data and the subsequent existing data at the data missing moment, obtains the second data to be filled at the data missing moment according to the first interpolation result, the second interpolation result and the third fitting value, and fills the data missing moment with the second data to be filled, thereby improving the accuracy of data filling for the missing data to be filled according to the industrial process data of the current dimension and related dimensions.
[0071] According to an industrial process data filling method provided by an embodiment of the present invention, the second data to be filled is represented as:
[0072]
[0073] Among them, M T Indicates the second data to be filled, M″ T represents the first interpolation result, represents the second interpolation result, M′ T represents the third fitting value.
[0074] The following is the derivation process of the above formula:
[0075] Assume that the industrial process data to be filled is represented as: Among them, T and T+1 are the missing data moments to be filled. The following takes the acquisition of the second data to be filled at moment T as an example to illustrate the derivation process of the above formula. The method of filling the data at moment T+1 is similar.
[0076] A fitting curve (represented by function f) is obtained based on the industrial process data of the current dimension and related dimensions, and a first fitting value of the previous existing data closest to the second data missing period of the industrial process data, a second fitting value of the subsequent existing data closest to the second data missing period of the industrial process data, and a third fitting value at the time when the data missing of the industrial process data is to be filled are fitted based on the fitting curve.
[0077] Among them, the previous existing data closest to the second data missing period of the industrial process data is M3, and the first fitting value is expressed as M′3=f(3); the subsequent existing data closest to the second data missing period of the industrial process data is M7, and the second fitting value is expressed as M′7=f(7). The third fitting value of the missing data time T of the industrial process data to be filled is expressed as M′ T =f(T).
[0078] The first interpolation result at the moment of missing data is obtained by interpolating the first fitting value and the second fitting value, which is expressed as M″ T ; Calculate the difference Δ′=M″ between the first interpolation result and the third fitting value at the time of data missing T -M′ T , as the first data fluctuation; calculating the second interpolation result of the previous existing data M3 and the subsequent existing data M7 at the time of data missing Calculate the difference between the second interpolation result and the second data to be filled at the time of data missing As the second data fluctuation. The ratio of the first data fluctuation to the third fitting value is equal to the ratio of the second data fluctuation to the second data to be filled, satisfying:
[0079]
[0080] according to get
[0081] Further conversion: Thus we get:
[0082]
[0083] In this way, the second data to be filled at the data missing moment can be obtained according to the first interpolation result, the second interpolation result and the third fitting value, so that the data missing moment can be filled using the second data to be filled.
[0084] The industrial process data filling method provided by the embodiment of the present invention further improves the accuracy of data filling for missing data to be filled based on industrial process data of the current dimension and related dimensions by providing a calculation expression for the second data to be filled.
[0085] It should be noted that the multiple preferred implementations given in this embodiment can be freely combined under the premise that they do not conflict with each other in logic or structure, and the present invention does not limit this.
[0086] The industrial process data filling device provided by an embodiment of the present invention is described below. The industrial process data filling device described below and the industrial process data filling method described above can be referenced to each other.
[0087] Figure 2 Schematic diagram of the structure of the industrial process data filling device provided by the embodiment of the present invention. Figure 2 As shown, the device includes a data missing type judgment module 10, an instantaneous data missing filling module 20 and a long-term data missing filling module 30, wherein: the data missing type judgment module 10 is used to: obtain the industrial process data to be filled, and judge whether the data missing type of the industrial process data is instantaneous data missing or long-term data missing; the instantaneous data missing filling module 20 is used to: in response to the data missing type being instantaneous data missing, further judge the stability of the industrial process data, if the industrial process data is stable data, use the industrial process data of the current dimension to fill the missing data to be filled, if the industrial process data is unstable data, then fill the missing data to be filled according to the industrial process data of the current dimension and related dimensions; the long-term data missing filling module 30 is used to: in response to the data missing type being long-term data missing, fill the missing data to be filled according to the industrial process data of the current dimension and related dimensions.
[0088] The industrial process data filling device provided by the embodiment of the present invention obtains the industrial process data to be filled, determines whether the data missing type of the industrial process data is instantaneous data missing or long-term data missing, and in response to the data missing type being instantaneous data missing, further determines the stability of the industrial process data. If the industrial process data is stable data, the industrial process data of the current dimension is used to fill the missing data to be filled. If the industrial process data is unstable data, the missing data to be filled is filled according to the industrial process data of the current dimension and related dimensions. In response to the data missing type being long-term data missing, the missing data to be filled is filled according to the industrial process data of the current dimension and related dimensions. The scheme of using single-dimensional data to fill the missing data is abandoned, and a data filling scheme for the missing data is obtained comprehensively according to the data missing type, data stability, etc., which can effectively realize the accurate prediction of the missing data, improve the accuracy of the missing data filling, thereby improving the quality of the industrial process data, and providing a good data foundation for subsequent data mining and modeling.
[0089] According to an industrial process data filling device provided by an embodiment of the present invention, the data missing type judgment module 10 is specifically used to: obtain the data missing time span according to the industrial process data; in response to the data missing time span being greater than the data sampling time interval and less than or equal to a preset time span threshold, the data missing type is instantaneous data missing; in response to the data missing time span being greater than the preset time span threshold, the data missing type is long-term data missing.
[0090] The industrial process data filling device provided in an embodiment of the present invention obtains the data missing time span based on the industrial process data. In response to the data missing time span being greater than the data sampling time interval and less than or equal to a preset time span threshold, the data missing type is instantaneous data missing. In response to the data missing time span being greater than the preset time span threshold, the data missing type is long-term data missing, thereby achieving accurate judgment of the data missing type.
[0091] According to an industrial process data filling device provided by an embodiment of the present invention, the instantaneous data missing filling module 20, when used to judge the stability of the industrial process data, is specifically used to: obtain data of the industrial process data in a preset time unit; based on the data of the industrial process data in the preset time unit, obtain the change trend of the existing data within a preset distance before and after the data missing period of the industrial process data; in response to knowing that the data changes monotonously and slowly according to the change trend, determine that the industrial process data is stable data, otherwise it is unstable data.
[0092] The industrial process data filling device provided by the embodiment of the present invention obtains data of the industrial process data in preset time units, and based on the data of the industrial process data in the preset time units, obtains the change trend of the existing data within a preset distance before and after the data missing period of the industrial process data. In response to knowing that the data changes monotonously and slowly according to the change trend, the industrial process data is determined to be stable data, otherwise it is unstable data, thereby improving the accuracy of the judgment of the stability of the industrial process data.
[0093] According to an industrial process data filling device provided by an embodiment of the present invention, when the instantaneous data missing filling module 20 is used to fill the missing data to be filled using the industrial process data of the current dimension, it is specifically used to: obtain a first data missing time period where data missing occurs according to the industrial process data; obtain the preceding and following existing data closest to the first data missing time period in the industrial process data, and obtain the first data to be filled for the first data missing time period by interpolation calculation according to the preceding and following existing data; and fill the first data missing time period according to the first data to be filled.
[0094] The industrial process data filling device provided by the embodiment of the present invention obtains the first data missing time period where data missing occurs based on the industrial process data, obtains the existing data before and after the first data missing time period of the industrial process data, obtains the first data to be filled in the first data missing time period by interpolation calculation based on the existing data before and after, and fills the first data missing time period with data based on the first data to be filled, thereby achieving fast and easy data filling under the condition of instantaneous data missing and stable data, and avoiding unnecessary complex calculations.
[0095] According to an industrial process data filling device provided by an embodiment of the present invention, when the instantaneous data missing filling module 20 and the long-term data missing filling module 30 are used to fill the missing data to be filled according to the industrial process data of the current dimension and the related dimensions, they are specifically used to: obtain a second data missing period where data missing occurs according to the industrial process data; wherein the second data missing period does not include the equipment shutdown and maintenance period; obtain a fitting curve through machine learning based on the industrial process data of the current dimension and the related dimensions; obtain the previous existing data and the subsequent existing data closest to the second data missing period of the industrial process data; obtain a first fitting value of the previous existing data, a second fitting value of the subsequent existing data, and a third fitting value of the industrial process data at the time of data missing to be filled according to the fitting curve; calculate a first interpolation result of the first fitting value and the second fitting value at the time of data missing, and calculate a second interpolation result of the previous existing data and the subsequent existing data at the time of data missing; obtain the second data to be filled at the time of data missing according to the first interpolation result, the second interpolation result and the third fitting value; and fill the data at the time of data missing using the second data to be filled.
[0096] An industrial process data filling device provided by an embodiment of the present invention obtains a second data missing period where data missing occurs according to the industrial process data, wherein the second data missing period does not include an equipment shutdown and maintenance period, obtains a fitting curve through machine learning based on the industrial process data of the current dimension and related dimensions, obtains the previous existing data and the subsequent existing data closest to the second data missing period of the industrial process data, obtains a first fitting value of the previous existing data, a second fitting value of the subsequent existing data, and a third fitting value of the industrial process data at the data missing moment to be filled according to the fitting curve, calculates a first interpolation result of the first fitting value and the second fitting value at the data missing moment, and calculates a second interpolation result of the previous existing data and the subsequent existing data at the data missing moment, obtains the second data to be filled at the data missing moment according to the first interpolation result, the second interpolation result and the third fitting value, and fills the data missing moment with the second data to be filled, thereby improving the accuracy of data filling of the missing data to be filled according to the industrial process data of the current dimension and related dimensions.
[0097] According to an industrial process data filling device provided by an embodiment of the present invention, the second data to be filled is represented as:
[0098]
[0099] Among them, M T Indicates the second data to be filled, M″ T represents the first interpolation result, represents the second interpolation result, M′ T represents the third fitting value.
[0100] The industrial process data filling device provided by the embodiment of the present invention further improves the accuracy of data filling for missing data to be filled based on industrial process data of the current dimension and related dimensions by providing a calculation expression for the second data to be filled.
[0101] Figure 3 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention, such as Figure 3As shown, the electronic device may include: a processor 310, a communication interface 320, a memory 330 and a communication bus 340, wherein the processor 310, the communication interface 320 and the memory 330 communicate with each other through the communication bus 340. The processor 310 may call the logic instructions in the memory 330 to execute the industrial process data filling method, which includes: obtaining the industrial process data to be filled, determining whether the data missing type of the industrial process data is instantaneous data missing or long-term data missing; in response to the data missing type being instantaneous data missing, further determining the stability of the industrial process data, if the industrial process data is stable data, then using the industrial process data of the current dimension to fill the missing data to be filled, if the industrial process data is unstable data, then filling the missing data to be filled according to the industrial process data of the current dimension and related dimensions; in response to the data missing type being long-term data missing, filling the missing data to be filled according to the industrial process data of the current dimension and related dimensions.
[0102] In addition, the logic instructions in the above-mentioned memory 330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.
[0103] On the other hand, an embodiment of the present invention further provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the industrial process data filling method provided by the above methods, which includes: obtaining industrial process data to be filled, and determining whether the data missing type of the industrial process data is instantaneous data missing or long-term data missing; in response to the data missing type being instantaneous data missing, further determining the stability of the industrial process data; if the industrial process data is stable data, using the industrial process data of the current dimension to fill the missing data to be filled; if the industrial process data is unstable data, filling the missing data to be filled according to the industrial process data of the current dimension and related dimensions; in response to the data missing type being long-term data missing, filling the missing data to be filled according to the industrial process data of the current dimension and related dimensions.
[0104] On the other hand, an embodiment of the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it is implemented to execute the industrial process data filling method provided by the above-mentioned methods. The method includes: obtaining industrial process data to be filled, and determining whether the data missing type of the industrial process data is instantaneous data missing or long-term data missing; in response to the data missing type being instantaneous data missing, further determining the stability of the industrial process data; if the industrial process data is stable data, using the industrial process data of the current dimension to fill the missing data to be filled; if the industrial process data is unstable data, filling the missing data to be filled according to the industrial process data of the current dimension and related dimensions; in response to the data missing type being long-term data missing, filling the missing data to be filled according to the industrial process data of the current dimension and related dimensions.
[0105] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.
[0106] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.
[0107] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for filling industrial process data, characterized in that: include: Acquire the industrial process data to be filled, and determine whether the data missing type of the industrial process data is instantaneous data missing or long-term data missing; In response to the data missing type being instantaneous data missing, further determining the stability of the industrial process data, if the industrial process data is stable data, using the industrial process data of the current dimension to fill in the missing data to be filled, and if the industrial process data is unstable data, filling in the missing data to be filled in according to the industrial process data of the current dimension and related dimensions; In response to the data missing type being long-term data missing, data filling is performed on the missing data to be filled according to industrial process data of the current dimension and related dimensions.
2. The industrial process data filling method according to claim 1, characterized in that: The determining whether the data missing type of the industrial process data is instantaneous data missing or long-term data missing includes: Acquire a data missing time span according to the industrial process data; In response to the data missing time span being greater than the data sampling time interval and less than or equal to a preset time span threshold, the data missing type is instantaneous data missing; In response to the data missing time span being greater than the preset time span threshold, the data missing type is long-term data missing.
3. The industrial process data filling method according to claim 1, characterized in that: The determining the stability of the industrial process data includes: Acquire the industrial process data in a preset time unit; Based on the data of the industrial process data in a preset time unit, obtaining a change trend of the existing data within a preset distance before and after the data missing period of the industrial process data; In response to knowing that the data changes monotonously and slowly according to the change trend, the industrial process data is determined to be stable data, otherwise it is unstable data.
4. The industrial process data filling method according to claim 1, characterized in that: The method of using the industrial process data of the current dimension to fill in the missing data to be filled includes: Acquire a first data missing time period in which data missing occurs according to the industrial process data; Acquire the existing data before and after the first data missing period in the industrial process data, and obtain the first data to be filled in the first data missing period by interpolation calculation based on the existing data before and after; The first data missing period is filled with data according to the first data to be filled.
5. The industrial process data filling method according to claim 1, characterized in that: The data filling of the missing data to be filled according to the industrial process data of the current dimension and related dimensions includes: Acquire a second data missing period in which data missing occurs according to the industrial process data; wherein the second data missing period does not include an equipment shutdown or maintenance period; Based on the industrial process data of the current dimension and related dimensions, the fitting curve is obtained through machine learning; Acquire the previous existing data and the subsequent existing data closest to the second data missing time period of the industrial process data; Acquire, according to the fitting curve, a first fitting value of the previous existing data, a second fitting value of the subsequent existing data, and a third fitting value of the industrial process data at a time when data is missing and to be filled; Calculating a first interpolation result of the first fitting value and the second fitting value at the time when the data is missing, and calculating a second interpolation result of the previous existing data and the subsequent existing data at the time when the data is missing; Obtain second data to be filled at the data missing moment according to the first interpolation result, the second interpolation result and the third fitting value; The data missing moment is filled using the second data to be filled.
6. The industrial process data filling method according to claim 5, characterized in that: The second data to be filled is expressed as: Among them, M T represents the second data to be filled, M' T ' represents the first interpolation result, represents the second interpolation result, M' T represents the third fitting value.
7. An industrial process data filling device, characterized in that: include: A data missing type judgment module is used to: obtain the industrial process data to be filled, and judge whether the data missing type of the industrial process data is instantaneous data missing or long-term data missing; a transient data missing filling module, for: in response to the data missing type being transient data missing, further determining the stability of the industrial process data; if the industrial process data is stable data, filling the missing data to be filled using the industrial process data of the current dimension; if the industrial process data is unstable data, filling the missing data to be filled according to the industrial process data of the current dimension and related dimensions; The long-term data missing filling module is used to: in response to the data missing type being long-term data missing, fill in the missing data to be filled according to the industrial process data of the current dimension and related dimensions.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the industrial process data filling method according to any one of claims 1 to 6 are implemented.
9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the industrial process data filling method according to any one of claims 1 to 6 are implemented.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the industrial process data filling method according to any one of claims 1 to 6 are implemented.