Industrial equipment data intelligent management method based on big data algorithm

Abstract

The invention discloses an industrial equipment data intelligent management method based on a big data algorithm. The method comprises the following steps: acquiring operation data of industrial equipment from a plurality of data sources; the missing part in the operation data is filled based on a data filling algorithm; performing format conversion and standardization processing on the multi-source heterogeneous data; positive and negative samples are balanced by adopting a resampling technology in a model training process; and carrying out large-scale data processing by utilizing a distributed computing framework. The industrial equipment data management method based on the big data algorithm is high in accuracy, high in real-time performance, simpler in data processing and high in data processing upper limit.

Description

Technical Field

[0001] The present invention relates to the field of intelligent management of industrial equipment data, and in particular to an intelligent management method of industrial equipment data based on big data algorithms. Background Art

[0002] The intelligent management method of industrial equipment data based on big data algorithms aims to optimize and manage the operating data of industrial equipment by utilizing big data technology and advanced algorithms, thereby realizing functions such as predictive maintenance, fault diagnosis and performance optimization. However, this method faces multiple challenges in practical applications: first, when the operating data of industrial equipment is incomplete or missing, the accuracy of the prediction model will be seriously affected due to insufficient data volume; second, data often encounters network delays and packet loss during transmission, which will cause the real-time performance of the system to be damaged; in addition, in the process of multi-source heterogeneous data fusion, the complexity of data processing is increased due to the inconsistency of data formats; during model training, the imbalance of positive and negative samples will cause the model to be skewed, affecting its reliability and accuracy; finally, in large-scale data processing, limited computing resources often become a performance bottleneck, restricting the overall efficiency of the system. These problems not only affect the system performance, but also limit the further application of big data technology in the industrial field. Summary of the invention

[0003] In view of this, the present invention discloses an intelligent management method for industrial equipment data based on a big data algorithm, which at least partially solves the problems existing in the prior art.

[0004] To achieve the above technical objectives, the technical solution of the present invention is implemented as follows:

[0005] The intelligent management method of industrial equipment data based on big data algorithm includes:

[0006] Obtaining operational data of industrial equipment from multiple data sources, including sensor data, maintenance records, and historical operation logs;

[0007] Fill in the missing parts of the running data based on the data completion algorithm to improve the accuracy of the prediction model;

[0008] Perform format conversion and standardization on multi-source heterogeneous data to reduce the complexity of data processing;

[0009] Resampling techniques are used during model training to balance positive and negative samples to avoid model skew;

[0010] Use distributed computing frameworks to process large-scale data, optimize the use of computing resources, and resolve performance bottlenecks.

[0011] Preferably, the method of filling the missing parts in the operating data based on the data completion algorithm specifically includes the following steps:

[0012] The Kalman filter algorithm is used to fill the missing parts of the sensor data;

[0013] The missing parts of the maintenance records were filled using a linear interpolation method based on time series;

[0014] Use similarity-based clustering algorithms to fill in the missing parts of historical operation logs;

[0015] Compare and verify the filled data with the original data to ensure the accuracy of the filled data.

[0016] Preferably, the method of filling the missing parts in the running data based on the data completion algorithm further comprises the following steps:

[0017] The missing values ​​in the sensor data are predicted based on the Kalman filter algorithm. The prediction formula is: X(k)=A*X(k1)+B*U(k1), where X(k) represents the sensor data at the kth moment, A represents the state transfer matrix, X(k1) represents the sensor data at the k1th moment, B represents the control input matrix, and U(k1) represents the control input at the k1th moment;

[0018] Conduct confidence assessment on the predicted value to ensure that it meets the requirements within the confidence interval;

[0019] If the confidence level of the predicted value is lower than the preset threshold, the mean of historical data is used for filling;

[0020] Verify the integrity and consistency of the padded data to ensure that the padded data can be used for model training.

[0021] Preferably, the method of filling the missing parts in the running data based on the data completion algorithm further comprises the following steps:

[0022] Based on the historical trends and periodic characteristics of sensor data, an adaptive filter is constructed to fill in the data;

[0023] Use the formulated conditional judgment statement to determine whether the filling value needs to be recalculated: if |Y(k)-P(k)|>T, then recalculate the filling value, where Y(k) represents the actual value, P(k) represents the predicted value, and T represents the threshold;

[0024] Perform a second check on the filled data to ensure data consistency and reliability;

[0025] Feedback the verification results to the system log for subsequent analysis and optimization.

[0026] Preferably, the method of filling the missing parts in the running data based on the data completion algorithm further comprises the following steps:

[0027] Data filling based on an integrated approach of multiple filling algorithms (such as Kalman filtering and linear interpolation);

[0028] The validity of the filling algorithm is judged by using a formulaic conditional statement: if |E1-E2|<δ, the prediction result of the integrated method is adopted, where E1 represents the result of Kalman filtering, E2 represents the result of linear interpolation, and δ represents the maximum allowable error;

[0029] Evaluate the final filling results to ensure that the filling effect meets the expected standards;

[0030] The key steps and results of the imputation process are saved in log files for subsequent analysis.

[0031] Preferably, the method of filling the missing parts in the running data based on the data completion algorithm further comprises the following steps:

[0032] Machine learning-based anomaly detection algorithms identify missing data patterns;

[0033] Use a formulaic conditional judgment statement to determine whether data needs to be recollected: if Q>M, recollection is triggered, where Q represents the number of abnormal data and M represents the preset abnormal threshold;

[0034] Supplement and fill in the re-collected data to ensure the completeness and accuracy of the data;

[0035] The results of filling and collection are recorded in the database for subsequent query and management.

[0036] Preferably, the method of filling the missing parts in the running data based on the data completion algorithm further comprises the following steps:

[0037] Predict missing values ​​in sensor data based on Bayesian inference methods;

[0038] Use a formulaic conditional judgment statement to judge the reliability of the predicted value: if L>λ, then use the Bayesian prediction value, where L represents the probability distribution function value of the predicted value and λ represents the reliability threshold;

[0039] Conduct statistical analysis on the filled data to ensure that the data distribution is within the normal range;

[0040] The filling results and their reliability indicators are saved to the system log for subsequent audit and improvement.

[0041] Preferably, filling the missing parts in the running data based on the data completion algorithm includes:

[0042] Supplement missing time point data through time series interpolation methods;

[0043] Use Kalman filter algorithm to smooth and predict missing data;

[0044] Generate the probability distribution of missing data through deep learning models;

[0045] After generating the possibility distribution, the data with the highest probability is selected as the filling result to improve the accuracy of the prediction model, especially when the industrial equipment operation data is incomplete or missing, to reduce the problem of decreased accuracy of the prediction model due to insufficient data.

[0046] Beneficial effects of the present invention:

[0047] 1. The present invention can solve the problem of decreased accuracy of prediction models due to insufficient data when the operating data of industrial equipment is incomplete or missing;

[0048] 2. The present invention can solve the problem of real-time impairment caused by network delay and packet loss during data transmission;

[0049] 3. The present invention can solve the problem of increased data processing complexity caused by non-uniform data formats during the fusion of multi-source heterogeneous data;

[0050] 4. The present invention can solve the model skew problem caused by the imbalance of positive and negative samples during model training;

[0051] 5. The present invention can solve the performance bottleneck problem caused by limited computing resources in large-scale data processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0053] The technical solutions in the embodiments of the present invention are described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0054] like Figure 1 As shown, according to an embodiment of the present invention, a method for intelligent management of industrial equipment data based on a big data algorithm is intended to solve multiple common challenges in the process of industrial equipment data management and analysis, specifically including: obtaining the operating data of industrial equipment from multiple data sources; filling in the missing parts of the operating data based on a data completion algorithm; format conversion and standardization of multi-source heterogeneous data; using resampling technology to balance positive and negative samples during model training; and using a distributed computing framework for large-scale data processing. The following is a detailed explanation of each step and how it solves the above problems.

[0055] Obtain operating data of industrial equipment from multiple data sources. The purpose of this step is to ensure the diversity and comprehensiveness of data sources, thereby improving the overall quality and credibility of the data. Specifically, data sources can include various sensors built into the equipment, historical maintenance records provided by the maintenance team, and factory operation logs. For example, in a wind power station, operation and maintenance personnel can obtain real-time wind speed, temperature, humidity, speed and other parameters through sensors installed on the turbines, and combine past equipment maintenance logs and the influence of environmental factors to more accurately evaluate the operating status of the equipment.

[0056] Based on the data completion algorithm, the missing parts in the operating data are filled to improve the accuracy of the prediction model. When industrial equipment runs for a long time, it is inevitable to encounter sensor failures, storage medium damage, etc., resulting in the loss of some data. The data completion algorithm analyzes the existing data and uses statistical methods (such as mean filling, linear interpolation, etc.) or more complex machine learning models (such as K nearest neighbors, Bayesian methods, etc.) to infer and fill these missing values. In one embodiment, in response to the missing data of wind power generation equipment, by analyzing the relationship between wind speed, humidity and speed in historical data, a multivariate regression model is used to infer the data of the missing period, thereby ensuring the integrity of the model input and significantly improving the accuracy of the equipment health status prediction model.

[0057] Format conversion and standardization of multi-source heterogeneous data to reduce the complexity of data processing. Information generated by different data sources is often encoded in different formats, and directly merging them may cause compatibility issues. To overcome this, data from various sources need to be converted into a common standard format, which usually involves character encoding specifications, unit unification, and data type definitions. For example, when processing information output by various testing equipment in a steel plant, the system can automatically identify and convert the measurement unit of the pressure value (such as kgf / cm 2 Automatically convert to kPa) to eliminate calculation errors caused by inconsistent units.

[0058] Resampling techniques are used during model training to balance positive and negative samples to avoid model skew. Imbalance of positive and negative samples means that the target class (usually indicating equipment abnormalities) is very scarce relative to the normal class sample size. If this type of imbalance is not handled when building a diagnostic model, the model will be more inclined to predict most common categories and ignore a few important but dangerous events. Balance the proportion of the two groups of data by implementing methods such as random undersampling, oversampling, or comprehensive strategies (SMOTE algorithm) to ensure that the algorithm can effectively distinguish between equipment in normal working conditions and those that may have problems. For example, when monitoring the working conditions of a reactor on a chemical production line, although the frequency of equipment failure is very low, we still need to fully train the AI ​​system to recognize potential danger signals so that it can respond in a timely manner in truly emergency situations.

[0059] Use distributed computing frameworks to process large-scale data, optimize the use of computing resources, and solve performance bottlenecks. With the continuous improvement of the informatization of the manufacturing industry, the amount of data generated per minute is growing exponentially, and the traditional single-machine processing model is difficult to cope with such a huge load demand. The use of distributed platforms such as Apache Hadoop and Spark can achieve the ability to process massive data in parallel across multiple nodes, which not only greatly accelerates the speed of preprocessing stages such as cleaning and aggregation, but also provides strong power support for subsequent complex algorithm calculations. Taking an oil refinery as an example, in the face of huge log files generated by continuous monitoring of thousands of wellheads, deploying Spark-based applications can complete preliminary screening and quickly extract valuable information in just a few seconds, greatly improving data mining efficiency.

[0060] Furthermore, the present invention fills the missing parts in the running data based on the data completion algorithm, specifically including the following steps:

[0061] The Kalman filter algorithm is used to fill the missing parts of the sensor data;

[0062] The missing parts of the maintenance records were filled using a linear interpolation method based on time series;

[0063] Use similarity-based clustering algorithms to fill in the missing parts of historical operation logs;

[0064] Compare and verify the filled data with the original data to ensure the accuracy of the filled data.

[0065] In the first step, the Kalman filter algorithm is used to fill in the missing parts of the sensor data. The Kalman filter is a recursive filter that predicts the future state of the system through state estimation and corrects the prediction based on new measurement data. In the case of missing sensor data, the algorithm uses the previous and next observations and the dynamic characteristics of the system model to infer the data at the missing time. For example, in a manufacturing plant, sensors record the operating temperature and pressure data of production equipment. When the sensor fails at a certain moment and the data is missing, the Kalman filter can infer the temperature and pressure values ​​at this moment through previous data trends and known physical models.

[0066] The second step is to use a linear interpolation method based on time series to fill in the missing parts of the maintenance records. Linear interpolation is to assume a linear relationship between two known data points and use this relationship to infer the value of the missing point. Maintenance records usually include information such as equipment maintenance time and the date of replacement of parts. If a section of the record is missing due to negligence or system failure, linear interpolation can be performed through the existing records before and after. In a specific example, the maintenance record of a certain equipment shows that a routine inspection is required every two weeks, but the record of a certain week is missing. By linearly interpolating the time points of two adjacent inspections, the specific time of the missing record can be reasonably inferred.

[0067] The third step is to use a similarity-based clustering algorithm to fill in the missing parts of the historical operation log. The algorithm divides the historical operation log into multiple clusters based on similarity, and the data in each cluster has a high degree of similarity. For missing data in a cluster, it can be inferred based on other data points in the cluster. For example, in a wind farm, the operation log of the wind turbine records the power generation, ambient wind speed, fault conditions and other information of each wind turbine. If the log data of a certain day is missing, the data of that day can be compared with the data under other similar conditions (such as the data in the same wind farm and similar wind speed conditions) to find the most similar set of data, and use these data to infer the missing part of the information.

[0068] The last step is to compare and verify the padded data with the original data to ensure the accuracy of the padded data. This step evaluates the rationality of the padded data through statistical methods and machine learning models. Specifically, the padded effect can be measured by calculating the mean square error (MSE) and mean absolute error (MAE) between the padded data and the existing data. At the same time, manual verification can also be performed through expert knowledge. For example, in the historical data verification process of a wind farm, if the daily power generation of a wind turbine after padded is far different from the actual log data it generates, the accuracy and consistency of the data can be improved by adjusting the parameters of the padded algorithm or re-padded.

[0069] In the above steps, the formula parameters involved and their meanings are as follows:

[0070] Kalman filter formula:

[0071] hat{x}_{k|k}=hat{x}_{k|k-1}+K_k(z_k-H*hat{x}_{k|k-1})

[0072] Where hat{x}_{k|k} is the current state estimate, hat{x}_{k|k-1} is the predicted state, K_k is the Kalman gain, which determines the proportion of measurement updates, z_k is the measurement value, and H is the observation matrix, which indicates how the state is converted to the measurement value. The Kalman gain K_k needs to vary between 0 and 1, and the optimal value is usually found through an optimization algorithm to ensure that the estimated value fluctuates less around the true value.

[0073] Linear interpolation formula:

[0074] y(t)=y_1+frac{(t-t_1)(y_2-y_1)}{t_2-t_1}

[0075] Where y(t) is the interpolated value, t_1 and t_2 are the times of the known data points, and y_1 and y_2 are the known values ​​at the corresponding times.

[0076] The time points and values ​​in the interpolation formula should be within a reasonable range to prevent the interpolation results from deviating too much from the actual values.

[0077] Mean Squared Error (MSE):

[0078] MSE=frac{1}{n}sum_{i=1}^n(y_i-hat{y}_i)^2)

[0079] n is the number of data points and y_i is the actual value.

[0080] The method of filling the missing parts in the running data based on the data completion algorithm of the present invention further includes the following steps:

[0081] Based on the Kalman filter algorithm, the missing values ​​in the sensor data are predicted. The prediction formula is: X(k) = A*X(k-1) + B*U(k-1), where X(k) represents the sensor data at the kth moment, A represents the state transfer matrix, X(k-1) represents the sensor data at the k-1th moment, B represents the control input matrix, and U(k-1) represents the control input at the k-1th moment. This formula is used to predict the most likely value at the location of the missing value in the sensor data. A is a matrix that usually represents the dynamic behavior of the system, and its value depends on the specific physical process. For example, in a motion system, it may represent the relationship between velocity and acceleration. B is a matrix used to convert the control input U(k-1) into an impact on the system state. In most cases, A and B are estimated based on historical data and system models. The choice of formula is based on the effectiveness of the Kalman filter, which can accurately estimate the state in a noisy environment, so it is widely used in data prediction of industrial equipment.

[0082] The confidence level of the predicted value is evaluated to ensure that it meets the requirements within the confidence interval. This step involves statistical methods to evaluate the reliability of the predicted value by calculating the standard error or confidence interval of the predicted value. For example, if the confidence interval of the predicted value is narrow and contains the actual value of the historical data, the predicted value is considered to have a high degree of confidence. Conversely, if the confidence interval is wide or does not contain the actual value, the reliability of the predicted value is considered to be low. In one embodiment, assuming that the confidence interval of the predicted value of a certain sensor is ±5%, and the confidence interval covers 95% of the actual values ​​of the historical data, the predicted value is considered to have a high degree of confidence.

[0083] If the confidence of the predicted value is lower than the preset threshold, the mean of the historical data is used for filling. This means that when the reliability of the predicted value is not high, the average value in the historical data is chosen to fill the missing value. The mean of the historical data is usually obtained by taking the average of all sensor data in a specific time period. This method is suitable for scenarios with low confidence in the predicted value because it provides a more robust alternative. Specifically, assuming that a device has been running smoothly for the past 100 hours and the sensor data has not fluctuated much, the average of all data points during this period can be calculated as the filling value.

[0084] Verify the integrity and consistency of the padded data to ensure that the padded data can be used for model training. This step involves a comprehensive check of the padded data to ensure the consistency and integrity of the data. The verification content includes whether there are outliers in the data, whether the padded data conforms to the logical and expected pattern, etc. For example, in a specific application scenario, assuming that the padded data shows that the temperature at a certain moment suddenly rises beyond the normal range, the rationality of the padded value needs to be further verified. If anomalies are found, the predicted value should be re-evaluated or other padded methods should be used again to ensure the quality of the final data. This ensures that the padded data can provide accurate and reliable input in subsequent model training.

[0085] The data-filling algorithm of the present invention is used to fill in the missing parts in the operating data, and further includes the following steps: based on the historical trends and periodic characteristics of the sensor data, an adaptive filter is constructed to perform data filling; a formulaic conditional judgment statement is used to determine whether the filling value needs to be recalculated: if |Y(k)-P(k)|>T, the filling value is recalculated, where Y(k) represents the actual value, P(k) represents the predicted value, and T represents the threshold; the filled data is double-checked to ensure the consistency and reliability of the data; and the verification results are fed back to the system log for subsequent analysis and optimization.

[0086] The first step is to construct an adaptive filter to fill in the data based on the historical trends and periodic characteristics of the sensor data. Specifically, this step involves analyzing the data recorded by the sensor over a long period of time to extract trends and periodic patterns in the data, such as the difference between weekdays and weekends or fluctuations in different time periods. Based on these patterns, the system can build an adaptive filter that can automatically adjust parameters based on historical data to better predict missing values. For example, in one embodiment, if an industrial equipment is shut down for maintenance every weekend, resulting in missing data, the adaptive filter will infer the weekend data based on the data pattern of the weekday.

[0087] Secondly, use the formulated conditional judgment statement to determine whether the filling value needs to be recalculated: if |Y(k)-P(k)|>T, then recalculate the filling value, where Y(k) represents the actual value, P(k) represents the predicted value, and T represents the threshold. In this formula, Y(k) is the actual measured value at a certain moment, P(k) is the predicted value at the same moment, and T is a set error threshold, which is usually a relatively small positive number. If the absolute difference between the predicted value and the actual value is greater than this threshold, the system considers that the prediction result is unreliable and the filling value needs to be recalculated. For example, T may be set to 0.1, indicating that when the deviation between the predicted value and the actual value exceeds 10%, recalculation is required. This can avoid using predicted data that deviates significantly from the actual value, thereby improving data accuracy.

[0088] Third, perform a second check on the padded data to ensure data consistency and reliability. This step verifies the padded data through multiple independent methods or algorithms to ensure data consistency and reliability. Specifically, you can use another different prediction model for comparison, or check based on business logic and physical rules to ensure there are no obvious anomalies. For example, if the padded data is obviously inconsistent with the business process or physical laws, the system will mark the data as suspicious and conduct a manual review.

[0089] Finally, the verification results are fed back to the system log for subsequent analysis and optimization. Specifically, all filling processes and verification results will be recorded in the system log, including the specific value of each prediction, the filled data points, the number of recalculations and reasons, etc. These log data are not only convenient for tracing the source of data quality problems, but also can be used as a basis for training and optimizing algorithms. For example, through regular analysis of log data, it is possible to identify which devices' data often need to recalculate the filled values, thereby optimizing algorithm parameters or improving the data collection process.

[0090] The method of filling the missing parts in the running data based on the data completion algorithm of the present invention further includes the following steps:

[0091] Data filling based on an integrated approach of multiple filling algorithms (such as Kalman filtering and linear interpolation);

[0092] The validity of the filling algorithm is judged by using a formulaic conditional statement: if |E1-E2|<δ, the prediction result of the integrated method is adopted, where E1 represents the result of Kalman filtering, E2 represents the result of linear interpolation, and δ represents the maximum allowable error;

[0093] Evaluate the final filling results to ensure that the filling effect meets the expected standards;

[0094] The key steps and results of the imputation process are saved in log files for subsequent analysis.

[0095] In the first step, multiple filling algorithms are integrated in the data filling process. For example, common filling algorithms include Kalman filtering and linear interpolation. These two algorithms have their own advantages. Kalman filtering is suitable for smoothing dynamic system data, while linear interpolation is suitable for simple, continuous data filling. In one embodiment, assuming that the temperature sensor data of a certain industrial equipment is missing, Kalman filtering can be used for prediction first, and then linear interpolation can be used to supplement the data points. Finally, the outputs of the two are weighted averaged as the final filling result.

[0096] In the second step, a formulated conditional statement is used to determine the effectiveness of the filling algorithm. If |E1-E2|<δ holds, the prediction result of the integrated method is used. Here, E1 and E2 represent the results of Kalman filtering and linear interpolation, respectively, and δ represents the maximum allowable error, which is usually a small positive number, such as 0.1 or 0.5, and the specific value depends on the application scenario. The meaning of this formula is that if the absolute difference between the results of the two algorithms is less than the preset error threshold δ, it is considered that the consistency between the two algorithms is high, and the result of the integrated method can be selected. This setting is intended to ensure the stability and reliability of the filling results.

[0097] In the third step, the final filling result needs to be evaluated to ensure that it meets the expected standards. For example, the filling effect can be evaluated by comparing the statistical characteristics of the filled data with the normal data (such as mean, variance, autocorrelation, etc.). In one embodiment, for the temperature data of the above industrial equipment, the root mean square error (RMSE) between the filled data and the actual recorded data can be calculated to ensure that the filling error is within an acceptable range.

[0098] In the fourth step, the key steps and results of the filling process are saved in the log file for subsequent analysis. This includes the filling algorithm used, the specific parameter configuration, the data changes before and after filling, the evaluation results and other information. Specifically, the log file may record the following: data filling using Kalman filtering and linear interpolation; the maximum allowable error δ = 0.5; the root mean square error RMSE after filling is 0.3; the mean temperature data before and after filling are 37.2℃ and 37.5℃ respectively. The purpose of this is to facilitate the backtracking of the filling process for troubleshooting or further optimization.

[0099] The method of filling the missing parts in the running data based on the data completion algorithm of the present invention further includes the following steps:

[0100] The anomaly detection algorithm based on machine learning identifies missing data patterns; uses a formulated conditional judgment statement to determine whether data needs to be recollected: if Q>M, recollection is triggered, where Q represents the number of abnormal data and M represents the preset abnormal threshold; the recollected data is supplemented and filled to ensure the integrity and accuracy of the data; the results of filling and collection are recorded in the database for subsequent query and management.

[0101] First, a machine learning-based anomaly detection algorithm identifies missing data patterns. In this step, supervised or unsupervised machine learning algorithms are used to identify and classify abnormal patterns in the data. These algorithms can be clustering algorithms, decision trees, support vector machines (SVMs), or neural networks. For example, in an industrial equipment monitoring system, an anomaly detection algorithm can be trained on historical data to identify data gaps or abnormal values ​​that occur in temperature sensors at specific points in time.

[0102] Secondly, use the formulaic conditional judgment statement to determine whether the data needs to be recollected: if Q>M, recollection is triggered. Here, Q represents the number of abnormal data detected, and M represents the preset abnormal threshold. The threshold M is selected based on the importance and reliability requirements of the data, usually between 1 and 10, depending on the application scenario. For example, in high-voltage electrical equipment monitoring, M may be set to 3, which means that when more than 3 abnormal data are detected within a certain period of time, it is considered a data quality problem and needs to be recollected.

[0103] Next, the re-collected data is supplemented to ensure the integrity and accuracy of the data. This step is performed by an automated data collection system. After cleaning and verification, the re-collected data is merged with the existing data to fill in the missing parts. For example, in the data monitoring system of a wind turbine generator set, once it is identified that the speed sensor of a certain generator frequently has abnormal values ​​within a certain period of time, the system will automatically start the re-collection process and obtain the correct data within that period from the backup data source.

[0104] Finally, the results of filling and collection are recorded in the database for subsequent query and management. All new data will be saved in the database and merged with the original data set. This not only helps to monitor the operating status of the system in real time, but also supports long-term historical data analysis. For example, in this way, maintenance personnel can easily retrieve detailed operating data from any time period in the past to help with fault analysis and preventive maintenance. In industrial equipment management, ensuring the integrity and accuracy of data is crucial to ensuring the stable operation of equipment.

[0105] Next, the method of filling the missing part in the running data based on the data completion algorithm of the present invention is described, further comprising the following steps:

[0106] Predict missing values ​​in sensor data based on Bayesian inference methods;

[0107] Use a formulaic conditional judgment statement to judge the reliability of the predicted value: if L>λ, then use the Bayesian prediction value, where L represents the probability distribution function value of the predicted value and λ represents the reliability threshold;

[0108] Conduct statistical analysis on the filled data to ensure that the data distribution is within the normal range;

[0109] The filling results and their reliability indicators are saved to the system log for subsequent audit and improvement.

[0110] The first step is to predict the missing values ​​in the sensor data based on the Bayesian inference method. This step uses the Bayesian inference method to estimate the missing values ​​using known data points. The Bayesian inference method calculates the posterior probability through the prior probability and the likelihood function to predict the most likely missing value. In one embodiment, assuming that missing data appears in the temperature sensor record of a certain industrial equipment, the most likely temperature value at the missing time point can be calculated by combining the temperature change trend in the previous time period and other relevant sensor data through the Bayesian inference method.

[0111] The second step is to use a formulated conditional judgment statement to determine the reliability of the predicted value: if L>λ, the Bayesian predicted value is adopted. In this process, the probability distribution function value L of the predicted value is first calculated, and then compared with the preset reliability threshold λ. If L is greater than λ, the predicted value is considered reliable, otherwise it is discarded and other methods are reselected or the data is kept missing. In a specific embodiment, assuming that λ is set to 0.95, it means that the predicted value is considered reliable only when its probability distribution function value is greater than 0.95. For example, if the calculated probability distribution function value of a temperature prediction value is 0.98, which is greater than 0.95, the predicted value is adopted.

[0112] The third step is to perform statistical analysis on the padded data to ensure that the data distribution is within the normal range. This step aims to verify the quality of the padded data by performing statistical analysis on the padded data to check whether its distribution is within the normal range of historical data. Specific analysis methods can include calculating statistical indicators such as mean, standard deviation, variance, and drawing visual graphics such as box plots and histograms to intuitively display the data distribution. For example, in industrial equipment operation monitoring, if the mean of the padded temperature data is close to the historical data and the fluctuation is within the normal range, the data can be considered to be successfully padded.

[0113] The fourth step is to save the filling results and their reliability indicators to the system log for subsequent audit and improvement. In this link, all filled data and their corresponding reliability indicators will be recorded in the system log for later audit and optimization. Specifically, the system log will contain information such as the timestamp of the filled data, the filled value, and the reliability indicator of the predicted value, which is convenient for operation and maintenance personnel to trace and improve data management strategies. For example, during a certain equipment maintenance, if some filled data is found to be abnormal, the operation and maintenance personnel can trace back the system log, view the corresponding reliability indicators, analyze the cause of the problem and adjust the filling algorithm.

[0114] The data completion algorithm of the present invention fills in the missing parts in the running data.

[0115] The missing time point data is supplemented by time series interpolation methods. This step uses the existing time point data to infer the missing values ​​based on the adjacent data points in the time sequence. Interpolation methods usually include linear interpolation, spline interpolation, and polynomial interpolation. These methods construct a continuous curve or function based on the relationship between known data points to estimate the value of unknown data points. Specifically, in the data management of industrial equipment, suppose that a certain equipment records the operating status every 10 minutes. If the data at a certain time point is missing, the data at the previous time point and the next time point can be linearly interpolated to obtain a reasonable estimate of the missing point. For example, if the data point at 10:30 is missing between 10:00 and 11:00, the data at 10:30 can be calculated by linear interpolation method as the average of these two points.

[0116] Use the Kalman filter algorithm to smooth and predict missing data. Kalman filtering is a recursive filter used to estimate the true state of a system from noisy measurement data. Kalman filtering combines the two steps of state prediction and measurement update to optimize the estimation results through continuous iteration. Specifically, in the operation of industrial equipment, Kalman filtering can be used to smooth sensor readings and predict the true state when data is missing. For example, if a sensor does not record data at a specific time due to a fault, the Kalman filter algorithm can be used to predict the sensor reading at this time. The formula for Kalman filtering is as follows:

[0117] hat{x}_{k|k}=hat{x}_{k|k-1}+K_k(z_k-H*hat{x}_{k|k-1})

[0118] Among them, hat{x}_{k|k-1} is the state prediction value, hat{x}_{k|k} is the updated state estimate, K_k is the Kalman gain, z_k is the observation value, and H is the observation matrix. The optimal value of parameter K_k depends on the state of the system and the characteristics of the observation noise, which is generally determined by experiments. Kalman filtering can effectively smooth and predict missing data and improve the continuity and accuracy of data.

[0119] Generate a probability distribution of missing data through a deep learning model. Deep learning models such as long short-term memory networks (LSTM) or variational autoencoders (VAE) can be used to generate multimodal distributions of missing data. Such models capture complex patterns and dependencies of data by learning from a large amount of historical data, thereby generating multiple possible fill-in values ​​and their probability distributions. Specifically, assuming that a device continuously records data for a period of time, but there are several missing values ​​in the middle, you can train an LSTM model, input the data of the previous and next time periods, and output multiple possible values ​​and corresponding probabilities of the missing points in the middle. For example, suppose a two-layer LSTM network is used, each layer has 128 neurons, the input sequence length is 50 time points, and the output sequence length is 10 time points. By training such a model, multiple possible values ​​and their probability distributions can be generated at the location of missing data.

[0120] After generating the possibility distribution, select the data with the highest probability as the filling result to improve the accuracy of the prediction model. After generating multiple possibilities, select the value with the highest probability as the final filling result, which helps to improve the accuracy of subsequent analysis and prediction models. Especially in the case of incomplete or missing data, reduce the prediction error caused by insufficient data. For example, in the above example, multiple possible values ​​and their probability distributions are generated by LSTM. Assume that there are two candidate values ​​A and B at the position of missing data, where the probability of A is 0.8 and the probability of B is 0.2. Finally, A is selected as the filling result. This method not only improves the integrity of the data, but also ensures the accuracy of subsequent analysis.

[0121] In summary, the various steps based on the data completion algorithm can effectively process the missing parts in the operation data of industrial equipment and improve the quality and reliability of the data.

[0122] The present invention uses the Kalman filter algorithm to smooth and predict missing data. In the intelligent management method of industrial equipment data based on big data algorithm, the Kalman filter is used to smooth and predict possible data missing or noise. The specific process is as follows:

[0123] Define the state equation x_{k+1}=A*x_k+B*u_k+w_k, where x_k represents the state vector of the system at the kth time step, A is the state transfer matrix, which reflects the change law of the system from one time point to another; B is the control input matrix, u_k is the control input applied externally to the system; w_k represents the process noise, which is assumed to be zero-mean Gaussian white noise. This equation describes the change process of the system state from k to k+1, as well as the internal dynamic characteristics of the system and the influence of external control.

[0124] Define the observation equation z_k=H*x_k+v_k, where z_k is the actual observation value of the system, H is the observation matrix, which maps the hidden state to the observation space; v_k represents the observation noise, which is also assumed to be zero-mean Gaussian white noise. This equation describes the conversion relationship from the implicit true system state to the measurable data, as well as the errors that may be introduced in the observation process.

[0125] Calculate the prior estimate hat{x}_{k|k-1}=A*hat{x}_{k-1|k-1}+B*u_k and the prior error covariance P_{k|k-1}=AP_{k-1|k-1}A^T+Q. Here, hat{x}_{k-1|k-1} is the posterior estimate from the previous step, representing the known system state; and P_{k-1|k-1} is the posterior error covariance matrix from the previous iteration, reflecting the uncertainty of the previous estimate. Formula Q represents the covariance of the process noise, which is usually determined by the model designer based on actual conditions. The purpose of this step is to make a preliminary prediction of the system state at the current moment based on the system dynamic model and previous estimates.

[0126] Perform a Kalman gain update K_k = P_{k|k-1}H^T(HP_{k|k-1}H^T+R)^{-1}, and then update the posterior estimate

[0127] hat{x}_{k|k}=hat{x}_{k|k-1}+K_k(z_k-H*hat{x}_{k|k-1}), where R is the covariance matrix of the observation noise. The Kalman gain K_k reflects the importance of the new observation data in correcting the prediction error, and it is dynamically adjusted by balancing the trust of the prediction model and the actual value of the observation. Finally, this gain is used to feed back the difference between the observed value z_k and the predicted value to the state estimate, thereby obtaining a more accurate system state estimate hat{x}_{k|k} and the corresponding error covariance P_{k|k}.

[0128] For example, in a specific embodiment, a factory has a complex equipment monitoring system that needs to continuously monitor the operating status of the equipment. However, due to sensor aging or other unforeseen factors, there will occasionally be missing or severely distorted data in certain periods of time in the system. The application of the above-mentioned Kalman filter process can effectively deal with these problems. Initially, reasonable state equation and observation equation parameters are set based on historical operating data and expert knowledge. Then, each time a new sensor reading is received or there is no new data after a period of time, the system state estimate is recursively updated according to the method described above to achieve real-time monitoring of the equipment status and predict future behavior trends. Specifically, for a specific motor temperature detection, even if a temperature sensor fails and the real data cannot be obtained in the short term, the temperature level and its changing trend at that point can still be inferred more accurately by using the Kalman filter combined with other sensor information.

[0129] The present invention selects the data with the highest probability as the filling result. First, based on the generated possibility distribution of missing data, the confidence score of each missing data point is calculated. This step is to determine the credibility of each possible filling value at the position of the missing data point to ensure that the selected filling value is a high-probability result. In this process, the historical data and the existing complete data points are analyzed using big data algorithms to establish a possibility model. For example, in the data acquisition system of industrial equipment, if the sensor fails to record data at a certain moment, the probability distribution of the missing point can be generated by the algorithm based on the data in the adjacent time period and the historical data of similar equipment, and the corresponding confidence is calculated for each possible filling value.

[0130] Afterwards, when the calculated confidence score C(x)>T, the data point is considered reliable enough to be selected as the filling result, where (C(x) represents the confidence score of a missing value x, and T represents a pre-defined confidence threshold, which is usually set according to application requirements and data analysis accuracy goals. For example, if T is set to 0.8, then all data points corresponding to C(x) greater than or equal to 0.8 will be selected. The purpose of this step is to filter out low-confidence filling options and reduce the possibility of introducing errors.

[0131] Next, the error after filling the data needs to be calculated, \(E(x) = |x_{\text{actual}} - x_{\text{imputed}}|\), where \(x_{\text{actual}}\) represents the original value that actually exists but is not observed, and \(x_{\text{imputed}}\) represents the data point estimated by the algorithm and used for filling. The error \(E(x)\) is used to measure the gap between the filled data and the theoretically true value. This information will be stored in the error matrix for later evaluation and optimization reference. This step not only helps to monitor the filling performance but also aids in identifying areas with poor filling quality.

[0132] Finally, if the calculated error \(E(x)\) is greater than the preset maximum acceptable error \(E_{\text{max}}\), it means that the currently selected data point is not ideal and other high-confidence data points need to be reconsidered. Specifically, in one embodiment, assume that \(E_{\text{max}}\) is set to 0.05. This means that any \(E(x)\) exceeding this value will be considered unacceptable, and a new round of screening process for high-score alternative solutions will be initiated. This mechanism effectively ensures high standards for the accuracy and reliability of the finally supplemented data. During the entire filling process, it is crucial to reasonably set and adjust these key parameters, which are directly related to the success rate of data repair and the overall performance of the application system.

[0133] The present invention calculates the confidence score for each missing data point based on the generated probability distribution of the missing data. This process involves the following steps: calculating the cumulative frequency of each missing data point in all generated probability distributions; based on the cumulative frequency \(F(x)\), using the formula \(C(x)=1 - e^{-\alpha F(x)}\) to calculate the confidence score, where \(\alpha\) is an adjustment parameter used to control the influence of frequency on confidence; if the confidence score \(C(x)<T\), regenerate the possible missing data points until a data point that meets the condition is found; store the data points that meet the condition and their corresponding confidence scores in the result matrix.

[0134] First, calculate the cumulative frequency of each missing data point in all generated probability distributions. This means that for each possible missing data point, by generating multiple possible distribution cases, the frequency of this data point appearing in these multiple distributions is statistically counted to form a cumulative distribution function. The higher the cumulative frequency, the greater the possibility of this data point appearing in the generated distribution. For example, in one embodiment, assume that 100 possible temperature data distributions are generated, and a specific temperature data point appears 60 times among them, then the cumulative frequency of this data point is 0.60.

[0135] Based on the cumulative frequency F(x), the confidence score is calculated using the formula C(x) = 1 - e^(-alpha * F(x)), where alpha is an adjustment parameter with a value range typically between 0 and 1, and the optimal value is determined according to the actual application situation and experimental results. The role of alpha is to control the influence of the cumulative frequency on the confidence score. When alpha is small, the change in the cumulative frequency has a small impact on the confidence score; when alpha is large, the change in the cumulative frequency has a large impact on the confidence score. The exponential part in the formula C(x) = 1 - e^(-alpha * F(x)) makes the confidence score C(x) gradually increase as the cumulative frequency F(x) increases, reflecting that a higher frequency corresponds to a higher confidence. For example, if alpha = 0.5, the confidence score C(x) for a data point with a cumulative frequency of 0.60 is 0.713.

[0136] If the calculated confidence score C(x) < T, regenerate the possible missing data points until a data point that meets the condition is found. Here, T is a preset threshold used to determine whether a data point can be considered reliable. If the confidence score of a certain data point is lower than this threshold, it is considered to have low credibility and new data points need to be found by regenerating the distribution to ensure that the finally selected data points have sufficient confidence. For example, in a specific embodiment, the threshold T is set to 0.70. If the confidence score of a certain data point after calculation is only 0.65, the possible temperature distribution needs to be regenerated until at least one data point with a confidence score not lower than 0.70 is found.

[0137] Finally, store the data points that meet the conditions and their corresponding confidence scores in the result matrix. The purpose of this step is to facilitate subsequent processing and analysis. The result matrix contains all the high-confidence data points and their corresponding confidence scores, providing reliable data support for subsequent data management and decision-making. For example, in an industrial equipment monitoring system, storing these data can help identify potential faults or abnormal conditions and take preventive maintenance measures in advance to ensure the normal operation of the equipment.

[0138] The present invention regenerates the possible missing data points. Generate an initial set of hypotheses for the missing data points based on historical data. The core of this step is to speculate on the possible missing data points through the existing historical data and form a set of initial hypotheses. For example, in the scenario of industrial equipment data management, historical data shows that a device uploads work status reports at regular intervals within a specific time period, but some reports on a certain day fail to be uploaded successfully. According to the previous data records and patterns, the system will assume the potential values of these un-uploaded data points and construct one or more possible combinations of data points.

[0139] Each hypothesis point is generated through multiple rounds of iterations, with new generated points added in each iteration until there are data points in the hypothesis set that meet the conditions. This step aims to ensure that the generated missing data points can conform to the inherent laws and statistical characteristics of the overall data set through repeated verification and updating. Specifically, the similarity and consistency between the hypothesis points generated in each round and the existing data are evaluated through machine learning algorithms, and the hypothesis points are continuously optimized until the newly generated data points are fully integrated into the existing data set, forming a consistent and reasonable data chain.

[0140] Based on the probability distribution of the hypothesis points, the formula p(x) = frac{1}{sqrt{2pisigma^2}}e^{-frac{(x-mu)^2}{2sigma^2}} is used to calculate the probability of each hypothesis point, where mu represents the mean or expected value, which is usually calculated based on known data; sigma represents the standard deviation, which measures the degree of data dispersion and should ideally be close to zero but not too strict; and x refers to the value of the hypothesis data point to be evaluated. This formula describes a probability density function of a normal distribution, which is used to quantify the possibility of each hypothesis point being a true missing value. Since the operation of equipment in actual industrial scenarios often exhibits normal distribution characteristics, this formula can be used to effectively screen out the hypothesis points that are closest to the true value. The formula is designed to ensure that the generated data points are consistent with the overall trend of statistical data and can avoid the introduction of outliers to a certain extent, thereby improving the quality and reliability of data recovery.

[0141] The industrial equipment data intelligent management method based on big data algorithm of the present invention includes:

[0142] First, this method obtains the operating data of industrial equipment from multiple data sources, including key information such as sensor data, maintenance records, and historical operation logs. These data sources are diverse and the data volume is huge, providing a rich data foundation for subsequent processing. However, during the data collection process, data is often missing due to equipment failure, network problems, or human negligence, resulting in a decrease in the accuracy of the prediction model. To address this problem, this method introduces a data completion algorithm. By using advanced techniques such as interpolation and statistical inference, the gaps in the data can be effectively filled, thereby improving the accuracy and reliability of the prediction model.

[0143] Secondly, in the process of data transmission, network delay and packet loss may lead to the loss of real-time performance. The present invention ensures the stable transmission of data by taking effective optimization measures at the network layer and the transport layer. For example, a reliable transmission protocol (such as TCP) is used and a data retransmission mechanism is implemented to ensure that the data packet can reach the destination in a timely and complete manner. At the same time, the reliability and real-time performance of data transmission are further improved by combining caching technology and forward error correction code.

[0144] In the data preprocessing stage, this method solves the problem of inconsistent formats of multi-source heterogeneous data. By converting and standardizing the data format, it ensures that all data can be analyzed and processed under the same framework. This process not only reduces the complexity of data processing, but also improves the consistency and availability of data. Specifically, a general data conversion tool can be used to convert data in different formats into standardized structured data, such as CSV or JSON format.

[0145] In the model training phase, in order to deal with the imbalance of positive and negative samples, this method uses resampling technology. By oversampling minority class samples or undersampling majority class samples, the ratio of positive and negative samples is balanced, thereby preventing the model from being skewed. This technology can significantly improve the robustness and generalization ability of the model, making the prediction results more accurate and reliable.

[0146] Finally, in terms of large-scale data processing, this method utilizes distributed computing frameworks such as Hadoop or Spark. By distributing data and computing tasks to multiple nodes, the utilization efficiency of computing resources is effectively improved, solving the performance bottleneck problem caused by limited computing resources. This method not only speeds up data processing, but also allows flexible expansion of computing clusters to adapt to the growing size of data.

[0147] In summary, the intelligent management method of industrial equipment data based on big data algorithm of the present invention systematically solves key technical problems such as data missing, unstable transmission, inconsistent formats, sample imbalance and resource bottleneck through a variety of advanced technical means, and realizes an efficient and reliable industrial equipment data management platform.

[0148] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium.

[0149] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples, unless they are contradictory.

[0150] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0151] Any process or method description in the flow chart or otherwise described herein can be understood to represent a module, fragment or portion of a code including one or more executable instructions for implementing the steps of a specific logical function or process. And the scope of the preferred embodiment of the present application includes other implementations, in which the functions may not be performed in the order shown or discussed, including in a substantially simultaneous manner or in a reverse order according to the functions involved.

[0152] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, which can be embodied in any computer-readable medium for use by an instruction execution system, apparatus or device (such as a computer-based system, a system including a processor or other system that can fetch instructions from an instruction execution system, apparatus or device and execute instructions), or used in combination with these instruction execution systems, apparatuses or devices.

[0153] It should be understood that the various parts of the present application can be implemented with hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented with software or firmware stored in a memory and executed by a suitable instruction execution system. All or part of the steps of the above embodiment method can be completed by instructing the relevant hardware through a program, which can be stored in a computer-readable storage medium, and when the program is executed, it includes one of the steps of the method embodiment or a combination thereof.

[0154] In addition, each functional unit in each embodiment of the present application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the above-mentioned integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The storage medium can be a read-only memory, a disk or an optical disk, etc.

[0155] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An intelligent management method for industrial equipment data based on big data algorithm, characterized in that: The method comprises the following steps: 1) Obtaining the operating data of industrial equipment from multiple data sources, including sensor data, maintenance records, and historical operation logs; 2) Fill in the missing parts in the running data based on the data completion algorithm; 3) Format conversion and standardization of multi-source heterogeneous data; 4) Use resampling techniques to balance positive and negative samples during model training; 5) Use distributed computing frameworks for large-scale data processing.

2. The method according to claim 1, characterized in that Step 2) specifically includes the following steps: The Kalman filter algorithm is used to fill the missing parts of the sensor data; The missing parts of the maintenance records were filled using a linear interpolation method based on time series; Use similarity-based clustering algorithms to fill in the missing parts of historical operation logs; Compare and verify the filled data with the original data to ensure the accuracy of the filled data.

3. The method according to claim 2, characterized in that Step 2) further comprises the following steps: Based on the Kalman filter algorithm, the missing values ​​in the sensor data are predicted. The prediction formula is: X(k) = A*X(k1) + B*U(k1) Among them, X(k) represents the sensor data at the kth moment, A represents the state transfer matrix, X(k1) represents the sensor data at the k1th moment, B represents the control input matrix, and U(k1) represents the control input at the k1th moment; Conduct confidence assessment on the predicted value to ensure that it meets the requirements within the confidence interval; If the confidence level of the predicted value is lower than the preset threshold, the mean of historical data is used for filling; Verify the integrity and consistency of the padded data to ensure that the padded data can be used for model training.

4. The method according to claim 3, characterized in that Step 2) further comprises the following steps: Based on the historical trends and periodic characteristics of sensor data, an adaptive filter is constructed to fill in the data; Use the formulated conditional judgment statement to determine whether the filling value needs to be recalculated: if |Y(k)-P(k)|>T, then recalculate the filling value, where Y(k) represents the actual value, P(k) represents the predicted value, and T represents the threshold; Perform a second check on the filled data to ensure data consistency and reliability; Feedback the verification results to the system log for subsequent analysis and optimization.

5. The method according to claim 4, characterized in that Step 2) further comprises the following steps: Data filling based on an integrated approach of multiple filling algorithms; The validity of the filling algorithm is judged by using a formulaic conditional statement: if |E1-E2|<δ, the prediction result of the integrated method is adopted, where E1 represents the result of Kalman filtering, E2 represents the result of linear interpolation, and δ represents the maximum allowable error; Evaluate the final filling results to ensure that the filling effect meets the expected standards; The key steps and results of the imputation process are saved in log files for subsequent analysis.

6. The method according to claim 5, characterized in that Step 2) further comprises the following steps: Machine learning-based anomaly detection algorithms identify missing data patterns; Use a formulaic conditional judgment statement to determine whether data needs to be recollected: if Q>M, recollection is triggered, where Q represents the number of abnormal data and M represents the preset abnormal threshold; Supplement and fill in the re-collected data to ensure the completeness and accuracy of the data; The results of filling and collection are recorded in the database for subsequent query and management.

7. The method according to claim 6, characterized in that Step 2) further comprises the following steps: Predict missing values ​​in sensor data based on Bayesian inference methods; Use a formulaic conditional judgment statement to judge the reliability of the predicted value: if L>λ, then use the Bayesian prediction value, where L represents the probability distribution function value of the predicted value and λ represents the reliability threshold; Conduct statistical analysis on the filled data to ensure that the data distribution is within the normal range; The filling results and their reliability indicators are saved to the system log for subsequent audit and improvement.

8. The method according to claim 1, characterized in that Step 2) further comprises the following steps: Supplement missing time point data through time series interpolation methods; Use Kalman filter algorithm to smooth and predict missing data; Generate the probability distribution of missing data through deep learning models; After generating the possibility distribution, the data with the highest probability is selected as the filling result.

Images