Steel mill production data management method and system

Abstract

The invention provides a steel mill production data management method and system, and relates to the technical field of data management. The system comprises a data receiving module, a data preprocessing module, a data analysis module, a data storage module, a data transmission management module and a model training module. According to the method, possible problems and trends in the production process are predicted, so that a decision maker can make a more accurate decision based on a data analysis result, guarantee is provided for the safety and integrity of data, the requirements of steel mill production management and decision are met, the transmission requirements of steel mill production data in different scenes are met, and the production efficiency is improved. And the reliability of data transmission is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data management, and in particular to a steel plant production data management method and system. Background Art

[0002] In the modern steel production process, efficient management of production data is crucial to ensure product quality, improve production efficiency and reduce operating costs. Traditional data management methods often face multiple challenges, such as incomplete data collection, inefficient data preprocessing, inaccurate data classification, and poor data storage and transmission security. These problems limit the ability of steel mills in production management and decision optimization, which in turn affects the competitiveness of enterprises. Summary of the invention

[0003] In view of the above technical problems, the present invention proposes a steel plant production data management method and system, aiming to achieve efficient management and utilization of steel plant production data by adopting advanced data processing technology and algorithms.

[0004] A first aspect of the present invention provides a steel plant production data management method, comprising: Step S1, receiving steel plant production data collected by sensors deployed in multiple production links of the steel plant, and preprocessing the steel plant production data through a data preprocessing unit to obtain steel plant production preprocessing data; Step S2: Performing in-depth analysis on the steel plant production preprocessing data through a data analysis unit to generate data analysis results; Step S3: classify and encode the data analysis results, store the classified and encoded data analysis results in the target database, perform data backup and set a recovery strategy for the stored data analysis results; Step S4: determine the data transmission protocol based on the data transmission request and encrypt the target data, compress the encrypted target data and transmit it, dynamically adjust the data transmission rate according to the transmission monitoring data, monitor the transmission process through error monitoring technology and data retransmission mechanism, and complete the transmission of the target data.

[0005] Preferably, step S1 specifically includes: S11. Receive production data of steel mills in multiple production links, including ironmaking, steelmaking, continuous casting, initial rolling, finishing rolling, cooling and finished product processing; S12. Perform data integrity assessment on the steel plant production data, predict missing values ​​based on the random forest model, generate predicted values ​​of missing values, fill missing values ​​with predicted values, and obtain the steel plant production data after data supplementation processing; S13, performing data visualization analysis on the steel plant production data after data supplementation based on the box-and-whisker plot, performing anomaly detection on the steel plant production data through a preset deviation threshold, and generating anomaly detection results; S14, deleting abnormal values ​​in the steel plant production data according to the abnormality detection result, and obtaining the steel plant production data processed by data supplementation and abnormal value deletion; S15. Convert the data type and unify the data unit of the steel plant production data, generate the steel plant production cleaning data, and record the process and results of the data cleaning.

[0006] Preferably, step S2 specifically includes: S21. Perform statistical feature extraction and time series analysis on the steel plant production cleaning data to generate statistical features and time series features, including calculating multiple statistics, quantiles, skewness and kurtosis of the steel plant production cleaning data to generate discrete features and distribution features, extracting trend feature data of the steel plant production cleaning data through a linear regression algorithm, extracting period feature data of the steel plant production cleaning data through an autocorrelation function, and analyzing the fluctuation characteristics of the steel plant production cleaning data through a rolling standard deviation; S22. Perform feature importance analysis on statistical features and time series features based on the gradient boosting tree, optimize the feature importance analysis results through cross-validation, rank the statistical features and time series features by importance, and generate feature importance analysis results.

[0007] Preferably, step S3 specifically includes: S31, linearly pull up the time series features and standardize the statistical features to obtain input data for the data classification model; S32, processing the input data through the data classification model to generate feature data classification results; S33, clustering statistical features and time series features according to the feature data classification results to obtain multiple feature clusters, performing one-hot encoding on the multiple feature clusters and writing them into the target database; S34, performing differential backup on multiple feature clusters in the target database, ranking the multiple feature clusters in the target database by importance based on the feature importance analysis result, and generating an importance ranking result; S35. According to the importance ranking results, the grayscale priorities of multiple feature clusters are set and written into the recovery execution script to improve the setting of the recovery strategy.

[0008] Preferably, step S4 specifically includes: S41, receiving a data transmission request, parsing the data transmission request, and generating basic information of the data request, including data type, data size, transmission security level, and transmission speed; S42, selecting a target data transmission protocol based on the data request basic information, and filtering out target data from a target database; S43, encrypting the target data based on the data encryption algorithm configured for the target data transmission protocol to generate target data ciphertext, and compressing the target data ciphertext using a data compression algorithm to generate target compressed data; S44, transmitting the target compressed data based on the target data transmission protocol, monitoring the data transmission process through a transmission monitoring module, and obtaining transmission monitoring data; S45, analyzing the transmission monitoring data, extracting multiple parameter characteristic data sequences, predicting multiple transmission parameters of the data transmission process based on the multiple parameter characteristic data sequences, generating parameter prediction results, and formulating a data transmission adjustment strategy based on the data request basic information and the parameter prediction results; S46, adjusting the configuration parameters of the transmission protocol and the sending rate of the regulated data packets through the data transmission adjustment strategy, and reallocating the network traffic through the load balancing technology; S47. Determine the data transmission monitoring indicator threshold based on the data request basic information, analyze the transmission monitoring data, use the threshold to identify transmission anomalies in the data transmission process, repair the transmission anomalies through a preset anomaly repair mechanism, and complete the transmission of the target data after the target data successfully reaches the receiving end.

[0009] Preferably, the data classification model in step S31 further includes: The data classification model is trained through the training data set, which includes time series feature data, statistical feature data and label data. The data classification model includes CNN layer, LSTM layer, fully connected layer and output layer. The time series feature data and statistical feature data in the training data set are normalized, the label data is converted through one-hot encoding, the processed training data set is input into the pre-built data classification model, the time series feature data is processed through the CNN layer, the time series feature data processed by the CNN layer is merged with the statistical feature data and input into the LSTM layer for processing, the data is classified through the softmax function, the model training process is detected through the callback function, and a trained data classification model is obtained.

[0010] A second aspect of the present invention provides a steel plant production data management system, which is used to implement the above-mentioned steel plant production data management method, comprising: A data receiving module is used to receive the steel plant production data collected by sensors deployed in multiple production links of the steel plant; A data preprocessing module is used to preprocess the steel plant production data through a data preprocessing unit to obtain the steel plant production preprocessing data; Data analysis module, used to conduct in-depth analysis of steel plant production pre-processing data through data analysis unit and generate data analysis results A data storage module is used to classify and encode data analysis results, store the classified and encoded data analysis results through a target database, perform data backup and set a recovery strategy for the stored data analysis results, wherein the data storage module classifies the data analysis results through a data classification model; The data transmission management module is used to determine the data transmission protocol based on the data transmission request and encrypt the target data, compress the encrypted target data and transmit it, dynamically adjust the data transmission rate according to the transmission monitoring data, and monitor the transmission process through error detection technology and data retransmission mechanism.

[0011] Preferably, it also includes: The model training module is used to train the data classification model using the training data set.

[0012] The present invention has the following beneficial effects: 1. The present invention ensures the accuracy and reliability of the data by performing data cleaning, data standardization, and data feature extraction on the collected steel plant production data, thereby providing a high-quality data basis for subsequent data analysis and decision support; the statistical characteristics and time series characteristics of the steel plant production preprocessing data are extracted through a data analysis unit, and feature importance analysis is performed on the statistical characteristics and time series characteristics to predict possible problems and trends in the production process, so that decision makers can make more accurate decisions based on the data analysis results.

[0013] 2. The present invention classifies the data by using appropriate classification standards based on the characteristics of the data analysis results, encodes the classification results, and converts them into a format that is easy for database storage and retrieval, so that a large amount of data analysis results are more orderly, easy to query and use, and support subsequent data-driven decision-making. It selects a suitable database system to store the classified and encoded data analysis results, and formulates a detailed data recovery strategy to quickly restore the data when it is lost or damaged, so as to ensure the security and integrity of the data and support the production management and decision-making needs of the steel plant.

[0014] 3. The present invention selects a suitable data transmission protocol by analyzing the characteristics of the data transmission request to protect the data transmission process and meet the needs of the data requester, uses an encryption algorithm to encrypt the target data to ensure the security of the data during transmission, monitors network bandwidth, delay, packet loss rate and other indicators in real time, automatically adjusts the data packet size or transmission rate, and uses error monitoring technology to implement an automatic error detection and retransmission mechanism to ensure rapid response and repair when data transmission errors are detected, thereby meeting the transmission needs of steel plant production data in different scenarios and improving the reliability of data transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 The figure is a flow chart of a method for managing production data of a steel plant in an embodiment of the present invention.

[0016] Figure 2 It is a structural schematic diagram of a steel plant production data management system in an embodiment of the present invention. DETAILED DESCRIPTION

[0017] In order to make those skilled in the art better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0018] See also Figure 1 , is a flow chart of a steel plant production data management method in an embodiment of the present invention. A steel plant production data management method provided by an embodiment of the present invention comprises the following steps: Step S1, receiving steel plant production data collected by sensors deployed in multiple production links of the steel plant, and preprocessing the steel plant production data through a data preprocessing unit to obtain steel plant production preprocessing data; Step S2: Performing in-depth analysis on the steel plant production preprocessing data through a data analysis unit to generate data analysis results; Step S3: classify and encode the data analysis results, store the classified and encoded data analysis results in the target database, perform data backup and set a recovery strategy for the stored data analysis results; Step S4: determine the data transmission protocol based on the data transmission request and encrypt the target data, compress the encrypted target data and transmit it, dynamically adjust the data transmission rate according to the transmission monitoring data, monitor the transmission process through error monitoring technology and data retransmission mechanism, and complete the transmission of the target data.

[0019] In this embodiment, the steel plant production data can be obtained by deploying various types of sensors in key production links in the steel plant, such as temperature sensors, pressure sensors, flow meters and quality detectors, etc., to collect key parameters in the production process in real time. By preprocessing the collected steel plant production data, such as data cleaning, data standardization, and data feature extraction, the accuracy and reliability of the data are guaranteed, providing a high-quality data foundation for subsequent data analysis and decision support.

[0020] In this embodiment, the data analysis unit performs statistical feature analysis and time series feature analysis on the steel plant production preprocessing data, extracts the statistical features and time series features of the steel plant production preprocessing data, and performs feature importance analysis on the statistical features and time series features. This can predict possible problems and trends that may arise in the production process, such as decreased production efficiency due to equipment aging, and the impact of changes in raw material quality on product quality. Feature importance analysis can identify the factors that have the greatest impact on the production process, focus on key features, and enable decision makers to make more accurate decisions based on data analysis results.

[0021] In this embodiment, the data is classified by using appropriate classification standards based on the characteristics of the data analysis results, for example, according to the production process, time, product type, etc. Then, the classification results are encoded and converted into a format that is easy to store and retrieve in the database, so that a large amount of data analysis results are more orderly, easy to query and use, and support subsequent data-driven decision-making. A suitable database system (such as relational database MySQL, PostgreSQL or non-relational database such as MongoDB) is selected to store the classified and encoded data analysis results, and a detailed data recovery strategy is formulated to quickly recover the data when it is lost or damaged, so as to ensure the security and integrity of the data and support the needs of production management and decision-making in the steel plant.

[0022] In this embodiment, by analyzing the characteristics of the data transmission request (such as data size, security requirements, real-time requirements, etc.), a suitable data transmission protocol is selected to protect the data transmission process and meet the needs of the data requester. The target data is encrypted using an encryption algorithm to ensure the security of the data during transmission. The network bandwidth, delay, packet loss rate and other indicators are monitored in real time, the data packet size or transmission rate is automatically adjusted, and the error monitoring technology is used to implement an automatic error detection and retransmission mechanism to ensure rapid response and repair when a data transmission error is detected, thereby meeting the transmission requirements of the steel plant's production data in different scenarios and improving the reliability of data transmission.

[0023] In an optional implementation process, step S1 specifically includes the following contents: S11. Receive production data of steel mills in multiple production links, including ironmaking, steelmaking, continuous casting, initial rolling, finishing rolling, cooling and finished product processing; S12. Perform data integrity assessment on the steel plant production data, predict missing values ​​based on the random forest model, generate predicted values ​​of missing values, fill missing values ​​with predicted values, and obtain the steel plant production data after data supplementation processing; S13, performing data visualization analysis on the steel plant production data after data supplementation based on the box-and-whisker plot, performing anomaly detection on the steel plant production data through a preset deviation threshold, and generating anomaly detection results; S14, deleting abnormal values ​​in the steel plant production data according to the abnormality detection result, and obtaining the steel plant production data processed by data supplementation and abnormal value deletion; S15. Convert the data type and unify the data unit of the steel plant production data, generate the steel plant production cleaning data, and record the process and results of the data cleaning.

[0024] In this embodiment, for the steel plant production data collected from different production links, taking into account reasons such as sensor failure and operational errors, these data are prone to missing values ​​and outliers, affecting the integrity and reliability of the data. By performing integrity assessment on these data, the problem of missing values ​​in the data is discovered. Specifically, a method based on a random forest model is used to predict the missing values, and the patterns and correlations of the existing data are used to generate predicted values ​​for the missing values, which are then filled with these predicted values. For example, if the temperature data at a certain point in time during the continuous casting process is missing, the missing temperature value can be predicted based on the temperature data at adjacent time points and other relevant parameters (such as flow rate, cooling water temperature, etc.); the characteristics of the data distribution can be intuitively displayed through box plots, and abnormal conditions in the production data can be detected by setting reasonable deviation thresholds. For example, a sudden deviation of the temperature from the normal range during the continuous casting process may be a signal of equipment failure; outliers in the production data are deleted to clean the data and ensure the accuracy of subsequent analysis and decision-making; the data type conversion and unit unification of the steel plant's production data are processed to further improve the quality and consistency of the data. For example, the temperature data unit is unified into degrees Celsius, and the timestamp data is uniformly converted into date and time types. At the same time, the process and results of data processing are recorded to provide experience and reference for data governance and analysis.

[0025] In an optional implementation process, step S2 specifically includes the following contents: S21. Perform statistical feature extraction and time series analysis on the steel plant production cleaning data to generate statistical features and time series features, including calculating multiple statistics, quantiles, skewness and kurtosis of the steel plant production cleaning data to generate discrete features and distribution features, extracting trend feature data of the steel plant production cleaning data through a linear regression algorithm, extracting period feature data of the steel plant production cleaning data through an autocorrelation function, and analyzing the fluctuation characteristics of the steel plant production cleaning data through a rolling standard deviation; S22. Perform feature importance analysis on statistical features and time series features based on the gradient boosting tree, optimize the feature importance analysis results through cross-validation, rank the statistical features and time series features by importance, and generate feature importance analysis results.

[0026] In this embodiment, by calculating multiple statistics, such as mean, median, standard deviation, and more advanced statistical indicators, such as quantile, skewness and kurtosis, the statistical characteristics of the steel plant production cleaning data are extracted to understand the extreme values ​​of the data, the fluctuation characteristics of the production volume and the frequency of occurrence of extreme values; the trend characteristics of the production data are extracted through a linear regression algorithm to facilitate the identification of the changing trends of indicators such as production volume and energy consumption over time; the periodic characteristics are analyzed through the autocorrelation function to discover the periodic change patterns in production activities, such as seasonal demand fluctuations; the rolling standard deviation is used to reveal the volatility of the data within a specific time window, which provides a basis for the stability analysis of the production process.

[0027] In this embodiment, by using the gradient boosting tree model to perform feature importance analysis and combining the cross-validation method to optimize the analysis results, the features can be effectively ranked by importance to reveal some features that have a greater impact on the steel plant production process, providing a scientific basis for further data analysis and production optimization. For example, if it is found that there is a strong correlation between the daily volatility of production volume and energy consumption, then reducing volatility by optimizing the production plan may lead to a reduction in energy consumption. By conducting in-depth analysis of some important features, some potential production problems and potential improvement opportunities can be discovered, thereby continuously promoting the optimization and innovation of the production process.

[0028] In an optional implementation process, step S3 specifically includes the following contents: S31, linearly pull up the time series features and standardize the statistical features to obtain input data for the data classification model; S32, processing the input data through the data classification model to generate feature data classification results; S33, clustering statistical features and time series features according to the feature data classification results to obtain multiple feature clusters, performing one-hot encoding on the multiple feature clusters and writing them into the target database; S34, performing differential backup on multiple feature clusters in the target database, ranking the multiple feature clusters in the target database by importance based on the feature importance analysis result, and generating an importance ranking result; S35. According to the importance ranking results, the grayscale priorities of multiple feature clusters are set and written into the recovery execution script to improve the setting of the recovery strategy.

[0029] In this embodiment, the time series features are linearly pulled up, for example, by applying linear interpolation to the time series data to fill in the missing time points, so that the data is more continuous and complete. The statistical features are standardized, such as converting the data into a distribution with a mean of 0 and a standard deviation of 1. Such processing enables statistical data of different scales and dimensions to be compared and used at the same scale, thereby obtaining data suitable for input into the data classification model. Using data classification models, such as support vector machines (SVM) or deep learning models, these pre-processed input data are processed to effectively classify the data into different categories and generate feature data classification results. For example, data classification based on production efficiency and energy consumption can help identify efficient and inefficient production modes.

[0030] In this embodiment, a data clustering algorithm, such as K-means, is used to further cluster the statistical features and time series features to obtain multiple feature clusters to reveal potential patterns and relationships in the production data. Exemplarily, the production patterns within a specific time period discovered through cluster analysis are used to indicate a better production scheduling strategy. These feature clusters are one-hot encoded and written into the target database to facilitate subsequent data retrieval and analysis, and to lay the foundation for machine learning applications of the data.

[0031] In this embodiment, differential backup is performed on the data, specifically, the data that has changed since the last backup is backed up, which saves storage space while ensuring the integrity of the data. Based on the feature importance analysis results, the feature clusters in the target database are sorted by importance, which can help determine which data has the greatest impact on the production process and which data should be restored first. This sorting result guides the setting of the recovery strategy, ensuring that the most critical data can be restored first when data loss or system failure occurs, and the grayscale priority of multiple feature clusters set according to the importance sorting results is recorded through the recovery execution script, ensuring that during the recovery process, the order of data recovery is carried out according to the importance of its impact on production. This meticulous recovery strategy not only improves the efficiency of data recovery, but also minimizes the impact of production interruptions, ensuring the continuity and stability of production.

[0032] In an optional implementation process, step S4 specifically includes the following contents: S41, receiving a data transmission request, parsing the data transmission request, and generating basic information of the data request, including data type, data size, transmission security level, and transmission speed; S42, selecting a target data transmission protocol based on the data request basic information, and filtering out target data from a target database; S43, encrypting the target data based on the data encryption algorithm configured for the target data transmission protocol to generate target data ciphertext, and compressing the target data ciphertext using a data compression algorithm to generate target compressed data; S44, transmitting the target compressed data based on the target data transmission protocol, monitoring the data transmission process through a transmission monitoring module, and obtaining transmission monitoring data; S45, analyzing the transmission monitoring data, extracting multiple parameter characteristic data sequences, predicting multiple transmission parameters of the data transmission process based on the multiple parameter characteristic data sequences, generating parameter prediction results, and formulating a data transmission adjustment strategy based on the data request basic information and the parameter prediction results; S46, adjusting the configuration parameters of the transmission protocol and the sending rate of the regulated data packets through the data transmission adjustment strategy, and reallocating the network traffic through the load balancing technology; S47. Determine the data transmission monitoring indicator threshold based on the data request basic information, analyze the transmission monitoring data, use the threshold to identify transmission anomalies in the data transmission process, repair the transmission anomalies through a preset anomaly repair mechanism, and complete the transmission of the target data after the target data successfully reaches the receiving end.

[0033] In this embodiment, for the received data transmission request, basic information of the data request is extracted, including but not limited to data type, data size, transmission security level, transmission speed and other information, which is used to indicate the needs of the data requester, so as to facilitate the formulation of corresponding data transmission strategies, and determine the appropriate data transmission protocol according to the basic information of the data request. For example, for data with high security requirements, HTTPS or SFTP protocols that support encryption are selected, while QUIC protocol may be selected for data with special requirements for transmission speed; the target data is encrypted by a preconfigured data encryption algorithm, such as AES or RSA algorithm, and the encrypted data is compressed by the GZIP data compression algorithm to improve transmission efficiency.

[0034] In this embodiment, during the data transmission process, a dedicated transmission monitoring module is used to monitor the state of data transmission in real time, collect transmission monitoring data, such as transmission rate, delay, packet loss rate, etc., extract key parameter feature data sequences based on these monitoring data, and use these features to predict key parameters in the transmission process, so as to formulate a more accurate data transmission adjustment strategy. For example, if it is predicted that network congestion may cause a decrease in transmission speed, the system can adjust the sending rate in advance or redistribute network traffic through load balancing technology to avoid potential transmission delays.

[0035] In this embodiment, appropriate data transmission monitoring index thresholds are determined based on the basic information of the data request and the historical transmission monitoring data, and these thresholds are used to judge and identify possible anomalies in the transmission process. Once an anomaly is detected, the system will automatically start the preset anomaly repair mechanism, such as retransmitting the lost data packet, to ensure the reliability of data transmission. The security and efficiency of data transmission are guaranteed, the intelligence level and adaptive ability of the data transmission process are improved, and the real-time monitoring, analysis and decision support system of steel plant production data are used as an important guarantee for data integrity, improving production efficiency and product quality.

[0036] In an optional implementation process, for the data classification model in step S31, it also includes: The data classification model is trained through the training data set, which includes time series feature data, statistical feature data and label data. The data classification model includes CNN layer, LSTM layer, fully connected layer and output layer. The time series feature data and statistical feature data in the training data set are normalized, the label data is converted through one-hot encoding, the processed training data set is input into the pre-built data classification model, the time series feature data is processed through the CNN layer, the time series feature data processed by the CNN layer is merged with the statistical feature data and input into the LSTM layer for processing, the data is classified through the softmax function, the model training process is detected through the callback function, and a trained data classification model is obtained.

[0037] In this embodiment, the training data set can be constructed based on the historical production data of the steel plant, and the time series data and statistical data in the historical production data can be extracted, such as temperature, hardness, output and other data in different time periods, output fluctuation data in different stages, etc. The extracted time series data and statistical data are classified, and multiple type labels are added to obtain corresponding label data.

[0038] In this embodiment, the time series feature data and statistical feature data collected from each production link of the steel plant in the training data set are first normalized. For example, the temperature is converted from degrees Celsius to a value between 0 and 1, and the proportion of chemical components is also normalized to the same order of magnitude. This can eliminate the impact of different dimensions and ensure that the model can learn the information of each feature equally. For label data, unique hot encoding is used for conversion. For example, if the final product quality is divided into three categories: high, medium, and low, then the label data will be converted from a single number to a three-element vector, such as [1,0,0] for high quality, [0,1,0] for medium quality, and [0,0,1] for low quality, so that the model can better perform multi-classification learning.

[0039] In this embodiment, for the training process of the model, after the processed training data set is input into the pre-built data classification model, the CNN layer first processes the time series feature data to extract local features in the time series, such as identifying abnormal fluctuations in the heating process by learning the pattern of temperature changes; then the time series feature data processed by the CNN layer is merged with the normalized statistical feature data, and input together into the LSTM layer. The LSTM layer can learn the long-term dependencies in the feature data, such as predicting the hardness grade of the final product by analyzing the changes in temperature and chemical composition ratio in the continuous production process; the merged feature data is classified by the softmax function; during the model training process, a callback function is used for performance monitoring, such as using early stopping to prevent overfitting, to ensure that the model also performs well on unseen data, thereby training a data classification model for classifying data.

[0040] See also Figure 2 , is a structural diagram of a steel plant production data management system in an embodiment of the present invention. Based on a steel plant production data management method provided in an embodiment of the present invention, an embodiment of the present invention further provides a steel plant production data management system, specifically comprising: A data receiving module is used to receive the steel plant production data collected by sensors deployed in multiple production links of the steel plant; A data preprocessing module is used to preprocess the steel plant production data through a data preprocessing unit to obtain the steel plant production preprocessing data; Data analysis module, used to conduct in-depth analysis of steel plant production pre-processing data through data analysis unit and generate data analysis results A data storage module is used to classify and encode data analysis results, store the classified and encoded data analysis results through a target database, perform data backup and set a recovery strategy for the stored data analysis results, wherein the data storage module classifies the data analysis results through a data classification model; The data transmission management module is used to determine the data transmission protocol based on the data transmission request and encrypt the target data, compress the encrypted target data and transmit it, dynamically adjust the data transmission rate according to the transmission monitoring data, and monitor the transmission process through error detection technology and data retransmission mechanism.

[0041] The model training module is used to train the data classification model using the training data set.

[0042] Among them, the data classification model is trained through the training data set, specifically including: The data classification model is trained through the training data set, which includes time series feature data, statistical feature data and label data. The data classification model includes CNN layer, LSTM layer, fully connected layer and output layer. The time series feature data and statistical feature data in the training data set are normalized, the label data is converted through one-hot encoding, the processed training data set is input into the pre-built data classification model, the time series feature data is processed through the CNN layer, the time series feature data processed by the CNN layer is merged with the statistical feature data and input into the LSTM layer for processing, the data is classified through the softmax function, the model training process is detected through the callback function, and a trained data classification model is obtained.

[0043] The above is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be obvious to those skilled in the art. Parts not described in detail in this specification belong to the prior art known to those skilled in the art.

Claims

1. A steel plant production data management method, characterized in that: include: Step S1, receiving steel plant production data collected by sensors deployed in multiple production links of the steel plant, and preprocessing the steel plant production data through a data preprocessing unit to obtain steel plant production preprocessing data; Step S2: Performing in-depth analysis on the steel plant production preprocessing data through a data analysis unit to generate data analysis results; Step S3: classify and encode the data analysis results, store the classified and encoded data analysis results in a target database, perform data backup and set a recovery strategy for the stored data analysis results; Step S4: determine the data transmission protocol based on the data transmission request and encrypt the target data, compress the encrypted target data and transmit it, dynamically adjust the data transmission rate according to the transmission monitoring data, monitor the transmission process through error monitoring technology and data retransmission mechanism, and complete the transmission of the target data.

2. A steel plant production data management method according to claim 1, characterized in that: Step S1 specifically includes: S11. Receive production data of steel mills in multiple production links, including ironmaking, steelmaking, continuous casting, initial rolling, finishing rolling, cooling and finished product processing; S12. Perform data integrity assessment on the steel plant production data, predict missing values ​​based on the random forest model, generate predicted values ​​of the missing values, fill the missing values ​​with the predicted values, and obtain the steel plant production data after data supplementation processing; S13, performing data visualization analysis on the steel plant production data after data supplementation based on the box-and-whisker plot, performing anomaly detection on the steel plant production data through a preset deviation threshold, and generating anomaly detection results; S14, deleting abnormal values ​​in the steel plant production data according to the abnormality detection result, and obtaining the steel plant production data processed by data supplementation and abnormal value deletion; S15. Convert the data type and unify the data unit of the steel plant production data, generate the steel plant production cleaning data, and record the process and results of the data cleaning.

3. A steel plant production data management method according to claim 2, characterized in that: Step S2 specifically includes: S21. Perform statistical feature extraction and time series analysis on the steel plant production cleaning data to generate statistical features and time series features, including calculating multiple statistics, quantiles, skewness and kurtosis of the steel plant production cleaning data to generate discrete features and distribution features, extracting trend feature data of the steel plant production cleaning data through a linear regression algorithm, extracting period feature data of the steel plant production cleaning data through an autocorrelation function, and analyzing the fluctuation characteristics of the steel plant production cleaning data through a rolling standard deviation; S22. Perform feature importance analysis on statistical features and time series features based on the gradient boosting tree, optimize the feature importance analysis results through cross-validation, rank the statistical features and time series features by importance, and generate feature importance analysis results.

4. A steel plant production data management method according to claim 3, characterized in that: Step S3 specifically includes: S31, linearly pull up the time series features and standardize the statistical features to obtain input data for the data classification model; S32, processing the input data through the data classification model to generate feature data classification results; S33, clustering statistical features and time series features according to the feature data classification results to obtain multiple feature clusters, performing one-hot encoding on the multiple feature clusters and writing them into the target database; S34, performing differential backup on multiple feature clusters in the target database, ranking the multiple feature clusters in the target database by importance based on the feature importance analysis result, and generating an importance ranking result; S35. According to the importance ranking results, the grayscale priorities of multiple feature clusters are set and written into the recovery execution script to improve the setting of the recovery strategy.

5. A steel plant production data management method according to claim 4, characterized in that: Step S4 specifically includes: S41, receiving a data transmission request, parsing the data transmission request, and generating basic information of the data request, including data type, data size, transmission security level, and transmission speed; S42, selecting a target data transmission protocol based on the data request basic information, and filtering out target data from a target database; S43, encrypting the target data based on the data encryption algorithm configured for the target data transmission protocol to generate target data ciphertext, and compressing the target data ciphertext using a data compression algorithm to generate target compressed data; S44, transmitting the target compressed data based on the target data transmission protocol, monitoring the data transmission process through a transmission monitoring module, and obtaining transmission monitoring data; S45, analyzing the transmission monitoring data, extracting multiple parameter characteristic data sequences, predicting multiple transmission parameters of the data transmission process based on the multiple parameter characteristic data sequences, generating parameter prediction results, and formulating a data transmission adjustment strategy based on the data request basic information and the parameter prediction results; S46, adjusting the configuration parameters of the transmission protocol and the sending rate of the regulated data packets through the data transmission adjustment strategy, and reallocating the network traffic through the load balancing technology; S47. Determine the data transmission monitoring indicator threshold based on the data request basic information, analyze the transmission monitoring data, use the threshold to identify transmission anomalies in the data transmission process, repair the transmission anomalies through a preset anomaly repair mechanism, and complete the transmission of the target data after the target data successfully reaches the receiving end.

6. A steel plant production data management method according to claim 4, characterized in that: For the data classification model in step S31, it also includes: The data classification model is trained through the training data set, which includes time series feature data, statistical feature data and label data. The data classification model includes CNN layer, LSTM layer, fully connected layer and output layer. The time series feature data and statistical feature data in the training data set are normalized, the label data is converted through one-hot encoding, the processed training data set is input into the pre-built data classification model, the time series feature data is processed through the CNN layer, the time series feature data processed by the CNN layer is merged with the statistical feature data and input into the LSTM layer for processing, the data is classified through the softmax function, the model training process is detected through the callback function, and a trained data classification model is obtained.

7. A steel plant production data management system, characterized in that: The system is used to implement a steel plant production data management method as described in any one of claims 1 to 6, comprising: A data receiving module is used to receive the steel plant production data collected by sensors deployed in multiple production links of the steel plant; A data preprocessing module is used to preprocess the steel plant production data through a data preprocessing unit to obtain the steel plant production preprocessing data; Data analysis module, used to conduct in-depth analysis of steel plant production pre-processing data through data analysis unit and generate data analysis results A data storage module is used to classify and encode data analysis results, store the classified and encoded data analysis results through a target database, perform data backup and set a recovery strategy for the stored data analysis results, wherein the data storage module classifies the data analysis results through a data classification model; The data transmission management module is used to determine the data transmission protocol based on the data transmission request and encrypt the target data, compress the encrypted target data and transmit it, dynamically adjust the data transmission rate according to the transmission monitoring data, and monitor the transmission process through error detection technology and data retransmission mechanism.

8. A steel plant production data management system according to claim 7, characterized in that: Also includes: The model training module is used to train the data classification model using the training data set.

Images