Machine room information data management method and system

Abstract

The invention relates to the technical field of data management, in particular to a machine room information data management method and system. The method comprises a database building step, a data acquisition step, a data preprocessing step, a missing value detection step, an identification step, an associated data extraction step, an association judgment step, a sorting step, a missing derivation step, a distance measurement step, a reliability evaluation step, a data screening step and a data storage step. The system comprises a database building module, a data acquisition module, a data preprocessing module, a missing value detection module, an identification module, an associated data extraction module, an association judgment module, a sorting module, a missing derivation module, a distance measurement module, a reliability evaluation module, a data screening module and a data storage module. The method has the effect of reducing the influence on the data integrity.

Description

Technical Field

[0001] The present application relates to the technical field of data management, and in particular to a method and system for managing computer room information data. Background Art

[0002] At present, with the continuous improvement of informatization, the computer room, as an important storage and processing place for data, plays a vital role in the decision-making and operation of the enterprise in the management of information data.

[0003] In the existing technology, in the actual operation process, the computer room information data often has missing problems, which not only affects the integrity of the data, but also may have an adverse impact on the operation decision-making and business continuity of the computer room. There are many reasons for missing data, such as sensor failure, data collection errors, data synchronization delays, etc. Therefore, how to effectively complete the missing parts in the computer room information data has become a technical problem that needs to be solved in the current computer room management field.

[0004] Traditional data completion methods mostly rely on simple interpolation algorithms or statistical models. These methods work well when processing linear or simple nonlinear data. However, when faced with complex, multidimensional and dynamically changing data sets such as computer room information data, it is often difficult to achieve ideal completion effects, which seriously affects the integrity of the data. Summary of the invention

[0005] In order to reduce the impact on data integrity, the present application provides a computer room information data management method and system.

[0006] In the first aspect, the present application provides a computer room information data management method, which adopts the following technical solution: A computer room information data management method comprises the following steps: Build a database: store the historical information data of the computer room and available real-time information data; Data collection: collect real-time information data of the computer room and store it as real-time information data to be processed; Data preprocessing: preprocessing the collected real-time information data to be processed to form first data; Missing value detection: using statistical methods or machine learning algorithms to detect missing values ​​in the first data; Identification: determining the type and pattern of missing values ​​in the first data based on the distribution characteristics of the historical data and the distribution characteristics of the missing values; Related data extraction: extracting other data associated with the first data to form the second data; Relevance judgment: judging whether the relevance of the second data is greater than a preset first threshold, and if so, executing a sorting step; Sorting: screening the second data and sorting them according to the relevance to form the third data; Missing inference: Use third-party data to train a machine learning model, apply the trained model to the missing data, infer the missing values, and store them as fill-in data; Distance measurement: Use distance measurement algorithm to calculate the similarity or distance between the supplementary data and the associated data and historical data; Reliability evaluation: Evaluate the reliability of the replacement data based on the similarity or distance and context information; Data screening: Based on the reliability evaluation results, filter the replacement data whose reliability is greater than the set reliability threshold; Data storage: The first data is supplemented with the filtered replacement data and stored in the database to form usable real-time information data.

[0007] By adopting the above technical solution, through data collection, preprocessing, missing value detection, identification and subsequent missing value deduction steps, the solution can effectively deal with the missing value problem in the computer room information data; using machine learning models for missing value deduction, missing values ​​can be inferred based on the correlation and historical distribution characteristics of the data, thereby improving the integrity and accuracy of the data.

[0008] By introducing the steps of associated data extraction and associated judgment, and extracting other data associated with the data to be processed and evaluating their degree of association, the intrinsic connections between the data can be more effectively utilized to deduce missing values. This method not only improves the efficiency of data processing, but also enhances the rationality of the derivation results.

[0009] By calculating the similarity or distance between the replacement data and the associated data and historical data through a distance measurement algorithm and performing reliability assessment in combination with contextual information, this solution can screen out replacement data with high reliability. This step ensures that the data finally stored in the database has high accuracy and credibility.

[0010] The solution makes full use of machine learning algorithms and statistical methods to process data, realizing the automation and intelligence of data processing, which can not only reduce manual intervention and improve processing efficiency, but also reduce the risk of human error to a certain extent.

[0011] By supplementing the first data with the screened replacement data and storing it in the database to form usable real-time information data, this solution provides effective support for the information data management of the computer room, which not only helps to improve the operating efficiency and safety of the computer room, but also provides a more accurate and comprehensive data foundation for subsequent data analysis, monitoring and early warning applications.

[0012] Through the above method, the data browsed by the user is completed, and the completed data is made closer to the real data, thereby ensuring the integrity and accuracy of the data, so that the user can obtain complete and accurate information.

[0013] Optionally, after the data storage step, an association adjustment step is further provided; Association adjustment: According to the feedback result of the reliability assessment, the association of the second data is adjusted. If the reliability of the replacement data is higher than the preset second threshold, the association of the corresponding second data is increased, and an association label is established for the adjusted associated data to identify its association and reliability; otherwise, the association is reduced.

[0014] By adopting the above technical solution, the association adjustment step allows the system to dynamically adjust the association of the second data according to the reliability of the replacement data, which means that the system can optimize its internal data association model in real time based on data quality. When the replacement data is proven to be highly reliable, the corresponding associated data will be given a higher association, which helps to more accurately utilize these associations when processing data in the future.

[0015] By establishing associated labels for the adjusted associated data, the system can make more effective use of the data. These labels not only identify the degree of association of the data, but also provide additional information about the reliability of the data; this helps the system prioritize data that is both highly associated and highly reliable when performing data analysis, prediction or decision-making.

[0016] The associated adjustment step enhances the system's adaptive ability, which enables the system to dynamically adjust its processing strategy based on the actual quality of the data, rather than relying on fixed, preset rules. This adaptive ability is essential for handling the complex and ever-changing computer room information data environment.

[0017] By continually monitoring and adjusting the relevance and reliability of the data, the system can form a feedback loop that continuously improves the quality of its data; as the amount of data processed by the system increases, this ability to continuously improve will become increasingly important because it helps the system maintain its accuracy and efficiency.

[0018] For users who rely on computer room information data to make decisions, the associated adjustment steps can bring a better user experience. Users can get more accurate and reliable data support, which helps them make more informed decisions.

[0019] Optionally, in the association determination step, it further includes: if the association degree of the second data is greater than a preset first threshold, executing a pending step; To be determined: the second data whose correlation degree is less than or equal to the first threshold is stored as the fourth data; Analysis: Analyze the fourth data, establish preliminary labels, and calculate the similarity between the preliminary labels and the associated labels, which is recorded as label similarity; Similarity judgment: judge whether the label similarity is greater than or equal to the third threshold, if so, perform the sorting intervention step; Sorting intervention: Intervene in the sorting step, establish a temporary connection between the fourth data and the associated tag, insert it into the third data, and attach a temporary authentication tag.

[0020] By adopting the above technical solution, through the pending steps, the system can store the second data whose correlation does not meet the preset first threshold separately as the fourth data. This subdivision processing helps the system to more accurately identify and process data of different qualities, avoiding the information loss or misjudgment that may be caused by a one-size-fits-all processing method.

[0021] Analyzing the fourth data and establishing preliminary labels is a reflection of the system's attempt to optimize the data processing process through label management; the establishment of preliminary labels helps the system to quickly identify certain characteristics or attributes of the data, providing convenience for subsequent data processing; at the same time, calculating the similarity between the preliminary labels and the associated labels provides a basis for further screening and processing of the data.

[0022] The similarity judgment step allows the system to flexibly decide whether to sort and intervene in the fourth data based on label similarity; this flexibility enables the system to adjust its processing strategy according to actual conditions, so as to better adapt to complex and changing data environments.

[0023] The sorting intervention step achieves data integration by establishing a temporary connection between the fourth data and the associated label and inserting it into the third data; at the same time, affixing a temporary authentication label helps the system track and monitor this part of the data so that further authentication or adjustment can be performed according to actual conditions in subsequent processing.

[0024] Overall, these new steps have improved the efficiency and quality of data processing by the system through data segmentation, label management, flexible judgment and data integration. They enable the system to more accurately identify and handle potential problems in the data, thereby providing users with more reliable and accurate data support for computer room information.

[0025] Optionally, after the sorting intervention step, a reliability verification step and a connection determination step are further provided; Reliability verification: determining whether the reliability of the third data with the temporary authentication tag is greater than a set fourth threshold, and if so, executing a contact determination step; Contact confirmation: remove the temporary authentication tag of the third data and establish a permanent contact with the associated tag.

[0026] By adopting the above technical solution, the reliability verification step provides a second level of quality assurance for the third data with a temporary authentication tag. By determining whether the reliability of these data is greater than the set fourth threshold, the system can further screen out high-quality data; this double protection mechanism helps to ensure that the data ultimately stored in the database is highly accurate and reliable.

[0027] The contact determination step optimizes the data processing flow by removing temporary authentication tags and establishing permanent connections with associated tags. This step not only simplifies the subsequent processing steps of the data, but also improves the readability and usability of the data. At the same time, by establishing permanent connections, the system can more efficiently utilize the correlation between the data, facilitating subsequent data analysis and application.

[0028] Through reliability verification and contact determination steps, the system can more accurately identify and handle potential problems in the data, thereby avoiding the impact of data errors or inconsistencies on system stability and robustness.

[0029] Optionally, in the reliability verification step, it further includes: if the reliability of the third data with the temporary authentication tag is less than or equal to a fourth threshold, executing a data backtracking step; Data backtracking: restore the third data to the fourth data and mark those that do not match the associated label for data screening in the associated data extraction step.

[0030] By adopting the above technical solution, when the reliability of the third data with the temporary authentication label does not meet the set fourth threshold, the data backtracking step can restore these data to the fourth data and mark them as not meeting the associated label; this error correction mechanism helps the system to promptly discover and handle problems in the data, thereby improving the accuracy and reliability of the data.

[0031] The data backtracking step not only corrects errors in the data processing process, but also optimizes the entire processing flow; by restoring data that does not meet the conditions to the original data set and providing a basis for subsequent data screening and processing, the system can more effectively utilize data resources and avoid unnecessary repeated processing.

[0032] Optionally, the data screening step further includes: determining whether the reliability of the high-reliability filling data is greater than a fifth threshold, and if so, executing the data storage step, otherwise, executing the history deduction step; Historical deduction: obtain the historical data of the first data, count the change trend of the historical data, and form new filling data according to the change trend, and then execute the data storage step.

[0033] By adopting the above technical solution, by judging whether the reliability of the replacement data is greater than the fifth threshold, the system can screen out high-quality data for storage; this strategy helps to ensure that the data stored in the database is highly accurate and reliable, providing a solid foundation for subsequent data analysis and application.

[0034] When the reliability of the filling data does not meet the set fifth threshold, the system will not abandon the data directly, but will execute the historical deduction step; this step obtains the historical data of the first data, counts its changing trend, and forms new filling data accordingly; this strategy not only makes full use of the information of historical data, but also helps the system to deal with the problem of missing data more flexibly.

[0035] By directly storing high-quality replacement data and using historical data to deduce new replacement data, the system can reduce unnecessary repeated processing and data verification steps, thereby improving data processing efficiency.

[0036] Optionally, the historical deduction includes: an abnormality analysis step, a trend correction step, and a elimination step; Abnormal analysis: Use time series analysis or machine learning prediction models to predict the trend of historical data and generate preliminary forecast trend data. During the trend prediction process, perform abnormal point detection to identify data points that are significantly inconsistent with the overall trend and record them as abnormal points. Perform cause analysis on the abnormal points to determine whether they are reasonable changes. If so, perform trend correction steps. Otherwise, perform elimination steps. Trend correction: keep the outlier point, and use the subsequent data of the outlier point for continuous trend analysis and form new filling data; Elimination: Remove the outlier from the predicted trend data, re-predict the trend, and form new filling data based on the predicted trend.

[0037] By adopting the above technical solutions, the anomaly analysis step predicts the changing trend of historical data through time series analysis or machine learning prediction models, and identifies anomalies that are significantly inconsistent with the overall trend; this capability helps the system to promptly discover and handle potential problems in the data, thereby improving the accuracy of the data.

[0038] For the identified anomalies, the system does not simply eliminate them, but instead analyzes the causes to determine whether they are reasonable changes. This processing method enables the system to respond more flexibly to anomalies in the data and enhance the robustness of the data.

[0039] The trend correction step retains the outliers and uses the subsequent data of the outliers for continuous trend analysis; this processing method helps the system to more accurately capture the changing trends of the data and form more reliable supplementary data.

[0040] By retaining reasonable outliers and performing trend corrections, and removing unreasonable outliers and re-predicting trends, the system can reduce unnecessary repeated processing and data verification steps, thereby improving data processing efficiency.

[0041] By accurately predicting data trends and identifying anomalies, the system can provide more reliable and accurate data support for decision-making on computer room information data; this helps users make more informed decisions and improve the operating efficiency and safety of the computer room.

[0042] Optionally, after the missing value detection step, an anomaly correction step is also provided; The anomaly correction step includes an anomaly detection step and an anomaly processing step; Outlier detection: Detect outliers in the real-time information data to be processed using statistical methods or machine learning algorithms; Outlier processing: Analyze the detected outliers to determine whether they are data entry errors. If so, record them as missing values ​​and perform the identification steps. Otherwise, retain the data.

[0043] By adopting the above technical solution, through missing value detection and outlier detection, the solution can comprehensively identify problem points in the data, thereby improving the overall quality of the data; the outlier processing step further ensures the accuracy of the data, and avoids the accidental deletion of important data by distinguishing between data entry errors and other types of outliers; the solution combines statistical methods and machine learning algorithms, which can efficiently detect outliers and improve the speed of data processing; the flexible judgment mechanism in the outlier processing step enables the system to make appropriate processing decisions according to specific circumstances, thereby enhancing the flexibility of the system.

[0044] By introducing machine learning algorithms, the solution realizes intelligent analysis of data and improves the degree of automation of data processing; the improvement of the intelligence level helps to reduce manual intervention and reduce the risk of human error.

[0045] Optionally, after the outlier processing step, an abnormal data marking step is also provided; Abnormal data marking: Mark the abnormal values ​​after retaining the data to distinguish them from data entry errors or other types of abnormal values; Manual verification: Based on the abnormal data marking, data retrieval and in-depth analysis are performed to verify the real cause of the abnormal value. If it is manually confirmed as real data, the log generation step is executed. Otherwise, it is recorded as a missing value and the identification step is executed; Log generation: Record data changes as historical data and store them in logs, mark the change points, and record the reasons for the data changes.

[0046] By adopting the above technical solution, the abnormal data marking step enables the system to clearly distinguish different types of abnormal values, providing a clear basis for subsequent processing; the manual authentication step ensures the authenticity and accuracy of the data through in-depth analysis and verification, and avoids data errors caused by misjudgment; the solution further improves the data processing process by adding steps such as abnormal data marking, manual authentication and log generation, so that the system can process and analyze data more comprehensively; these new steps enhance the flexibility and adaptability of the system, enabling it to better cope with complex and changing data environments.

[0047] The log generation step records data changes as historical data and stores them in logs, making the data traceable. By marking change points and recording reasons for data changes, the system can clearly display the evolution of the data, which helps with data auditing and compliance checks. At the same time, the storage of logs also helps with data recovery and backup, improving data security.

[0048] In the second aspect, the present application provides a computer room information data management system, which adopts the following technical solution: A computer room information data management system, comprising the following modules: Database: used to store the historical information data of the computer room and available real-time information data; Data acquisition module: The output end is connected to the input end of the database, used to collect real-time information data of the computer room and store it as real-time information data to be processed and stored in the database; Data preprocessing module: the input end is connected to the output end of the data acquisition module, and the output end is connected to the input end of the database, and is used to preprocess the collected real-time information data to be processed to form the first data stored in the data; Missing value detection module: an input end is connected to the output end of the data preprocessing module, and is used to detect missing values ​​in the first data; Identification module: an input end is connected to the output end of the missing value detection module, and is used to determine the type and pattern of missing values ​​in the first data; Related data extraction module: the input end is connected to the output end of the identification module, and is used to extract other data associated with the first data to form second data; Relevance judgment module: the input end is connected to the output end of the associated data extraction module, and is used to judge whether the relevance of the second data is greater than a preset first threshold; Sorting module: the input end is connected to the output end of the association module, and is used to screen the second data and sort them according to the degree of association to form third data; Missing inference module: The input end is connected to the output end of the sorting module, used to infer missing values ​​and store them as fill-in data; Distance measurement module: The input end is connected to the output end of the missing derivation module and is used to calculate the similarity or distance between the supplementary data and the associated data and historical data; Reliability evaluation module: The input end is connected to the output end of the distance measurement module and is used to evaluate the reliability of the filling data; Data screening module: the input end is connected to the output end of the reliability evaluation module, and is used to screen the high-reliability filling data; Data storage module: The input end is connected to the output end of the data screening module, and is used to supplement the first data with the screened supplementary data to form usable real-time information data, and store it in the database.

[0049] By adopting the above technical solution, the data acquisition module is responsible for collecting information data in real time from the computer room environment, such as temperature, humidity, voltage, current, etc., and storing these data as real-time information data to be processed in the database; the data preprocessing module performs preprocessing operations such as cleaning and formatting on the received real-time information data to be processed to form first data, and stores it in the database again; the missing value detection module scans the first data, identifies and marks the missing values ​​therein; the recognition module analyzes the type and pattern of the missing values, such as random missing, systematic missing or conditional missing, etc.; the associated data extraction module extracts other data associated with the first data from the database according to the recognition result, such as historical data, adjacent sensor data, etc., to form second data; the associated judgment module The block calculates the correlation between the second data and the first data, and determines whether it is greater than a preset first threshold; the sorting module screens the second data and sorts them from high to low according to the correlation to form third data; the missing derivation module uses the information in the third data to derive missing values ​​through interpolation, regression and other methods to form supplementary data; the distance measurement module calculates the similarity or distance between the supplementary data and the associated data and historical data to evaluate the accuracy of the supplementary data; the reliability evaluation module evaluates the reliability of the supplementary data according to the calculation results of the similarity or distance; the data screening module screens out the supplementary data with high reliability; the data storage module supplements the first data with the screened supplementary data to form complete and available real-time information data, and updates it to the database.

[0050] Through steps such as missing value detection, associated data extraction and missing value derivation, the system can effectively identify and fill in missing values ​​in the data, thereby improving the integrity and availability of the data; using mechanisms such as association judgment, similarity or distance calculation, and reliability assessment, the system can ensure the accuracy and reliability of the filled-in data and reduce the impact of data errors and noise; the entire data processing process is highly automated and intelligent, reducing the possibility of human intervention and errors, and improving data processing efficiency and accuracy; the system can collect and process computer room information data in real time, fill in missing values ​​in a timely manner, and ensure the real-time and timeliness of the data; the complete and available real-time information data provides strong data support for computer room management, maintenance and decision-making, and helps to improve the operating efficiency and safety of the computer room; the modular design of the system makes it easy to expand and upgrade, and can adapt to the information data management needs of computer rooms of different scales and complexities.

[0051] In summary, the present application includes the following beneficial technical effects: through data collection, preprocessing, missing value detection, identification and subsequent missing value derivation steps, the scheme can effectively handle the missing value problem in the computer room information data; using machine learning models for missing value derivation, it is possible to infer missing values ​​based on the correlation and historical distribution characteristics of the data, thereby improving the integrity and accuracy of the data; introducing associated data extraction and associated judgment steps, by extracting other data associated with the data to be processed and evaluating their correlation, it is possible to more effectively utilize the intrinsic connection between the data to derive missing values. This method not only improves the efficiency of data processing, but also enhances the rationality of the derivation results. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 is a flow chart of the computer room information data management method in Example 1 of the present application; Figure 2 It is a flow chart from the pending step to the sorting intervention step in Example 2 of the present application; Figure 3 It is a flowchart from the reliability verification step to the contact determination step in Example 2 of the present application; Figure 4 is a detailed flow chart of the historical deduction steps in Example 2 of the present application; Figure 5 It is a detailed flow chart of the abnormality correction step in Example 2 of the present application. DETAILED DESCRIPTION

[0053] The following combination Figures 1 to 5 This application is described in further detail.

[0054] This embodiment discloses a method for managing computer room information data.

[0055] Example 1: Reference Figure 1, a computer room information data management method, comprising the following steps: Build a database: store the historical information data of the computer room and available real-time information data; Specifically, select a suitable database management system (DBMS), such as MySQL, Oracle, etc.; design the database structure, including tables, fields, indexes, etc., to store the historical information data and available real-time information data of the computer room; configure the security and performance parameters of the database to ensure the integrity and security of the data.

[0056] Data collection: collect real-time information data of the computer room and store it as real-time information data to be processed; Specifically, sensors, monitoring software and other tools are used to collect real-time information data of the computer room, and the collected data is transmitted to the database for storage through API interfaces, data streams, etc.

[0057] Data preprocessing: preprocessing the collected real-time information data to be processed to form first data; Specifically, the collected data is cleaned to remove duplicate and invalid data; the data is formatted to meet the requirements of subsequent processing.

[0058] Missing value detection: using statistical methods or machine learning algorithms to detect missing values ​​in the first data; Specifically, statistical methods (such as descriptive statistics, visualization, etc.) or machine learning algorithms (such as clustering algorithms, classification algorithms, etc.) are used to detect missing values ​​in the data. This embodiment uses Python's Pandas library for descriptive statistics, calculates the number and proportion of missing values ​​for each variable, and uses visualization tools (such as Matplotlib, Seaborn, etc.) to draw a missing value matrix or heat map to intuitively display the distribution of missing values.

[0059] Identification: determining the type and pattern of missing values ​​in the first data based on the distribution characteristics of the historical data and the distribution characteristics of the missing values; Specifically, clustering algorithms are used to perform cluster analysis on missing values ​​to determine the types and patterns of missing values.

[0060] Related data extraction: extracting other data associated with the first data to form the second data; Relevance judgment: judging whether the relevance of the second data is greater than a preset first threshold, and if so, executing a sorting step; Sorting: screening the second data and sorting them according to the relevance to form the third data; Missing inference: Use third-party data to train a machine learning model, apply the trained model to the missing data, infer the missing values, and store them as fill-in data; Distance measurement: Use distance measurement algorithm to calculate the similarity or distance between the supplementary data and the associated data and historical data; Reliability evaluation: Evaluate the reliability of the replacement data based on the similarity or distance and context information; Data screening: Based on the reliability evaluation results, filter the replacement data whose reliability is greater than the set reliability threshold; Data storage: The first data is supplemented with the filtered replacement data and stored in the database to form usable real-time information data.

[0061] Specifically, other data associated with the first data, such as device information, operating status, etc., are extracted from the database; for example, based on fields such as device ID or timestamp, device information and operating status data associated with missing values ​​are extracted from the database.

[0062] The correlation degree of the second data is calculated using correlation analysis, mutual information and other methods to determine whether the correlation degree is greater than a preset first threshold value; the specific implementation method is as follows: Correlation analysis: Use the correlation_matrix function in the Scikit-learn library of Python or the corr method in the Pandas library to calculate the correlation matrix of the second data; each element in the correlation matrix represents the correlation coefficient between two variables (such as the Pearson correlation coefficient); the strength of the correlation between the variables is determined based on the absolute value of the correlation coefficient. The closer the absolute value is to 1, the stronger the correlation is; the closer it is to 0, the weaker the correlation is.

[0063] Mutual information: Use the mutual_info_score function in the Scikit-learn library or related libraries to calculate the mutual information value between the variables in the second data; the larger the mutual information value, the more shared information there is between the two variables, that is, the stronger the association.

[0064] After calculating the correlation, we compare it with the preset first threshold. If the correlation is greater than the first threshold, it is considered that the data has a strong correlation with the first data, and the subsequent sorting steps should be performed. The first threshold is determined by performing data statistical analysis based on the correlation between historical data.

[0065] The sorting stage mainly sorts the second data according to the relevance, which can be achieved through Python's built-in sorting function or the sorting method in the Pandas library; use Pandas' sort_values ​​method to sort in descending order according to the relevance column (assuming that the higher the relevance, the more important the data).

[0066] In the missing inference stage, we use machine learning algorithms to train the model and apply the model to infer the missing values ​​of missing data; data preparation: extract samples containing complete data sets from the database as training data, and preprocess the training data, including data cleaning, feature selection, etc.; model training: select appropriate machine learning algorithms according to the nature of the problem (such as regression algorithms for continuous variable prediction, classification algorithms for discrete variable prediction), and use relevant models in the Scikit-learn library (such as linear regression, decision tree regression, random forest regression, etc.) for training; evaluate the performance of the model through methods such as cross-validation, and adjust the model parameters to optimize the prediction results.

[0067] Missing value derivation: Apply the trained model to a data set containing missing values ​​(i.e., the first data or the real-time information data to be processed after preprocessing). The model will predict the missing values ​​based on the known data (i.e., the associated data in the second data), and store the predicted values ​​as the supplementary data for use in subsequent steps.

[0068] In the distance measurement stage, we use a distance measurement algorithm to calculate the similarity or distance between the filled-in data and the associated data and historical data; select a suitable distance measurement algorithm (such as Euclidean distance, Manhattan distance, cosine similarity, etc.) according to the nature of the data and the needs of the problem; this embodiment uses relevant functions in the NumPy library (such as numpy.linalg.norm for calculating Euclidean distance) to calculate the distance or similarity value, and evaluates the closeness between the filled-in data and the associated data based on the calculated distance or similarity value.

[0069] In the reliability assessment stage, we evaluate the reliability of the replacement data based on the distance measurement results and contextual information; set a distance threshold or similarity threshold, which is determined according to the requirements of the problem; compare the distance or similarity value between the replacement data and the associated data with the threshold. If the distance is less than the distance threshold or the similarity is greater than the similarity threshold, the replacement data is considered to have a higher reliability; otherwise, its reliability is considered to be lower; combine the contextual information (such as the source of the data, the collection time, etc.) to further verify and adjust the reliability assessment results. For example, if the source of the replacement data is reliable and the collection time is relatively recent, then its reliability can be considered to be high; conversely, if the source of the replacement data is unclear or the collection time is relatively long, then its reliability assessment result may need to be lowered.

[0070] In the data screening stage, we screen the replacement data with high reliability based on the reliability assessment results; set a reliability threshold, determined according to the needs of the problem, and compare the reliability of the replacement data with the threshold. If the reliability is greater than the threshold, the replacement data will be retained. If there are multiple replacement data to be retained, the one with the highest reliability will be retained and provided to users, and the rest will be stored as data backup; otherwise, it will be eliminated. The screened replacement data will be used for subsequent data storage and analysis steps.

[0071] The setting of the reliability threshold can use historical data to count the distance or similarity distribution between the filling data and the associated data, and set a reasonable threshold range based on the distribution characteristics. For example, a quantile that allows most of the filling data to be considered reliable can be selected as the threshold.

[0072] In the data storage stage, we supplement the first data with the filtered filling data and store them in the database. We use the Pandas library to merge the filtered filling data with the first data, and further clean and format the merged data to ensure that it meets the requirements of the database; use API interfaces, data streams, or database connection tools (such as SQLAlchemy, MySQLdb, etc.) to store the merged data in the MySQL database.

[0073] Example 1 of the present application supplements missing values ​​by associating data and the correlation, and then verifies the reliability based on historical data and context information to complete the supplement of missing values ​​and achieve data integrity.

[0074] Example 2: Reference Figure 2 , the difference between this embodiment and embodiment 1 is that, after the data storage step, an association adjustment step is also provided; Association adjustment: According to the feedback result of the reliability assessment, the association of the second data is adjusted. If the reliability of the replacement data is higher than the preset second threshold, the association of the corresponding second data is increased, and an association label is established for the adjusted associated data to identify its association and reliability; otherwise, the association is reduced.

[0075] Specifically, according to the feedback result of the reliability assessment, the relevance of the second data is adjusted. If the reliability of the replacement data is higher than a preset second threshold (this threshold is usually higher than the first threshold to ensure that the adjustment of the relevance is more cautious and is obtained through historical data statistics), the relevance of the corresponding second data is increased. Conversely, if the reliability of the replacement data is lower than the first threshold or higher than the first threshold but lower than the second threshold, the relevance of the corresponding second data is reduced, and an associated label is established for the adjusted associated data. The label should contain specific numerical values ​​or level information of the relevance and reliability to facilitate subsequent data analysis and application.

[0076] Reference Figure 2 In other embodiments, in the association determination step, it further includes: if the association degree of the second data is greater than a preset first threshold, executing a pending step; To be determined: the second data whose correlation degree is less than or equal to the first threshold is stored as the fourth data; Analysis: Analyze the fourth data, establish preliminary labels, and calculate the similarity between the preliminary labels and the associated labels, which is recorded as label similarity; Similarity judgment: judge whether the label similarity is greater than or equal to the third threshold, if so, perform the sorting intervention step; Sorting intervention: Intervene in the sorting step, establish a temporary connection between the fourth data and the associated tag, insert it into the third data, and attach a temporary authentication tag.

[0077] Specifically, the fourth data with a smaller correlation is analyzed to extract its key features or information. According to the analysis results, a preliminary label is established for the fourth data. After the preliminary label is established, the similarity between the preliminary label and the existing associated label is analyzed, which is recorded as label similarity. It is determined whether the label similarity is greater than or equal to a third threshold (by analyzing historical data, understanding the distribution, mean, standard deviation and other statistical characteristics of the data, and setting it according to the statistical characteristics of the data). If the label similarity is greater than or equal to the third threshold, it is considered that there is a certain correlation or similarity between the fourth data and the associated data, and a sorting intervention step is performed. If the label similarity is less than the third threshold, it is possible that the correlation judgment is correct, and the fourth data is poorly correlated with the first data.

[0078] The sorting step is intervened, a temporary connection is established between the fourth data and the associated label, the temporarily connected data is inserted into the third data (the third data is a sorted or filtered data set), and then the missing deduction step is performed.

[0079] For example, we set the first threshold to 0.8 (indicating the minimum requirement for correlation), check the correlation between the maintenance record of a certain device and the real-time monitoring data, and if the correlation is greater than 0.8, continue to process the maintenance record of the device; if the correlation is less than or equal to 0.8, store the maintenance record as the fourth data; store the maintenance records (fourth data) with a correlation less than or equal to 0.8 separately; conduct in-depth analysis of the fourth data, extract key features (such as fault type, fault time, etc.), and based on the analysis results, establish preliminary labels for the fourth data (such as "potential faults", "frequent maintenance", etc.), and calculate the similarity between the preliminary labels and existing associated labels (such as "high failure rate", "stable operation", etc.). The third threshold is set to 0.75 (indicating the minimum requirement for label similarity). If the similarity between the preliminary label and a related label is greater than or equal to 0.75, it is considered that there is a certain correlation between the fourth data and the related data. The sorting step is intervened to establish a temporary connection between the fourth data and the related label. The data with this temporary connection is inserted into the third data to deduce missing values, and a temporary authentication label (such as "fault to be verified") is attached to these data for subsequent tracking and processing.

[0080] Reference Figure 3 , in other embodiments, after the sorting intervention step, a reliability verification step and a contact determination step are also provided; Reliability verification: determining whether the reliability of the third data with the temporary authentication tag is greater than a set fourth threshold, and if so, executing a contact determination step; Contact confirmation: remove the temporary authentication tag of the third data and establish a permanent contact with the associated tag.

[0081] Specifically, in this step, we establish a temporary connection between the fourth data and the associated label according to a certain sorting or screening condition, and insert it into the third data, and at the same time attach a temporary authentication label. These third data with temporary authentication labels are now regarded as potential associated data; reliability verification is a process of in-depth evaluation of the third data with temporary authentication labels to determine whether its reliability is high enough to establish a permanent connection or be used as associated data for other data; a fourth threshold is set, which represents the minimum standard of reliability. This standard can be set based on multiple dimensions such as the source, integrity, timeliness, and accuracy of the data; if the reliability of the third data is greater than the fourth threshold, the data is considered to be reliable, and the subsequent connection determination step can be executed or used as associated data for other data; in the connection determination step, we remove the temporary label of the third data with the temporary authentication label and establish a permanent connection with the associated label; the derived replacement data also needs to be judged for reliability. We can set the same reliability standard as the fourth threshold, or set different standards according to the specific situation; if the reliability of the replacement data is greater than or equal to the set threshold, the replacement data is considered to be reliable and can be used as associated data for the first data or for other analysis.

[0082] Reference Figure 3 , in other embodiments, in the reliability verification step, it further includes: if the reliability of the third data with the temporary authentication tag is less than or equal to a fourth threshold, executing a data backtracking step; Data backtracking: restore the third data to the fourth data and mark those that do not match the associated label for data screening in the associated data extraction step.

[0083] Specifically, reliability verification is a process of in-depth evaluation of the third data with a temporary authentication tag to determine whether its reliability is high enough to establish a permanent connection or be used as associated data for other data; if the reliability of the third data is greater than a fourth threshold, the data is considered reliable and can perform subsequent contact determination steps or be used as associated data for other data; in the contact determination step, we remove the temporary label of the third data with a temporary authentication tag and establish a permanent connection with the associated label.

[0084] Reference Figure 4 In other embodiments, in the data screening step, it further includes: determining whether the reliability of the high-reliability filling data is greater than a fifth threshold, if so, executing the data storage step, otherwise, executing the historical deduction step; Historical deduction: obtain the historical data of the first data, count the change trend of the historical data, and form new filling data according to the change trend, and then execute the data storage step.

[0085] Specifically, when the reliability of the replacement data does not meet the fifth threshold requirement, execute the historical deduction step; obtain the historical data of the first data, which can be similar data or related data in the previous time period; perform statistical analysis on the historical data to identify the changing trend or pattern of the data; form new replacement data based on the identified changing trend or pattern; perform reliability evaluation on the new replacement data, and if it meets the fifth threshold requirement, execute the data storage step, wherein the setting of the fifth threshold can be higher than or equal to the reliability threshold, which is used as an accuracy judgment, so as to facilitate the selection of historical data trend deduction to estimate missing values ​​or related data derivation, so as to improve the accuracy of data completion.

[0086] Reference Figure 4 ,In other implementation methods, the historical deduction includes: an abnormal analysis step, a trend correction step, and a elimination step; Abnormal analysis: Use time series analysis or machine learning prediction models to predict the trend of historical data and generate preliminary forecast trend data. During the trend prediction process, perform abnormal point detection to identify data points that are significantly inconsistent with the overall trend and record them as abnormal points. Perform cause analysis on the abnormal points to determine whether they are reasonable changes. If so, perform trend correction steps. Otherwise, perform elimination steps. Trend correction: keep the outlier point, and use the subsequent data of the outlier point for continuous trend analysis and form new filling data; Elimination: Remove the outlier from the predicted trend data, re-predict the trend, and form new filling data based on the predicted trend.

[0087] Specifically, time series analysis or machine learning prediction models are used to predict the changing trends of historical data and generate preliminary prediction trend data; in the trend prediction process, outlier detection is performed, and outliers refer to data points that are significantly inconsistent with the overall trend; the causes of the identified outliers are analyzed to determine whether they are reasonable changes; if the outliers are judged to be reasonable changes (for example, changing component models), the trend correction step is executed; if the outliers are judged to be unreasonable changes (for example, anomalies caused by internal factors such as data entry errors and equipment failures), the elimination step is executed.

[0088] Trend correction: retain outliers and conduct continuous trend analysis based on the subsequent data of the outliers; based on the subsequent stable and continuous trend analysis, form new filling data, which will better reflect the real change trend of historical data, and all data points including outliers are taken into account.

[0089] Elimination: Remove outliers from the predicted trend data and re-predict the trend to generate new predicted trend data that does not contain outliers; based on the new predicted trend data, form new fill-in data; since the outliers have been eliminated, the new fill-in data will more smoothly reflect the overall trend of the historical data.

[0090] Reference Figure 5 , in other implementations, after the missing value detection step, an anomaly correction step is also provided; The anomaly correction step includes an anomaly detection step and an anomaly processing step; Outlier detection: Detect outliers in the real-time information data to be processed using statistical methods or machine learning algorithms; Outlier processing: Analyze the detected outliers to determine whether they are data entry errors. If so, record them as missing values ​​and perform the identification steps. Otherwise, retain the data.

[0091] Specifically, in this step, we check the real-time information data to be processed and identify the missing values, which may be caused by sensor failure, data transmission error, etc.; Following the missing value detection step, we use statistical methods (such as the 3σ principle, box plot, etc.) or machine learning algorithms (such as isolation forest, LOF, etc.) to detect outliers on the detected non-missing data; outliers refer to those data points that are significantly different from most data points, which may be caused by data entry errors, equipment failures, abnormal events, etc.; the detected outliers are analyzed to determine whether they are data entry errors. This can be based on factors such as the contextual information of the data, historical data, external events, etc.; if the outlier is judged to be a data entry error, it is treated as a missing value and the subsequent identification steps are performed (i.e., identifying the type and cause of the missing value).

[0092] If the outlier is judged to be not a data entry error (e.g., an outlier or replacement component due to an unusual event), then the data point is retained because it may contain useful information that may have a significant impact on subsequent analysis or prediction.

[0093] Reference Figure 5 , in other embodiments, after the outlier processing step, an abnormal data marking step is also provided; Abnormal data marking: Mark the abnormal values ​​after retaining the data to distinguish them from data entry errors or other types of abnormal values; Manual verification: Based on the abnormal data marking, data retrieval and in-depth analysis are performed to verify the real cause of the abnormal value. If it is manually confirmed as real data, the log generation step is executed. Otherwise, it is recorded as a missing value and the identification step is executed; Log generation: Record data changes as historical data and store them in logs, mark the change points, and record the reasons for the data changes.

[0094] Specifically, in this step, we use statistical methods or machine learning algorithms to detect outliers in the data to be processed, and perform preliminary classification processing on these outliers, such as determining whether they are data entry errors; mark the outliers that are still retained after the preliminary classification processing to distinguish them from outliers that are considered missing values ​​due to data entry errors; the marking method can be to add specific labels, color codes or annotations to easily identify these outliers in subsequent data processing and analysis; based on the abnormal data marking, data retrieval and in-depth analysis are performed to verify the real cause of the outliers.

[0095] If it is manually confirmed to be real data (that is, the outlier is caused by a real event or condition, the condition may be the replacement of a component), the log generation step is executed; if it is manually confirmed to be a data entry error or other types of erroneous data, it is re-recorded as a missing value and the subsequent identification steps are executed; the data changes are recorded as historical data and stored in a log. The log should contain detailed information on the data changes, such as the change point, change time, data values ​​before and after the change, and the reason for the data change.

[0096] The embodiment of the present application also discloses a computer room information data management system.

[0097] The computer room information data management system includes the following modules: Database: used to store the historical information data of the computer room and available real-time information data; Data acquisition module: The output end is connected to the input end of the database, used to collect real-time information data of the computer room and store it as real-time information data to be processed and stored in the database; Specifically, real-time information data of the computer room is collected through sensors, API interfaces, log files, etc.; scheduled tasks or event trigger mechanisms are used to collect data regularly or in real time, and the collected data is converted into a format recognizable by the system and stored as real-time information data to be processed.

[0098] Data preprocessing module: the input end is connected to the output end of the data acquisition module, and the output end is connected to the input end of the database, and is used to preprocess the collected real-time information data to be processed to form the first data stored in the data; Missing value detection module: an input end is connected to the output end of the data preprocessing module, and is used to detect missing values ​​in the first data; Identification module: an input end is connected to the output end of the missing value detection module, and is used to determine the type and pattern of missing values ​​in the first data; Specifically, the first data is traversed to check whether there are missing values, and a specific algorithm or rule is used to identify the location and type of the missing values. According to the type and pattern of the missing values, a machine learning algorithm or rule is used to identify and classify the missing values, such as missing values ​​of numerical type, missing values ​​of character type, etc.

[0099] Related data extraction module: the input end is connected to the output end of the identification module, and is used to extract other data associated with the first data to form second data; Relevance judgment module: the input end is connected to the output end of the associated data extraction module, and is used to judge whether the relevance of the second data is greater than a preset first threshold; Sorting module: the input end is connected to the output end of the association module, and is used to screen the second data and sort them according to the degree of association to form third data; Missing inference module: The input end is connected to the output end of the sorting module, used to infer missing values ​​and store them as fill-in data; Distance measurement module: The input end is connected to the output end of the missing derivation module and is used to calculate the similarity or distance between the supplementary data and the associated data and historical data; Reliability evaluation module: The input end is connected to the output end of the distance measurement module and is used to evaluate the reliability of the filling data; Data screening module: the input end is connected to the output end of the reliability evaluation module, and is used to screen the high-reliability filling data and determine whether the reliability of the high-reliability filling data is greater than the fifth threshold; Data storage module: The input end is connected to the output end of the data screening module, and is used to supplement the first data with the screened supplementary data to form usable real-time information data, and store it in the database.

[0100] Relevance adjustment module: The input end is connected to the output end of the data storage module and is used to adjust the degree of relevance according to the reliability of the filling data.

[0101] To-be-determined module: the input end is connected to the output end of the association judgment module, and stores the fourth data other than the second data whose association degree is less than or equal to the first threshold; Analysis module: the input end is connected to the output end of the pending module, and is used to analyze the fourth data, establish a preliminary label, and analyze the similarity between the preliminary label and the associated label; Similarity judgment module: the input end is connected to the output end of the analysis module, and is used to judge whether the tag similarity is greater than or equal to a third threshold; Sorting intervention module: the input end is connected to the output end of the affinity judgment module, and is used to establish a temporary connection between the data and the associated label and add a temporary authentication label after the label similarity is greater than or equal to the third threshold, and merge it into the third data for the data source of missing value derivation; Reliability verification module: the input end is connected to the output end of the sorting intervention module, and is used to determine the reliability of identifying the temporary authentication tag; Contact determination: The input end is connected to the output end of the reliability verification module to remove the temporary authentication tag from the third data with the temporary authentication tag that meets the reliability requirements, and establish a permanent connection with the associated tag; Historical deduction module: The input end is connected to the output end of the data screening module, which is used to analyze abnormal points, and according to the abnormal causes, predict the trend of changes, and make trend corrections or remove abnormal point data; Anomaly correction module: The input end is connected to the output end of the missing value detection module, which is used to determine the type of abnormal value, perform manual authentication and identification, and generate logs to record data changes.

[0102] The above are all preferred embodiments of the present application, and the protection scope of the present application is not limited thereto. Therefore, any equivalent changes made according to the structure, shape, and principle of the present application should be included in the protection scope of the present application.

Claims

1. A computer room information data management method, characterized in that: The following steps are involved: Build a database: store the historical information data of the computer room and available real-time information data; Data collection: collect real-time information data of the computer room and store it as real-time information data to be processed; Data preprocessing: preprocessing the collected real-time information data to be processed to form first data; Missing value detection: using statistical methods or machine learning algorithms to detect missing values ​​in the first data; Identification: determining the type and pattern of missing values ​​in the first data based on the distribution characteristics of the historical data and the distribution characteristics of the missing values; Related data extraction: extracting other data associated with the first data to form the second data; Relevance judgment: judging whether the relevance of the second data is greater than a preset first threshold, and if so, executing a sorting step; Sorting: screening the second data and sorting them according to the relevance to form the third data; Missing inference: Use third-party data to train a machine learning model, apply the trained model to the missing data, infer the missing values, and store them as fill-in data; Distance measurement: Use distance measurement algorithm to calculate the similarity or distance between the supplementary data and the associated data and historical data; Reliability evaluation: Evaluate the reliability of the replacement data based on the similarity or distance and context information; Data screening: Based on the reliability evaluation results, filter the replacement data whose reliability is greater than the set reliability threshold; Data storage: The first data is supplemented with the filtered replacement data and stored in the database to form usable real-time information data.

2. The computer room information data management method according to claim 1, characterized in that: After the data storage step, a correlation adjustment step is also provided; Association adjustment: According to the feedback result of the reliability assessment, the association of the second data is adjusted. If the reliability of the replacement data is higher than the preset second threshold, the association of the corresponding second data is increased, and an association label is established for the adjusted associated data to identify its association and reliability; otherwise, the association is reduced.

3. The computer room information data management method according to claim 2, characterized in that: In the association determination step, it also includes: if the association degree of the second data is greater than a preset first threshold, executing a pending step; To be determined: the second data whose correlation degree is less than or equal to the first threshold is stored as the fourth data; Analysis: Analyze the fourth data, establish preliminary labels, and calculate the similarity between the preliminary labels and the associated labels, which is recorded as label similarity; Similarity judgment: judge whether the label similarity is greater than or equal to the third threshold, if so, perform the sorting intervention step; Sorting intervention: Intervene in the sorting step, establish a temporary connection between the fourth data and the associated tag, insert it into the third data, and attach a temporary authentication tag.

4. The computer room information data management method according to claim 3, characterized in that: After the sorting intervention step, a reliability verification step and a contact determination step are also provided; Reliability verification: determining whether the reliability of the third data with the temporary authentication tag is greater than a set fourth threshold, and if so, executing a contact determination step; Contact confirmation: remove the temporary authentication tag of the third data and establish a permanent contact with the associated tag.

5. The computer room information data management method according to claim 4, characterized in that: In the reliability verification step, it also includes: if the reliability of the third data with the temporary authentication tag is less than or equal to a fourth threshold, executing a data backtracking step; Data backtracking: restore the third data to the fourth data and mark those that do not match the associated label for data screening in the associated data extraction step.

6. The computer room information data management method according to claim 4, characterized in that: The data screening step also includes: determining whether the reliability of the high-reliability filling data is greater than a fifth threshold, and if so, executing the data storage step, otherwise, executing the history deduction step; Historical deduction: obtain the historical data of the first data, count the change trend of the historical data, and form new filling data according to the change trend, and then execute the data storage step.

7. The computer room information data management method according to claim 6, characterized in that: The historical deduction includes: abnormal analysis step, trend correction step and elimination step; Abnormal analysis: Use time series analysis or machine learning prediction models to predict the trend of historical data and generate preliminary forecast trend data. During the trend prediction process, perform abnormal point detection to identify data points that are significantly inconsistent with the overall trend and record them as abnormal points. Perform cause analysis on the abnormal points to determine whether they are reasonable changes. If so, perform trend correction steps. Otherwise, perform elimination steps. Trend correction: keep the outlier point, and use the subsequent data of the outlier point for continuous trend analysis and form new filling data; Elimination: Remove the outlier from the predicted trend data, re-predict the trend, and form new filling data based on the predicted trend.

8. The computer room information data management method according to claim 1, characterized in that: After the missing value detection step, an anomaly correction step is also set; The anomaly correction step includes an anomaly detection step and an anomaly processing step; Outlier detection: Detect outliers in the real-time information data to be processed using statistical methods or machine learning algorithms; Outlier processing: Analyze the detected outliers to determine whether they are data entry errors. If so, record them as missing values ​​and perform the identification steps. Otherwise, retain the data.

9. The computer room information data management method according to claim 8, characterized in that: After the outlier processing step, an abnormal data marking step is also provided; Abnormal data marking: Mark the abnormal values ​​after retaining the data to distinguish them from data entry errors or other types of abnormal values; Manual verification: Based on the abnormal data marking, data retrieval and in-depth analysis are performed to verify the real cause of the abnormal value. If it is manually confirmed as real data, the log generation step is executed. Otherwise, it is recorded as a missing value and the identification step is executed; Log generation: Record data changes as historical data and store them in logs, mark the change points, and record the reasons for the data changes.

10. A system for implementing the computer room information data management method according to any one of claims 1 to 9, characterized in that: Includes the following modules: Database: used to store the historical information data of the computer room and available real-time information data; Data acquisition module: The output end is connected to the input end of the database, used to collect real-time information data of the computer room and store it as real-time information data to be processed and stored in the database; Data preprocessing module: the input end is connected to the output end of the data acquisition module, and the output end is connected to the input end of the database, and is used to preprocess the collected real-time information data to be processed to form the first data stored in the data; Missing value detection module: an input end is connected to the output end of the data preprocessing module, and is used to detect missing values ​​in the first data; Identification module: an input end is connected to the output end of the missing value detection module, and is used to determine the type and pattern of missing values ​​in the first data; Related data extraction module: the input end is connected to the output end of the identification module, and is used to extract other data associated with the first data to form second data; Relevance judgment module: the input end is connected to the output end of the associated data extraction module, and is used to judge whether the relevance of the second data is greater than a preset first threshold; Sorting module: the input end is connected to the output end of the association module, and is used to screen the second data and sort them according to the degree of association to form third data; Missing inference module: The input end is connected to the output end of the sorting module, used to infer missing values ​​and store them as fill-in data; Distance measurement module: The input end is connected to the output end of the missing derivation module and is used to calculate the similarity or distance between the supplementary data and the associated data and historical data; Reliability evaluation module: The input end is connected to the output end of the distance measurement module and is used to evaluate the reliability of the filling data; Data screening module: the input end is connected to the output end of the reliability evaluation module, and is used to screen the high-reliability filling data; Data storage module: The input end is connected to the output end of the data screening module, and is used to supplement the first data with the screened supplementary data to form usable real-time information data, and store it in the database.

Images