Data center data management method and system based on TensorFlow ecology

Abstract

The invention relates to the technical field of data management, in particular to a data center data management method and system based on TensorFlow ecology, and the method comprises the steps: collecting and preprocessing historical multi-source heterogeneous data and real-time multi-source heterogeneous data in a data center, and obtaining first historical multi-source data and first real-time multi-source data; a data partition management model is constructed to analyze the first historical multi-source data, a data partition result is obtained, and a gradient baseline is generated; calculating a real-time gradient baseline based on the first real-time multi-source data, calculating and obtaining a reconstruction error of the gradient baseline, dynamically adjusting a threshold value in real time, and obtaining an abnormal TensorFlow ecological gradient; an abnormity analysis management model is constructed to analyze the abnormal TensorFlow ecological gradient, a TensorFlow ecological abnormity index vector is obtained, and specific abnormity factors generating abnormity are obtained; and evaluating and alarming the abnormal factors.

Description

Technical Field

[0001] The present invention relates to the field of data management technology, and specifically to a data center data management method and system based on the TensorFlow ecosystem. Background Art

[0002] With the rapid development of cloud computing and artificial intelligence technologies, the demand for massive data processing carried by data centers has shown an exponential growth. Traditional data management methods face severe challenges in dealing with large-scale heterogeneous data processing, real-time analysis, and machine learning model integration. In existing technologies, data centers usually use a layered architecture to handle data storage, computing, and analysis tasks, but there are defects such as fragmented data processing pipelines, inefficient resource scheduling, and insufficient support for machine learning workflows. Especially in the context of the widespread application of the TensorFlow ecosystem, it is difficult for traditional systems to effectively integrate the pipelined data processing of TensorFlow Extended (TFX), the dynamic data distribution of TensorFlow Data Service, and the heterogeneous computing resource scheduling capabilities of TensorFlow Runtime, resulting in significant IO bottlenecks and idle computing resources in the data processing and model training links. In addition, existing solutions lack a unified management framework in terms of data version control, feature engineering automation, and distributed training collaborative optimization, making it difficult to achieve closed-loop management of data-model collaborative optimization.

[0003] To this end, a data center data management method and system based on the TensorFlow ecosystem is proposed. Summary of the invention

[0004] The purpose of the present invention is to provide a data center data management method and system based on the TensorFlow ecosystem, by collecting historical multi-source heterogeneous data and real-time multi-source heterogeneous data in the data center and preprocessing, to obtain first historical multi-source data and first real-time multi-source data; construct a data partition management model to analyze the first historical multi-source data, obtain data partition results, and generate a gradient baseline; based on the first real-time multi-source data, calculate the real-time gradient baseline, calculate and obtain the reconstruction error of the gradient baseline, dynamically adjust the threshold in real time, and obtain an abnormal TensorFlow ecological gradient; construct an abnormal analysis management model to analyze the abnormal TensorFlow ecological gradient, obtain a TensorFlow ecological abnormality indicator vector, and obtain the specific abnormal factors that cause the abnormality; evaluate and warn the abnormal factors.

[0005] To achieve the above object, the present invention provides the following technical solutions: Data center data management methods based on the TensorFlow ecosystem include: S1. Collect historical multi-source heterogeneous data and real-time multi-source heterogeneous data in the data center and pre-process them to obtain first historical multi-source data and first real-time multi-source data; S2. Constructing a data partition management model to analyze the first historical multi-source data to obtain data partition results; generating a gradient baseline based on the data partition results; S3. Calculate the real-time gradient baseline based on the first real-time multi-source data; obtain the reconstruction error of the gradient baseline by calculating the gradient baseline and the real-time gradient baseline; update the reconstruction error based on the real-time gradient baseline, dynamically adjust the threshold in real time, and obtain the abnormal TensorFlow ecological gradient; S4. Construct an exception analysis management model to analyze the abnormal TensorFlow ecological gradient and obtain the TensorFlow ecological abnormality indicator vector; based on the analysis of the TensorFlow ecological abnormality indicator vector, obtain the specific abnormal factors that cause the abnormality; evaluate and alarm the abnormal factors.

[0006] Preferably, the historical multi-source heterogeneous data includes historical server performance indicators, historical network traffic data, historical application logs and historical device status data; the real-time multi-source heterogeneous data includes real-time server performance indicators, real-time network traffic data, real-time application logs and real-time device status data; the preprocessing includes cleaning, normalization and spatiotemporal alignment processing; wherein the preprocessing utilizes the data processing components in the TensorFlow ecosystem to construct a data flow graph to achieve efficient parallel processing and conversion of multi-source heterogeneous data.

[0007] Preferably, the data partition management model includes a data partition layer, a data feature extraction layer and a gradient baseline generation layer; The data partitioning layer generates preliminary data partitions by performing cluster analysis on historical server performance indicators and historical application logs in the first historical multi-source data; performs feature correlation analysis on the historical application logs, historical network traffic data, historical application logs and historical device status data to obtain the first TensorFlow ecological variable features; modifies the preliminary data partitions based on the first TensorFlow ecological variable features to obtain data partitioning results; the data feature extraction layer obtains the TensorFlow ecological feature vector by performing feature extraction on the first TensorFlow ecological variable features and the data partitioning results; the gradient baseline generation layer obtains a preliminary theoretical TensorFlow ecological baseline by performing multidimensional regression analysis on the TensorFlow ecological feature vector and the first real-time multi-source data; nonlinearly maps the TensorFlow ecological feature vector through an autoencoder model to extract potential features in the TensorFlow ecosystem, reconstructs the original TensorFlow ecological gradient data based on the learned potential features, and generates a gradient baseline.

[0008] Preferably, the abnormal TensorFlow ecological gradient generation step includes: Real-time gradient baseline calculation, based on the first real-time multi-source data, combined with the data partitioning results and the TensorFlow ecological feature vector, uses multidimensional regression analysis and autoencoder models to perform feature learning and reconstruction on the real-time collected TensorFlow ecological data to generate a real-time gradient baseline; reconstruction error calculation, uses the autoencoder to perform nonlinear mapping and reconstruction on the real-time gradient baseline to obtain a reconstructed gradient baseline, and calculates the reconstruction error; abnormal TensorFlow ecological gradient identification, updates the reconstruction error based on the real-time gradient baseline, and dynamically adjusts the preset threshold in real time. When the reconstruction error is greater than the preset threshold, the abnormal TensorFlow ecological gradient is judged to be abnormal.

[0009] Preferably, the anomaly analysis management model includes a TensorFlow ecological anomaly indicator vector acquisition layer and an anomaly factor acquisition layer; The TensorFlow ecological anomaly index vector acquisition layer compares the abnormal TensorFlow ecological gradient with the gradient baseline generated based on historical multi-source data, calculates the deviation of each variable, and calculates the TensorFlow ecological anomaly index vector in combination with the feature weight; The abnormal factor acquisition layer attributes the ecological factors to the input TensorFlow ecological abnormality index vector using principal component regression analysis to obtain abnormal factors.

[0010] Preferably, the specific formula for evaluating the abnormal factors is: ; in, For each intervention program, For the plan The cost, For the plan The estimated reduction in abnormal risk, is the cost-estimate trade-off coefficient, When the objective function is minimized The variable value of .

[0011] A data center data management system based on the TensorFlow ecosystem, including: A data collection module is used to collect historical multi-source heterogeneous data and real-time multi-source heterogeneous data in the data center and perform preprocessing to obtain first historical multi-source data and first real-time multi-source data; A gradient baseline acquisition module is used to construct a data partition management model to analyze the first historical multi-source data to obtain data partition results; and generate a gradient baseline based on the data partition results; An abnormal TensorFlow ecological gradient acquisition module is used to calculate the real-time gradient baseline based on the first real-time multi-source data; obtain the reconstruction error of the gradient baseline by calculating the gradient baseline and the real-time gradient baseline; update the reconstruction error based on the real-time gradient baseline, dynamically adjust the threshold in real time, and obtain the abnormal TensorFlow ecological gradient; The abnormal factor acquisition module is used to build an abnormal analysis management model to analyze the abnormal TensorFlow ecological gradient and obtain the TensorFlow ecological abnormal indicator vector; based on the analysis of the TensorFlow ecological abnormal indicator vector, the specific abnormal factors that cause the abnormality are obtained; and the abnormal factors are evaluated and alarmed.

[0012] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention constructs a digital management platform based on the TensorFlow ecosystem and uses the data processing components in the TensorFlow ecosystem to construct a data flow graph, thereby realizing efficient parallel processing of multi-source heterogeneous data such as historical server performance indicators, network traffic data, application logs, and device status data. The autoencoder model is used to perform nonlinear mapping on feature vectors, which can automatically extract potential correlation features that are difficult to capture with traditional methods. The multi-dimensional feature fusion capability improves the accuracy of data partitioning, laying a high-quality data foundation for subsequent anomaly detection.

[0013] 2. The present invention constructs a dynamic adaptive anomaly detection system, and improves the detection sensitivity through the dynamic reconstruction mechanism of the gradient baseline. The multidimensional regression analysis and autoencoder dual-mode driving technology are adopted to make the generation of real-time gradient baselines adaptive to time series, which can effectively cope with the dynamic changes in the operating status of the data center. Through the online update algorithm of the reconstruction error, the threshold adjustment response time is shortened to milliseconds.

[0014] 3. Through abnormal analysis and intervention planning, the present invention deeply analyzes the causes of abnormal TensorFlow ecological gradients, and accurately attributes them through principal component regression to achieve accurate identification of abnormal factors. Comprehensively evaluate each intervention plan based on cost, effect and trade-off coefficient, automatically generate the optimal intervention strategy, and warn of abnormal risks in real time, so as to achieve data-driven ecological risk control and scientific decision-making, and improve overall management efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A schematic diagram of the process flow of the data center data management method based on the TensorFlow ecosystem provided by the present invention; Figure 2 A schematic diagram of the structure of a data center data management system based on the TensorFlow ecosystem provided by the present invention; Figure 3 A schematic diagram of ecological data anomaly assessment provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0016] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0017] Embodiment 1 See also Figure 1 to Figure 2 The present invention provides a data center data management method based on the TensorFlow ecosystem and is applied to a data center data management system based on the TensorFlow ecosystem. The technical solution is as follows: As an embodiment of the present invention, refer to Figure 1 S1 in the figure is applied to the data acquisition module of the data center data management system of the TensorFlow ecosystem. The data acquisition module is used to collect historical multi-source heterogeneous data and real-time multi-source heterogeneous data in the data center and perform preprocessing to obtain first historical multi-source data and first real-time multi-source data.

[0018] S1. Collect historical multi-source heterogeneous data and real-time multi-source heterogeneous data in the data center and pre-process them to obtain first historical multi-source data and first real-time multi-source data; The historical multi-source heterogeneous data includes historical server performance indicators, historical network traffic data, historical application logs and historical device status data; the real-time multi-source heterogeneous data includes real-time server performance indicators, real-time network traffic data, real-time application logs and real-time device status data; the preprocessing includes cleaning, normalization and spatiotemporal alignment processing; wherein the preprocessing uses the data processing components in the TensorFlow ecosystem to construct a data flow graph to achieve efficient parallel processing and conversion of multi-source heterogeneous data.

[0019] This implementation plan introduces historical and real-time multi-source heterogeneous data, and uses TensorFlow data flow graphs to build an efficient parallel processing platform to achieve data cleaning, normalization, and time-space alignment, ensuring data standardization, time-space consistency, and integrity. This provides high-quality data support for subsequent ecological gradient baseline generation, anomaly detection, and intelligent analysis.

[0020] As an embodiment of the present invention, refer to Figure 1 S2 in the figure is applied to the gradient baseline acquisition module of the data center data management system of the TensorFlow ecosystem. The gradient baseline acquisition module is used to build a data partition management model to analyze the first historical multi-source data to obtain data partition results; and generate a gradient baseline through the data partition results.

[0021] S2. Constructing a data partition management model to analyze the first historical multi-source data to obtain data partition results; generating a gradient baseline based on the data partition results; The data partition management model includes a data partition layer, a data feature extraction layer and a gradient baseline generation layer; The data partitioning layer generates preliminary data partitions by performing cluster analysis on historical server performance indicators and historical application logs in the first historical multi-source data; performs feature correlation analysis on the historical application logs, historical network traffic data, historical application logs and historical device status data to obtain the first TensorFlow ecological variable features; modifies the preliminary data partitions based on the first TensorFlow ecological variable features to obtain data partitioning results; the data feature extraction layer obtains the TensorFlow ecological feature vector by performing feature extraction on the first TensorFlow ecological variable features and the data partitioning results; the gradient baseline generation layer obtains a preliminary theoretical TensorFlow ecological baseline by performing multidimensional regression analysis on the TensorFlow ecological feature vector and the first real-time multi-source data; nonlinearly maps the TensorFlow ecological feature vector through an autoencoder model to extract potential features in the TensorFlow ecosystem, reconstructs the original TensorFlow ecological gradient data based on the learned potential features, and generates a gradient baseline.

[0022] The present invention constructs a data partition management model, uses clustering and feature correlation analysis to perform preliminary partition correction on historical data, and realizes accurate data partition division; obtains TensorFlow ecological feature vectors through data feature extraction and combines real-time data for multidimensional regression, generates a theoretical TensorFlow ecological baseline, and uses autoencoder nonlinear mapping to extract potential features and reconstruct ecological gradient data; this method greatly improves data partition accuracy and baseline modeling reliability, laying a solid foundation for ecological environment monitoring.

[0023] As an embodiment of the present invention, refer to Figure 1 S3 in the example, S3 is applied to the abnormal TensorFlow ecological gradient acquisition module of the data center data management system of the TensorFlow ecology. The abnormal TensorFlow ecological gradient acquisition module is used to calculate the real-time gradient baseline based on the first real-time multi-source data; obtain the reconstruction error of the gradient baseline by calculating the gradient baseline and the real-time gradient baseline; update the reconstruction error based on the real-time gradient baseline, dynamically adjust the threshold in real time, and obtain the abnormal TensorFlow ecological gradient.

[0024] S3. Calculate the real-time gradient baseline based on the first real-time multi-source data; obtain the reconstruction error of the gradient baseline by calculating the gradient baseline and the real-time gradient baseline; update the reconstruction error based on the real-time gradient baseline, dynamically adjust the threshold in real time, and obtain the abnormal TensorFlow ecological gradient; The abnormal TensorFlow ecological gradient generation steps include: Real-time gradient baseline calculation, based on the first real-time multi-source data, combined with the data partitioning results and the TensorFlow ecological feature vector, uses multidimensional regression analysis and autoencoder models to perform feature learning and reconstruction on the real-time collected TensorFlow ecological data to generate a real-time gradient baseline; reconstruction error calculation, uses the autoencoder to perform nonlinear mapping and reconstruction on the real-time gradient baseline to obtain a reconstructed gradient baseline, and calculates the reconstruction error; abnormal TensorFlow ecological gradient identification, updates the reconstruction error based on the real-time gradient baseline, and dynamically adjusts the preset threshold in real time. When the reconstruction error is greater than the preset threshold, the abnormal TensorFlow ecological gradient is judged to be abnormal.

[0025] This implementation scheme uses multidimensional regression and autoencoders to perform feature learning and reconstruction on real-time collected ecological data to generate a real-time gradient baseline, calculate the reconstruction error through nonlinear mapping, and dynamically update the preset threshold to achieve accurate identification of abnormal TensorFlow ecological gradients; this method effectively overcomes the limitations of traditional static thresholds, improves real-time monitoring and abnormal warning capabilities, and provides timely and accurate response guarantees for ecological risk prevention and control, thereby realizing efficient operation of real-time data monitoring and risk warning systems.

[0026] As an embodiment of the present invention, refer to Figure 1 S4 in the example is applied to the abnormal factor acquisition module of the data center data management system of the TensorFlow ecosystem. The abnormal factor acquisition module is used to construct an abnormal analysis management model to analyze the abnormal TensorFlow ecosystem gradient and obtain the TensorFlow ecosystem abnormal indicator vector; based on the analysis of the TensorFlow ecosystem abnormal indicator vector, the specific abnormal factors that cause the abnormality are obtained; and the abnormal factors are evaluated and warned.

[0027] S4. Construct an abnormal analysis management model to analyze the abnormal TensorFlow ecological gradient and obtain the TensorFlow ecological abnormality indicator vector; obtain the specific abnormal factors that cause the abnormality based on the analysis of the TensorFlow ecological abnormality indicator vector; evaluate and warn the abnormal factors; The anomaly analysis management model includes a TensorFlow ecological anomaly indicator vector acquisition layer and an anomaly factor acquisition layer; The TensorFlow ecological anomaly index vector acquisition layer compares the abnormal TensorFlow ecological gradient with the gradient baseline generated based on historical multi-source data, calculates the deviation of each variable, and calculates the TensorFlow ecological anomaly index vector in combination with the feature weight; The abnormal factor acquisition layer attributes the ecological factors to the input TensorFlow ecological abnormality index vector using principal component regression analysis to obtain abnormal factors; This implementation plan builds an abnormal analysis management model, compares the abnormal TensorFlow ecological gradient with the historical baseline to calculate the variable deviation, and generates an abnormal indicator vector by combining the feature weight; realizes ecological factor attribution through principal component regression analysis and accurately identifies abnormal factors; this method effectively improves the accuracy of abnormal diagnosis, clarifies the cause of abnormalities, and provides a scientific basis for subsequent precise intervention and risk mitigation. At the same time, it optimizes the utilization efficiency of ecological monitoring data and the accuracy of abnormal warning.

[0028] The specific formula for evaluating the abnormal factors is: ; in, For each intervention program, For the plan The cost, For the plan The estimated reduction in abnormal risk, is the cost-estimate trade-off coefficient, When the objective function is minimized The variable value of .

[0029] This implementation plan constructs an abnormal factor evaluation formula to quantitatively measure the cost and estimated effect of each intervention plan, and introduces a cost-effectiveness trade-off coefficient to achieve a scientific balance between economic benefits and risk reduction. This formula provides a quantitative basis for the selection of abnormal risk intervention measures, effectively avoids subjective evaluation errors, improves the feasibility and execution efficiency of intervention plans, and thus provides a scientific basis for intervention decisions to achieve optimal risk.

[0030] The data center data management method based on the TensorFlow ecosystem of the present invention realizes the high efficiency and intelligence of data management by integrating advanced data processing and analysis technologies. First, by using the data processing components in the TensorFlow ecosystem, a data flow graph is constructed to efficiently process and convert multi-source heterogeneous data in parallel, greatly improving the data processing efficiency. Secondly, by constructing an abnormal analysis management model, various indicators of the data center can be monitored in real time, abnormal factors can be accurately identified and analyzed, potential problems can be warned in time, and system stability can be ensured. At the same time, based on multidimensional regression analysis and autoencoder models, the threshold can be dynamically adjusted and the data gradient baseline can be optimized, thereby improving the accuracy of abnormal detection. In addition, machine learning technology is used to predict and optimize the energy consumption of the data center in real time, so as to realize intelligent allocation of energy and energy saving and consumption reduction. Overall, the present invention provides a flexible, efficient and accurate solution, which effectively improves the operating efficiency, resource utilization and security of the data center, and provides strong support for the intelligent management of the data center.

[0031] Embodiment 2

[0032] In this embodiment, the mountain ecological data is managed by the data center based on the TensorFlow ecology.

[0033] With the rapid development of information technology and the Internet of Things, mountain ecological environment monitoring is gradually moving towards data multi-source and real-time. Traditional mountain ecological data management systems mostly rely on a single data source or use simple statistical analysis methods. It is difficult to fully integrate various data such as historical geography, climate, soil, vegetation, topography, and spatial remote sensing, resulting in great limitations in spatiotemporal alignment, data cleaning, and normalization. On the other hand, mountainous areas have complex terrain and changeable climate, and their ecosystems show highly dynamic and nonlinear characteristics. Traditional methods are difficult to meet actual needs in identifying ecological anomalies and predicting ecological changes. At the same time, the emergence of deep learning platforms such as TensorFlow has provided new opportunities for large-scale data fusion and high-precision modeling, but the current research on its application in the construction of mountain ecological data centers, real-time data processing, and anomaly detection is still insufficient. Therefore, there is an urgent need for a data center data management method and system based on the TensorFlow ecosystem, which integrates multi-source heterogeneous data, realizes precise preprocessing, and efficient data fusion, so as to build an intelligent platform that can dynamically monitor and predict abnormal changes in mountain ecology.

[0034] As an embodiment of the present invention, refer to Figure 1 S1 in which historical multi-source heterogeneous data and real-time multi-source heterogeneous data in the mountain ecological area are collected and preprocessed to obtain first historical multi-source data and first real-time multi-source data; The historical multi-source heterogeneous data includes historical geographic data, historical climate data, historical soil data, historical vegetation data, historical terrain data and historical spatial remote sensing data; the real-time multi-source heterogeneous data includes real-time geographic data, real-time climate data, real-time soil data, real-time vegetation data, real-time terrain data and real-time spatial remote sensing data; the preprocessing includes cleaning, normalization and spatiotemporal alignment processing; wherein the preprocessing utilizes the data processing components in the TensorFlow ecosystem to construct a data flow graph to achieve efficient parallel processing and conversion of multi-source heterogeneous data.

[0035] Multi-source heterogeneous data includes multiple levels of space, time, and ecological environment. The main characteristic dimensions are as follows: Climate data: temperature, humidity, precipitation, wind speed; Soil data: soil moisture, soil nutrients (such as nitrogen, phosphorus, potassium content) and soil type; Vegetation data: vegetation index (NDVI), species diversity and vegetation coverage; Terrain data: altitude, slope, aspect (the angle of the slope toward the sun); Spatial remote sensing data: high-resolution remote sensing image features, LiDAR point cloud data features, and drone video data features (obtained through deep learning feature extraction); As an embodiment of the present invention, refer to Figure 1 S2, constructing a mountain zoning management model to analyze the first historical multi-source data based on the altitude vertical zoning to obtain mountain vertical zoning results; generating a gradient baseline through the mountain vertical zoning results; The mountain zoning management model includes a vertical zone zoning layer, a vertical zone feature extraction layer and an ecological gradient baseline generation layer; The vertical zone partition layer generates preliminary mountain partitions by clustering the historical terrain data and historical geographic data in the first historical multi-source data; performs feature correlation analysis on the historical geographic data, historical climate data, historical soil data, historical vegetation data, historical terrain data and historical spatial remote sensing data to obtain first ecological variable features; wherein the first ecological variable features include first features, second features, third features and fourth features; The first feature is strongly correlated with temperature, humidity, and precipitation, reflecting the climate gradient; The second feature is strongly correlated with altitude, slope, and aspect, reflecting the topographic gradient; The third characteristic is strongly correlated with NDVI and vegetation coverage, reflecting the ecological characteristics of vegetation; The fourth characteristic is strongly correlated with soil moisture and nutrient content, reflecting the soil environment.

[0036] The data after dimensionality reduction can better highlight the significant ecological differences between different regions, reduce the interference of noise and redundant features, and thus provide a clearer feature space for hierarchical partitioning.

[0037] The preliminary mountain zoning is revised based on the first ecological variable characteristics to obtain the mountain vertical zone zoning results; the mountain vertical zone zoning results include each zone containing its specific ecological characteristics (for example, temperature and humidity distribution patterns) and spatial boundaries. The spatial features are extracted from high-resolution remote sensing images and drone video data through convolutional neural networks; the dynamic change trends of temperature, humidity, etc. are analyzed from the time dimension through time series analysis; and the features that have the most significant impact on ecological gradient changes are screened out through feature selection algorithms.

[0038] The vertical zone feature extraction layer extracts features from the first ecological variable features and the mountain vertical zone partition results to obtain an ecological feature vector; The ecological gradient baseline generation layer obtains a preliminary theoretical ecological baseline by performing multidimensional regression analysis on the ecological feature vector and the first real-time multi-source data; nonlinear mapping is performed on the ecological feature vector through an autoencoder model to extract potential features in the ecosystem, and the original ecological gradient data is reconstructed according to the learned potential features to generate a gradient baseline, wherein the effectiveness complementary relationship between the multidimensional regression analysis and the autoencoder is shown in Table 1; As an embodiment of the present invention, refer to Figure 1 S3 in which a real-time ecological gradient baseline is calculated based on the first real-time multi-source data; a reconstruction error of the ecological gradient baseline is calculated by analyzing the ecological gradient baseline and the real-time ecological gradient baseline; the reconstruction error is updated based on the real-time ecological gradient baseline, and a threshold is dynamically adjusted in real time to obtain an abnormal ecological gradient;

[0039] The abnormal ecological gradient generation step comprises: Real-time ecological gradient baseline calculation, based on the first real-time multi-source data, combined with the mountain vertical zone zoning results and ecological feature vectors, using multidimensional regression analysis and autoencoder model to perform feature learning and reconstruction on the real-time collected ecological data to generate a real-time ecological gradient baseline; The real-time ecological gradient baseline calculation process is as follows: Based on the first real-time multi-source heterogeneous data Multidimensional ecological gradient baseline generated from historical data , using multidimensional regression analysis to establish a prediction model and calculate the real-time ecological gradient baseline , the calculation formula is as follows: ; in, is the real-time ecological gradient baseline, is an activation function to enhance nonlinear mapping capabilities. is the bias term, is the regression coefficient, For the feature, is the number of features; in this embodiment is 4; Reconstruction error calculation, using an autoencoder to perform nonlinear mapping and reconstruction on the real-time ecological gradient baseline, obtain a reconstructed ecological gradient baseline, and calculate the reconstruction error; The specific process of reconstructing the error is: Use autoencoder to learn the potential features of ecological feature vectors, and reconstruct the reconstructed ecological baseline ; The reconstruction process is: ; in, For the encoder, For the decoder; By reconstructing the ecological baseline and real-time ecological gradient baseline Calculating reconstruction error , the specific calculation formula is: ; Abnormal ecological gradient identification, updating the reconstruction error based on the real-time ecological gradient baseline, dynamically adjusting the preset threshold in real time, and when the reconstruction error is greater than the preset threshold, the abnormal ecological gradient is determined to be abnormal; The abnormal ecological gradient identification process is: The reconstruction error With dynamic threshold Compare and adjust thresholds in real time based on the latest data updates: ; in, is the mean of the historical reconstruction error, is the preset sensitivity factor, is the standard deviation of the historical reconstruction error; An abnormal ecological gradient is determined when the following conditions are met: ; At this point, the data is marked as an abnormal ecological gradient and input into the anomaly analysis management model to analyze the specific abnormal factors.

[0040] As an embodiment of the present invention, refer to Figure 1S4, constructing an abnormal analysis management model to analyze the abnormal ecological gradient and obtain a mountain ecological abnormality index vector; obtaining specific abnormal factors that cause the abnormality based on the analysis of the mountain ecological abnormality index vector; and evaluating and warning the abnormal factors; The anomaly analysis management model includes a mountain ecological anomaly index vector acquisition layer and an anomaly factor acquisition layer; The mountain ecological anomaly index vector acquisition layer compares the abnormal ecological gradient with the ecological gradient baseline generated based on historical multi-source data, calculates the deviation of each ecological variable, and calculates the mountain ecological anomaly index vector in combination with the feature weight; The abnormal factor acquisition layer attributes the input mountain ecological abnormality index vector to ecological factors by using principal component regression analysis to obtain abnormal factors.

[0041] The specific formula for evaluating the abnormal factors is: ; ; in, For each intervention program, For the plan The cost, For the plan The estimated reduction in abnormal risk, is the cost-estimate trade-off coefficient, It is the average reconstruction error between the theoretical ecological gradient baseline generated based on historical multi-source data and the real-time ecological gradient baseline without intervention, reflecting the initial ecological abnormality risk. To apply the intervention plan After that, the reconstruction error reduction value measured by reconstructing the ecological characteristics through the autoencoder is When the objective function is minimized The variable value of .

[0042] This formula quantitatively reflects the solution The relative improvement effect in reducing the risk of ecological abnormalities, the larger the value, the more significant the intervention effect.

[0043] This invention proposes a data center data management method based on the TensorFlow ecosystem, aiming to improve the accuracy and efficiency of mountain ecological environment monitoring. By collecting and preprocessing historical and real-time multi-source heterogeneous data, a mountain zoning management model is constructed to achieve accurate characterization of ecological gradients. Using multidimensional regression analysis and autoencoder models, real-time ecological gradient baselines are dynamically generated to detect abnormal ecological gradients in a timely manner. Furthermore, an anomaly analysis management model is used to deeply analyze abnormal indicators, determine specific abnormal factors, and provide effective intervention plans through evaluation and alarm mechanisms. This method makes full use of the powerful computing power and flexibility of TensorFlow to achieve all-round, real-time monitoring and management of mountain ecosystems, providing a scientific basis for ecological protection and resource management. For the overall process, please refer to Figure 3 , see Table 2 for specific data.

[0044]

[0045] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A data center data management method based on the TensorFlow ecosystem, characterized in that: Construct a digital management platform, the digital management platform includes: S1. Collect historical multi-source heterogeneous data and real-time multi-source heterogeneous data in the data center and pre-process them to obtain first historical multi-source data and first real-time multi-source data; S2. Constructing a data partition management model to analyze the first historical multi-source data to obtain data partition results; generating a gradient baseline based on the data partition results; S3. Calculate the real-time gradient baseline based on the first real-time multi-source data; obtain the reconstruction error of the gradient baseline by calculating the gradient baseline and the real-time gradient baseline; update the reconstruction error based on the real-time gradient baseline, dynamically adjust the threshold in real time, and obtain the abnormal TensorFlow ecological gradient; S4. Construct an exception analysis management model to analyze the abnormal TensorFlow ecological gradient and obtain the TensorFlow ecological abnormality indicator vector; based on the analysis of the TensorFlow ecological abnormality indicator vector, obtain the specific abnormal factors that cause the abnormality; evaluate and alarm the abnormal factors.

2. The data center data management method based on TensorFlow ecology according to claim 1 is characterized in that: The historical multi-source heterogeneous data includes historical server performance indicators, historical network traffic data, historical application logs and historical device status data; the real-time multi-source heterogeneous data includes real-time server performance indicators, real-time network traffic data, real-time application logs and real-time device status data; the pre-processing includes cleaning, normalization and spatiotemporal alignment processing; The preprocessing uses the data processing components in the TensorFlow ecosystem to build a data flow graph to achieve efficient parallel processing and conversion of multi-source heterogeneous data.

3. The data center data management method based on TensorFlow ecology according to claim 1 is characterized in that: The data partition management model includes a data partition layer, a data feature extraction layer and a gradient baseline generation layer; The data partitioning layer generates preliminary data partitions by performing cluster analysis on historical server performance indicators and historical application logs in the first historical multi-source data; performs feature correlation analysis on the historical application logs, historical network traffic data, historical application logs and historical device status data to obtain the first TensorFlow ecological variable features; modifies the preliminary data partitions based on the first TensorFlow ecological variable features to obtain data partitioning results; the data feature extraction layer obtains the TensorFlow ecological feature vector by performing feature extraction on the first TensorFlow ecological variable features and the data partitioning results; the gradient baseline generation layer obtains a preliminary theoretical TensorFlow ecological baseline by performing multidimensional regression analysis on the TensorFlow ecological feature vector and the first real-time multi-source data; nonlinearly maps the TensorFlow ecological feature vector through an autoencoder model to extract potential features in the TensorFlow ecosystem, reconstructs the original TensorFlow ecological gradient data based on the learned potential features, and generates a gradient baseline.

4. The data center data management method based on TensorFlow ecology according to claim 1 is characterized in that: The abnormal TensorFlow ecological gradient generation steps include: Real-time gradient baseline calculation, based on the first real-time multi-source data, combined with the data partitioning results and the TensorFlow ecological feature vector, uses multidimensional regression analysis and autoencoder models to perform feature learning and reconstruction on the real-time collected TensorFlow ecological data to generate a real-time gradient baseline; reconstruction error calculation, uses the autoencoder to perform nonlinear mapping and reconstruction on the real-time gradient baseline to obtain a reconstructed gradient baseline, and calculates the reconstruction error; abnormal TensorFlow ecological gradient identification, updates the reconstruction error based on the real-time gradient baseline, and dynamically adjusts the preset threshold in real time. When the reconstruction error is greater than the preset threshold, the abnormal TensorFlow ecological gradient is judged to be abnormal.

5. The data center data management method based on TensorFlow ecology according to claim 1, characterized in that: The anomaly analysis management model includes a TensorFlow ecological anomaly indicator vector acquisition layer and an anomaly factor acquisition layer; The TensorFlow ecological anomaly index vector acquisition layer compares the abnormal TensorFlow ecological gradient with the gradient baseline generated based on historical multi-source data, calculates the deviation of each variable, and calculates the TensorFlow ecological anomaly index vector in combination with the feature weight; The abnormal factor acquisition layer attributes the ecological factors to the input TensorFlow ecological abnormality index vector using principal component regression analysis to obtain abnormal factors.

6. The data center data management method based on TensorFlow ecology according to claim 1 is characterized in that: The specific formula for evaluating the abnormal factors is: ; in, For each intervention program, For the plan The cost, For the plan The estimated reduction in abnormal risk, is the cost-estimate trade-off coefficient, When the objective function is minimized The variable value of .

7. Data center data management system based on TensorFlow ecosystem, characterized by: include: A data collection module is used to collect historical multi-source heterogeneous data and real-time multi-source heterogeneous data in the data center and perform preprocessing to obtain first historical multi-source data and first real-time multi-source data; A gradient baseline acquisition module is used to construct a data partition management model to analyze the first historical multi-source data and obtain a data partition result; Generate a gradient baseline through the data partitioning result; An abnormal TensorFlow ecological gradient acquisition module is used to calculate the real-time gradient baseline based on the first real-time multi-source data; obtain the reconstruction error of the gradient baseline by calculating the gradient baseline and the real-time gradient baseline; update the reconstruction error based on the real-time gradient baseline, dynamically adjust the threshold in real time, and obtain the abnormal TensorFlow ecological gradient; The abnormal factor acquisition module is used to build an abnormal analysis management model to analyze the abnormal TensorFlow ecological gradient and obtain the TensorFlow ecological abnormal indicator vector; based on the analysis of the TensorFlow ecological abnormal indicator vector, the specific abnormal factors that cause the abnormality are obtained; and the abnormal factors are evaluated and alarmed.

Images