Intelligent evaluation and dynamic deployment management system for big data center construction

Abstract

The invention discloses an intelligent evaluation and dynamic deployment management system for big data center construction, and relates to the technical field of big data centers, and the system comprises an intelligent scheduling module which is used for resource transfer when big data center equipment breaks down; the intelligent scheduling module comprises a resource evaluation unit, a resource scheduling unit and a resource execution unit, the resource evaluation unit is connected with the resource scheduling unit through a communication signal, and the resource scheduling unit is connected with the resource execution unit through a communication signal. By designing the intelligent scheduling module, a resource transfer function when the big data center equipment fails is realized, the problem of service interruption caused by the failure of the big data center equipment is solved, and the stability and reliability of the system are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of big data centers, and in particular to an intelligent evaluation and dynamic allocation management system for the construction of a big data center. Background Art

[0002] With the rapid development of information technology, especially the development of emerging technologies such as artificial intelligence, cloud computing, and big data, global companies have begun to embark on the road of digital transformation. Among them, the construction of big data centers has become an indispensable pillar. Big data centers are not only responsible for the important tasks of data storage, processing, and analysis, but also provide companies with powerful computing and communication capabilities.

[0003] Traditional big data centers lack intelligent scheduling mechanisms and cannot flexibly respond to emergencies such as equipment failures or changes in resource demand. It is difficult to automatically trigger resource transfer and reallocation processes when equipment fails, resulting in some resources being idle, thereby reducing the resource utilization efficiency of the entire data center. Therefore, an intelligent evaluation and dynamic allocation management system for the construction of big data centers is extremely important.

[0004] Patent document CN116029701B discloses a method, system, device and electronic equipment for energy consumption assessment of a big data center. The above patent realizes the resource demand level reflecting multiple working conditions, multiple scenarios, different clean energy utilization rates, etc. of the big data center from a multi-dimensional and full-scenario perspective, but the above patent cannot realize the resource transfer function when the big data center equipment fails.

[0005] Patent document CN110673860B discloses a system and method for the construction and operation deployment of an intelligent cloud big data center. The above patent achieves the effect of intensive storage resources, but the above patent cannot realize the early warning function when the big data center is abnormal.

[0006] Patent document CN117236778B discloses an intelligent settlement management system for flexible employment based on big data. The above patent realizes direct adaptive matching of high-quality user terminals and assigns work tasks to them, but the above patent cannot realize the performance evaluation function after the transfer of big data center resources.

[0007] Patent document CN107301048B discloses an internal control management system with an application-responsive shared application architecture. The above patent realizes unified resource pooling, and the platform resource pool is shared among various government departments to maximize the use of platform resources. However, the above patent cannot realize the data backup function of the big data center.

[0008] In summary, the above patents cannot realize the resource transfer function when the big data center equipment fails, cannot realize the early warning function when the big data center is abnormal, cannot realize the performance evaluation function after the big data center resource transfer, and cannot realize the data backup function of the big data center, resulting in service interruption and data loss when the big data center equipment fails, the system administrator cannot discover the problem in time, and the resource utilization is inefficient after the resource transfer; To this end, this application proposes an intelligent evaluation and dynamic allocation management system for the construction of a big data center that realizes the resource transfer function when the big data center equipment fails, realizes the early warning function when the big data center is abnormal, realizes the performance evaluation function after the big data center resource transfer, and realizes the data backup function of the big data center. Summary of the invention

[0009] The purpose of the present invention is to provide an intelligent evaluation and dynamic allocation management system for the construction of a big data center, so as to solve the technical problems proposed in the above background technology that the resource transfer function when the big data center equipment fails, the early warning function when the big data center is abnormal, the performance evaluation function after the big data center resource transfer, and the data backup function of the big data center cannot be realized, resulting in service interruption and data loss when the big data center equipment fails, the system administrator cannot discover the problem in time, and the resource utilization is inefficient after the resource transfer.

[0010] To achieve the above-mentioned object, the present invention provides the following technical solutions: an intelligent evaluation and dynamic allocation management system for the construction of a big data center, comprising an intelligent scheduling module, the intelligent scheduling module being used for resource transfer when a failure occurs in a big data center device; The intelligent scheduling module includes: a resource evaluation unit, a resource scheduling unit and a resource execution unit, wherein the resource evaluation unit is connected to the resource scheduling unit via a communication signal, and the resource scheduling unit is connected to the resource execution unit via a communication signal; In the event of a large data center equipment failure, the resource evaluation unit evaluates the impact of the failed equipment on system tasks and system performance, determines the type and quantity of affected resources, analyzes the performance changes of the system after the resources are transferred, and generates a resource evaluation result; The resource scheduling unit selects appropriate resource transfer targets according to the resource evaluation results, identifies idle devices in the big data center, formulates resource transfer strategies, and determines the type, quantity, and sequence of resources to be transferred; The resource execution unit transfers the affected resources from the faulty device to the target device according to the resource transfer strategy, generates a resource transfer report after the resources are completed, records the process and results of the resource transfer, and feeds back the resource transfer report to the system administrator.

[0011] Preferably, the intelligent scheduling module is connected to a monitoring and early warning module through a communication signal, and the monitoring and early warning module is used to monitor various indicators of the big data center in real time and issue an early warning in time when an abnormal situation occurs; The monitoring and early warning module includes: a data acquisition unit, a monitoring and analysis unit and an early warning trigger unit, the data acquisition unit is connected to the monitoring and analysis unit through a communication signal, and the monitoring and analysis unit is connected to the early warning trigger unit through a communication signal; The data collection unit collects real-time operation data from the servers, storage devices and network devices of the big data center through the API interface, integrates and standardizes the operation data, and establishes the correlation between the data; The monitoring and analysis unit performs real-time analysis on the processed data, identifies the performance indicators of the equipment and the abnormal conditions of the equipment status, determines the location, type and number of abnormal equipment, analyzes the severity of the abnormal conditions, and generates abnormal signals; After receiving the abnormal signal, the early warning trigger unit controls the transfer of resources of the abnormal equipment and triggers the early warning mechanism at the same time, and promptly notifies the system administrator of the early warning information via SMS, email and APP.

[0012] Preferably, the intelligent scheduling module is connected to a verification and optimization module via a communication signal, the verification and optimization module is connected to a monitoring and early warning module via a communication signal, and the verification and optimization module is used to verify the result of resource transfer and optimize and adjust the resource transfer; The verification and optimization module includes: a resource verification unit, a performance evaluation unit and an optimization execution unit, the resource verification unit is connected to the performance evaluation unit through a communication signal, and the performance evaluation unit is connected to the optimization execution unit through a communication signal; After the resource transfer is completed, the resource verification unit checks the resource transfer status, compares the data between the source device and the target device, analyzes the synchronization status of the data between the source device and the target device, and verifies the resource transfer result; The performance evaluation unit collects the real-time operation data of the big data center after the resource transfer is completed, analyzes the various performance indicators of the big data center, evaluates the current performance of the system, and compares the current performance of the system with the expected results; The optimization execution unit formulates optimization strategies based on the results of performance evaluation, system load and resource usage, and automatically adjusts the allocation of computing resources, storage resources and network resources in the big data center.

[0013] Preferably, the intelligent scheduling module is connected to the data storage module via a communication signal, and the monitoring and early warning module is connected to the data storage module via a communication signal, and the data storage module is used for data backup; The data storage module includes: a data compression unit, a data backup unit and an encryption storage unit, the data compression unit is connected to the data backup unit via a communication signal, and the data backup unit is connected to the encryption storage unit via a communication signal; The data compression unit receives the real-time operation data of the big data center, identifies the type of data, and compresses the data using a compression algorithm; The data backup unit receives the compressed data, formulates and adjusts the data backup strategy according to business requirements, data importance and data change frequency, backs up the data according to the backup strategy, and verifies the backed-up data after the backup is completed; The encryption storage unit generates an encryption key, uses an encryption algorithm to encrypt the backed-up data, stores the encrypted data on multiple nodes of the distributed storage system, and regularly cleans up expired data based on the retention period of the backup policy.

[0014] Preferably, the intelligent scheduling module is connected to the visualization module via a communication signal, and the monitoring and early warning module is connected to the visualization module via a communication signal, and the visualization module is used to display the operation data of the big data center; The visualization module includes: a model construction unit, a data display unit and a user verification unit, the model construction unit is connected to the data display unit by communication signals, and the data display unit is connected to the user verification unit by communication signals; The model building unit collects the real-time operation data of the big data center, builds the digital twin model of the big data center through the real-time operation data, and transmits and updates the operation data to the digital twin model in real time; The data display unit uses 3D technology to build a three-dimensional model of the digital twin model, displays the operating status and key indicator data of the digital twin model through three-dimensional modeling, and provides users with an entry for query, screening and detection through a visual interface; The user verification unit sets different access rights according to the sensitivity and importance of the data, defines different permission levels according to the user's responsibilities, distributes the corresponding permissions to users, verifies the user's identity through a visual interface, and records the user's access events.

[0015] Preferably, the monitoring and early warning module is connected to a resource priority module via a communication signal, and the resource priority module is used to analyze and determine the priority of resources and guide the priority allocation and transfer of resources; The resource priority module includes: a priority analysis unit, a priority setting unit and a strategy generation unit, the priority analysis unit is connected to the priority setting unit through a communication signal, and the priority setting unit is connected to the strategy generation unit through a communication signal; The priority analysis unit collects resource usage data in the big data center, analyzes resource usage frequency, peak and trough distribution, historical usage trends, and analyzes resource usage frequency and load in different time periods; The priority setting unit sets resources to different priorities based on the results of resource analysis, and sets resource priorities for different time periods based on the distribution of resource peak and trough periods; The policy generation unit generates a policy for preferentially allocating and transferring high-priority resources according to resource priorities when a device in a large data center fails.

[0016] Preferably, the selection of the resource transfer target includes the following steps: Step 1: Analyze the compatibility between the target device and the source device to ensure that the resources will not be changed after the transfer; Step 2: Detect the status of the target device, confirm that the target device is normal and available, and analyze the remaining computing, storage, and network resources of the target device to meet the resource requirements that need to be transferred; Step 3: Predict the impact of resource transfer on target device performance.

[0017] Preferably, the monitoring and early warning module is connected to a load balancing module via a communication signal. The load balancing module analyzes the load conditions, performance indicators and network status of each server in the big data center according to the real-time operation data of the big data center, and dynamically allocates the workload to different servers.

[0018] Preferably, the data storage module is connected to a backup recovery module via a communication signal, and the backup recovery module is connected to an intelligent scheduling module via a communication signal. When equipment in the large data center fails, the backup recovery module restores system data and verifies the restored data.

[0019] Preferably, the visualization interface in the data display unit displays big data center equipment failure information and resource transfer reports to system administrators.

[0020] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention is designed with an intelligent scheduling module to realize the resource transfer function when the big data center equipment fails, solve the problem of service interruption caused by big data center equipment failure, and improve the stability and reliability of the system; 2. The present invention is designed with a monitoring and early warning module to realize the early warning function when the big data center is abnormal, realize the real-time collection of equipment operation data in the big data center, and improve the overall stability of the big data center; 3. The present invention is designed with a verification and optimization module to realize the performance evaluation function after the big data center resource transfer, realize the optimal allocation of resources, improve resource utilization, and solve the problem of service interruption or performance degradation caused by resource transfer; 4. The present invention realizes the data backup function of the big data center by designing a data storage module, solves the problem of data loss when the big data center equipment fails, ensures the security and integrity of the data, and improves the operating efficiency of the data center. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a schematic diagram of the intelligent scheduling module of the present invention; Figure 2 This is a schematic diagram of the monitoring and early warning module of the present invention; Figure 3 It is a schematic diagram of the verification optimization module of the present invention; Figure 4 A schematic diagram of a data storage module of the present invention; Figure 5 It is a schematic diagram of a visualization module of the present invention; Figure 6 It is a schematic diagram of a resource priority module of the present invention; Figure 7 It is a schematic diagram of the working process of the intelligent scheduling module of the present invention; Figure 8 Schematic diagram of the system workflow of the present invention. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] Example 1, please refer to Figure 1 , Figure 2 , Figure 7 and Figure 8 , an intelligent evaluation and dynamic allocation management system for the construction of a big data center, including an intelligent scheduling module, the intelligent scheduling module is used for resource transfer when a big data center device fails; the intelligent scheduling module includes: a resource evaluation unit, a resource scheduling unit and a resource execution unit, the resource evaluation unit is connected to the resource scheduling unit through a communication signal, and the resource scheduling unit is connected to the resource execution unit through a communication signal; In the event of a device failure in the big data center, the resource evaluation unit evaluates the impact of the failed device on the system tasks and system performance, determines the type and quantity of affected resources, analyzes the performance changes of the system after the resource transfer, and generates a resource evaluation result; the resource scheduling unit selects a suitable resource transfer target based on the resource evaluation result, identifies idle devices in the big data center, formulates a resource transfer strategy, and determines the type, quantity and sequence of resources to be transferred; the resource execution unit transfers the affected resources from the failed device to the target device based on the resource transfer strategy, generates a resource transfer report after the resources are completed, records the process and results of the resource transfer, and feeds back the resource transfer report to the system administrator; The intelligent dispatching module is connected to the monitoring and early warning module through communication signals. The monitoring and early warning module is used to monitor various indicators of the big data center in real time and issue early warnings in time when abnormal situations occur. The monitoring and early warning module includes: a data acquisition unit, a monitoring and analysis unit and an early warning trigger unit. The data acquisition unit is connected to the monitoring and analysis unit through communication signals, and the monitoring and analysis unit is connected to the early warning trigger unit through communication signals. The data collection unit collects real-time operation data from the servers, storage devices and network devices of the big data center through the API interface, integrates and standardizes the operation data, and establishes the correlation between the data; the monitoring and analysis unit analyzes the processed data in real time, identifies the performance indicators of the equipment and the abnormal conditions of the equipment status, determines the location, type and number of abnormal equipment, analyzes the severity of the abnormal conditions, and generates abnormal signals; after receiving the abnormal signal, the early warning trigger unit controls the transfer of resources of the abnormal equipment and triggers the early warning mechanism at the same time, and promptly notifies the system management personnel of the early warning information through SMS, email and APP; Furthermore, the data collection unit in the monitoring and early warning module collects real-time operation data from various devices in the big data center. The data collection unit integrates and standardizes the data to improve the accuracy of the data. When the monitoring and analysis unit identifies abnormal equipment performance indicators or equipment status through data analysis, the monitoring and analysis unit will generate an abnormal signal and transmit the abnormal signal to the early warning trigger unit. After receiving the abnormal signal, the early warning trigger unit will trigger the early warning mechanism and notify the system management personnel. At the same time, the early warning trigger unit will trigger the start of the intelligent scheduling module through a communication signal, so that the intelligent scheduling module can perform resource evaluation and resource scheduling processes, quickly locate the affected resources and formulate a transfer strategy to ensure that business continuity is not affected; The resource evaluation unit in the intelligent scheduling module receives the real-time operation data and abnormal analysis data collected by the monitoring and early warning module. The resource evaluation unit evaluates the faulty equipment. The resource scheduling unit formulates a resource transfer strategy based on the evaluation results, including selecting appropriate resource transfer targets, resource transfer quantities and transfer sequences, etc. The resource execution unit executes the resource transfer operation according to the strategy formulated by the resource scheduling unit to ensure that the affected resources are stably transferred from the faulty equipment to the target equipment. After the resource transfer is completed, the resource execution unit will generate a resource transfer report to record the transfer process and results. The resource execution unit will feed back the resource transfer report to the system management personnel and the monitoring and early warning module. The monitoring and early warning module records the resource transfer report to provide guidance for subsequent data analysis. At the same time, the monitoring and early warning module updates the system operation data in a timely manner, thereby continuously monitoring the system status after the resource transfer.

[0024] Example 2, please refer to Figure 1 , Figure 3 , Figure 7 and Figure 8 , an intelligent evaluation and dynamic allocation management system for the construction of a big data center, including an intelligent scheduling module, the intelligent scheduling module is used for resource transfer when a big data center device fails; the intelligent scheduling module includes: a resource evaluation unit, a resource scheduling unit and a resource execution unit, the resource evaluation unit is connected to the resource scheduling unit through a communication signal, and the resource scheduling unit is connected to the resource execution unit through a communication signal; In the event of a device failure in the big data center, the resource evaluation unit evaluates the impact of the failed device on the system tasks and system performance, determines the type and quantity of affected resources, analyzes the performance changes of the system after the resource transfer, and generates a resource evaluation result; the resource scheduling unit selects a suitable resource transfer target based on the resource evaluation result, identifies idle devices in the big data center, formulates a resource transfer strategy, and determines the type, quantity and sequence of resources to be transferred; the resource execution unit transfers the affected resources from the failed device to the target device based on the resource transfer strategy, generates a resource transfer report after the resources are completed, records the process and results of the resource transfer, and feeds back the resource transfer report to the system administrator; The intelligent scheduling module is connected to the verification and optimization module through communication signals, and the verification and optimization module is connected to the monitoring and early warning module through communication signals. The verification and optimization module is used to verify the result of resource transfer and optimize and adjust the resource transfer; the verification and optimization module includes: a resource verification unit, a performance evaluation unit and an optimization execution unit, the resource verification unit is connected to the performance evaluation unit through communication signals, and the performance evaluation unit is connected to the optimization execution unit through communication signals; After the resource transfer is completed, the resource verification unit checks the resource transfer status, compares the data between the resource device and the target device, analyzes the data synchronization between the resource device and the target device, and verifies the resource transfer result; the performance evaluation unit collects the real-time operation data of the big data center after the resource transfer is completed, analyzes the various performance indicators of the big data center, evaluates the current performance of the system, and compares the current performance of the system with the expected results; the optimization execution unit formulates an optimization strategy based on the results of the performance evaluation, the system load and the resource usage, and automatically adjusts the allocation of computing resources, storage resources and network resources of the big data center; Furthermore, after the resource execution unit in the intelligent scheduling module completes the transfer of resources from the faulty device to the target device, the resource execution unit transmits a signal of resource transfer completion to the verification optimization module. After the resource verification unit in the verification optimization module receives the signal, the resource verification unit starts to check the actual transfer of resources, compares whether the data between the resource device and the target device are consistent, and verifies the synchronization and integrity of the data, thereby ensuring the correctness and reliability of resource transfer; The performance evaluation unit analyzes various performance indicators of the big data center based on the information transmitted by the resource verification unit and the real-time operation data transmitted by the monitoring and early warning module. If the evaluation results show that the system's performance indicators do not meet expectations or there are problems such as incorrect resource allocation, the performance evaluation unit will pass the evaluation results to the optimization execution unit. The optimization execution unit will formulate and execute optimization strategies based on the evaluation results. The optimization execution unit automatically adjusts the allocation of computing resources, storage resources, and network resources in the big data center, thereby improving the system's performance indicators and optimizing resource utilization efficiency.

[0025] Embodiment 3, Figure 1 , Figure 4 , Figure 7 and Figure 8 , an intelligent evaluation and dynamic allocation management system for the construction of a big data center, including an intelligent scheduling module, the intelligent scheduling module is used for resource transfer when a big data center device fails; the intelligent scheduling module includes: a resource evaluation unit, a resource scheduling unit and a resource execution unit, the resource evaluation unit is connected to the resource scheduling unit through a communication signal, and the resource scheduling unit is connected to the resource execution unit through a communication signal; The intelligent scheduling module is connected to the data storage module through a communication signal, and the monitoring and early warning module is connected to the data storage module through a communication signal, and the data storage module is used for data backup; the data storage module includes: a data compression unit, a data backup unit and an encryption storage unit, the data compression unit is connected to the data backup unit through a communication signal, and the data backup unit is connected to the encryption storage unit through a communication signal; The data compression unit receives the real-time operation data of the big data center, identifies the type of data, and compresses the data using a compression algorithm; the data backup unit receives the compressed data, formulates and adjusts the data backup strategy according to business needs, data importance, and data change frequency, backs up the data according to the backup strategy, and verifies the backed-up data after the backup is completed; the encryption storage unit generates an encryption key, encrypts the backed-up data using an encryption algorithm, stores the encrypted data on multiple nodes of the distributed storage system, and regularly cleans up expired data according to the retention period of the backup strategy; The data storage module is connected to a backup recovery module via a communication signal, and the backup recovery module is connected to an intelligent scheduling module via a communication signal. When a device in a large data center fails, the backup recovery module restores the system data and verifies the restored data; Furthermore, when a large data center device fails, the intelligent scheduling module performs a resource transfer operation. In the process of transferring the affected resources from the failed device to the target device, the backup recovery module performs a data recovery task. The backup recovery module searches for relevant backup data from the data storage module and restores the backup data to the target device. During the recovery process, the backup recovery module verifies the recovered data to ensure the integrity and accuracy of the data, thereby avoiding the problem of business interruption caused by data loss or errors during resource transfer. During the resource transfer process, the data storage module ensures the security and privacy of the data and provides data backup support for the intelligent scheduling module.

[0026] Example 4, please refer to Figure 2 , Figure 4 , Figure 7 and Figure 8 , an intelligent evaluation and dynamic allocation management system for the construction of a big data center, wherein the intelligent scheduling module is connected to a monitoring and early warning module through a communication signal, and the monitoring and early warning module is used to monitor various indicators of the big data center in real time and issue an early warning in time when an abnormal situation occurs; the monitoring and early warning module includes: a data acquisition unit, a monitoring and analysis unit and an early warning trigger unit, the data acquisition unit is connected to the monitoring and analysis unit through a communication signal, and the monitoring and analysis unit is connected to the early warning trigger unit through a communication signal; The data collection unit collects real-time operation data from the servers, storage devices and network devices of the big data center through the API interface, integrates and standardizes the operation data, and establishes the correlation between the data; the monitoring and analysis unit analyzes the processed data in real time, identifies the performance indicators of the equipment and the abnormal conditions of the equipment status, determines the location, type and number of abnormal equipment, analyzes the severity of the abnormal conditions, and generates abnormal signals; after receiving the abnormal signal, the early warning trigger unit controls the transfer of resources of the abnormal equipment and triggers the early warning mechanism at the same time, and promptly notifies the system management personnel of the early warning information through SMS, email and APP; The intelligent scheduling module is connected to the data storage module through a communication signal, and the monitoring and early warning module is connected to the data storage module through a communication signal, and the data storage module is used for data backup; the data storage module includes: a data compression unit, a data backup unit and an encryption storage unit, the data compression unit is connected to the data backup unit through a communication signal, and the data backup unit is connected to the encryption storage unit through a communication signal; The data compression unit receives the real-time operation data of the big data center, identifies the type of data, and compresses the data using a compression algorithm; the data backup unit receives the compressed data, formulates and adjusts the data backup strategy according to business needs, data importance, and data change frequency, backs up the data according to the backup strategy, and verifies the backed-up data after the backup is completed; the encryption storage unit generates an encryption key, encrypts the backed-up data using an encryption algorithm, stores the encrypted data on multiple nodes of the distributed storage system, and regularly cleans up expired data according to the retention period of the backup strategy; Furthermore, the monitoring and early warning module monitors various performance indicators of the big data center in real time, and transmits the collected, integrated and standardized data to the data storage module in real time. After receiving the data, the data storage module compresses the data, thereby reducing the storage space occupied by the data and improving the transmission efficiency. The data storage module backs up the compressed data, and the data backup unit in the data storage module names the backup data, records the storage location, backup time and other information, and the encryption storage unit encrypts the data to prevent the leakage of the backup data during the recovery process. The monitoring and early warning module analyzes relevant data to discover or predict potential failures and safety hazards. When the monitoring and early warning module detects an abnormal situation, the monitoring and early warning module will send a signal to the data storage module. After receiving the signal, the data storage module will immediately back up the data to prevent data loss or damage, thereby ensuring that key data is backed up to a safe location before a failure occurs, avoiding data loss during resource transfer.

[0027] Example 5, please refer to Figure 1 , Figure 2 , Figure 5 , Figure 7 and Figure 8 , an intelligent evaluation and dynamic allocation management system for the construction of a big data center, including an intelligent scheduling module, the intelligent scheduling module is used for resource transfer when a big data center device fails; the intelligent scheduling module includes: a resource evaluation unit, a resource scheduling unit and a resource execution unit, the resource evaluation unit is connected to the resource scheduling unit through a communication signal, and the resource scheduling unit is connected to the resource execution unit through a communication signal; The intelligent scheduling module is connected to the monitoring and early warning module through communication signals. The monitoring and early warning module is used to monitor various indicators of the big data center in real time and issue early warnings in time when abnormal situations occur; the monitoring and early warning module includes: a data acquisition unit, a monitoring and analysis unit and an early warning trigger unit. The data acquisition unit is connected to the monitoring and analysis unit through communication signals, and the monitoring and analysis unit is connected to the early warning trigger unit through communication signals; the data acquisition unit collects real-time operation data from the server, storage device and network device of the big data center through an API interface, integrates and standardizes the operation data, and establishes a correlation relationship between the data; The intelligent scheduling module is connected to the visualization module through communication signals, and the monitoring and early warning module is connected to the visualization module through communication signals. The visualization module is used to display the operation data of the big data center; the visualization module includes: a model building unit, a data display unit and a user verification unit, the model building unit is connected to the data display unit by communication signals, and the data display unit is connected to the user verification unit by communication signals; The model building unit collects the real-time operation data of the big data center, builds the digital twin model of the big data center through the real-time operation data, and transmits and updates the operation data to the digital twin model in real time; the data display unit uses 3D technology to build a three-dimensional model of the digital twin model, and displays the operation status and key indicator data of the digital twin model through three-dimensional modeling, and provides users with query, screening and detection entrances through a visual interface; the user verification unit sets different access rights according to the sensitivity and importance of the data, defines different permission levels according to the user's responsibilities, distributes the corresponding permissions to users, verifies the user identity through a visual interface, and records the user's access events; Furthermore, the data acquisition unit and the monitoring and analysis unit in the monitoring and early warning module collect and analyze data. The monitoring and early warning module passes the data to the visualization module. The model building unit in the visualization module builds a digital twin model based on the real-time operation data. The model reflects the real-time operation status and key indicators of the big data center. The data display unit provides a three-dimensional visualization interface to intuitively display the operation status and various key indicators of the data center to the user, so that the user can intuitively view the various data of the system through the visualization interface of the data display unit. At the same time, system managers can understand the relevant information of the equipment failure of the big data center from the visualization interface, so as to solve the failure problem in time. The user verification unit sets different access rights according to the sensitivity and importance of the data, and sets different authority levels according to the user's responsibilities, thereby ensuring the security and privacy of the data and preventing unauthorized access.

[0028] Example 6, please refer to Figure 1 , Figure 2 , Figure 6 , Figure 7 and Figure 8 , an intelligent evaluation and dynamic allocation management system for the construction of a big data center, including an intelligent scheduling module, the intelligent scheduling module is used for resource transfer when a big data center device fails; the intelligent scheduling module includes: a resource evaluation unit, a resource scheduling unit and a resource execution unit, the resource evaluation unit is connected to the resource scheduling unit through a communication signal, and the resource scheduling unit is connected to the resource execution unit through a communication signal; The intelligent scheduling module is connected to the monitoring and early warning module through communication signals; the monitoring and early warning module includes: a data acquisition unit, a monitoring and analysis unit and an early warning trigger unit, the data acquisition unit is connected to the monitoring and analysis unit through communication signals, and the monitoring and analysis unit is connected to the early warning trigger unit through communication signals; the data acquisition unit collects real-time operation data from the server, storage device and network device of the big data center through the API interface, integrates and standardizes the operation data, and establishes the correlation relationship between the data; The monitoring and early warning module is connected to a resource priority module via a communication signal, and the resource priority module is used to analyze and determine the priority of resources and guide the priority allocation and transfer of resources; the resource priority module includes: a priority analysis unit, a priority setting unit and a strategy generation unit, the priority analysis unit is connected to the priority setting unit via a communication signal, and the priority setting unit is connected to the strategy generation unit via a communication signal; The priority analysis unit collects resource usage data in the big data center, analyzes resource usage frequency, peak and trough distribution, historical usage trends, and analyzes resource usage frequency and load conditions in different time periods; the priority setting unit sets resources to different priorities based on the results of resource analysis, and sets resource priorities in different time periods based on the distribution of resource peak and trough periods; the policy generation unit generates policies for preferential allocation and transfer of high-priority resources based on resource priorities when equipment in the big data center fails; Furthermore, the data acquisition unit in the monitoring and early warning module collects real-time operating data of the equipment in the big data center, and passes this data to the priority analysis unit of the resource priority module. The priority analysis unit analyzes key information such as the frequency of resource use, the distribution of peak and trough periods, and historical usage trends based on the data. The priority setting unit divides the resources into different priorities based on the analysis results of the priority analysis unit to ensure that the operation of key businesses or high-priority applications can be given priority when resources are tight. The policy generation unit automatically generates a policy for prioritizing the allocation and transfer of high-priority resources based on the resource priority set by the priority setting unit. The resource scheduling unit of the intelligent scheduling module receives the policy formulated by the resource priority module, and the resource scheduling unit further formulates a resource transfer policy in the event of equipment failure based on the policy formulated by the resource priority module.

[0029] Working principle: The monitoring and early warning module collects key data such as the load, performance indicators and network status of each server in the big data center in real time. The monitoring and early warning module analyzes the potential hidden dangers and abnormal information of the big data center equipment based on the data. After the monitoring and early warning module detects an abnormality, it notifies the data storage module to perform data backup and notifies the intelligent scheduling module to perform resource transfer; During the resource transfer process of the intelligent scheduling module, the resource priority module and the load balancing module determine the priority and load of resources by analyzing the data collected by the monitoring and early warning module. The intelligent scheduling module formulates corresponding resource transfer strategies based on the priority and load of resources, and the backup and recovery module restores the backup data in the data storage module to the target device; After the resource transfer is completed, the verification and optimization module evaluates the various performance indicators of the big data center. The verification and optimization module automatically adjusts the resource allocation according to the evaluation results. The visualization module builds a three-dimensional model of the digital twin model based on the big data center operation data collected in real time by the monitoring and early warning module, and displays it to the user through a visualization interface, so that the user can understand the real-time status of the big data center through the visualization interface, including key indicators, equipment status, abnormal information, etc.

[0030] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. An intelligent evaluation and dynamic allocation management system for the construction of a big data center, characterized by: It includes an intelligent scheduling module, which is used for resource transfer when a failure occurs in a large data center device; The intelligent scheduling module includes: a resource evaluation unit, a resource scheduling unit and a resource execution unit, wherein the resource evaluation unit is connected to the resource scheduling unit via a communication signal, and the resource scheduling unit is connected to the resource execution unit via a communication signal; In the event of a large data center equipment failure, the resource evaluation unit evaluates the impact of the failed equipment on system tasks and system performance, determines the type and quantity of affected resources, analyzes the performance changes of the system after the resources are transferred, and generates a resource evaluation result; The resource scheduling unit selects appropriate resource transfer targets according to the resource evaluation results, identifies idle devices in the big data center, formulates resource transfer strategies, and determines the type, quantity, and sequence of resources to be transferred; The resource execution unit transfers the affected resources from the faulty device to the target device according to the resource transfer strategy, generates a resource transfer report after the resources are completed, records the process and results of the resource transfer, and feeds back the resource transfer report to the system administrator.

2. According to claim 1, the intelligent evaluation and dynamic allocation management system for the construction of a big data center is characterized by: The intelligent scheduling module is connected to the monitoring and early warning module through communication signals. The monitoring and early warning module is used to monitor various indicators of the big data center in real time and issue early warnings in time when abnormal situations occur; The monitoring and early warning module includes: a data acquisition unit, a monitoring and analysis unit and an early warning trigger unit, the data acquisition unit is connected to the monitoring and analysis unit through a communication signal, and the monitoring and analysis unit is connected to the early warning trigger unit through a communication signal; The data collection unit collects real-time operation data from the servers, storage devices and network devices of the big data center through the API interface, integrates and standardizes the operation data, and establishes the correlation between the data; The monitoring and analysis unit performs real-time analysis on the processed data, identifies the performance indicators of the equipment and the abnormal conditions of the equipment status, determines the location, type and number of abnormal equipment, analyzes the severity of the abnormal conditions, and generates abnormal signals; After receiving the abnormal signal, the early warning trigger unit controls the transfer of resources of the abnormal equipment and triggers the early warning mechanism at the same time, and promptly notifies the system administrator of the early warning information via SMS, email and APP.

3. According to claim 1, the intelligent evaluation and dynamic allocation management system for the construction of a big data center is characterized by: The intelligent scheduling module is connected to the verification and optimization module through a communication signal, and the verification and optimization module is connected to the monitoring and early warning module through a communication signal. The verification and optimization module is used to verify the result of resource transfer and optimize and adjust the resource transfer; The verification and optimization module includes: a resource verification unit, a performance evaluation unit and an optimization execution unit, the resource verification unit is connected to the performance evaluation unit through a communication signal, and the performance evaluation unit is connected to the optimization execution unit through a communication signal; After the resource transfer is completed, the resource verification unit checks the resource transfer status, compares the data between the source device and the target device, analyzes the synchronization status of the data between the source device and the target device, and verifies the resource transfer result; The performance evaluation unit collects the real-time operation data of the big data center after the resource transfer is completed, analyzes the various performance indicators of the big data center, evaluates the current performance of the system, and compares the current performance of the system with the expected results; The optimization execution unit formulates optimization strategies based on the results of performance evaluation, system load and resource usage, and automatically adjusts the allocation of computing resources, storage resources and network resources in the big data center.

4. According to claim 1, the intelligent evaluation and dynamic allocation management system for the construction of a big data center is characterized by: The intelligent scheduling module is connected to the data storage module through communication signals, and the monitoring and early warning module is connected to the data storage module through communication signals, and the data storage module is used for data backup; The data storage module includes: a data compression unit, a data backup unit and an encryption storage unit, the data compression unit is connected to the data backup unit via a communication signal, and the data backup unit is connected to the encryption storage unit via a communication signal; The data compression unit receives the real-time operation data of the big data center, identifies the type of data, and compresses the data using a compression algorithm; The data backup unit receives the compressed data, formulates and adjusts the data backup strategy according to business requirements, data importance and data change frequency, backs up the data according to the backup strategy, and verifies the backed-up data after the backup is completed; The encryption storage unit generates an encryption key, uses an encryption algorithm to encrypt the backed-up data, stores the encrypted data on multiple nodes of the distributed storage system, and regularly cleans up expired data based on the retention period of the backup policy.

5. According to claim 1, the intelligent evaluation and dynamic allocation management system for the construction of a big data center is characterized by: The intelligent scheduling module is connected to the visualization module via a communication signal, and the monitoring and early warning module is connected to the visualization module via a communication signal, and the visualization module is used to display the operation data of the big data center; The visualization module includes: a model building unit, a data display unit and a user verification unit, the model building unit is connected to the data display unit by communication signals, and the data display unit is connected to the user verification unit by communication signals; The model building unit collects the real-time operating data of the big data center, builds the digital twin model of the big data center through the real-time operating data, and transmits and updates the operating data to the digital twin model in real time; The data display unit uses 3D technology to build a three-dimensional model of the digital twin model, displays the operating status and key indicator data of the digital twin model through three-dimensional modeling, and provides users with an entry for query, screening and detection through a visual interface; The user verification unit sets different access permissions based on the sensitivity and importance of the data, defines different permission levels based on the user's responsibilities, distributes the corresponding permissions to users, verifies user identities through a visual interface, and records user access events.

6. According to claim 2, the intelligent evaluation and dynamic allocation management system for the construction of a big data center is characterized by: The monitoring and early warning module is connected to a resource priority module via a communication signal, and the resource priority module is used to analyze and determine the priority of resources and guide the priority allocation and transfer of resources; The resource priority module includes: a priority analysis unit, a priority setting unit and a strategy generation unit, the priority analysis unit is connected to the priority setting unit through a communication signal, and the priority setting unit is connected to the strategy generation unit through a communication signal; The priority analysis unit collects resource usage data in the big data center, analyzes resource usage frequency, peak and trough distribution, historical usage trends, and analyzes resource usage frequency and load in different time periods; The priority setting unit sets resources to different priorities based on the results of resource analysis, and sets resource priorities for different time periods based on the distribution of resource peak and trough periods; The policy generation unit generates a policy for preferentially allocating and transferring high-priority resources according to resource priorities when a device in a large data center fails.

7. According to claim 1, the intelligent evaluation and dynamic allocation management system for the construction of a big data center is characterized by: The selection of the resource transfer target comprises the following steps: Step 1: Analyze the compatibility between the target device and the source device to ensure that the resources will not be changed after the transfer; Step 2: Detect the status of the target device, confirm that the target device is normal and available, and analyze the remaining computing, storage, and network resources of the target device to meet the resource requirements that need to be transferred; Step 3: Predict the impact of resource transfer on target device performance.

8. According to claim 2, the intelligent evaluation and dynamic allocation management system for the construction of a big data center is characterized by: The monitoring and early warning module is connected to the load balancing module through a communication signal. The load balancing module analyzes the load conditions, performance indicators and network status of each server in the big data center according to the real-time operation data of the big data center, and dynamically allocates the workload to different servers.

9. According to claim 4, the intelligent evaluation and dynamic allocation management system for the construction of a big data center is characterized by: The data storage module is connected to a backup recovery module via a communication signal, and the backup recovery module is connected to an intelligent scheduling module via a communication signal. When a device in a large data center fails, the backup recovery module recovers system data and verifies the recovered data.

10. The intelligent evaluation and dynamic allocation management system for the construction of a big data center according to claim 5, characterized in that: The visualization interface in the data display unit displays big data center equipment failure information and resource transfer reports to system administrators.

Images