Functional partitioning container-type data center and data processing method

Abstract

The present application provides a functional partitioned container-type data center and data processing method, which relates to the field of data center design technology, including: obtaining basic scale information, collecting basic equipment information, performing data mining, obtaining a configuration memory library, matching in the configuration memory library, obtaining a target configuration record, constructing a computing power-power source container mapping relationship, combining the number of computing power container configurations, obtaining the number of power source container configurations, laying out a topological network based on the computing power container and the number of power source container configurations, obtaining a power source container layout topological network, constructing a data center, and performing data processing. The present application can solve the technical problem in the prior art that the traditional data center lacks flexibility in configuration and expansion, which makes it difficult to adapt to the ever-changing data processing needs. Through data mining and configuration memory libraries, intelligent configuration of containers is realized, achieving the technical effect of improving the overall efficiency of the data center.

Description

Technical Field

[0001] The present application relates to the technical field of data center design, and in particular to a functionally partitioned containerized data center and a data processing method. Background Art

[0002] Containerized data centers are mobile, flexible and fast-deployable data centers that have been widely used. Containerized data centers integrate data center infrastructure such as servers, storage, network equipment, and cooling systems in a standard container. This design allows data centers to be quickly deployed at any location, and is particularly suitable for military, disaster recovery, temporary activities or remote areas. Existing containerized data centers mainly set up the power supply part and the data cabinet part in a container. Although this integrated design is easy to deploy, it has limitations in scalability and energy consumption management. At present, traditional data centers are usually relatively fixed in configuration and expansion, and it is difficult to quickly adapt to changes in business types, processing data volume, and response needs, resulting in waste of resources or insufficient processing capacity.

[0003] In summary, the prior art has a technical problem that traditional data centers lack flexibility in configuration and expansion, making it difficult to adapt to ever-changing data processing needs. Summary of the invention

[0004] The purpose of this application is to provide a functionally partitioned containerized data center and a data processing method to solve the technical problem in the prior art that traditional data centers lack flexibility in configuration and expansion, making it difficult to adapt to changing data processing needs.

[0005] In view of the above problems, the present application provides a functionally partitioned container-type data center and a data processing method.

[0006] In the first aspect, the present application provides a functionally partitioned container-type data center, wherein the functionally partitioned container-type data center includes: a basic scale information acquisition module, the basic scale information acquisition module is used to acquire the basic scale information of the target data center, wherein the basic scale information includes business type, processing data volume and response requirements; a basic equipment information collection module, the basic equipment information collection module is used to collect basic equipment information of the computing power container, wherein the basic equipment information includes computing capacity, storage capacity and network architecture; a data mining module, the data mining module is used to perform data mining with the computing capacity, storage capacity and network architecture as indexes to obtain a configuration memory library, wherein the configuration memory library includes N mining configuration records; a matching module, the matching module is used to mine with the business type , with the goal of processing data volume and responding to requirements, matching is performed among the N mining configuration records in the configuration memory to obtain a target configuration record, wherein the target configuration record includes the computing power container configuration quantity and the computing power container layout topology network; a mapping relationship construction module, the mapping relationship construction module is used to construct a computing power-power source container mapping relationship, and obtain the power source container configuration quantity in combination with the computing power container configuration quantity; a layout topology network determination module, the layout topology network determination module is used to obtain the power source container layout topology network based on the computing power container layout topology network and the power source container configuration quantity; a data processing module, the data processing module is used to construct a data center according to the computing power container layout topology network and the power source container layout topology network, and use the data center for data processing.

[0007] In a second aspect, the present application also provides a data processing method, wherein the data processing method includes: obtaining basic scale information of a target data center, wherein the basic scale information includes business type, processing data volume and response requirements; collecting basic equipment information of a computing power container, wherein the basic equipment information includes computing capacity, storage capacity and network architecture; performing data mining with the computing capacity, storage capacity and network architecture as indexes to obtain a configuration memory library, wherein the configuration memory library includes multiple configuration records; taking the business type, processing data volume and response requirements as targets, matching among multiple configuration records in the configuration memory library to obtain a target configuration record, wherein the target configuration record includes a computing power container configuration quantity and a computing power container layout topology network; constructing a computing power-power source container mapping relationship, and combining the computing power container configuration quantity to obtain the power source container configuration quantity; based on the computing power container layout topology network and the power source container configuration quantity, obtaining the power source container layout topology network; constructing a data center according to the computing power container layout topology network and the power source container layout topology network, and using the data center for data processing.

[0008] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0009] Through the basic scale information acquisition module, the basic scale information acquisition module is used to obtain the basic scale information of the target data center, wherein the basic scale information includes business type, processing data volume and response requirements; the basic equipment information acquisition module, the basic equipment information acquisition module is used to collect the basic equipment information of the computing power container, wherein the basic equipment information includes computing capacity, storage capacity and network architecture; the data mining module, the data mining module is used to perform data mining with the computing capacity, storage capacity and network architecture as indexes to obtain a configuration memory library, wherein the configuration memory library includes N mining configuration records; the matching module, the matching module is used to target the business type, processing data volume and response requirements, and in the configuration memory library N mining configuration records are matched to obtain the target configuration record, wherein the target configuration record includes the number of computing power container configurations and the computing power container layout topology network; a mapping relationship construction module, the mapping relationship construction module is used to construct a computing power-power source container mapping relationship, and obtain the number of power source container configurations in combination with the computing power container configuration number; a layout topology network determination module, the layout topology network determination module is used to obtain the power source container layout topology network based on the computing power container layout topology network and the power source container configuration number; a data processing module, the data processing module is used to construct a data center based on the computing power container layout topology network and the power source container layout topology network, and use the data center for data processing. In other words, through data mining and configuration memory, the intelligent configuration of containers is realized, and the technical effect of improving the overall efficiency of the data center is achieved.

[0010] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented according to the contents of the specification, and in order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are specifically cited below. It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easy to understand through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the present application or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0012] Figure 1 This is a structural diagram of a functionally partitioned containerized data center for this application;

[0013] Figure 2 This is a flowchart of a data processing method for this application.

[0014] Explanation of the reference numerals: basic scale information acquisition module 11, basic equipment information collection module 12, data mining module 13, matching module 14, mapping relationship construction module 15, topology network determination module 16, data processing module 17. DETAILED DESCRIPTION

[0015] This application provides a functional partitioned containerized data center and data processing method, which solves the technical problem in the prior art that traditional data centers lack flexibility in configuration and expansion, making it difficult to adapt to changing data processing needs. Through data mining and configuration memory, intelligent configuration of containers is achieved, achieving the technical effect of improving the overall efficiency of the data center.

[0016] Below, the technical solutions in the present application will be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments of the present application. It should be understood that the present application is not limited to the example embodiments described herein. Based on the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application. It should also be noted that, for the convenience of description, only the parts related to the present application are shown in the accompanying drawings, rather than all of them.

[0017] For example, please refer to the attached Figure 1 The present application provides a functionally zoned containerized data center, wherein the functionally zoned containerized data center comprises:

[0018] The basic scale information acquisition module 11 is used to acquire basic scale information of the target data center, wherein the basic scale information includes business type, processing data volume and response requirements.

[0019] Specifically, basic scale information of the target data center is collected, including business type, processing data volume, and response requirements. Business type is directly related to the type of data that the data center needs to process and the way the data is processed. For example, processing large-scale computing tasks requires more computing resources; processing big data analysis requires larger storage space and specific data analysis tools. Processing data volume refers to the amount of data that the data center needs to process, ranging from a few TB to several PB. There are significant differences in the demand for storage and computing resources, which determines the scale of the data center, including the required computing power, storage space, and network bandwidth. Response requirements involve performance indicators of the data center, such as processing speed, data transmission speed, and system availability. Some businesses may require low-latency real-time data processing, while other businesses may be able to tolerate higher latency. For example, consider a cloud computing service provider whose business types include providing virtual machines, cloud storage, and big data analysis services. The amount of data that the data center needs to process may be very large, including user data, application data, and log data. For such a data center, its response requirements may be very high, and it is necessary to ensure fast data processing and transmission speeds, as well as high availability and reliability. By obtaining this basic scale information, a clear direction can be provided for the design and configuration of the data center, helping to ensure that the resource configuration of the data center can meet actual business needs and avoid waste or shortage of resources.

[0020] The basic equipment information collection module 12 is used to collect the basic equipment information of the computing power container, wherein the basic equipment information includes computing capacity, storage capacity and network architecture.

[0021] Specifically, the computing resources, storage capacity and network architecture in the container are investigated and recorded in detail. Among them, the computing capacity refers to the sum of the computing resources in the container, including the number of CPUs, cores, GPUs, etc., which determines the processing capacity of the container, evaluates the performance of the server in the container and the number of concurrent tasks that can be processed. The storage capacity involves data storage capacity, including hard disk type, capacity and speed, which determines the data storage and processing capacity of the container. It is necessary to consider the type of storage (such as SSD, HDD), capacity, read and write speed, and data redundancy and backup strategy. The network architecture involves the network connection method inside and outside the container, including network bandwidth, latency and topology, which determines the efficiency of data transmission inside and between containers. Accurate collection of equipment information helps to optimize resource allocation and improve the overall performance of the data center.

[0022] The data mining module 13 is used to perform data mining with the computing amount, storage amount and network architecture as indexes to obtain a configuration memory library, wherein the configuration memory library includes N mining configuration records.

[0023] Specifically, using the initial weight ratio, data mining is performed on the computing amount, storage amount and network architecture to find the optimal configuration solution from historical data and existing configurations. The initial weight ratio is fine-tuned and optimized according to the preset adjustment method, and data mining is performed again to find the optimal configuration solution after adjusting the weight. The adjustment method of the new weight ratio is added to the banned list to avoid reusing the same adjustment method in the next iteration, thereby falling into the local optimal solution. During the data mining process, the system generates multiple configuration records, each of which contains a set of specific computing amount, storage amount and network architecture parameters, as well as related performance indicators and business requirements. These configuration records are collected to form a configuration memory. The configuration memory is a database that stores various configuration records, each of which represents the optimal configuration solution under a specific weight ratio. By storing and analyzing a large number of configuration records, the data center can quickly find the optimal configuration solution suitable for specific business needs, improving the deployment efficiency of the data center.

[0024] A matching module 14 is used to match the N mining configuration records in the configuration memory library with the business type, processing data volume and response requirements as the target, and obtain a target configuration record, wherein the target configuration record includes the number of computing power container configurations and the computing power container layout topology network.

[0025] Specifically, a matching space is constructed based on the N mining configuration records in the configuration memory, and the business type, processing data volume and corresponding requirements are input into the matching space to obtain the target point. The distance from each matching point to the target point is calculated, and the calculated results are sorted from small to large. The first k matching points that can meet the target requirements are extracted, and K matching mining configuration records are mapped to obtain the configuration scheme that performs best in meeting the target requirements. The mean of the K matching mining configuration records is calculated to obtain the target configuration record. The target configuration record is obtained by combining the advantages of the K matching mining configuration records and represents the average optimal configuration. The target configuration record includes the configuration number of computing power containers and the layout topology network of computing power containers. The configuration number of computing power containers represents the total amount of computing resources required by the data center, including the number of CPU cores and GPUs. The layout topology network of computing power containers describes the connection between these computing power containers, including network topology, data transmission path, etc. Match and determine the target configuration record to ensure that its configuration can meet key business requirements, such as processing speed, data storage capacity and system reliability.

[0026] The mapping relationship building module 15 is used to build a computing power-power source container mapping relationship, and obtain the power source container configuration quantity in combination with the computing power container configuration quantity.

[0027] Specifically, the smart computing power container is used to provide data computing power services. It is equipped with refrigeration equipment inside, which can intelligently adjust the temperature and humidity in the container. The power supply container is connected to the smart computing power container to provide power for the smart computing power container. At the same time, its box is also equipped with refrigeration equipment, which can intelligently adjust the temperature and humidity in the container. In the construction of the data center, the number of smart computing power containers can be determined according to the scale of the data center. If the data center is large, multiple power supply containers can also be set up, and each power supply container corresponds to multiple smart computing power containers. The boxes of the smart computing power container and the power supply container can be 20-foot standard containers or other standard containers. Determine the number of smart computing power containers required according to the scale and business needs of the data center. Evaluate the overall power demand based on the number of smart computing power containers and the power demand of each container. According to the power demand, construct a mapping relationship between the smart computing power container and the power supply container, and allocate one or more power supply containers to each smart computing power container to ensure power supply. Determine the number of power source containers required based on the configuration quantity and mapping relationship of the computing power container. Building a computing power-power source container mapping relationship ensures that the data center's power supply matches computing needs and improves the data center's power efficiency.

[0028] The topology network deployment determination module 16 is used to obtain the power source container deployment topology network based on the computing power container deployment topology network and the power source container configuration quantity.

[0029] Specifically, the computing power requirements of the data center are clarified, that is, the number and layout of the computing power containers. According to the number and layout of the computing power containers, the power requirements of each computing power container are evaluated. According to the power demand, the number and layout of the required power source containers are determined, and the needs for redundant and backup power supply are considered to ensure the stable operation of the data center. Based on the computing power container layout topology network and the number of power source containers, the power source container layout topology network is constructed, which describes the connection method and power supply path between the power source containers. Obtaining the power source container layout topology network ensures that the power supply of the data center matches the computing demand, improves the power efficiency of the data center, and reduces power waste.

[0030] The data processing module 17 is used to construct a data center according to the computing power container layout topology network and the power source container layout topology network, and use the data center to perform data processing.

[0031] Specifically, according to the topological network layout of the computing power container and the topological network layout of the power source container, design the physical layout of the data center, including the location and connection method of the computing power container and the power source container. According to the design layout, install the computing power container, the power source container and other related equipment, such as network equipment, cooling system, etc. Configure the network connection between the computing power containers, and the connection between the computing power containers and the external network. At the same time, configure the power supply path between the power source containers. Deploy data processing applications in the data center, such as big data analysis, cloud computing services, etc. Through the network in the data center, transfer the data to the computing power container for processing. Monitor and manage the data center in real time to ensure the efficiency and security of data processing. Build a data center and perform data processing. By reasonably configuring the computing power container and the power source container, it can ensure that the energy supply of the data center matches the computing needs, thereby improving the performance and efficiency of the data center.

[0032] Furthermore, the data mining module 13 in the functional partitioned containerized data center is also used for:

[0033] Data mining is performed on the computing amount, storage amount and network architecture according to the initial weight ratio to determine a first mining configuration record; fine-tuning and optimizing the initial weight ratio according to a preset adjustment method to obtain a first adjusted weight ratio, and data mining is performed on the computing amount, storage amount and network architecture based on the first adjusted weight ratio to obtain a second mining configuration record; the adjustment method for obtaining the first adjusted weight ratio is added to a disabled list, wherein the adjustment method in the disabled list has a disabled number of times; fine-tuning and optimizing is performed according to the disabled list, the initial weight ratio, the first mining configuration record and the second mining configuration record to obtain a configuration memory library.

[0034] Specifically, based on expert opinions, historical data or industry best practices, initial weights are pre-assigned to computing capacity, storage capacity and network architecture to determine the importance of each factor in the data mining process. Historical data and existing configurations are analyzed to identify which configurations perform best under specific weights. Complex algorithms and models, such as machine learning and artificial intelligence technologies, are used to process large amounts of data and find patterns and associations. The first mining configuration record is the optimal configuration solution obtained through data mining after considering the initial weight ratio. Based on multiple strategies, such as changes in business needs, technological advances or cost-benefit analysis, the adjustment method is pre-set. For example, if new technological developments improve the performance of storage devices, the weight of storage capacity is increased. If business needs change, such as increased demand for real-time data processing, the weight of network architecture is increased. According to the preset adjustment method, the weight ratio of different factors (computing capacity, storage capacity and network architecture) is adjusted, including increasing or decreasing the weight of a certain factor, exploring the impact of different configurations on overall performance and finding a better configuration solution. Based on the first adjusted weight ratio, data mining is performed again, and historical data and existing configurations are re-analyzed to determine the optimal configuration solution under the new weight, and the second mining configuration record is obtained. The second mining configuration record reflects how to better configure computing resources, storage capabilities, and network architecture to meet business needs after adjusting the weights.

[0035] After obtaining the first adjustment weight ratio through fine-tuning optimization, the obtained adjustment method is added to the banned list. In the next iteration, this specific adjustment method will not be used again until it is removed from the banned list. The taboo search algorithm maintains a banned list to record the adjustment methods that have been used and prohibits the use of these adjustment methods within a certain number of iterations to encourage the exploration of new solutions. The taboo search algorithm is used to avoid repeatedly using the same adjustment method in the optimization process, thereby falling into the local optimal solution. Each adjustment method in the banned list has a ban count, which determines the number of iterations that the adjustment method stays in the banned list. This strategy helps the algorithm jump out of the local optimum and explore a wider solution space. For example, if the weight of the network architecture is increased in one iteration, this adjustment will be recorded in the banned list and may be prohibited from use in the next few iterations. In this way, the algorithm will not repeat the same adjustment in consecutive iterations, but try other adjustment methods, such as increasing the weight of the amount of calculation or storage.

[0036] Fine-tune and optimize according to the banned list, initial weight ratio, first mining configuration record and second mining configuration record, iteratively adjust the weight ratio, and consider the adjustment method in the banned list, constantly try different weight combinations, and find the configuration solution that can provide the best performance. Each iteration generates a new configuration record and is added to the configuration memory. The configuration memory is a database that stores various configuration records, each of which represents the optimal configuration solution under a specific weight ratio. For example, in the first iteration, the algorithm finds that increasing the weight of the network architecture can significantly improve the performance of the data center. This adjustment will be recorded and added to the configuration memory. However, since this adjustment has been added to the banned list, the algorithm will not repeat this adjustment in the next few iterations, but will try other adjustments, such as increasing the weight of computing or storage. Continuously iterate and optimize weights to ensure that the configuration of the data center can best meet business needs. By storing and analyzing a large number of configuration records, the data center can quickly find the optimal configuration solution that suits specific business needs, which helps to improve the configuration efficiency of the data center.

[0037] Furthermore, the data mining module 13 in the functional partitioned containerized data center is also used for:

[0038] The initial weight ratio is fine-tuned and optimized by using a preset adjustment method other than the adjustment method in the disabled list to obtain a second adjusted weight ratio and a third mining configuration record; it is determined whether the second similarity between the third mining configuration record and the configuration record set consisting of the first mining configuration record and the second mining configuration record is less than or equal to the preset similarity, and if so, the third mining configuration record is added to the configuration record set, and the adjustment method for obtaining the second adjusted weight ratio is added to the disabled list, and fine-tuning and optimization are continued; after multiple fine-tuning and optimization, when the preset number of times is met, a configuration record set is obtained, wherein the configuration record set includes N mining configuration records; the N mining configuration records are stored in a database to obtain the configuration memory.

[0039] Specifically, the banned list records the most recently used adjustment methods, which are prohibited from being used within a certain number of iterations. Outside the banned list, the preset adjustment methods are used to fine-tune the initial weight ratio to explore new adjustment methods, avoid falling into local optimal solutions, and find global optimal solutions that may be overlooked. When fine-tuning and optimizing, the algorithm will select those methods that are not in the banned list from the preset adjustment methods to adjust the weight ratio. These preset adjustment methods include increasing or decreasing the weight of a factor, or changing the weight adjustment strategy. Through this strategy, the algorithm can obtain the second adjusted weight ratio, and perform data mining based on this new weight ratio to obtain a third mining configuration record. The third mining configuration record reflects how to better configure computing resources, storage capabilities, and network architecture to meet business needs under the new weight ratio.

[0040] Use the similarity calculation function to calculate the similarity between the third mining configuration record and the previous configuration records (the first and second mining configuration records). If the second similarity is less than or equal to the preset similarity threshold, that is, the third mining configuration record has a low similarity with the configuration records in the set, then the third mining configuration record is considered valuable, providing new or different information, and is added to the configuration record set. At the same time, the adjustment method that obtains the second adjustment weight ratio will be added to the disabled list. In the next iteration, this particular adjustment method will not be used again until it is removed from the disabled list. After multiple iterations, each time fine-tuning and optimization will be performed based on the current weight ratio and the adjustment method in the disabled list. Each iteration will generate a new configuration record. If the new configuration record has a low similarity with the existing configuration records in the set, that is, less than or equal to the preset similarity, then the new configuration record will be added to the configuration record set.

[0041] The preset number of iterations is an important parameter that affects the efficiency of the algorithm and the quality of the final solution. If the number of iterations is too small, a sufficiently good solution may not be found; if the number of iterations is too large, computing resources may be wasted and the quality of the solution may not be significantly improved. Set a number of iterations based on the complexity of the problem, available computing resources, time constraints, and quality requirements for the solution. This number can be fixed or dynamically adjusted based on certain conditions. When the number of iterations meets the preset number, a configuration record set containing N mining configuration records and a disabled list containing N adjustment methods will be obtained. This configuration record set is stored in a database to obtain a configuration memory. The configuration memory is a database that stores various configuration records, each of which represents the optimal configuration solution under a specific weight ratio. When optimizing configuration records, it is necessary to consider the differences between configurations, and increase the possibility of finding a better configuration by iteration and adjusting the weight ratio to avoid premature convergence and falling into a local optimal solution.

[0042] Furthermore, the functionally partitioned containerized data center further includes a fine-tuning optimization module for:

[0043] A first similarity between the first mining configuration record and the second mining configuration record is calculated using a similarity calculation function; and it is determined whether the first similarity is less than or equal to a preset similarity. If not, the adjustment method for obtaining the first adjustment weight ratio is eliminated, and fine-tuning is performed again to find the best.

[0044] Specifically, a similarity calculation function is used to evaluate the similarity between the first mining configuration record and the second mining configuration record. The similarity calculation function can be based on a variety of indicators, such as the amount of computing, storage, and configuration differences of the network architecture. According to specific needs and scenarios, a preset similarity is set to determine the threshold of the similarity between two data points, records, or configurations. The first similarity is an indicator that measures the similarity between the first mining configuration record and the second mining configuration record. If the first similarity is greater than the preset similarity, it means that the two configuration records are too similar, indicating that the weight adjustment does not bring enough changes, or the optimization process may have converged to a local optimal solution, which will cause the optimization process to fall into a local optimal solution. At this time, it is necessary to eliminate the adjustment method that obtains the first adjustment weight ratio, that is, remove this adjustment method from the disabled list, allowing the algorithm to reconsider this adjustment method in the next iteration, thereby increasing the exploration of the understanding space. Re-fine-tuning the optimization can help the algorithm find a better configuration solution and improve the configuration efficiency of the data center. Using the similarity calculation function to calculate the first similarity improves the diversity and global search capability of the optimization process. By avoiding reusing the same adjustments, the algorithm is able to better explore the solution space and find global optimal solutions that may be overlooked by other search strategies.

[0045] Furthermore, the matching module 14 in the functional partitioned containerized data center is also used for:

[0046] A matching space is constructed based on the N mining configuration records, wherein the matching space includes N matching points; the business type, processing data volume and response requirements are input into the matching space to obtain a target point; the distance from the N matching points to the target point is calculated, and the calculation results are sorted from small to large, and the matching points located in the first k positions are extracted, and K matching mining configuration records are mapped to obtain; the mean of the K matching mining configuration records is calculated to obtain a target configuration record.

[0047] Specifically, N mining configuration records are converted into points in a multidimensional space, each of which represents a configuration record. This multidimensional space is the matching space, which visualizes and quantifies various configuration schemes of the data center. The matching space is constructed based on the three-dimensional space coordinate system and N matching points, where the x-axis of the three-dimensional space coordinate system is the business type, the y-axis is the amount of processed data, and the z-axis is the response requirement. Next, the business type, the amount of processed data, and the response requirement are input into the matching space to obtain the target point, which represents the optimal or most suitable configuration scheme of the data center under the specific business requirements. For example, if the business type of a data center is mainly big data analysis, the amount of processed data is large, and fast response is required, then in the matching space, these requirements will correspond to those configuration points that can provide powerful computing power, large storage capacity, and fast network response. Based on some criteria, such as the ability of the configuration record to meet business requirements, performance indicators, etc., a variety of metrics are used to calculate the distance from each mining configuration record point (i.e., N matching points) in the matching space to the target point, such as Euclidean distance, Manhattan distance, etc. The calculated distances are sorted from small to large to find the configuration record that best meets the target requirements. Extract the first k matching points, which represent the configuration records that best meet the target requirements.

[0048] Map the extracted k matching points back to the original mining configuration records to obtain K matching mining configuration records, which represent the configuration solutions that perform best in meeting the target requirements. Collect key information of these K configuration records, such as the number of CPUs, storage capacity, network bandwidth, etc. Perform mean calculation on these key information to obtain the parameters of the target configuration record. For example, if each of the K configuration records has a different number of CPU cores, then the number of CPU cores of the target configuration record will be the average of these numbers. Based on the calculated mean parameters, generate the target configuration record, which represents the average optimal configuration of the K matching mining configuration records. By matching actual business needs with configuration solutions, data centers can ensure that their configurations meet key business requirements, such as processing speed, data storage capacity, and system reliability, which helps improve the overall performance and efficiency of data centers.

[0049] Furthermore, the functionally partitioned containerized data center further comprises a matching space module for:

[0050] Obtain a three-dimensional space coordinate system, wherein the x-axis of the three-dimensional space coordinate system is the business type, the y-axis is the amount of processed data, and the z-axis is the response demand; input the N mining configuration records into the three-dimensional space coordinate system to obtain N coordinates, and use the locations of the N coordinates as N matching points; and construct the matching space according to the three-dimensional space coordinate system and the N matching points.

[0051] Specifically, a three-dimensional spatial coordinate system is defined, where each axis represents a key dimension of a configuration record. The x-axis is the business type, which represents different types of business handled by the data center, such as big data analysis, cloud computing services, machine learning, etc. Different business types have different requirements for the resource configuration of the data center. For example, big data analysis may require more storage and computing resources. The y-axis is the amount of processed data, which represents the amount of data that the data center needs to process. The size of the data volume directly affects the storage capacity and computing power requirements of the data center. For example, processing large-scale data sets requires more storage space and more powerful computing power. The z-axis is the response demand, which represents the response speed requirement of the data center to the processing request. The response demand includes indicators such as processing speed, latency, and throughput. For example, for real-time data analysis applications, the response demand is very high, requiring the resources of the data center to be able to process and respond quickly.

[0052] Input N configuration mining records into the three-dimensional space coordinates, and convert the business type, processing data volume, and response requirements in the configuration records into numerical or categorical values ​​so that they can be represented in the coordinate system. Each configuration record will be converted into a point in the three-dimensional space, that is, a coordinate. In the three-dimensional space coordinate system, the coordinate of each mining configuration record represents a matching point, reflecting the characteristics of different configuration records in terms of business type, processing data volume, and response requirements. Using these coordinates as N matching points helps the data center to understand different configuration schemes more intuitively. By locating different configuration records in the three-dimensional space, the data center can more easily identify which configurations can best meet specific business requirements. For example, if the data center needs to choose the best configuration for a specific big data analysis application, then those matching points that are close to "big data analysis" on the x-axis, close to "large data processing" on the y-axis, and close to "fast response" on the z-axis can be found in the three-dimensional space. The configuration records represented by these points may be the best choice. Construct a matching space based on the three-dimensional space coordinate system and N matching points. Constructing the matching space provides an intuitive way to understand and compare different configuration schemes. By locating different configuration records in three-dimensional space, data centers can more easily identify which configurations can best meet specific business needs, helping to improve the configuration efficiency of the data center.

[0053] Furthermore, the layout topology network determination module 16 in the functional partitioned container-type data center is also used for:

[0054] Construct a power source configuration network layer; use the power source configuration network layer to identify the computing power container deployment topology network and the power source container configuration quantity, and obtain the power source container deployment topology network.

[0055] Specifically, the power source requirements of the data center are clarified, including power supply, cooling system, etc. According to the power source requirements, a network layer is constructed, which includes the configuration information of all power source containers. The power source configuration network layer is an abstract concept that represents the network structure of energy supply in the data center, including the configuration information of all power source containers, such as power supply, cooling system, etc. Using the power source configuration network layer, the computing power container layout topology network and the number of power source container configurations are identified, and the layout and connection methods of the computing power containers, as well as their correspondence with the power source containers, are analyzed. Based on the identification results, the power source container layout topology network, the connection methods between the power source containers, and the power supply path are obtained. Constructing the power source configuration network layer provides an intuitive way to understand and compare different power source configuration schemes.

[0056] In summary, the functionally partitioned containerized data center provided by the present application has the following technical effects:

[0057] Through the basic scale information acquisition module, the basic scale information acquisition module is used to obtain the basic scale information of the target data center, wherein the basic scale information includes business type, processing data volume and response requirements; the basic equipment information acquisition module, the basic equipment information acquisition module is used to collect the basic equipment information of the computing power container, wherein the basic equipment information includes computing capacity, storage capacity and network architecture; the data mining module, the data mining module is used to perform data mining with the computing capacity, storage capacity and network architecture as indexes to obtain a configuration memory library, wherein the configuration memory library includes N mining configuration records; the matching module, the matching module is used to target the business type, processing data volume and response requirements, and in the configuration memory library N mining configuration records are matched to obtain the target configuration record, wherein the target configuration record includes the number of computing power container configurations and the computing power container layout topology network; a mapping relationship construction module, the mapping relationship construction module is used to construct a computing power-power source container mapping relationship, and obtain the number of power source container configurations in combination with the computing power container configuration number; a layout topology network determination module, the layout topology network determination module is used to obtain the power source container layout topology network based on the computing power container layout topology network and the power source container configuration number; a data processing module, the data processing module is used to construct a data center based on the computing power container layout topology network and the power source container layout topology network, and use the data center for data processing. In other words, through data mining and configuration memory, the intelligent configuration of containers is realized, and the technical effect of improving the overall efficiency of the data center is achieved.

[0058] Embodiment 2: Based on the same inventive concept as the functional partitioned container data center in the above embodiment, the present application also provides a data processing method, please refer to the attached Figure 2 , the data processing method comprises:

[0059] Obtain basic scale information of the target data center, wherein the basic scale information includes business type, processing data volume and response requirements; collect basic equipment information of the computing power container, wherein the basic equipment information includes computing capacity, storage capacity and network architecture; perform data mining with the computing capacity, storage capacity and network architecture as indexes to obtain a configuration memory library, wherein the configuration memory library includes multiple configuration records; take the business type, processing data volume and response requirements as targets, perform matching in multiple configuration records of the configuration memory library to obtain a target configuration record, wherein the target configuration record includes the computing power container configuration quantity and the computing power container layout topology network; construct a computing power-power source container mapping relationship, and obtain the power source container configuration quantity in combination with the computing power container configuration quantity; obtain the power source container layout topology network based on the computing power container layout topology network and the power source container configuration quantity; construct a data center according to the computing power container layout topology network and the power source container layout topology network, and use the data center for data processing.

[0060] Furthermore, the data processing method further includes:

[0061] Data mining is performed on the computing amount, storage amount and network architecture according to the initial weight ratio to determine a first mining configuration record; fine-tuning and optimizing the initial weight ratio according to a preset adjustment method to obtain a first adjusted weight ratio, and data mining is performed on the computing amount, storage amount and network architecture based on the first adjusted weight ratio to obtain a second mining configuration record; the adjustment method for obtaining the first adjusted weight ratio is added to a disabled list, wherein the adjustment method in the disabled list has a disabled number of times; fine-tuning and optimizing is performed according to the disabled list, the initial weight ratio, the first mining configuration record and the second mining configuration record to obtain a configuration memory library.

[0062] Furthermore, the data processing method further includes:

[0063] The initial weight ratio is fine-tuned and optimized by using a preset adjustment method other than the adjustment method in the disabled list to obtain a second adjusted weight ratio and a third mining configuration record; it is determined whether the second similarity between the third mining configuration record and the configuration record set consisting of the first mining configuration record and the second mining configuration record is less than or equal to the preset similarity, and if so, the third mining configuration record is added to the configuration record set, and the adjustment method for obtaining the second adjusted weight ratio is added to the disabled list, and fine-tuning and optimization are continued; after multiple fine-tuning and optimization, when the preset number of times is met, a configuration record set is obtained, wherein the configuration record set includes N mining configuration records; the N mining configuration records are stored in a database to obtain the configuration memory.

[0064] Furthermore, the data processing method further includes:

[0065] A first similarity between the first mining configuration record and the second mining configuration record is calculated using a similarity calculation function; and it is determined whether the first similarity is less than or equal to a preset similarity. If not, the adjustment method for obtaining the first adjustment weight ratio is eliminated, and fine-tuning is performed again to find the best.

[0066] Furthermore, the data processing method further includes:

[0067] A matching space is constructed based on the N mining configuration records, wherein the matching space includes N matching points; the business type, processing data volume and response requirements are input into the matching space to obtain a target point; the distance from the N matching points to the target point is calculated, and the calculation results are sorted from small to large, and the matching points located in the first k positions are extracted, and K matching mining configuration records are mapped to obtain; the mean of the K matching mining configuration records is calculated to obtain a target configuration record.

[0068] Furthermore, the data processing method further includes:

[0069] Obtain a three-dimensional space coordinate system, wherein the x-axis of the three-dimensional space coordinate system is the business type, the y-axis is the amount of processed data, and the z-axis is the response demand; input the N mining configuration records into the three-dimensional space coordinate system to obtain N coordinates, and use the locations of the N coordinates as N matching points; and construct the matching space according to the three-dimensional space coordinate system and the N matching points.

[0070] Furthermore, the data processing method further includes:

[0071] Construct a power source configuration network layer; use the power source configuration network layer to identify the computing power container deployment topology network and the power source container configuration quantity, and obtain the power source container deployment topology network.

[0072] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. Figure 1 A functionally partitioned container data center and specific examples in Example 1 are also applicable to a data processing method in this embodiment. Through the above detailed description of a functionally partitioned container data center, technical personnel in this field can clearly understand a data processing method in this embodiment, so for the sake of brevity of the specification, it will not be described in detail here.

[0073] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

[0074] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application belong to the scope of the present application and its equivalent technology, the present application is also intended to include these modifications and variations.

Claims

1. A functionally zoned containerized data center, characterized in that: include: A basic scale information acquisition module, wherein the basic scale information acquisition module is used to acquire basic scale information of a target data center, wherein the basic scale information includes business type, processing data volume, and response requirements; A basic equipment information collection module, which is used to collect basic equipment information of the computing power container, wherein the basic equipment information includes computing capacity, storage capacity and network architecture; A data mining module, the data mining module is used to perform data mining with the computing amount, storage amount and network architecture as indexes to obtain a configuration memory library, wherein the configuration memory library includes N mining configuration records; A matching module, wherein the matching module is used to match the N mining configuration records in the configuration memory with the business type, the processing data volume and the response requirement as the target, and obtain a target configuration record, wherein the target configuration record includes the number of computing power container configurations and the computing power container layout topology network; A mapping relationship building module, the mapping relationship building module is used to build a computing power-power source container mapping relationship, and obtain the power source container configuration quantity in combination with the computing power container configuration quantity; A topology network deployment determination module, the topology network deployment determination module is used to obtain a power source container deployment topology network based on the computing power container deployment topology network and the power source container configuration quantity; A data processing module, the data processing module is used to build a data center according to the computing power container layout topology network and the power source container layout topology network, and use the data center to perform data processing; The data mining module is used to: Performing data mining on the computing amount, storage amount, and network architecture according to the initial weight ratio to determine a first mining configuration record; Fine-tune and optimize the initial weight ratio according to a preset adjustment method to obtain a first adjusted weight ratio, and perform data mining on the computing amount, storage amount, and network architecture based on the first adjusted weight ratio to obtain a second mining configuration record; Adding the adjustment method for obtaining the first adjustment weight ratio into a prohibited list, wherein the adjustment methods in the prohibited list have a prohibited number of times; Perform fine-tuning and optimization according to the banned list, the initial weight ratio, the first mining configuration record, and the second mining configuration record to obtain a configuration memory library; Using a preset adjustment method other than the adjustment method in the prohibited list to fine-tune the initial weight ratio to obtain a second adjusted weight ratio and a third mining configuration record; Determine whether the second similarity between the third mining configuration record and the configuration record set consisting of the first mining configuration record and the second mining configuration record is less than or equal to a preset similarity, and if so, add the third mining configuration record to the configuration record set, and add the adjustment method for obtaining the second adjustment weight ratio to the disabled list, and continue to perform fine-tuning and optimization; After multiple fine-tuning optimizations, when a preset number of times is met, a configuration record set is obtained, wherein the configuration record set includes N mining configuration records; Storing the N mining configuration records in a database to obtain the configuration memory library; Calculating a first similarity between the first mining configuration record and the second mining configuration record using a similarity calculation function; It is determined whether the first similarity is less than or equal to a preset similarity. If not, the adjustment method for obtaining the first adjustment weight ratio is eliminated, and fine-tuning is performed again to find the best result.

2. A functionally zoned containerized data center as claimed in claim 1, characterized in that: The matching module is used for: Constructing a matching space based on the N mining configuration records, wherein the matching space includes N matching points; Input the business type, processing data volume and response requirements into the matching space to obtain a target point; Calculate the distances from the N matching points to the target point, sort the calculation results from small to large, extract the first k matching points, and map to obtain K matching mining configuration records; The K matching mining configuration records are averaged to obtain a target configuration record.

3. A functionally zoned containerized data center as claimed in claim 2, characterized in that: The matching module is used for: Obtain a three-dimensional space coordinate system, wherein the x-axis of the three-dimensional space coordinate system is the business type, the y-axis is the amount of processed data, and the z-axis is the response demand; Input the N mining configuration records into the three-dimensional space coordinate system to obtain N coordinates, and use the positions of the N coordinates as N matching points; The matching space is constructed according to the three-dimensional space coordinate system and the N matching points.

4. A functionally zoned containerized data center as claimed in claim 1, characterized in that: The topology network determination module is used for: Build the power source configuration network layer; The power source configuration network layer is used to identify the computing power container deployment topology network and the power source container configuration quantity to obtain the power source container deployment topology network.

5. A data processing method, characterized in that: The data processing method comprises: Obtaining basic scale information of the target data center, wherein the basic scale information includes business type, processing data volume, and response requirements; Collect basic equipment information of the computing power container, wherein the basic equipment information includes computing capacity, storage capacity and network architecture; Performing data mining with the computing amount, storage amount and network architecture as indexes to obtain a configuration memory library, wherein the configuration memory library includes a plurality of configuration records; Taking the business type, processing data volume and response requirements as the target, matching is performed among multiple configuration records in the configuration memory to obtain a target configuration record, wherein the target configuration record includes the number of computing power container configurations and the computing power container layout topology network; Construct a computing power-power source container mapping relationship, and obtain the power source container configuration quantity in combination with the computing power container configuration quantity; Based on the computing power container deployment topology network and the number of power source container configurations, a power source container deployment topology network is obtained; Building a data center according to the computing power container layout topology network and the power source container layout topology network, and using the data center for data processing; The data processing method further comprises: Performing data mining on the computing amount, storage amount and network architecture according to the initial weight ratio to determine a first mining configuration record; fine-tuning and optimizing the initial weight ratio according to a preset adjustment method to obtain a first adjusted weight ratio, and performing data mining on the computing amount, storage amount and network architecture based on the first adjusted weight ratio to obtain a second mining configuration record; adding the adjustment method obtained by the first adjusted weight ratio to a disabled list, wherein the adjustment method in the disabled list has a disabled number of times; performing fine-tuning and optimizing according to the disabled list, the initial weight ratio, the first mining configuration record and the second mining configuration record to obtain a configuration memory library; The initial weight ratio is fine-tuned and optimized by using a preset adjustment method other than the adjustment method in the disabled list to obtain a second adjusted weight ratio and a third mining configuration record; it is determined whether the second similarity between the third mining configuration record and the configuration record set consisting of the first mining configuration record and the second mining configuration record is less than or equal to the preset similarity, and if so, the third mining configuration record is added to the configuration record set, and the adjustment method for obtaining the second adjusted weight ratio is added to the disabled list, and the fine-tuning and optimization is continued; after multiple fine-tuning and optimization, when the preset number of times is met, a configuration record set is obtained, wherein the configuration record set includes N mining configuration records; the N mining configuration records are stored in a database to obtain the configuration memory library; A first similarity between the first mining configuration record and the second mining configuration record is calculated using a similarity calculation function; and it is determined whether the first similarity is less than or equal to a preset similarity. If not, the adjustment method for obtaining the first adjustment weight ratio is eliminated, and fine-tuning is performed again to find the best.

Images