Data load processing method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to a data load processing method and device, computer equipment, a readable storage medium and a program product. The method comprises the steps of obtaining a to-be-processed data load of a target data center, determining a data load type of the to-be-processed data load, and performing delay data processing on the to-be-processed data load on the basis of the data load type under the condition that a preset data load delay processing condition is met, or performing delay data processing on the to-be-processed data load on the basis of the data load type under the condition that a preset data load space transfer condition is met. And transferring the to-be-processed data load from the target data center to a remote data center, wherein the remote data center is used for carrying out data processing on the obtained to-be-processed data load. By adopting the method, data load processing can be efficiently carried out.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data load processing method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0002] In recent years, with the continuous development of technology, the scale and number of major Internet devices, such as data centers, have increased year by year, and the overall energy consumption has also gradually increased. Therefore, it is necessary to process the data load of Internet devices to reduce the load pressure of Internet devices.

[0003] In traditional technology, taking data centers as an example, the main technical means for data centers to participate in data load processing is through the "energy storage + renewable energy + IDC (Internet Data Center)" model, that is, to carry out certain modifications to the microgrid of the data center and add energy storage equipment. Through the joint action of the added energy storage equipment and the modified microgrid, the load peak of the data center can be achieved and the load pressure of Internet equipment can be reduced.

[0004] However, the current data load processing method is not efficient enough. Summary of the invention

[0005] Based on this, it is necessary to provide an efficient data load processing method, device, computer equipment, computer readable storage medium and computer program product to address the above technical problems.

[0006] In a first aspect, the present application provides a data load processing method, comprising:

[0007] Obtaining the data load to be processed in the target data center;

[0008] determining a data payload type of the data payload to be processed;

[0009] Based on the data load type, when the preset data load delay processing conditions are met, delayed data processing is performed on the data load to be processed, or, when the preset data load space transfer conditions are met, the data load to be processed is transferred from the target data center to an off-site data center, and the off-site data center is used to perform data processing on the acquired data load to be processed.

[0010] In one embodiment, based on the data load type, when a preset data load delay processing condition is met, delay data processing is performed on the data load to be processed, or, when a preset data load space transfer condition is met, the data load to be processed is transferred to a remote data center, including:

[0011] If the data load type is a first data load type, then, if a preset data load delay processing condition is met, delayed data processing is performed on the data load to be processed, or, if a preset data load space transfer condition is met, the data load to be processed is transferred from the target data center to a remote data center, wherein the first data load type is a data load type that is not time-sensitive;

[0012] If the data load type is the second data load type, then the data load to be processed is transferred from the target data center to the remote data center when the preset data load space transfer conditions are met, wherein the second data load type is a time-sensitive data load type.

[0013] In one embodiment, based on the data load type, when a preset data load delay processing condition is met, delayed data processing is performed on the data load to be processed, or, when a preset data load space transfer condition is met, the data load to be processed is transferred from the target data center to the remote data center, and the remote data center is used to perform data processing on the acquired data load to be processed, including:

[0014] Obtaining a first sending data load and a first receiving data load of the front-end portal server, a second receiving data load of the target data center, and a second sending data load sent to the target data center by other data centers except the target data center within the same time period;

[0015] When the first sent data load is equal to the first received data load and the second sent data load is equal to the second received data load, based on the data load type, delayed data processing is performed on the data load to be processed, or the data load to be processed is transferred from the target data center to an off-site data center.

[0016] In one embodiment, based on the data load type, when a preset data load delay processing condition is met, delayed data processing is performed on the data load to be processed, or, when a preset data load space transfer condition is met, the data load to be processed is transferred from the target data center to a remote data center, and the remote data center is used to perform data processing on the acquired data load to be processed, including:

[0017] Obtain the average residence time of the data load to be processed in the target data center;

[0018] On the basis that the average stay time is less than or equal to the preset maximum delay time, based on the data load type, delayed data processing is performed on the data load to be processed, or the data load to be processed is transferred from the target data center to a remote data center.

[0019] In one embodiment, based on the data load type, when a preset data load delay processing condition is met, delayed data processing is performed on the data load to be processed, or, when a preset data load space transfer condition is met, the data load to be processed is transferred from the target data center to a remote data center, and the remote data center is used to perform data processing on the acquired data load to be processed, including:

[0020] Obtain the total data load of the target data center;

[0021] On the basis that the total amount of data load is less than or equal to the preset maximum data computing resource, based on the data load type, delayed data processing is performed on the data load to be processed, or the data load to be processed is transferred from the target data center to an off-site data center.

[0022] In one embodiment, based on the data load type, when a preset data load delay processing condition is met, delay data processing is performed on the data load to be processed, or, when a preset data load space transfer condition is met, the data load to be processed is transferred from the target data center to the remote data center, including:

[0023] Obtaining the data load change of the target data center from the previous time period to the current time period, the initial data load of the previous time period, the current data load of the current time period, and the delayed processing time;

[0024] When the sum of the data load change and the initial data load is equal to the current data load, the current data load is less than or equal to the preset maximum storage capacity and greater than or equal to the preset ideal processing completed data load, and the delayed processing time is greater than or equal to the delay tolerance time, based on the data load type, delayed data processing is performed on the data load to be processed, or the data load to be processed is transferred from the target data center to an off-site data center.

[0025] In one embodiment, when a preset data load space transfer condition is met, transferring the data load to be processed from the target data center to the remote data center includes:

[0026] Obtaining the outgoing data load of the target data center and the incoming data load of other data centers except the target data center;

[0027] When the outgoing data load is equal to the incoming data load, the data load to be processed is transferred from the target data center to the remote data center.

[0028] In a second aspect, the present application further provides a data load processing device, comprising:

[0029] An acquisition module, used for acquiring the data load to be processed;

[0030] A determination module, used to determine the data load type of the data load to be processed;

[0031] A processing module is used to perform delayed data processing on the data load to be processed based on the data load type and when the preset data load delay processing conditions are met, or to transfer the data load to be processed from the target data center to an off-site data center when the preset data load space transfer conditions are met. The off-site data center is used to perform data processing on the acquired data load to be processed.

[0032] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0033] Obtaining the data load to be processed in the target data center;

[0034] determining a data payload type of the data payload to be processed;

[0035] Based on the data load type, when the preset data load delay processing conditions are met, delayed data processing is performed on the data load to be processed, or, when the preset data load space transfer conditions are met, the data load to be processed is transferred from the target data center to an off-site data center, and the off-site data center is used to perform data processing on the acquired data load to be processed.

[0036] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the following steps are implemented:

[0037] Obtaining the data load to be processed in the target data center;

[0038] determining a data payload type of the data payload to be processed;

[0039] Based on the data load type, when the preset data load delay processing conditions are met, delayed data processing is performed on the data load to be processed, or, when the preset data load space transfer conditions are met, the data load to be processed is transferred from the target data center to an off-site data center, and the off-site data center is used to perform data processing on the acquired data load to be processed.

[0040] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0041] Obtaining the data load to be processed in the target data center;

[0042] determining a data payload type of the data payload to be processed;

[0043] Based on the data load type, when the preset data load delay processing conditions are met, delayed data processing is performed on the data load to be processed, or, when the preset data load space transfer conditions are met, the data load to be processed is transferred from the target data center to an off-site data center, and the off-site data center is used to perform data processing on the acquired data load to be processed.

[0044] The above-mentioned data load processing method, device, computer equipment, computer-readable storage medium and computer program product, according to the data load type of the data load to be processed, perform delayed data processing on the data load to be processed when the preset data load delay processing conditions are met, or transfer the data load to be processed from the target data center to the remote data center when the preset data load space transfer conditions are met. During the whole process, there is no need to transform the microgrid of the target data center, nor to add additional energy storage equipment to the target data center. The processing of data loads to be processed of different data load types can be fully considered, thereby reducing the load pressure of the target data center and greatly improving the data load processing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments of the present application or the related technical descriptions will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying any creative work.

[0046] Figure 1 is an application environment diagram of a data load processing method in one embodiment;

[0047] Figure 2 is a flow chart of a data load processing method in one embodiment;

[0048] Figure 3 is a flow chart of a data load processing method in another embodiment;

[0049] Figure 4 is a structural block diagram of a data load processing device in one embodiment;

[0050] Figure 5 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are used to explain the present application and are not used to limit the present application.

[0052] For data centers, the fundamental source of their power load is network load. For network loads that are not single processors (data centers), they can be distributed to data centers located at different nodes in the power grid through data links for processing. Since there is a corresponding relationship between the energy consumption of data centers and the network load, this application proposes a data load processing method that uses the transfer of network loads to replace the long-distance transmission of electrical energy.

[0053] The data load processing method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown, multiple data centers 102 communicate with the server 104 through the network, and the data load of each data center 102 can also be transferred to each other. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers.

[0054] When the load pressure of the target data center among the multiple data centers 102 exceeds the preset load pressure threshold, the data load to be processed of the target data center 102 is obtained. The server 104 determines the data load type of the data load to be processed, and based on the data load type, performs delayed data processing on the data load to be processed when the preset data load delay processing condition is met, or, when the preset data load space transfer condition is met, transfers the data load to be processed from the target data center to an off-site data center, where the off-site data center is used to perform data processing on the acquired data load to be processed, and the off-site data center refers to a data center that does not belong to the same region or the same power system as the target data center.

[0055] The server 104 may be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services.

[0056] In an exemplary embodiment, Figure 2 As shown, a data load processing method is provided, which is applied to Figure 1 The server 104 in FIG. 1 is used as an example for explanation. Among them:

[0057] S200, obtaining the data load to be processed of the target data center.

[0058] Specifically, the data load received by the target data center comes from the front-end portal server on the one hand, and from the data load transferred from other data centers on the other hand. When processing the data load for the target data center, the data load to be processed of the target data center can be obtained by the background server to accurately determine the data load type of the data load to be processed.

[0059] S400: Determine a data load type of a data load to be processed.

[0060] Specifically, the data load type of the data load to be processed is determined. The data load type can be divided into many types. For example, according to the differences in user data network requirements, the data load is divided into batch data load and interactive data load, etc. For another example, according to whether the data is time-sensitive, it can be divided into a time-insensitive data load type and a time-sensitive data load type. Among them, time sensitivity means that if the data is not processed for a certain period of time, it is easy to cause data failure.

[0061] S600, based on the data load type, when the preset data load delay processing conditions are met, the data load to be processed is delayed data processed, or, when the preset data load space transfer conditions are met, the data load to be processed is transferred from the target data center to a remote data center.

[0062] The off-site data center is used to process the acquired data load to be processed.

[0063] Specifically, based on the data load type, it can be determined whether to perform delayed data processing on the data load to be processed or to perform spatial transfer processing on the data load to be processed.

[0064] More specifically, delayed data processing of the data load to be processed means that the target data center can first store the received data load to be processed, and then process the stored data load to be processed after a period of delay, so as to realize the time transfer of the data load in the same data center, and the process of delayed data processing needs to meet the preset data load delayed processing conditions.

[0065] The spatial transfer processing of the data load to be processed means that the target data center can transfer the data load to be processed received from the front-end portal server to an off-site data center for processing, thereby realizing spatial transfer. The spatial transfer process needs to meet the preset data load spatial transfer conditions, reduce the power consumption of the target data center per unit time, and avoid excessive regional power load. Among them, the off-site data center refers to a data center that crosses regions or power systems.

[0066] Based on the above technical means, for each data load type to be processed, its time and space adjustment capability is fully utilized to reduce the impact of a large number of distributed power sources on the distribution system, balance the network line load, and improve the system source, grid and load complementary operation characteristics.

[0067] In an exemplary embodiment, based on the data load type, it can be determined whether to perform delayed data processing on the data load to be processed or to perform spatial transfer processing on the data load to be processed, including: for interactive data loads, it is usually immediate, such as searching through a browser, retrieving data from a server for observation, etc. In this regard, it requires strict response to the user's immediate needs, and time adjustment cannot be performed. Fast data analysis and high-speed interaction are required. Therefore, the target data center needs to perform fast and delay-free processing. For batch data loads, it comes from operations such as indexing and data mining run by the background server, and can be delayed in batches. That is to say, the batch data load can be transferred to different data centers for processing within a certain period of time, or delayed in the same data center.

[0068] In the above data load processing method, according to the data load type of the data load to be processed, when the preset data load delay processing conditions are met, the data load to be processed is delayed data processing, or, when the preset data load space transfer conditions are met, the data load to be processed is transferred from the target data center to an off-site data center. During the whole process, there is no need to transform the microgrid of the target data center, nor to add additional energy storage equipment to the target data center. Different data load types can be fully considered to reduce the load pressure of the target data center, thereby greatly improving the data load processing efficiency.

[0069] In an exemplary embodiment, Figure 3 As shown, S600 includes:

[0070] S620: If the data load type is the first data load type, then when the preset data load delay processing conditions are met, delayed data processing is performed on the data load to be processed, or, when the preset data load space transfer conditions are met, the data load to be processed is transferred from the target data center to a remote data center.

[0071] S640: If the data load type is the second data load type, the data load to be processed is transferred from the target data center to the remote data center when a preset data load space transfer condition is met.

[0072] The first data load type is a data load type that is not sensitive to time, and the second data load type is a data load type that is sensitive to time.

[0073] Specifically, since the first data load type is a data load type that is not sensitive to time, therefore, when the preset data load delay processing conditions are met, the data load to be processed of the first data load type can be subjected to delayed data processing or spatial transfer processing. That is to say, if the data load type is the first data load type, then when the preset data load delay processing conditions are met, the data load to be processed is temporarily stored in the target data center, and after a period of time, that is, after the current peak-shaving time period ends, data processing is performed on the temporarily stored data load to be processed, or, when the preset data load spatial transfer conditions are met, the data load to be processed is transferred from the target data center to a remote data center, and the remote data center performs data processing on the data load to be processed.

[0074] However, since the second data load type is a time-sensitive data load type, in order to ensure the timeliness of data processing, delayed data processing cannot be performed on the pending data load of the first data load type. Instead, spatial transfer processing is performed on the pending data load of the second data load type. That is to say, if the data load type is the second data load type, then when the preset data load spatial transfer conditions are met, the pending data load will be transferred from the target data center to a remote data center, and the remote data center will perform data processing on the pending data load.

[0075] Generally, the first data payload type may be a delay-tolerant payload in a batch data payload, and the second data payload type may be a delay-sensitive payload and an interactive data payload in a batch data payload.

[0076] In this embodiment, by subdividing the data load processing operations performed on each data load type, the data load processing operations corresponding to each data load type can be performed more accurately, reducing processing errors in the data load processing process and improving the accuracy of data load processing.

[0077] In an exemplary embodiment, based on the data load type, when a preset data load delay processing condition is met, delayed data processing is performed on the data load to be processed, or, when a preset data load space transfer condition is met, the data load to be processed is transferred from the target data center to the remote data center, and the remote data center is used to perform data processing on the acquired data load to be processed, including:

[0078] Obtain the first sent data load and the first received data load of the front-end portal server, the second received data load of the target data center, and the second sent data load sent by other data centers except the target data center to the target data center within the same time period; when the first sent data load is equal to the first received data load and the second sent data load is equal to the second received data load, based on the data load type, perform delayed data processing on the data load to be processed, or transfer the data load to be processed from the target data center to an off-site data center.

[0079] Specifically, the preset data load space transfer condition and the preset data load delay processing condition both include data load balancing. Data load balancing includes the following two points:

[0080] First, the total amount of data load allocated by the front-end portal server to each data center is balanced with the data load received by the front-end portal server.

[0081] More specifically, the data load received by the data center comes from the front-end portal server on the one hand, and from the load transferred from other data centers on the other hand. Therefore, it is necessary to ensure that the total data load allocated to each data center by the front-end portal server is balanced with the data load received by the front-end portal server.

[0082] For the f-type data load, the balance equation between the total amount of f-type data load allocated by the front-end portal server to each data center and the amount of f-type data load received by the front-end portal server is as follows:

[0083]

[0084] Where N is the total number of data centers, is the front-end portal server code, f is the data type code, t is the time period, The front-end portal server in time period t The amount of data load of type f sent to all data centers, The front-end portal server in time period t The amount of data payload of type f received.

[0085] Furthermore, for all types of data loads, the total amount of different types of data loads allocated by the front-end portal server to each data center is equal to the amount of different types of data loads received by the front-end portal server.

[0086] Second, the data load received by the target data center from other data centers is equal to the sum of the data loads sent by the remaining data centers to the target data center.

[0087] The constraint equations established are as follows:

[0088]

[0089] Among them, i is the code of the target data center receiving the data load, and j is the code of the other data center sending the data load. —The amount of data load of type f sent by data center j to target data center i during time period t, —The amount of data load of type f received by data center i from other data centers.

[0090] That is to say, the first sending data load amount allocated by the front-end portal server to each data center and the first receiving data load amount received by the front-end portal server in the same time period are obtained, and the second receiving data load amount received by the target data center from each other data center and the second sending data load amount sent by other data centers except the target data center to the target data center are obtained.

[0091] When the first sent data load is equal to the first received data load, and the second sent data load is equal to the second received data load, it is considered that the data load balance is met at this time, and then based on the data load type of the data load to be processed, the data load to be processed is delayed data processing, or the data load to be processed is transferred from the target data center to a remote data center.

[0092] In this embodiment, when the total amount of data load allocated to each data center by the front-end portal server is balanced with the data load received by the front-end portal server, and the data load received by the target data center from other data centers is equal to the sum of the data loads sent by the remaining data centers to the target data center, that is, when the data load balance is satisfied, it is more accurate and reliable to perform delayed data processing or spatial transfer processing on the data load to be processed.

[0093] In an exemplary embodiment, based on the data load type, when a preset data load delay processing condition is met, delayed data processing is performed on the data load to be processed, or, when a preset data load space transfer condition is met, the data load to be processed is transferred from the target data center to a remote data center, and the remote data center is used to perform data processing on the acquired data load to be processed, including:

[0094] The average residence time of the data load to be processed in the target data center is obtained. On the basis that the average residence time is less than or equal to the preset maximum delay time, based on the data load type, delayed data processing is performed on the data load to be processed, or the data load to be processed is transferred from the target data center to an off-site data center.

[0095] Specifically, for the time-sensitive second data load type of pending data load, assuming that the pending data load allocated to the target data center is evenly distributed to each server in active operation, the average residence time of the pending data load in the target data center (not exceeding the length of a time period) is estimated by the M / M / 1 queuing model, where the M / M / 1 model is the simplest model in queuing theory, M means that the arrival rate and service rate are both exponentially distributed, 1 represents a service desk, which refers to a single service desk, a queuing system in which customer arrivals follow a Poisson distribution and service time follows an exponential distribution.

[0096] Furthermore, the average residence time of the data load to be processed needs to not exceed the preset maximum delay time allowed, which is expressed as follows:

[0097]

[0098] in, is the total number of front-end portal servers, is the average service rate of each running server in time period t, The target data center i is from The sum of the data loads of type f received by the front-end portal servers, is the sum of the data load of type f sent by other data centers j to the target data center i during time period t, is the amount of data transferred by target data center i due to data load scheduling during time period t, The maximum delay allowed for data payloads of type f, It is the minimum number of active front-end portal servers in time period t.

[0099] On the basis that the average stay time is less than or equal to the preset maximum delay time, based on the data load type, delayed data processing is performed on the data load to be processed, or the data load to be processed is transferred from the target data center to a remote data center.

[0100] Furthermore, in this application, it is assumed that data transfer between data centers must rely on server coordination. When the number of active servers is insufficient, cross-region coordinated spatial transfer may not be possible. Therefore, the above formula is transformed into the same form to obtain the minimum number of active front-end portal servers required to process data load within the delay range, which is expressed as follows:

[0101]

[0102] In this embodiment, by meeting the data load delay requirement while performing delayed data processing or spatial transfer processing, that is, satisfying that the average residence time is less than or equal to the preset maximum delay time, the peak load of the data to be processed can be more accurately and reliably shaving.

[0103] In an exemplary embodiment, based on the data load type, when a preset data load delay processing condition is met, delayed data processing is performed on the data load to be processed, or, when a preset data load space transfer condition is met, the data load to be processed is transferred from the target data center to a remote data center, and the remote data center is used to perform data processing on the acquired data load to be processed, including:

[0104] Obtain the total amount of data load of the target data center, and on the basis that the total amount of data load is less than or equal to the preset maximum data computing resources, based on the data load type, perform delayed data processing on the data load to be processed, or transfer the data load to be processed from the target data center to an off-site data center. Perform delayed data processing on the data load to be processed, or transfer the data load to be processed from the target data center to an off-site data center.

[0105] Specifically, since the number of servers configured in the data center and the data computing capacity of each server are limited, it is necessary to ensure that the data center has sufficient data computing resources to handle the data load. In other words, the total amount of data load processed by the target data center is greater than or equal to the preset maximum data computing resource. The specific expression is as follows:

[0106]

[0107] in, is the data load of type f processed by target data center i during time period t, is the total number of data payload types, It is the preset maximum data computing resource that the target data center i can provide within the time period t.

[0108] Furthermore, the maximum data computing resources are preset It can be obtained based on the number of active servers corresponding to the target data center and the average service rate of each server. The specific expression is as follows:

[0109]

[0110] in, is the minimum number of active front-end portal servers in time period t, The average service rate of each running server configured in the target data center i during time period t.

[0111] On the basis that the total amount of data load processed by the target data center is greater than or equal to the preset maximum data computing resource, based on the data load type, delayed data processing is performed on the data load to be processed, or the data load to be processed is transferred from the target data center to an off-site data center.

[0112] In this embodiment, when the total amount of data load processed by the target data center is greater than or equal to the preset maximum data computing resources, the data load to be processed is peak-loaded based on the data load type, which improves the accuracy of data load processing and makes the peak-load process more reliable.

[0113] In an exemplary embodiment, based on the data load type, when a preset data load delay processing condition is met, delayed data processing is performed on the data load to be processed, or, when a preset data load space transfer condition is met, the data load to be processed is transferred from the target data center to the remote data center, including:

[0114] Obtain the data load change from the previous time period to the current time period, the initial data load of the previous time period, the current data load of the current time period, and the delayed processing time of the target data center. When the sum of the data load change and the initial data load is equal to the current data load, the current data load is less than or equal to the preset maximum storage capacity and greater than or equal to the preset ideal processing completed data load, and the delayed processing time is greater than or equal to the delay tolerance time, based on the data load type, perform delayed data processing on the data load to be processed, or transfer the data load to be processed from the target data center to an off-site data center.

[0115] Specifically, delayed data processing of the data load to be processed also needs to meet the time transfer constraint, which is a kind of preset data load delayed processing condition. In addition, it should be noted that during the process of spatial transfer of data, there may be time for data transfer and recall. At the same time, if the time-insensitive delayable data load is spatially transferred, after the spatial transfer, delayed processing can also be performed in data centers in other regions, so the time transfer constraint also needs to be met. Therefore, the transfer of the data load to be processed from the target data center to the remote data center also needs to meet the time transfer constraint.

[0116] For the to-be-processed data load corresponding to the first data load type that is not sensitive to time, the target data center may perform an appropriate delay therefor, for example, a delay greater than a time period, so as to achieve a time load demand response.

[0117] During the delay process, the following expression constraint models can be established:

[0118] 1 The change in data load stored in the target data center is:

[0119]

[0120] in, It is the change in the data load of type f during the time period t when the target data center i performs time transfer.

[0121] 2 The data load relationship of the target data center storage in different time periods is:

[0122] Among them, E i，t is the data load stored in the target data center i during the time period t, E i，t-1 is the data load stored in the target data center i during the time period t-1, is the change in the load of all types of data in the target data center i during the time period t.

[0123] Combining Expression 1-2, it can be obtained that the sum of the change in data load of all types in the target data center and the initial data load is equal to the current data load.

[0124] 3. In order to ensure that the target data center stores the same amount of data load at the beginning and end, that is, to ensure that the data load processed by the target data center will not be reduced under time delay, the target data center is required to complete all the received data loads within the operation cycle to meet the following requirements:

[0125] Among them, E i，T is the data load of the target data center after the delay processing is completed, The amount of data load at the target data center before delayed processing begins.

[0126] 4 Ensure that the data load stored in the target data center does not exceed the preset maximum storage capacity:

[0127] in, The preset maximum storage capacity of the target data center.

[0128] Furthermore, for the target data center, the data load processing of the target data center must also meet QoS (Quality of Service), that is, the data load must be processed within the QoS time constraint, and cannot be delayed indefinitely. In other words, the current data load must be less than or equal to the preset maximum storage capacity and greater than or equal to the preset ideal processing completed data load, and the delayed processing time must be greater than or equal to the delay tolerance time. The specific QoS constraint expression is as follows:

[0129] in, For At the end of the time period, the processing should be completed The preset ideal processing capacity of the type is the target data center. The amount of data load received before time; for The delay tolerance of the data payload type.

[0130] Furthermore, on the basis of satisfying the above constraint model, based on the data load type, delayed data processing is performed on the data load to be processed, or the data load to be processed is transferred from the target data center to a remote data center.

[0131] In this embodiment, by setting time transfer constraints, when the sum of the data load change and the initial data load is equal to the current data load, the current data load is less than or equal to the preset maximum storage capacity and greater than or equal to the preset ideal processing completed data load, and the delayed processing time is greater than or equal to the delay tolerance time, it is possible to accurately and reliably perform delayed data processing on the data load to be processed, or transfer the data load to be processed from the target data center to a remote data center.

[0132] In an exemplary embodiment, when a preset data load space transfer condition is met, transferring the data load to be processed from the target data center to the remote data center includes:

[0133] The outgoing data load of the target data center and the incoming data load of other data centers except the target data center are obtained. When the outgoing data load is equal to the incoming data load, the data load to be processed is transferred from the target data center to the remote data center.

[0134] Specifically, the spatial transfer processing of the data load to be processed also needs to meet the spatial transfer constraint condition, which is a type of preset data load spatial transfer condition.

[0135] For each data center, the data load transferred from the data center should be equal to the sum of the loads received by the remaining data centers. The specific expression is as follows:

[0136]

[0137] in, is the amount of data load of type f transferred out by target data center i due to data load scheduling during time period t, is the amount of incoming data load of type f sent by other data centers j to target data center i during time period t.

[0138] Therefore, the outgoing data load of the target data center and the incoming data load of other data centers except the target data center are obtained. When the outgoing data load is equal to the incoming data load, the data load to be processed is transferred from the target data center to the remote data center, and the remote data center processes the data load.

[0139] In this embodiment, by setting spatial transfer constraints, when the outgoing data load is equal to the incoming data load, the data load to be processed is peak-loaded based on the data load type, thereby improving the accuracy of data load processing and making the peak-load process more reliable.

[0140] In an exemplary embodiment, different types of data loads to be processed are processed in batches with delayed processing or cross-regional assisted processing according to the data load types processed by the data center. On the basis of traditional data centers, by screening the adjustable data, the data center can deeply participate in the peak load regulation of the external power grid by only scheduling the data load to be processed.

[0141] Specifically, data load processing methods can be divided into the following two adjustment technologies, including (1) data center load time adjustment technology and (2) data center load space adjustment technology.

[0142] (1) Data center load time adjustment technology

[0143] The time adjustment of data load in data center is mainly realized by adjusting the time-insensitive data load in batch data load. Under the time constraint of service quality, data center will temporarily store the batch data load that can be processed late in a part of time period, and process the data after the peak period, so as to realize the time transfer of data load in the same data center.

[0144] (2) Data center load space adjustment technology

[0145] The spatial regulation of data load in data centers is mainly achieved by adjusting the time-sensitive data load and interactive data load in batch data load. During the peak regulation period, the data center transfers part of the load received from the front-end portal server to other data centers for processing. The other data centers transferred to are often data processing bases across regions or cross-regional power networks. This can achieve spatial transfer, reduce the power consumption of data centers per unit time, and avoid excessive regional power load.

[0146] When the data center load time adjustment technology is adopted, the data load processing process also needs to meet the following four constraints in addition to the spatial transfer requirements. When the data center load space adjustment technology is adopted, the data load processing process also needs to meet the following five constraints. Specifically:

[0147] Constraint 1: Data load balance. The total amount of data load allocated by the front-end portal server to each data center is balanced with the data load received by the front-end portal server. The data load received by the data center from other data centers is equal to the sum of the data loads sent to the data center by other data centers.

[0148] Constraint 2: Data load delay requirement. The average residence time of the data load to be processed in the target data center is less than or equal to the preset maximum delay time. Among them, the average residence time of the data load to be processed in the target data center = the average service rate of the front-end portal servers in operation configured in the target data center - (the amount of data load received by the target data center from each front-end portal server in the time period t + the amount of data load received by the target data center from other data centers in the time period t - the amount of data load transferred by the target data center due to load scheduling in the time period t) / the minimum number of active front-end portal servers configured in the target data center in the time period t.

[0149] Constraint 3: Data computing capacity requirements. In order to balance energy consumption and server operation, the data computing resources provided by the target data center in time period t are greater than or equal to the data load of all types processed by the target data center in time period t.

[0150] Constraint 4: Time transfer constraint. The sum of the change in the target data center's data load from the previous time period to the current time period and the initial data load of the previous time period is equal to the current data load of the current time period; the current data load of the current time period is less than or equal to the preset maximum storage capacity and greater than or equal to the preset ideal processing completed data load; the delayed processing time is greater than or equal to the delay tolerance time. Furthermore, in the case of time delay, the data load of the target data center before the delay and the data load after the delay remain unchanged.

[0151] Constraint 5: Spatial transfer constraint: For each data center, the data load transferred out of the data center should be equal to the sum of the loads received by the remaining data centers.

[0152] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0153] Based on the same inventive concept, the embodiment of the present application also provides a data load processing device for implementing the data load processing method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more data load processing device embodiments provided below can refer to the limitations on the data load processing method above, and will not be repeated here.

[0154] In an exemplary embodiment, Figure 4 As shown, a data load processing device 800 is provided, comprising: an acquisition module 200, a determination module 400 and a processing module 600, wherein:

[0155] An acquisition module 200 is used to acquire a data load to be processed;

[0156] A determination module 400, configured to determine a data load type of a data load to be processed;

[0157] Processing module 600 is used to perform delayed data processing on the data load to be processed based on the data load type and when the preset data load delay processing conditions are met, or to transfer the data load to be processed from the target data center to an off-site data center when the preset data load space transfer conditions are met. The off-site data center is used to perform data processing on the acquired data load to be processed.

[0158] In one embodiment, the processing module 600 is also used to, if the data load type is a first data load type, perform delayed data processing on the data load to be processed while satisfying a preset data load delay processing condition, or, if a preset data load space transfer condition is satisfied, transfer the data load to be processed from the target data center to an off-site data center, wherein the first data load type is a data load type that is not sensitive to time; and if the data load type is a second data load type, then, if the preset data load space transfer condition is satisfied, transfer the data load to be processed from the target data center to the off-site data center, wherein the second data load type is a data load type that is sensitive to time.

[0159] In one of the embodiments, the processing module 600 is also used to obtain the first sent data load and the first received data load of the front-end portal server, the second received data load of the target data center, and the second sent data load sent by other data centers except the target data center to the target data center within the same time period. When the first sent data load is equal to the first received data load and the second sent data load is equal to the second received data load, based on the data load type, delayed data processing is performed on the data load to be processed, or the data load to be processed is transferred from the target data center to an off-site data center.

[0160] In one embodiment, the processing module 600 is also used to obtain the average residence time of the data load to be processed in the target data center, and based on the average residence time being less than or equal to the preset maximum delay time, the data load to be processed is delayed data processing based on the data load type, or the data load to be processed is transferred from the target data center to an off-site data center.

[0161] In one embodiment, the processing module 600 is also used to obtain the total data load of the target data center, and on the basis that the total data load is less than or equal to the preset maximum data computing resource, based on the data load type, delayed data processing is performed on the data load to be processed, or the data load to be processed is transferred from the target data center to an off-site data center.

[0162] In one embodiment, the processing module 600 is also used to obtain the data load change of the target data center from the previous time period to the current time period, the initial data load of the previous time period, the current data load of the current time period, and the delayed processing time. When the sum of the data load change and the initial data load is equal to the current data load, the current data load is less than or equal to the preset maximum storage capacity and greater than or equal to the preset ideal processing completed data load, and the delayed processing time is greater than or equal to the delay tolerance time, based on the data load type, delayed data processing is performed on the data load to be processed, or the data load to be processed is transferred from the target data center to an off-site data center.

[0163] In one embodiment, the processing module 600 is also used to obtain the outgoing data load of the target data center and the incoming data load of other data centers except the target data center, and when the outgoing data load is equal to the incoming data load, the data load to be processed is transferred from the target data center to the remote data center.

[0164] Each module in the above data load processing device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module.

[0165] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 5 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as data load to be processed. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a data load processing method is implemented.

[0166] Those skilled in the art will understand that Figure 5The structure shown in the figure is a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0167] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0168] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0169] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0170] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0171] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0172] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0173] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A data load processing method, characterized in that: The method comprises: Obtaining the data load to be processed in the target data center; Determining a data load type of the data load to be processed; Based on the data load type, when the preset data load delay processing conditions are met, delayed data processing is performed on the data load to be processed, or, when the preset data load space transfer conditions are met, the data load to be processed is transferred from the target data center to an off-site data center, and the off-site data center is used to perform data processing on the acquired data load to be processed.

2. The method according to claim 1, characterized in that The method of performing delayed data processing on the data load to be processed based on the data load type and when a preset data load delay processing condition is met, or transferring the data load to be processed to a remote data center when a preset data load space transfer condition is met, includes: If the data load type is the first data load type, then, if a preset data load delay processing condition is met, delay data processing is performed on the data load to be processed, or, if a preset data load space transfer condition is met, the data load to be processed is transferred from the target data center to a remote data center, wherein the first data load type is a data load type that is not time-sensitive; If the data load type is the second data load type, then the data load to be processed is transferred from the target data center to a remote data center when a preset data load space transfer condition is met, wherein the second data load type is a time-sensitive data load type.

3. The method according to claim 1, characterized in that Based on the data load type, when a preset data load delay processing condition is met, delay data processing is performed on the data load to be processed, or, when a preset data load space transfer condition is met, the data load to be processed is transferred from the target data center to a remote data center, and the remote data center is used to perform data processing on the acquired data load to be processed, including: Acquire a first sending data load and a first receiving data load of the front-end portal server, a second receiving data load of the target data center, and a second sending data load sent to the target data center by other data centers except the target data center within the same time period; When the first sent data load is equal to the first received data load and the second sent data load is equal to the second received data load, based on the data load type, delayed data processing is performed on the data load to be processed, or the data load to be processed is transferred from the target data center to an off-site data center.

4. The method according to claim 1, characterized in that: Based on the data load type, when a preset data load delay processing condition is met, delay data processing is performed on the data load to be processed, or, when a preset data load space transfer condition is met, the data load to be processed is transferred from the target data center to a remote data center, and the remote data center is used to perform data processing on the acquired data load to be processed, including: Obtaining an average residence time of the to-be-processed data load in the target data center; On the basis that the average stay time is less than or equal to the preset maximum delay time, based on the data load type, delayed data processing is performed on the data load to be processed, or the data load to be processed is transferred from the target data center to a remote data center.

5. The method according to claim 1, characterized in that: Based on the data load type, when a preset data load delay processing condition is met, delay data processing is performed on the data load to be processed, or, when a preset data load space transfer condition is met, the data load to be processed is transferred from the target data center to a remote data center, and the remote data center is used to perform data processing on the acquired data load to be processed, including: Obtaining the total amount of data load of the target data center; On the basis that the total amount of data load is less than or equal to the preset maximum data computing resource, based on the data load type, delayed data processing is performed on the data load to be processed, or the data load to be processed is transferred from the target data center to an off-site data center.

6. The method according to claim 1, characterized in that The method of performing delayed data processing on the data load to be processed based on the data load type and when a preset data load delay processing condition is met, or transferring the data load to be processed from the target data center to a remote data center when a preset data load space transfer condition is met, includes: Obtaining a data load change amount of the target data center from a previous time period to a current time period, an initial data load amount of the previous time period, a current data load amount of the current time period, and a delayed processing time; When the sum of the data load change and the initial data load is equal to the current data load, the current data load is less than or equal to the preset maximum storage capacity and greater than or equal to the preset ideal processing completed data load, and the delayed processing time is greater than or equal to the delay tolerance time, based on the data load type, delayed data processing is performed on the data load to be processed, or the data load to be processed is transferred from the target data center to an off-site data center.

7. The method according to claim 1, characterized in that The transferring of the to-be-processed data load from the target data center to the remote data center when the preset data load space transfer condition is met includes: Obtaining the outgoing data load of the target data center and the incoming data load of other data centers except the target data center; When the outgoing data load is equal to the incoming data load, the to-be-processed data load is transferred from the target data center to the remote data center.

8. A data load processing device, characterized in that: The device comprises: An acquisition module, used for acquiring the data load to be processed; A determination module, used to determine the data load type of the data load to be processed; A processing module is used to perform delayed data processing on the data load to be processed based on the data load type and when a preset data load delay processing condition is met, or to transfer the data load to be processed from the target data center to an off-site data center when a preset data load space transfer condition is met, and the off-site data center is used to perform data processing on the acquired data load to be processed.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images