Double-center data distribution method, device, medium and system

Abstract

The invention provides a double-center data distribution method and device, a medium and a system. According to the method, the data size of the to-be-transmitted data in the task, the preset priority level, the unique identifier of the processing center and the enqueue time are added into consideration of the priority degree of the task in the priority queue, so that calculation of the priority degree better meets the actual working condition requirement; each task in the priority queue is distributed to the first processing center and the second processing center in the double processing centers, so that the data transmission request can be preferentially sent to the nearby data center, a large amount of delay caused by a physical distance is avoided, the loads of the two centers are kept relatively balanced, the distribution of double-center data processing is more reasonable, and the data transmission efficiency is improved. Therefore, the problem of unreasonable distribution of existing double-center data processing is solved.

Description

Technical Field

[0001] The present application relates to the technical field of data task allocation, and more particularly, to a dual-center data allocation method, apparatus, medium, and system. Background Art

[0002] Existing off-site dual-center computer rooms are prepared for disaster recovery, but data transmission usually distributes through a global reverse proxy to one of the two computer rooms, which may cause the following three problems.

[0003] First, due to the randomness of computer room selection, data may be allocated to the center with a greater physical distance, resulting in an extended processing time, that is, a significant increase in transmission delay.

[0004] Second, in the case of large-volume data transmission, the strategy of randomly allocating computer rooms may lead to an increased load on one computer room while the other computer room runs idle. For example, distributing two 4GB file transmission tasks to computer room A and two 4KB file transmission tasks to computer room B. Although the number of requests is the same, the total data volume is vastly different, resulting in vastly different load conditions in the two computer rooms.

[0005] Third, data transmission tasks usually also have priorities. A simple reverse proxy can only strictly send data transmission tasks to the back-end server for processing according to the arrival time sequence of requests, and cannot meet the requirements of priority processing.

[0006] That is, the existing dual-center data processing allocation is unreasonable. Summary of the Invention

[0007] The main object of the present application is to provide a dual-center data allocation method, apparatus, medium, and system to at least solve the problem of unreasonable allocation in existing dual-center data processing.

[0008] To achieve the above object, according to one aspect of the present application, a dual-center data allocation method is provided, the method comprising: obtaining weight parameters of each task in a priority queue, the weight parameters including the data size of the data to be transmitted in the task, a preset priority level, a unique identifier of a processing center, and an enqueue time; determining a final priority parameter according to the weight parameters of the task, the final priority parameter representing the priority degree of the task in the priority queue; and distributing each task in the priority queue to a first processing center and a second processing center in a dual-processing center according to the final priority parameter and the unique identifier of the processing center, so that the first processing center and the second processing center process the corresponding tasks respectively.

[0009] Optionally, according to the final priority parameter and the unique identifier of the processing center, distributing each task in the priority queue to a first processing center and a second processing center in a dual-processing center, including: when the unique identifier of the processing center is a first preset identifier, distributing them to the first processing center in the order of the final priority parameter; when the unique identifier of the processing center is a second preset identifier, distributing them to the second processing center in the order of the final priority parameter.

[0010] Optionally, determining the final priority parameter according to the weight parameter of the task, including: determining the level and value as the data scale level corresponding to the data size, the sum of the preset priority level and the enqueue time; when the unique identifier of the processing center is a first preset identifier, determining multiple preset bits in the final priority parameter as the level and value, and determining one preset bit in the final priority parameter as a first preset value; when the unique identifier of the processing center is a second preset identifier, determining multiple preset bits in the final priority parameter as the level and value, and determining one of the final priority parameters as a second preset value, where the second preset value is greater than the first preset value.

[0011] Optionally, each processing center in the dual-processing center is respectively provided with multiple lines; distributing each task in the priority queue to a first processing center and a second processing center in the dual-processing center, including: determining the priority of each line according to the transmission speed of each line, where the priority of the line is inversely proportional to the transmission speed, and the priority of the line is idle or busy; when the number of lines with the priority of the line being idle exceeds one, distributing each task in the priority queue to the target line in the dual-processing center, where the target line is the idle line with the maximum average transmission speed within a preset time period; when the number of lines with the priority of the line being idle is one, distributing each task in the priority queue to the idle line in the dual-processing center.

[0012] Optionally, the method further includes: when the enqueue time of the task in the priority queue is greater than or equal to the time threshold and the preset priority level of the task is lower than the priority level threshold, removing the task from the priority queue; when the enqueue time of the task in the priority queue is greater than or equal to the time threshold and the preset priority level of the task is higher than the priority level threshold, preferentially allocating the task.

[0013] Optionally, the method further includes: when the enqueue time of a task in the waiting queue is greater than or equal to a time threshold, replacing the task with a target task in the priority queue, where multiple tasks are stored in the waiting queue, and the target task is the task in the priority queue whose preset priority level is lower than a priority level threshold.

[0014] Optionally, when the number of tasks in the priority queue and the waiting queue changes, the method further includes: re - sorting the tasks in the priority queue and the waiting queue according to the preset priority level.

[0015] According to another aspect of the present application, there is provided a dual - center data distribution device, which includes: an acquisition unit, configured to acquire the weight parameters of each task in the priority queue, where the weight parameters include the data size of the data to be transmitted in the task, a preset priority level, the unique identifier of the processing center, and the enqueue time; a determination unit, configured to determine a final priority parameter according to the weight parameters of the task, where the final priority parameter represents the priority degree of the task in the priority queue; a first processing unit, configured to distribute each task in the priority queue to a first processing center and a second processing center in a dual - processing center according to the final priority parameter and the unique identifier of the processing center, so that the first processing center and the second processing center process the corresponding tasks respectively.

[0016] According to yet another aspect of the present application, there is provided a computer - readable storage medium, where the computer - readable storage medium includes a stored program, and when the program runs, it controls the device where the computer - readable storage medium is located to execute any one of the above - mentioned methods.

[0017] According to still another aspect of the present application, there is provided a dual - center data distribution system, which includes: one or more processors, a memory, and one or more programs, where the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include those for executing any one of the above - mentioned methods.

[0018] Applying the technical solution of the present application, by considering the data size, preset priority level, unique identifier of the processing center, and enqueue time of the data to be transmitted in the task when calculating the priority of the task in the priority queue, the calculation of the priority is made more in line with the actual working condition requirements. According to the final priority parameter and the unique identifier of the processing center, each task in the priority queue is distributed to the first processing center and the second processing center in the dual processing centers, so that the data transfer request can be preferentially sent to the nearest data center, avoiding a large amount of delay caused by the physical distance, ensuring that the loads of the two centers remain relatively balanced, making the distribution of dual-center data processing more reasonable, and thus solving the problem of unreasonable distribution of existing dual-center data processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The specification drawings forming a part of the present application are used to provide a further understanding of the present application. The schematic embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation to the present application. In the drawings:

[0020] Figure 1 shows a schematic flowchart of a dual-center data allocation method provided according to an embodiment of the present application;

[0021] Figure 2 shows a schematic structural diagram of a bidirectional queue and a selector provided according to an embodiment of the present application;

[0022] Figure 3 shows a selection schematic diagram of a selector provided according to an embodiment of the present application;

[0023] Figure 4 shows a schematic flowchart of another dual-center data allocation method provided according to an embodiment of the present application;

[0024] Figure 5 shows a schematic block diagram of a dual-center data allocation device provided according to an embodiment of the present application;

[0025] Figure 6 shows a schematic architecture diagram of a dual-center data allocation system according to the present application;

[0026] Figure 7 shows a schematic architecture diagram of a dual-center data allocation system of the existing solution. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0027] It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments can be combined with each other. The present application will be described in detail below with reference to the drawings and in combination with the embodiments.

[0028] To enable those skilled in the art to better understand the solution of this application, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of this application.

[0029] It should be noted that the terms "first", "second", etc. in the description and claims of this application and the above-mentioned accompanying drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so as to describe the embodiments of this application here. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0030] As introduced in the background art, the existing off-site dual-core computer rooms are prepared for disaster recovery, but data transmission usually needs to be distributed to one of the two computer rooms through a global reverse proxy, which may cause the following three problems. First, due to the randomness of computer room selection, data may be assigned to the center with a greater physical distance, resulting in an extended processing time, that is, a significant increase in transmission delay. Second, in the case of a large volume of data transmission, the strategy of randomly assigning computer rooms may lead to an increased load on one computer room while the other computer room is running idle. For example, two 4GB file transmission tasks are distributed to computer room A, and two other 4KB file transmission tasks are distributed to computer room B. Although the number of requests is the same, the total amount of data is vastly different, resulting in a huge difference in the load conditions of the two computer rooms. Third, data transmission tasks usually also have priorities. A simple reverse proxy can only strictly send data transmission tasks to the back-end server for processing according to the arrival time sequence of requests, and cannot meet the requirement of priority processing. To solve the problem of unreasonable allocation in the existing dual-core data processing, the embodiments of this application provide a dual-core data allocation method, device, medium and system.

[0031] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0032] In this embodiment, a dual - center data allocation method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer - executable instructions. And although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0033] Figure 1 It is a schematic flowchart of a dual - center data allocation method provided according to an embodiment of the present application. As Figure 1 shown, the method includes the following steps:

[0034] Step S101, obtain the weight parameters of each task in the priority queue. The above - mentioned weight parameters include the data size of the data to be transmitted in the above - mentioned task, the preset priority level, the unique identifier of the processing center, and the enqueue time;

[0035] Step S102, determine the final priority parameter according to the above - mentioned weight parameters of the above - mentioned task. The above - mentioned final priority parameter characterizes the priority degree of the above - mentioned task in the above - mentioned priority queue;

[0036] Among them, determining the final priority parameter according to the above - mentioned weight parameters of the above - mentioned task includes:

[0037] Determine that the level - sum value is the data - scale level corresponding to the above - mentioned data size, the sum of the above - mentioned preset priority level and the above - mentioned enqueue time;

[0038] When the unique identifier of the above - mentioned processing center is the first preset identifier, determine that multiple preset bits in the above - mentioned final priority parameter are the above - mentioned level - sum value, and determine that one preset bit in the above - mentioned final priority parameter is the first preset value;

[0039] When the unique identifier of the above - mentioned processing center is the second preset identifier, determine that multiple preset bits in the above - mentioned final priority parameter are the above - mentioned level - sum value, and determine that one of the above - mentioned final priority parameters is the second preset value, and the above - mentioned second preset value is greater than the above - mentioned first preset value.

[0040] The first preset value can be 1, and the second preset value can be 9.

[0041] Specifically, for example, first sum up the data - scale level corresponding to the above - mentioned data size, the above - mentioned preset priority level and the above - mentioned enqueue time to obtain the level - sum value, use the level - sum value as the lower five bits of the final priority parameter, and then determine the first bit of the final priority parameter according to the unique identifier of the above - mentioned processing center.

[0042] The architecture of the present application is as Figure 2As shown, tasks are submitted to the waiting queue and the task weight parameters are recorded; the priority calculator recalculates the task weights of all tasks in the priority queue; at the same time, the priority calculator calculates the priorities of all tasks in the waiting queue; the priority calculator reorders the tasks in the above two queues according to their priorities and puts them into the priority queue, clearing the waiting queue.

[0043] The weight parameters of a data transmission task include: data size (Size, in MB), custom priority level (Level, 0-9999, higher priority), computer room sign (Sign, 1 or 9, representing two computer rooms respectively), and queue entry time (Time, in minutes, a total of four digits, obtained by subtracting the current time from the queue entry time). Then the final priority parameter (Priority) of the task can be expressed as:

[0044] Priority=function(Size,Level,Sign,Time);

[0045] As an example, a specific implementation of a calculation method is given below:

[0046] Map the data size to four data scale levels from 0 to 3. [0,10] is 3, (10,100] is 2, (100,500] is 1, and (500,infinity) is 0. The unit is MB. Add the queue duration, data scale level, and custom priority level, and the resulting number is used as the lower 5 bits of the final Priority. Determine the computer room flag. If the computer room flag is 1, directly use 1 as the sixth-to-last digit to get the Priority. Otherwise, use 99999 to subtract the obtained Priority lower five digits, and use the difference to replace the aforementioned Priority lower five digits. Then use 9 as the sixth digit to get the final Priority.

[0047] Taking data A (Size = 10, Level = 1, Sign = 1, Time = 120) and data B (Size = 501, Level = 100, Sign = 9, Time = 20) as examples, the calculated weights are: Priority A = 100124, Priority B = 999878.

[0048] The design of priority weights allows data transfer requests to be sent to the nearest data center first, avoiding large delays caused by physical distance. The design of priority weights takes into account factors such as file size and business priority, and can flexibly adjust the order of tasks, improving the flexibility of task processing timing.

[0049] Step S103: Distribute each task in the above priority queue to the first processing center and the second processing center in the dual - processing center according to the above final priority parameter and the unique identifier of the above processing center, so that the first processing center and the second processing center process the corresponding above tasks respectively.

[0050] In the above steps, by taking into account the data size of the data to be transmitted in the task, the preset priority level, the unique identifier of the processing center, and the enqueue time in the priority of the task in the priority queue, the calculation of the priority degree is made more in line with the actual working condition requirements. According to the above final priority parameter and the unique identifier of the above processing center, each task in the above priority queue is distributed to the first processing center and the second processing center in the dual - processing center, so that the data transfer request can be preferentially sent to the nearby data center, avoiding a large amount of delay caused by the physical distance, ensuring that the loads of the two centers remain relatively balanced, making the distribution of dual - center data processing more reasonable, and thus solving the problem of unreasonable distribution of existing dual - center data processing.

[0051] The design of the dual - center shared task queue allows the two centers to process the low - priority tasks of the other party when the tasks in their own centers are completed, ensuring that the loads of the two centers remain relatively balanced. When new tasks are added to a center, the tasks of its own center can be preferentially processed again. This can effectively prevent the situation where one computer room is idle while the other is under high pressure. The selector then optimally selects a line within a center to complete the transmission task.

[0052] Among them, distributing each task in the above priority queue to the first processing center and the second processing center in the dual - processing center according to the above final priority parameter and the unique identifier of the above processing center includes:

[0053] When the unique identifier of the above processing center is the first preset identifier, distribute it to the above first processing center in the order of the above final priority parameter;

[0054] When the unique identifier of the above processing center is the second preset identifier, distribute it to the above second processing center in the order of the above final priority parameter.

[0055] For example, the unique identifier of the first processing center is 1 (i.e., the first preset identifier), and the unique identifier of the second processing center is 0 (i.e., the second preset identifier); then in the case of the first processing center 1, distribute it to the above first processing center in the order of the above final priority parameter (from high to low), and in the case where the unique identifier of the above processing center is 0, distribute it to the above second processing center in the order of the above final priority parameter, so as to achieve that the tasks close to the first processing center are assigned to the first processing center and the tasks close to the second processing center are assigned to the second processing center.

[0056] In an embodiment of the present application, multiple lines are respectively set in each of the above-mentioned dual processing centers; distributing each task in the above-mentioned priority queue to the first processing center and the second processing center in the dual processing center includes:

[0057] Determine the priority of each of the above-mentioned lines according to the transmission speed of each of the above-mentioned lines. The priority level of the above-mentioned line is inversely proportional to the above-mentioned transmission speed, and the priority of the above-mentioned line is idle or busy;

[0058] When the number of lines with the above-mentioned line priority being idle exceeds one, distribute each task in the above-mentioned priority queue to the target line in the above-mentioned dual processing center. The above-mentioned target line is the idle line with the maximum average transmission speed within a preset time period (which can be 30 min);

[0059] When the number of lines with the above-mentioned line priority being idle is one, distribute each task in the above-mentioned priority queue to the idle line in the above-mentioned dual processing center.

[0060] Specifically, as Figure 3 shown, each selector will respectively read tasks from the head and tail of the priority queue each time and add them to the selector queue until the selector queue reaches the capacity limit (for example, 5). The selector dequeues the data transmission tasks in ascending order of data and assigns them to the application for execution. When the selector queue is empty, repeat the above steps. Such a design is to reduce the number of interactions between the two queues. The shared tasks in the dual computer rooms can achieve the effect of load balancing between the computer rooms.

[0061] Subsequently, the selector will select one line from all the lines in the back line. Steps for determining the line priority: The priority of the idle line is higher than that of the busy line; when the line occupancy is the same (that is, the number of idle or busy lines exceeds one), the line with a greater average transmission speed in the past 30 min is preferred.

[0062] In an embodiment of the present application, the above-mentioned method further includes:

[0063] When the enqueue time of the above-mentioned task in the above-mentioned priority queue is greater than or equal to the time threshold, and the preset priority level of the above-mentioned task is lower than the priority level threshold, remove the above-mentioned task from the above-mentioned priority queue;

[0064] When the enqueue time of the above-mentioned task in the above-mentioned priority queue is greater than or equal to the time threshold, and the preset priority level of the above-mentioned task is higher than the priority level threshold, preferentially allocate the above-mentioned task.

[0065] Specifically, for example, if the time threshold is 10 minutes, then when the enqueue time of the above tasks in the priority queue exceeds 10 minutes and the preset priority level of the task is lower than the priority level threshold (e.g., 1000), it is determined that the task is not a task that must be completed, and the above task is removed from the above priority queue. When the enqueue time of the above tasks in the above priority queue is greater than or equal to the time threshold and the preset priority level of the above task is higher than the above priority level threshold, it is determined that the task is a task that must be completed, and then the execution order of the task is advanced.

[0066] In an embodiment of the present application, the above method further includes: when the enqueue time of the tasks in the waiting queue is greater than or equal to the time threshold, replacing the above tasks with the target tasks in the above priority queue. There are multiple tasks stored in the waiting queue, and the target task is the above task in the above priority queue whose preset priority level is lower than the priority level threshold.

[0067] Among them, when the enqueue time of the tasks in the waiting queue is greater than or equal to the time threshold (e.g., 10 minutes), first, pop the tasks from the priority queue to find the target tasks that need to be replaced. Then, determine the target tasks that need to be replaced, which can be selected according to the priority of the tasks or other metrics. The target tasks to be replaced are removed from the priority queue, and then the new tasks are inserted into the priority queue. Determine their positions in the queue according to their priority or other metrics. After the replacement process, the order of the tasks in the priority queue may change, and it is necessary to re-adjust the order of the tasks in the queue. Finally, ensure that the new tasks are correctly inserted and processed in the order of priority.

[0068] In an embodiment of the present application, when the number of tasks in the priority queue and the waiting queue changes, the above method further includes: re-ordering the tasks in the above priority queue and the above waiting queue according to the above preset priority level.

[0069] Among them, by re-ordering the tasks in the above priority queue and the above waiting queue according to the above preset priority level, it is convenient to improve the speed of traversing all tasks according to the priority degree subsequently.

[0070] In order to enable those skilled in the art to more clearly understand the technical solution of the present application, the implementation process of the dual-center data distribution method of the present application will be described in detail below in combination with specific embodiments.

[0071] This embodiment relates to a specific dual-center data distribution method, as Figure 4 shown, including:

[0072] Obtain the weight parameters of each task in the priority queue. The weight parameters include the data size of the data to be transmitted in the task, a preset priority level, the unique identifier of the processing center, and the enqueue time;

[0073] Determine that the level sum value is the sum of the data scale level corresponding to the data size, the preset priority level, and the enqueue time;

[0074] When the unique identifier of the processing center is the first preset identifier, determine that multiple preset bits in the final priority parameter are the level sum value, and determine that one preset bit in the final priority parameter is the first preset value;

[0075] When the unique identifier of the processing center is the second preset identifier, determine that multiple preset bits in the final priority parameter are the level sum value, and determine that one of the final priority parameters is the second preset value. The second preset value is greater than the first preset value. The final priority parameter characterizes the priority degree of the task in the priority queue;

[0076] When the unique identifier of the processing center is the first preset identifier, distribute them to the first processing center in the order of the final priority parameter;

[0077] When the unique identifier of the processing center is the second preset identifier, distribute them to the second processing center in the order of the final priority parameter.

[0078] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions. And although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0079] The embodiment of the present application also provides a dual - center data distribution device. It should be noted that the dual - center data distribution device of the embodiment of the present application can be used to execute the dual - center data distribution method provided by the embodiment of the present application. The device is used to implement the above - mentioned embodiments and preferred implementation manners, and those that have been described will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that realizes a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and contemplated.

[0080] The following introduces the dual - center data distribution device provided by the embodiment of the present application.

[0081] Figure 5 It is a structural block diagram of a dual - center data distribution device provided according to an embodiment of the present application. As Figure 5 shown, the device includes:

[0082] An acquisition unit 51 for acquiring the weight parameters of each task in the priority queue, where the weight parameters include the data size of the data to be transmitted in the task, a preset priority level, a unique identifier of the processing center, and the enqueue time;

[0083] A determination unit 52 for determining a final priority parameter according to the weight parameters of the task, where the final priority parameter represents the priority degree of the task in the priority queue;

[0084] A first processing unit 53 for distributing each task in the priority queue to a first processing center and a second processing center in a dual processing center according to the final priority parameter and the unique identifier of the processing center, so that the first processing center and the second processing center process the corresponding tasks respectively.

[0085] In the above device, by taking into account the data size of the data to be transmitted in the task, the preset priority level, the unique identifier of the processing center, and the enqueue time in the priority degree of the task in the priority queue, the calculation of the priority degree is made more in line with the actual working condition requirements. According to the final priority parameter and the unique identifier of the processing center, each task in the priority queue is distributed to the first processing center and the second processing center in the dual processing center, so that the data transfer request can be preferentially sent to the nearest data center, avoiding a large amount of delay caused by the physical distance, ensuring that the loads of the two centers remain relatively balanced, making the distribution of dual-center data processing more reasonable, and thus solving the problem of unreasonable distribution of existing dual-center data processing.

[0086] In an embodiment of the present application, the first processing unit includes a first processing module and a second processing module. The first processing module is used to distribute to the first processing center in the order of the final priority parameter when the unique identifier of the processing center is a first preset identifier; the second processing module is used to distribute to the second processing center in the order of the final priority parameter when the unique identifier of the processing center is a second preset identifier.

[0087] In an embodiment of the present application, the determination unit includes a first determination module, a second determination module, and a third determination module. The first determination module is used to determine the level and value as the data scale level corresponding to the above data size, the sum of the above preset priority levels and the above enqueue time; the second determination module is used to determine that when the unique identifier of the above processing center is the first preset identifier, multiple preset bits in the above final priority parameter are the above level and value, and determine that one preset bit in the above final priority parameter is the first preset value; the third determination module is used to determine that when the unique identifier of the above processing center is the second preset identifier, multiple preset bits in the above final priority parameter are the above level and value, and determine that one of the above final priority parameters is the second preset value, and the above second preset value is greater than the above first preset value.

[0088] In an embodiment of the present application, each processing center in the above dual processing center is respectively provided with multiple lines; the first processing unit includes a fourth determination module, a third processing module, and a fourth processing module; the fourth determination module is used to determine the priority of each of the above lines according to the transmission speed of each of the above lines, the priority degree of the above line is inversely proportional to the above transmission speed, and the priority of the above line is idle or busy; the third processing module is used to, when the number of lines with the above line priority being idle exceeds one, distribute each task in the above priority queue to the target line in the above dual processing center, and the above target line is the idle line with the maximum average transmission speed within a preset time period; the fourth processing module is used to, when the number of lines with the above line priority being idle is one, distribute each task in the above priority queue to the idle line in the above dual processing center.

[0089] In an embodiment of the present application, the above device further includes a second processing unit and a third processing unit. The second processing unit is used to remove the above task from the above priority queue when the enqueue time of the above task in the above priority queue is greater than or equal to the time threshold and the preset priority level of the above task is lower than the priority level threshold; the third processing unit is used to preferentially allocate the above task when the enqueue time of the above task in the above priority queue is greater than or equal to the time threshold and the preset priority level of the above task is higher than the above priority level threshold.

[0090] In an embodiment of the present application, the above device further includes a fourth processing unit; the fourth processing unit is used to replace the above task with the target task in the above priority queue when the enqueue time of the task in the waiting queue is greater than or equal to the time threshold, and multiple tasks are stored in the waiting queue, and the above target task is the above task in the above priority queue with a preset priority level lower than the priority level threshold.

[0091] In one embodiment of the present application, the above-mentioned device further includes a fifth processing unit, which is used to reorder the tasks in the above-mentioned priority queue and the waiting queue according to the above-mentioned preset priority levels when the number of tasks in the priority queue and the waiting queue changes.

[0092] The above-mentioned dual-center data distribution device includes a processor and a memory. The above-mentioned acquisition unit, determination unit, first processing unit, etc. are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to implement corresponding functions. The above-mentioned modules are all located in the same processor; or, the above-mentioned each module is located in different processors in any combined form.

[0093] The processor contains a kernel, and the kernel retrieves the corresponding program units from the memory. One or more kernels can be set, and by adjusting the kernel parameters, the problem of unreasonable distribution in the existing dual-center data processing can be solved.

[0094] The memory may include non-permanent memory in a computer-readable medium, forms such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash memory (flash RAM), and the memory includes at least one storage chip.

[0095] An embodiment of the present invention provides a computer-readable storage medium. The above-mentioned computer-readable storage medium includes a stored program, wherein when the above-mentioned program runs, it controls the device where the above-mentioned computer-readable storage medium is located to execute the above-mentioned dual-center data distribution method.

[0096] An embodiment of the present invention provides a processor, which is used to run a program. When the above-mentioned program runs, it executes the above-mentioned dual-center data distribution method.

[0097] An embodiment of the present invention provides a device. The device includes a processor, a memory, and a program stored on the memory and executable on the processor. When the processor executes the program, it implements at least the following steps: obtaining the weight parameters of each task in the priority queue, where the above-mentioned weight parameters include the data size of the data to be transmitted in the above-mentioned task, the preset priority level, the unique identifier of the processing center, and the enqueue time; determining the final priority parameter according to the above-mentioned weight parameters of the above-mentioned task, where the above-mentioned final priority parameter represents the priority degree of the above-mentioned task in the above-mentioned priority queue; distributing each task in the above-mentioned priority queue to the first processing center and the second processing center in the dual processing center according to the above-mentioned final priority parameter and the unique identifier of the above-mentioned processing center, so that the above-mentioned first processing center and the above-mentioned second processing center respectively process the corresponding above-mentioned tasks. The device herein can be a server, a PC, a PAD, a mobile phone, etc.

[0098] The present application also provides a computer program product, which, when executed on a data processing device, is adapted to execute a program initialized with at least the following method steps: obtaining the weight parameters of each task in the priority queue, where the weight parameters include the data size of the data to be transmitted in the task, a preset priority level, the unique identifier of the processing center, and the enqueue time; determining a final priority parameter according to the weight parameters of the task, where the final priority parameter characterizes the priority degree of the task in the priority queue; and distributing each task in the priority queue to a first processing center and a second processing center in a dual processing center according to the final priority parameter and the unique identifier of the processing center, so that the first processing center and the second processing center process the corresponding tasks respectively.

[0099] The present application also provides a dual-center data distribution system, which includes: one or more processors, a memory, and one or more programs, where the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include those for executing any one of the above methods. By taking into account the data size of the data to be transmitted in the task, the preset priority level, the unique identifier of the processing center, and the enqueue time when considering the priority degree of the task in the priority queue, the calculation of the priority degree is made more in line with the actual working condition requirements. According to the final priority parameter and the unique identifier of the processing center, each task in the priority queue is distributed to the first processing center and the second processing center in the dual processing center, so that the data transfer request can be preferentially sent to the nearest data center, avoiding a large amount of delay caused by the physical distance, ensuring that the loads of the two centers remain relatively balanced, making the distribution of dual-center data processing more reasonable, and thus solving the problem of unreasonable distribution of existing dual-center data processing.

[0100] As Figure 6 shown, a schematic diagram of the architecture of the dual-center data distribution system of the present application is presented and will not be elaborated here; as Figure 7 shown, in the existing solution, data will be randomly distributed to the reverse proxies in two computer rooms at the global reverse proxy, and then distributed by them to the machines in the corresponding computer rooms.

[0101] Existing solutions are applicable to the application architecture design of a single data center. However, in the design of a dual-data-center architecture, there is an indicator that cannot be ignored, namely the physical latency of requests reaching the server. Although signals travel at the speed of light in optical fibers, there is still a non-negligible delay when spanning thousands of kilometers. Therefore, it is necessary to design a load balancing method that takes this factor into account. In the general architecture of existing solutions, the global reverse proxy cannot determine the current load conditions of the two data centers. As a result, the load conditions of the data centers may vary greatly, which may affect the business processing efficiency and cause some applications to crash. Secondly, simply using simple load balancing cannot take into account the adjustment of priorities and can only process tasks strictly according to objective indicators. In extreme cases, tasks with high business priorities may be delayed due to the previous request being blocked for too long. Finally, requests on the public Internet may span a long distance, and combined with the distance of off-site disaster recovery between the two data centers, if the requests take a roundabout route, the inevitable physical delay may lead to a decrease in system throughput.

[0102] Obviously, those skilled in the art should understand that the various modules or steps of the present invention described above can be implemented by a general-purpose computing device. They can be concentrated on a single computing device or distributed on a network composed of multiple computing devices. They can be implemented by program code executable by the computing device. Thus, they can be stored in a storage device and executed by the computing device. And in some cases, the steps shown or described can be executed in a sequence different from that here, or they can be separately fabricated into individual integrated circuit modules, or multiple modules or steps among them can be fabricated into a single integrated circuit module for implementation. In this way, the present invention is not limited to any specific combination of hardware and software.

[0103] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0104] This application is described with reference to the flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It should be understood that each flow and / or block in the flowchart and / or block diagram, and combinations of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processors of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions executed by the processor of the computer or other programmable data processing device generate means for implementing the functions specified in one Figure 1 flow or multiple flows and / or blocks Figure 1 or multiple blocks.

[0105] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory produce a manufacture including instruction means that implement the functions specified in one Figure 1 flow or multiple flows and / or blocks Figure 1 or multiple blocks.

[0106] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one Figure 1 flow or multiple flows and / or blocks Figure 1 or multiple blocks.

[0107] In a typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and memory.

[0108] The memory may include non-permanent memory in the form of computer-readable media, random access memory (RAM), and / or non-volatile memory such as read-only memory (ROM) or flash memory (flash RAM). The memory is an example of computer-readable media.

[0109] A computer-readable medium includes both permanent and non-permanent, removable and non-removable media and can implement information storage by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory, or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD), or other optical storage, magnetic cassettes, magnetic disk storage, or other magnetic storage devices, or any other non-transitory medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer-readable medium does not include transitory computer-readable media, such as modulated data signals and carrier waves.

[0110] It should also be noted that the term "comprising", "including", or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the statement "comprising one..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus comprising the element.

[0111] From the above description, it can be seen that the above embodiments of the present application achieve the following technical effects:

[0112] 1) In the dual-center data distribution method of the present application, by considering the data size, preset priority level, unique identifier of the processing center, and enqueue time of the data to be transmitted in the task when calculating the priority of the task in the above priority queue, the calculation of the priority is made more in line with the actual working condition requirements. According to the above final priority parameter and the unique identifier of the processing center, each task in the above priority queue is distributed to the first processing center and the second processing center in the dual processing centers, so that the data transfer request can be preferentially sent to the nearest data center, avoiding a large amount of delay caused by the physical distance, ensuring that the loads of the two centers remain relatively balanced, and making the distribution of dual-center data processing more reasonable, thereby solving the problem of unreasonable distribution in existing dual-center data processing.

[0113] 2) The dual-center data distribution device of the present application considers the data size, preset priority level, unique identifier of the processing center, and enqueue time of the data to be transmitted in the task when calculating the priority of the task in the above-mentioned priority queue, so that the calculation of the priority is more in line with the actual working condition requirements. According to the above-mentioned final priority parameter and the unique identifier of the processing center, each task in the above-mentioned priority queue is distributed to the first processing center and the second processing center in the dual processing centers, so that the data transfer request can be preferentially sent to the nearest data center, avoiding a large amount of delay caused by the physical distance, ensuring that the loads of the two centers remain relatively balanced, making the distribution of dual-center data processing more reasonable, and thus solving the problem of unreasonable distribution of existing dual-center data processing.

[0114] The above are only the preferred embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the protection scope of the present application.

Claims

1. A dual-center data distribution method, characterized in that Including: Obtain the weight parameters of each task in the priority queue, where the weight parameters include the data size of the data to be transmitted in the task, a preset priority level, the unique identifier of the processing center, and the enqueue time; Determine the final priority parameter according to the weight parameter of the task, where the final priority parameter represents the priority degree of the task in the priority queue; Distribute each task in the priority queue to the first processing center and the second processing center in the dual processing center according to the final priority parameter and the unique identifier of the processing center, so that the first processing center and the second processing center process the corresponding tasks respectively.

2. The method according to claim 1, wherein Distributing each task in the priority queue to the first processing center and the second processing center in the dual processing center according to the final priority parameter and the unique identifier of the processing center includes: When the unique identifier of the processing center is the first preset identifier, distribute it to the first processing center in the order of the final priority parameter; When the unique identifier of the processing center is the second preset identifier, distribute it to the second processing center in the order of the final priority parameter.

3. The method according to claim 1, characterized in that Determining the final priority parameter according to the weight parameter of the task includes: Determine that the level and value are the data scale level corresponding to the data size, the sum of the preset priority level and the enqueue time; When the unique identifier of the processing center is the first preset identifier, determine that multiple preset bits in the final priority parameter are the level and value, and determine that one preset bit in the final priority parameter is the first preset value; When the unique identifier of the processing center is the second preset identifier, determine that multiple preset bits in the final priority parameter are the level and value, and determine that one of the final priority parameters is the second preset value, where the second preset value is greater than the first preset value.

4. The method according to claim 1, wherein Each processing center in the dual processing center is respectively provided with multiple lines; distributing each task in the priority queue to the first processing center and the second processing center in the dual processing center includes: Determine the priority of each line according to the transmission speed of each line, where the priority degree of the line is inversely proportional to the transmission speed, and the priority of the line is idle or busy; When the number of lines with the priority of the line being idle exceeds one, distribute each task in the priority queue to the target line in the dual processing center, where the target line is the idle line with the maximum average transmission speed within a preset time period; When the number of lines with the priority of the line being idle is one, distribute each task in the priority queue to the idle line in the dual processing center.

5. The method according to claim 1, wherein The method further includes: When the enqueue time of the task in the priority queue is greater than or equal to the time threshold and the preset priority level of the task is lower than the priority level threshold, remove the task from the priority queue. When the enqueue time of the task in the priority queue is greater than or equal to the time threshold and the preset priority level of the task is higher than the priority level threshold, the task is preferentially allocated.

6. The method according to claim 1, characterized in that The method further includes: When the enqueue time of the task in the waiting queue is greater than or equal to the time threshold, the task is replaced with a target task in the priority queue. There are multiple tasks stored in the waiting queue, and the target task is the task in the priority queue with a preset priority level lower than the priority level threshold.

7. The method according to any one of claims 1 to 6, characterized in that, When the number of tasks in the priority queue and the waiting queue changes, the method further includes: Re - sorting the tasks in the priority queue and the waiting queue according to the preset priority level.

8. A dual-center data distribution device, characterized in that, It includes: An acquisition unit, configured to acquire the weight parameters of each task in the priority queue. The weight parameters include the data size of the data to be transmitted in the task, the preset priority level, the unique identifier of the processing center, and the enqueue time. A determination unit, configured to determine the final priority parameter according to the weight parameters of the task. The final priority parameter represents the priority degree of the task in the priority queue. A first processing unit, configured to distribute each task in the priority queue to the first processing center and the second processing center in the dual - processing center according to the final priority parameter and the unique identifier of the processing center, so that the first processing center and the second processing center process the corresponding tasks respectively.

9. A computer-readable storage medium, characterized in that, The computer - readable storage medium includes a stored program. When the program runs, it controls the device where the computer - readable storage medium is located to execute the method according to any one of claims 1 to 7.

10. A dual - center data distribution system, characterized in that, It includes: One or more processors, a memory, and one or more programs. The one or more programs are stored in the memory and are configured to be executed by the one or more processors. The one or more programs include those for executing the method according to any one of claims 1 to 7.

Images