A Delayed Message Queue Scheduling Method, System and Storage Medium Based on a Dynamic Time Window

Abstract

The present invention provides a delay message queue scheduling method, system and storage medium based on a dynamic time window for a payment transaction system. The method includes: dividing the delay messages in the delay message queue into N scheduling queues based on the delay duration and the priority of the delay messages, dynamically dividing M i time windows for the i-th scheduling queue among the N scheduling queues based on the delay duration of the delay messages, constructing N linear storage structures as a primary storage area to store M i time windows in the corresponding N scheduling queues, constructing a dynamic circular buffer for each linear storage structure as a secondary storage area to store the delay messages within the current j-th time window to be processed, generating N processing threads corresponding to the N scheduling queues, obtaining the number of processing cores K of the current processor, and executing the N processing threads on the K processing cores to process the delay messages in the N scheduling queues. The present invention improves the execution efficiency of the processing cores.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing and task scheduling, and particularly relates to a delay message queue scheduling method, system and storage medium based on a dynamic time window. Background Art

[0002] In a payment transaction system, the timely processing of delay messages in a delay message queue in a distributed system (such as order payment timeout, timed tasks, etc.) plays a key role in the performance of the system. The main existing processing methods for delay messages are as follows: (1) The time wheel algorithm, which uses a fixed time scale, resulting in waste of resources in sparse scenarios and thus relatively low CPU utilization; (2) The implementation based on Redis ZSET: large serialization overhead and relatively large cross-node synchronization delay. Therefore, there is an urgent need for an efficient scheduling and processing method for delay message queues. Summary of the Invention

[0003] In view of one or more of the above technical defects in the prior art, the present invention proposes the following technical solutions.

[0004] A delay message queue scheduling method based on a dynamic time window, for a payment transaction system, the method includes:

[0005] A processing step of dividing delay messages in the delay message queue into N scheduling queues based on the delay duration and the priority of the delay messages, where N≥2;

[0006] An allocation step of dynamically dividing M i time windows for the i-th scheduling queue among the N scheduling queues based on the delay duration of the delay message, where M i ≥2 and 1≤i≤N;

[0007] A storage step of constructing N linear storage structures as a primary storage area to store M i time windows corresponding to the N scheduling queues, and constructing a dynamic circular buffer for each linear storage structure as a secondary storage area to store the delay messages within the j-th time window to be processed currently, where 1≤j≤M i ;

[0008] A scheduling step of generating N processing threads corresponding to the N scheduling queues, obtaining the number of processing cores K of the current processing machine, and executing the N processing threads on the K processing cores to process the delay messages in the N scheduling queues, where K≥2.

[0009] Further, the operation of constructing a dynamic circular buffer for each linear storage structure as a secondary storage area to store the delay messages in the current j-th time window to be processed is as follows: when j≥2, when processing the delay messages in the (j - 1)-th time window, obtain the number num of delay messages in the j-th time window mes , based on num mes apply for a dynamic circular buffer of the corresponding size. After the delay messages in the (j - 1)-th time window are processed, destroy the dynamic circular buffer corresponding to the (j - 1)-th time window. When j = 1, obtain the number num of delay messages in the j-th time window mes , based on num mes apply for a dynamic circular buffer of the corresponding size.

[0010] Further, the linear storage structure is an array storage structure.

[0011] Further, the operation of dividing the delay messages in the delay message queue into N scheduling queues based on the delay duration and the delay message priority is as follows: initialize N scheduling queues based on the number of delay message priorities; read each delay message from the delay message queue, and allocate it to the k-th scheduling queue based on the delay duration and priority of the delay message until all the delay messages in the delay message queue are allocated to obtain N scheduling queues. The initial queue priority of the k-th scheduling queue is k, where 1≤k≤N, and the determination method of k is as follows

[0012] ;

[0013] represents the priority of the delay message represents the delay duration of the delay message represents the first time threshold.

[0014] Further, the operation of the allocation step is as follows: obtain the delay duration of the delay messages in the i-th scheduling queue, divide the delay messages with the same delay duration into one time window, and a total of M i ’ time windows are obtained. Determine whether the number of delay messages in each time window is greater than the second threshold. If so, split the time window into two time windows, and use the two split time windows to replace the original time window to obtain M i ’’ time windows; then determine M iWhether the number of delayed messages in each time window within a time window is less than a third threshold. If so, compare the number of delayed messages in two time windows adjacent to this time window, and determine the time window with fewer delayed messages as the target time window. If this time window is the first time window, determine the second time window as the target time window. If this time window is the M i ’’th time window, determine the M i ’’-1th time window as the target time window, merge the delayed messages in this time window into the target time window, and destroy this time window. Finally, obtain M i time windows.

[0015] Further, the operation of the scheduling step is as follows: The priorities of N processing threads correspond to the initial queue priorities of N scheduling queues. Obtain the current status of each of the K processing cores, where the current status indicates the idle situation of the processing cores. If K≥N, select N processing cores from the K processing cores based on the current status and allocate them to the N processing threads in sequence. Among them, the processing core allocated to the processing thread corresponding to the Nth scheduling queue has the most idle current status, and so on. If N >K, calculate the current scheduling priority of each scheduling queue based on the total delay duration of the delayed messages in each scheduling queue and the initial queue priority of this scheduling queue. Select the top K scheduling queues in the current scheduling priority ranking, and allocate the K processing cores to the K processing threads corresponding to the top K scheduling queues in the current scheduling priority ranking for parallel processing. Among them, the processing core allocated to the thread corresponding to the scheduling queue with the highest current scheduling priority is the most idle, and so on. For the remaining N-K threads, when a processing core becomes idle, allocate them to the idle processing core based on the current scheduling priorities of the N-K threads. Among them, the current scheduling priority of the kth scheduling queue is calculated as follows:

[0016] ;

[0017] where k represents the kth scheduling queue and its initial queue priority, represents the total delay duration of the delayed messages in the kth scheduling queue, represents the median of the delay times of the delayed messages in the kth scheduling queue.

[0018] The present invention also proposes a delayed message queue scheduling device based on a dynamic time window for a payment transaction system. The device includes:

[0019] A processing unit divides the delay messages in the delay message queue into N scheduling queues based on the delay duration and the priority of the delay messages, where N≥2;

[0020] An allocation unit dynamically divides M time windows for the i-th scheduling queue among the N scheduling queues based on the delay duration of the delay message, where M≥2, 1≤i≤N; i time windows for the i-th scheduling queue among the N scheduling queues based on the delay duration of the delay message, where M i ≥2, 1≤i≤N;

[0021] A storage unit constructs N linear storage structures as a primary storage area to store the M time windows corresponding to the N scheduling queues, and constructs a dynamic circular buffer for each linear storage structure as a secondary storage area to store the delay messages within the j-th time window to be processed currently, where 1≤j≤M i ; i ;

[0022] A scheduling unit generates N processing threads corresponding to the N scheduling queues, obtains the number K of processing cores of the current processor, and executes the N processing threads on the K processing cores to process the delay messages in the N scheduling queues, where K≥2.

[0023] Further, the operation of constructing a dynamic circular buffer for each linear storage structure as a secondary storage area to store the delay messages within the j-th time window to be processed currently is: when j≥2, when processing the delay messages in the (j - 1)-th time window, obtain the number num of delay messages in the j-th time window mes , and apply for a dynamic circular buffer of the corresponding size based on num mes , after the delay messages in the (j - 1)-th time window are processed, destroy the dynamic circular buffer corresponding to the (j - 1)-th time window, when j = 1, obtain the number num of delay messages in the j-th time window mes , and apply for a dynamic circular buffer of the corresponding size based on num mes .

[0024] Further, the linear storage structure is an array storage structure.

[0025] Further, the operation of dividing the delay messages in the delay message queue into N scheduling queues based on the delay duration and the priority of the delay messages is: initialize N scheduling queues based on the number of priorities of the delay messages; read each delay message from the delay message queue, and allocate it to the k-th scheduling queue based on the delay duration and priority of the delay message until all the delay messages in the delay message queue are allocated to obtain N scheduling queues, and the initial queue priority of the k-th scheduling queue is k, where 1≤k≤N, and the determination method of k is:

[0026] ;

[0027] represents the priority of the delayed message, represents the delay duration of the delayed message, represents the first time threshold.

[0028] Further, the operation of the allocation unit is as follows: obtain the delay duration of the delayed messages in the i-th scheduling queue, divide the delayed messages with the same delay duration into one time window, and a total of M i ' time windows are obtained. Determine whether the number of delayed messages in each time window is greater than the second threshold. If so, split the time window into two time windows, and replace the original time window with the two split time windows to obtain M i '' time windows; then determine whether the number of delayed messages in each of the M i '' time windows is less than the third threshold. If so, compare the number of delayed messages in the two time windows adjacent to this time window, and determine the time window with fewer delayed messages as the target time window. If this time window is the first time window, then determine the second time window as the target time window. If this time window is the M i ''-th time window, then determine the M i ''-1-th time window as the target time window, merge the delayed messages in this time window into the target time window, and destroy this time window. Finally, M i time windows are obtained.

[0029] Further, the operation of the scheduling step is as follows: the priorities of N processing threads correspond to the initial queue priorities of N scheduling queues. Obtain the current state of each of the K processing cores. The current state represents the idle situation of the processing core. If K≥N, select N processing cores from the K processing cores based on the current state and allocate them to the N processing threads in sequence. Among them, the processing core allocated to the processing thread corresponding to the N-th scheduling queue has the most idle current state, and so on; if N K, calculate the current scheduling priority of each scheduling queue based on the total delay duration of the delayed messages in each scheduling queue and the initial queue priority of the scheduling queue, select the top K scheduling queues with the current scheduling priority sorted, and allocate K processing cores to the K processing threads corresponding to the top K scheduling queues with the current scheduling priority sorted for parallel processing based on the current scheduling priority. Among them, the processing core allocated to the thread corresponding to the scheduling queue with the highest current scheduling priority is the most idle, and so on. For the remaining N - K threads, when a processing core is idle, allocate them to the idle processing core based on the current scheduling priority of the N - K threads. Among them, the calculation method of the current scheduling priority of the kth scheduling queue is as follows:

[0030] ;

[0031] where k represents the kth scheduling queue and its initial queue priority, represents the total delay duration of the delayed messages in the kth scheduling queue, represents the median of the delay times of the delayed messages in the kth scheduling queue.

[0032] The present invention also proposes a computer-readable storage medium, on which computer program code is stored. When the computer program code is executed by a computer, the method described above is executed.

[0033] The technical effect of the present invention is as follows: A method, system and storage medium for scheduling a delayed message queue based on a dynamic time window according to the present invention. The method includes: processing step S101, dividing the delayed messages in the delayed message queue into N scheduling queues based on the delay duration and the delay message priority, where N ≥ 2; allocation step S102, dynamically dividing M i time windows for the ith scheduling queue among the N scheduling queues based on the delay duration of the delayed messages, where M i ≥ 2, 1 ≤ i ≤ N; storage step S103, constructing N linear storage structures as a primary storage area to store M i time windows corresponding to the N scheduling queues, and constructing a dynamic circular buffer for each linear storage structure as a secondary storage area to store the delayed messages within the current jth time window to be processed, where 1 ≤ j ≤ M i; Scheduling step S104, generating N processing threads corresponding to N scheduling queues, obtaining the number K of processing cores of the current processor (i.e., the available number of processing cores), and executing the N processing threads on the K processing cores to process the delay messages in the N scheduling queues, where K≥2. The present invention creatively proposes to divide the delay message queue into multiple scheduling queues based on the delay duration and the priority of the delay message. The initial scheduling priority of each scheduling queue is the same. Since both the delay duration and the priority of the delay message are considered when allocating the delay message to the corresponding scheduling queue, each delay message can be processed efficiently. In the present invention, the time window is dynamically divided to ensure that the number of messages in each time window is neither too large nor too small, so that the proposed two-level memory structure of the present invention can be well combined. N linear storage structures are used as the first-level storage area to store M in the N scheduling queues i time windows. The linear storage structure improves the query efficiency during window execution. Using a dynamic circular buffer as the second-level storage area to store the delay messages in the current j-th time window to be processed can efficiently utilize the memory space and reduce the memory occupancy rate. In order to minimize the time wasted by locking and unlocking operations as much as possible, by corresponding threads to processing cores, the lock operations are reduced, the execution efficiency of the processing cores is improved, the efficiency of processing delay messages is improved, and the memory occupancy is greatly reduced. Description of the Drawings

[0034] Other features, objects, and advantages of the present application will become more apparent by reading the detailed description of the non-limiting embodiments with reference to the following drawings.

[0035] Figure 1 is a flowchart of a method for scheduling a delay message queue based on a dynamic time window according to an embodiment of the present invention.

[0036] Figure 2 is a structural diagram of a device for scheduling a delay message queue based on a dynamic time window according to an embodiment of the present invention. Detailed Embodiment

[0037] The present application will be further described in detail below with reference to the drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the related invention and are not intended to limit the invention. In addition, it should be noted that, for the sake of description, only parts related to the relevant invention are shown in the drawings.

[0038] It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments can be combined with each other. The present application will be described in detail below with reference to the drawings and embodiments.

[0039] Figure 1A delay message queue scheduling method based on a dynamic time window of the present invention is shown, which is used for a payment transaction system. The method includes:

[0040] Processing step S101: Divide the delay messages in the delay message queue into N scheduling queues based on the delay duration and the delay message priority, where N≥2;

[0041] Allocation step S102: Dynamically divide M i time windows for the i-th scheduling queue among the N scheduling queues based on the delay duration of the delay message, where M i ≥2 and 1≤i≤N;

[0042] Storage step S103: Construct N linear storage structures as the primary storage area to store the M i time windows corresponding to the N scheduling queues, and construct a dynamic circular buffer for each linear storage structure as the secondary storage area to store the delay messages within the currently to-be-processed j-th time window, where 1≤j≤M i ;

[0043] Scheduling step S104: Generate N processing threads corresponding to the N scheduling queues, obtain the number K of processing cores of the current processor (i.e., the available number of processing cores), and execute the N processing threads on the K processing cores to process the delay messages in the N scheduling queues, where K≥2.

[0044] In the present invention, in order to enable the delayed messages to be processed as soon as possible, it is creatively proposed to first divide the delay message queue into multiple scheduling queues based on the delay duration and the delay message priority. The initial scheduling priorities of each scheduling queue are the same. Since both the delay duration and the delay message priority of the delay message are considered when allocating the delay message to the corresponding scheduling queue, each delayed message can be processed efficiently. Further, in order to improve the processing efficiency of the delayed messages, M i time windows are dynamically divided for the i-th scheduling queue among the N scheduling queues based on the delay duration of the delay message. In the present invention, the method of dynamically dividing time windows is adopted to ensure that the messages in each time window are neither too many nor too few, so that the secondary memory structure proposed in the present invention can be well combined. N linear storage structures are used as the primary storage area to store the M iFor a time window, the linear storage structure improves the query efficiency during window execution. Using a dynamic circular buffer as a secondary storage area to store the delayed messages within the j-th time window to be processed currently can efficiently utilize the memory space and reduce the memory occupancy rate. To minimize the time wasted by locking and unlocking operations as much as possible, by the method of corresponding threads to processing cores, the lock operations are reduced, and the execution efficiency of the processing cores is improved. That is, by generating N processing threads corresponding to N scheduling queues and obtaining the number K of processing cores of the current processor, the K processing cores execute the N processing threads to process the delayed messages in the N scheduling queues. Through this scheduling method of the present invention, the efficiency of processing delayed messages is improved, and the memory occupancy is greatly reduced. This is an important inventive concept of the present invention.

[0045] In one embodiment, the operation of constructing a dynamic circular buffer as a secondary storage area for each linear storage structure to store the delayed messages within the j-th time window to be processed currently is as follows: when j≥2, when processing the delayed messages in the (j - 1)-th time window, obtain the number num of the delayed messages within the j-th time window mes , based on num mes apply for a dynamic circular buffer of the corresponding size. After the delayed messages in the (j - 1)-th time window are processed, destroy the dynamic circular buffer corresponding to the (j - 1)-th time window. When j = 1, obtain the number num of the delayed messages within the j-th time window mes , based on num mes apply for a dynamic circular buffer of the corresponding size.

[0046] In the present invention, by the method of pre-constructing a dynamic circular buffer, after the messages in the (j - 1)-th (j≥2) window are processed, the delayed messages in the j-th window have been loaded into the dynamic circular buffer, and the dynamic circular buffer corresponding to the (j - 1)-th time window is destroyed. That is, apply for the corresponding memory size as the dynamic circular buffer based on the number of delayed messages to be processed within each time window, and apply in advance to avoid the processing core waiting for the loading of the delayed messages, improve the overall efficiency of the system, and improve the memory utilization rate. Through this secondary storage method, when processing the delayed messages, the memory fragmentation rate is reduced by 72%, and the pause time of the processing core is shortened to within 3 ms, greatly improving the system performance.

[0047] In one embodiment, the linear storage structure is an array storage structure, and the query time of the array is O(n), which greatly improves the search speed of the time window.

[0048] In one embodiment, the operation of dividing the delay messages in the delay message queue into N scheduling queues based on the delay duration and the delay message priority is as follows: Initialize N scheduling queues based on the number of delay message priorities; Read each delay message from the delay message queue, and allocate it to the k-th scheduling queue based on the delay duration and priority of the delay message until all the delay messages in the delay message queue are allocated to obtain N scheduling queues. The initial queue priority of the k-th scheduling queue is k, and the larger the value of k, the higher the priority. Where 1 ≤ k ≤ N, the determination method of k is:

[0049] ;

[0050] represents the priority of the delay message, represents the delay duration of the delay message, represents the first time threshold.

[0051] An important link in implementing the present invention is how to divide the delay messages in the delay queue into corresponding scheduling queues. In the prior art, some are divided simply according to the delay duration, and some are divided according to the priority of the delay message. Dividing the scheduling queue simply based on the delay duration or priority may result in low-priority delay messages not being processed for a long time, or there being a large difference in the number of delay messages in each scheduling queue. To overcome these defects, the present invention proposes to initialize N scheduling queues based on the number of delay message priorities. That is, at the beginning, each scheduling queue is empty. Read a message from the delay message queue and calculate which queue it should be allocated to based on the delay duration and its priority, so as to basically ensure that the number of messages in each scheduling queue is the same, and each scheduling queue is given a corresponding initial queue priority, that is, regardless of the original priority of the delay message in this scheduling queue, subsequent processing is based on the initial queue priority of this queue, improving the processing efficiency and avoiding the defect that low-priority delay messages are difficult to process. Moreover, the present invention proposes a specific method for allocating delay messages to the scheduling queue, improving the speed of dividing delay messages, which is another important inventive point of the present invention.

[0052] In one embodiment, the operation of the allocation step S102 is as follows: Obtain the delay duration of the delay messages in the i-th scheduling queue, divide the delay messages with the same delay duration into one time window, and a total of M i ’ time windows are obtained. Determine whether the number of delay messages in each time window is greater than the second threshold. If so, split the time window into two time windows, and use the two split time windows to replace the original time window to obtain M i ’’ time windows; Then determine M iWhether the number of delayed messages in each time window within a time window is less than a third threshold. If so, compare the number of delayed messages in the two time windows adjacent to this time window, and determine the time window with fewer delayed messages as the target time window. If this time window is the first time window, determine the second time window as the target time window. If this time window is the M i ’’th time window, then determine the M i ’’-1th time window as the target time window, merge the delayed messages in this time window into the target time window, and destroy this time window. Finally, obtain M i time windows.

[0053] Another important inventive concept of the present invention is to propose a method of dynamically dividing the scheduling queue into M i time windows. To improve the processing efficiency of delayed messages, in the present invention, the initial time window with too many delayed messages is split into two. Of course, it can also be split into multiple, but the time cost of maintaining multiple queues is relatively high, so it is generally split into two. Then, scan the number of messages in all time windows. If the number of messages in a certain time window is too small, merge the delayed messages in it into the adjacent time window with fewer delayed messages, and delete the original time window, so as to make full use of the processing resources of the processing kernel within a time window and avoid waste of processing resources. This is another important inventive point of the present invention.

[0054] In one embodiment, the operation of the scheduling step S104 is as follows: The priorities of N processing threads correspond to the initial queue priorities of N scheduling queues. Obtain the current state of each of the K processing cores, where the current state represents the idle situation of the processing core. If K≥N, select N processing cores from the K processing cores based on the current state and allocate them to the N processing threads in sequence. Among them, the processing core allocated to the processing thread corresponding to the Nth scheduling queue has the most idle current state, and so on; if N >K, calculate the current scheduling priority of each scheduling queue based on the total delay duration of the delayed messages in each scheduling queue and the initial queue priority of the scheduling queue. Select the K scheduling queues with the top K current scheduling priorities, and allocate the K processing cores to the K processing threads corresponding to the K scheduling queues with the top K current scheduling priorities for parallel processing based on the current scheduling priority. Among them, the processing core allocated to the thread corresponding to the scheduling queue with the highest current scheduling priority is the most idle, and so on. For the remaining N-K threads, wait until a processing core is idle, and allocate them to the idle processing core based on the current scheduling priorities of the N-K threads. Among them, the current scheduling priority of the kth scheduling queue is calculated as follows:

[0055] ;

[0056] Wherein, k represents the k-th scheduling queue and the initial queue priority of this queue. represents the total delay duration of the delayed messages in the k-th scheduling queue. represents the median of the delay time of the delayed messages in the k-th scheduling queue.

[0057] In the present invention, in order to solve the problem that the frequent switching of processing cores leads to frequent locks (locking and unlocking), which causes low system performance, the available processing cores are bound to the corresponding processing threads, that is, one processing core is responsible for processing the threads of one scheduling queue, and the most idle processing core processes the scheduling queue with the highest priority. And when the number of processing cores is less than the number of scheduling queues, the current scheduling priority of each scheduling queue is calculated based on the total delay duration of the delayed messages in each scheduling queue and the initial queue priority of this scheduling queue, and the threads of the K scheduling queues are allocated to the K processing cores for processing based on the current scheduling priority, and the threads of the remaining N-K scheduling queues wait for processing core resources in a polling manner, that is, at this time it is a hybrid scheduling method, and a specific method for calculating the current scheduling priority of each scheduling queue is proposed. Through this scheduling method, the processing speed of the delayed messages is greatly improved, which is another important inventive concept of the present invention.

[0058] The method of the present invention is specifically designed for the payment transaction system. Through actual tests in a certain e-commerce order system, it can process 230 million order payment timeout transactions per day on average, and the present invention can reduce the fault recovery time from the minute level to the sub-second level.

[0059] In the simulation test of 100 million delayed messages (the time distribution conforms to the Poisson process) in the simulated payment transaction system, the performance is compared with the prior art as follows:

[0060]

[0061] It can be seen from the above table that compared with the existing methods, the present invention has obvious improvements in indicators such as throughput, delay time, and CPU utilization. It can be seen that the technical effect of the present invention is significantly better than the prior art.

[0062] Figure 2 Shows a delayed message queue scheduling device based on a dynamic time window of the present invention for a payment transaction system, including:

[0063] A processing unit 201 divides the delayed messages in the delayed message queue into N scheduling queues based on the delay duration and the priority of the delayed messages, where N≥2;

[0064] The allocation unit 202 dynamically divides M time windows for the i-th scheduling queue among the N scheduling queues based on the delay duration of the delay messages, where M≥2 and 1≤i≤N. i The storage unit 203 constructs N linear storage structures as the primary storage area to store the M time windows corresponding to the N scheduling queues, and constructs a dynamic circular buffer for each linear storage structure as the secondary storage area to store the delay messages within the currently to-be-processed j-th time window, where 1≤j≤M. i ≥2, 1≤i≤N;

[0065] The storage unit 203 constructs N linear storage structures as the primary storage area to store the M time windows corresponding to the N scheduling queues, and constructs a dynamic circular buffer for each linear storage structure as the secondary storage area to store the delay messages within the currently to-be-processed j-th time window, where 1≤j≤M. i The scheduling unit 204 generates N processing threads corresponding to the N scheduling queues, obtains the number K of processing cores of the current processor (i.e., the available number of processing cores), and executes the N processing threads on the K processing cores to process the delay messages in the N scheduling queues, where K≥2. i ;

[0066] In the present invention, in order to process the delay messages as soon as possible, it is creatively proposed to divide the delay message queue into multiple scheduling queues based on the delay duration and the priority of the delay messages. The initial scheduling priority of each scheduling queue is the same. Since the delay duration and the priority of the delay messages are considered when allocating the delay messages to the corresponding scheduling queues, each delay message can be processed efficiently. Further, in order to improve the processing efficiency of the delay messages, M time windows are dynamically divided for the i-th scheduling queue among the N scheduling queues based on the delay duration of the delay messages. In the present invention, the method of dynamically dividing time windows is adopted to ensure that the messages in each time window are neither too many nor too few, so that the secondary memory structure proposed in the present invention can be well combined. N linear storage structures are used as the primary storage area to store the M time windows corresponding to the N scheduling queues. The linear storage structure improves the query efficiency during window execution. Using a dynamic circular buffer as the secondary storage area to store the delay messages within the currently to-be-processed j-th time window can efficiently utilize the memory space and reduce the memory occupancy rate. In order to minimize the time wasted by lock and unlock operations as much as possible, by the way of corresponding threads to processing cores, the lock operations are reduced, and the execution efficiency of the processing cores is improved, that is, by generating N processing threads corresponding to the N scheduling queues, and obtaining the number K of processing cores of the current processor, and executing the N processing threads on the K processing cores to process the delay messages in the N scheduling queues. Through this scheduling method of the present invention, the processing efficiency of the delay messages is improved, and the memory occupancy is greatly reduced. This is an important inventive concept of the present invention.

[0067] In the present invention, in order to process the delay messages as soon as possible, it is creatively proposed to divide the delay message queue into multiple scheduling queues based on the delay duration and the priority of the delay messages. The initial scheduling priority of each scheduling queue is the same. Since the delay duration and the priority of the delay messages are considered when allocating the delay messages to the corresponding scheduling queues, each delay message can be processed efficiently. Further, in order to improve the processing efficiency of the delay messages, M time windows are dynamically divided for the i-th scheduling queue among the N scheduling queues based on the delay duration of the delay messages. i In the present invention, the method of dynamically dividing time windows is adopted to ensure that the messages in each time window are neither too many nor too few, so that the secondary memory structure proposed in the present invention can be well combined. N linear storage structures are used as the primary storage area to store the M time windows corresponding to the N scheduling queues. The linear storage structure improves the query efficiency during window execution. Using a dynamic circular buffer as the secondary storage area to store the delay messages within the currently to-be-processed j-th time window can efficiently utilize the memory space and reduce the memory occupancy rate. In order to minimize the time wasted by lock and unlock operations as much as possible, by the way of corresponding threads to processing cores, the lock operations are reduced, and the execution efficiency of the processing cores is improved, that is, by generating N processing threads corresponding to the N scheduling queues, and obtaining the number K of processing cores of the current processor, and executing the N processing threads on the K processing cores to process the delay messages in the N scheduling queues. Through this scheduling method of the present invention, the processing efficiency of the delay messages is improved, and the memory occupancy is greatly reduced. This is an important inventive concept of the present invention. i The scheduling unit 204 generates N processing threads corresponding to the N scheduling queues, obtains the number K of processing cores of the current processor (i.e., the available number of processing cores), and executes the N processing threads on the K processing cores to process the delay messages in the N scheduling queues, where K≥2.

[0068] In one embodiment, the operation of constructing a dynamic circular buffer for each linear storage structure as a secondary storage area to store the delay messages within the current j-th time window to be processed is as follows: When j ≥ 2, when processing the delay messages in the (j - 1)-th time window, obtain the number num of delay messages within the j-th time window mes , based on num mes apply for a dynamic circular buffer of the corresponding size. After the delay messages in the (j - 1)-th time window are processed, destroy the dynamic circular buffer corresponding to the (j - 1)-th time window. When j = 1, obtain the number num of delay messages within the j-th time window mes , based on num mes apply for a dynamic circular buffer of the corresponding size.

[0069] In the present invention, by adopting the method of pre-constructing a dynamic circular buffer, after the messages in the (j - 1)-th (j ≥ 2) window are processed, the delay messages in the j-th window have been loaded into the dynamic circular buffer, and the dynamic circular buffer corresponding to the (j - 1)-th time window is destroyed, that is, apply for the corresponding memory size as the dynamic circular buffer based on the number of delay messages to be processed within each time window, and apply in advance, which avoids the processing kernel waiting for the loading of delay messages, improves the overall efficiency of the system, and improves the memory utilization rate. Through this secondary storage method, the memory fragmentation rate is reduced by 72% during the processing of delay messages, and the pause time of the processing kernel is shortened to within 3 ms, greatly improving the system performance.

[0070] In one embodiment, the linear storage structure is an array storage structure, and the query time of the array is O(n), which greatly improves the search speed of the time window.

[0071] In one embodiment, the operation of dividing the delay messages in the delay message queue into N scheduling queues based on the delay duration and the delay message priority is as follows: Initialize N scheduling queues based on the number of delay message priorities; Read each delay message from the delay message queue, and allocate it to the k-th scheduling queue based on the delay duration and priority of the delay message until all the delay messages in the delay message queue are allocated to obtain N scheduling queues. The initial queue priority of the k-th scheduling queue is k, and the larger the value of k, the higher the priority. Among them, 1 ≤ k ≤ N, and the determination method of k is as follows

[0072] ;

[0073] represents the priority of the delay message represents the delay duration of the delay message represents the first time threshold.

[0074] An important step in implementing the present invention is how to divide the delayed messages in the delay queue into the corresponding scheduling queues. In the prior art, some divide them simply according to the delay duration, and some according to the priority of the delayed messages. Dividing the scheduling queues simply based on the delay duration or priority may result in low-priority delayed messages not being processed for a long time, or a large difference in the number of delayed messages in each scheduling queue. To overcome these deficiencies, the present invention proposes to initialize N scheduling queues based on the quantity of the priority of the delayed messages. That is, at the beginning, each scheduling queue is empty. Read a message from the delayed message queue, and calculate which queue it should be assigned to based on the delay duration and its priority, so as to basically ensure that the number of messages in each scheduling queue is consistent, and each scheduling queue is given a corresponding initial queue priority, that is, regardless of the original priority of the delayed messages in this scheduling queue, subsequent processing is carried out based on the initial queue priority of this queue, which improves the processing efficiency and avoids the deficiency that low-priority delayed messages are difficult to be processed. Moreover, the present invention proposes a specific method for assigning the delayed messages to the scheduling queues, which improves the speed of dividing the delayed messages. This is another important inventive point of the present invention.

[0075] In one embodiment, the operation of the allocation unit 202 is as follows: obtain the delay duration of the delayed messages in the i-th scheduling queue, divide the delayed messages with the same delay duration within a time window, and a total of M i ’ time windows are obtained. Determine whether the number of delayed messages in each time window is greater than a second threshold. If so, split the time window into two time windows, and replace the original time window with the two split time windows to obtain M i ’’ time windows; then determine whether the number of delayed messages in each of the M i ’’ time windows is less than a third threshold. If so, compare the number of delayed messages in the two time windows adjacent to this time window, and determine the time window with fewer delayed messages as the target time window. If this time window is the first time window, then determine the second time window as the target time window. If this time window is the M i ’’-th time window, then determine the M i ’’-1-th time window as the target time window, merge the delayed messages in this time window into the target time window, and destroy this time window. Finally, M i time windows are obtained.

[0076] Another important inventive concept of the present invention is to propose to dynamically divide the scheduling queues into M iIn the way of time windows, in order to improve the processing efficiency of delayed messages, in the present invention, the initial time window with excessive delayed messages is split into two. Of course, it can also be split into multiple, but the time cost of maintaining multiple queues is relatively high, so it is generally split into two. Then, the number of messages in all time windows is scanned. If the number of messages in a certain time window is too small, the delayed messages therein are merged into the adjacent time window with a relatively small number of delayed messages, and the original time window is deleted. Thus, within one time window, the processing resources of the processing kernel can be fully utilized, avoiding waste of processing resources. This is another important inventive point of the present invention.

[0077] In one embodiment, the operation of the scheduling unit 204 is as follows: The priorities of N processing threads correspond to the initial queue priorities of N scheduling queues. The current state of each of the K processing kernels is obtained, and the current state indicates the idle situation of the processing kernel. If K≥N, N processing kernels are selected from the K processing kernels based on the current state and are sequentially assigned to N processing threads. Among them, the processing kernel assigned to the processing thread corresponding to the Nth scheduling queue has the most idle current state, and so on; if N <K, based on the total delay duration of the delayed messages in each scheduling queue and the initial queue priority of the queue, the current scheduling priority of each scheduling queue is calculated. The K scheduling queues with the top K current scheduling priorities are selected, and the K processing kernels are assigned to the K processing threads corresponding to the K scheduling queues with the top K current scheduling priorities for parallel processing based on the current scheduling priority. Among them, the processing kernel assigned to the thread corresponding to the scheduling queue with the highest current scheduling priority is the most idle, and so on. For the remaining N-K threads, when a processing kernel becomes idle, they are assigned to the idle processing kernel based on the current scheduling priorities of the N-K threads. Among them, the current scheduling priority of the kth scheduling queue is calculated as follows:

[0078] ;

[0079] where k represents the kth scheduling queue and its initial queue priority, represents the total delay duration of the delayed messages in the kth scheduling queue, represents the median of the delay times of the delayed messages in the kth scheduling queue.

[0080] In the present invention, to solve the problem that the frequent switching of processing cores leads to frequent locking (locking and unlocking), which causes low system performance, the available processing cores are bound to the corresponding processing threads, that is, one processing core is responsible for processing the threads of one scheduling queue, and the most idle processing core processes the scheduling queue with the highest priority. When the number of processing cores is less than the number of scheduling queues, the current scheduling priority of each scheduling queue is calculated based on the total delay duration of the delayed messages in each scheduling queue and the initial queue priority of the scheduling queue, and the threads of K scheduling queues are allocated to K processing cores for processing based on the current scheduling priority. The threads of the remaining N - K scheduling queues wait for processing core resources in a polling manner, that is, a hybrid scheduling method is used at this time. A specific method for calculating the current scheduling priority of each scheduling queue is proposed. Through this scheduling method, the processing speed of delayed messages is greatly improved, which is another important inventive concept of the present invention.

[0081] For the convenience of description, the above device is described by dividing it into various units according to functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0082] From the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the device described in each embodiment or some parts of the embodiments of the present application.

[0083] Finally, it should be noted that the above embodiments are only used to illustrate rather than limit the technical solution of the present invention. Although the present invention has been described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that the present invention can still be modified or equivalently replaced. Any modification or partial replacement without departing from the spirit and scope of the present invention shall be covered by the scope of the claims of the present invention.

Claims

1. A delay message queue scheduling method based on a dynamic time window, which is used in a payment transaction system, and is characterized in that, The method includes: A processing step of dividing the delay messages in the delay message queue into N scheduling queues based on the delay duration and the delay message priority, where N≥2, and initializing N scheduling queues based on the number of delay message priorities; reading each delay message from the delay message queue, and allocating it to the k-th scheduling queue based on the delay duration and priority of the delay message until all the delay messages in the delay message queue are allocated to obtain N scheduling queues. The initial queue priority of the k-th scheduling queue is k, where 1≤k≤N, and the determination method of k is: ； Indicates the priority of the delayed message, Indicates the delay duration of the delayed message, Indicates the first time threshold; Allocation step, dynamically divide M time windows based on the delay duration of the delay message for the i-th scheduling queue among N scheduling queues, where M ≥ 2, 1 ≤ i ≤ N, obtain the delay duration of the delay messages in the i-th scheduling queue, divide the delay messages with the same delay duration into one time window, and a total of M' time windows are obtained. Determine whether the number of delay messages in each time window is greater than the second threshold. If so, split the time window into two time windows, and use the two split time windows to replace the original time window to obtain M'' time windows. Then determine whether the number of delay messages in each of the M'' time windows is less than the third threshold. If so, compare the number of delay messages in the two time windows adjacent to this time window, and determine the time window with fewer delay messages as the target time window. If this time window is the first time window, determine the second time window as the target time window. If this time window is the M''-th time window, determine the M''-1-th time window as the target time window, merge the delay messages in this time window into the target time window, and destroy this time window. Finally, M time windows are obtained. i time windows, where M i ≥ 2, 1 ≤ i ≤ N, obtain the delay duration of the delay messages in the i-th scheduling queue, divide the delay messages with the same delay duration into one time window, and a total of M i ’ time windows are obtained. Determine whether the number of delay messages in each time window is greater than the second threshold. If so, split the time window into two time windows, and use the two split time windows to replace the original time window to obtain M i ’’ time windows. Then determine whether the number of delay messages in each of the M i ’’ time windows is less than the third threshold. If so, compare the number of delay messages in the two time windows adjacent to this time window, and determine the time window with fewer delay messages as the target time window. If this time window is the first time window, determine the second time window as the target time window. If this time window is the M i ’’-th time window, then determine the M i ’’ - 1-th time window as the target time window, merge the delay messages in this time window into the target time window, and destroy this time window. Finally, M i time windows are obtained. Storage step, constructing N linear storage structures as the primary storage area to store M in the corresponding N scheduling queues i time windows, constructing a dynamic circular buffer for each linear storage structure as the secondary storage area to store the delay messages within the current j-th time window to be processed, where 1 ≤ j ≤ M i ; A scheduling step of generating N processing threads corresponding to the N scheduling queues, obtaining the number K of processing cores of the current processor, and executing the N processing threads on the K processing cores to process the delay messages in the N scheduling queues, where K≥2.

2. The method according to claim 1, characterized in that The linear storage structure is an array storage structure.

3. The method according to claim 2, characterized in that, The operations of the scheduling steps are as follows: The priorities of N processing threads correspond to the initial queue priorities of N scheduling queues. Obtain the current state of each of the K processing cores, where the current state indicates the idle situation of the processing core. If K ≥ N, select N processing cores from the K processing cores based on the current state and allocate them to the N processing threads in sequence. Among them, the processing core allocated to the processing thread corresponding to the Nth scheduling queue has the most idle current state, and so on; if N > K, calculate the current scheduling priority of each scheduling queue based on the total delay duration of the delayed messages in each scheduling queue and the initial queue priority of the scheduling queue. Select the top K scheduling queues in the current scheduling priority ranking, and allocate the K processing cores to the K processing threads corresponding to the top K scheduling queues in the current scheduling priority ranking for parallel processing. Among them, the processing core allocated to the thread corresponding to the scheduling queue with the highest current scheduling priority is the most idle, and so on. For the remaining N - K threads, wait until a processing core is idle, and allocate them to the idle processing core based on the current scheduling priority of the N - K threads. Among them, the current scheduling priority of the kth scheduling queue is calculated as follows: ； where k represents the k-th scheduling queue and its initial queue priority represents the total delay duration of the delayed messages in the k-th scheduling queue represents the median of the delay times of the delayed messages in the k-th scheduling queue 4. A delay message queue scheduling device based on a dynamic time window, which is used for a payment transaction system, and is characterized in that The apparatus includes: A processing unit that divides the delay messages in the delay message queue into N scheduling queues based on the delay duration and the delay message priority, where N≥2, and initializes N scheduling queues based on the number of delay message priorities; reads each delay message from the delay message queue, and allocates it to the k-th scheduling queue based on the delay duration and priority of the delay message until all the delay messages in the delay message queue are allocated to obtain N scheduling queues. The initial queue priority of the k-th scheduling queue is k, where 1≤k≤N, and the determination method of k is: ； Indicates the priority of the delayed message, Indicates the delay duration of the delayed message, Indicates the first time threshold; Allocation unit, based on the delay duration of the delay message, dynamically divides M time windows for the i-th scheduling queue among N scheduling queues, where M ≥ 2, 1 ≤ i ≤ N. Obtain the delay duration of the delay messages in the i-th scheduling queue, divide the delay messages with the same delay duration into one time window, and obtain M' time windows in total. Determine whether the number of delay messages in each time window is greater than the second threshold. If so, split the time window into two time windows, and use the two split time windows to replace the original time window to obtain M'' time windows. Then determine whether the number of delay messages in each of the M'' time windows is less than the third threshold. If so, compare the number of delay messages in the two time windows adjacent to this time window, and determine the time window with fewer delay messages as the target time window. If this time window is the first time window, determine the second time window as the target time window. If this time window is the M''-th time window, determine the (M'' - 1)-th time window as the target time window, merge the delay messages in this time window into the target time window, and destroy this time window. Finally, obtain M time windows. i time windows, where M i ≥ 2, 1 ≤ i ≤ N, obtain the delay duration of the delay messages in the i-th scheduling queue, divide the delay messages with the same delay duration into one time window, and obtain M i ’ time windows in total. Determine whether the number of delay messages in each time window is greater than the second threshold. If so, split the time window into two time windows, and use the two split time windows to replace the original time window to obtain M i ’’ time windows. Then determine whether the number of delay messages in each of the M i ’’ time windows is less than the third threshold. If so, compare the number of delay messages in the two time windows adjacent to this time window, and determine the time window with fewer delay messages as the target time window. If this time window is the first time window, determine the second time window as the target time window. If this time window is the M i ’’-th time window, then determine the (M i ’’ - 1)-th time window as the target time window, merge the delay messages in this time window into the target time window, and destroy this time window. Finally, obtain M i time windows; A storage unit constructs N linear storage structures as a primary storage area to store M time windows in the corresponding N scheduling queues i For each linear storage structure, a dynamic circular buffer is constructed as a secondary storage area to store the delay messages within the current j-th time window to be processed, where 1 ≤ j ≤ M i ; A scheduling unit that generates N processing threads corresponding to the N scheduling queues, obtains the number K of processing cores of the current processor, and executes the N processing threads on the K processing cores to process the delay messages in the N scheduling queues, where K≥2.

5. The device according to claim 4, characterized in that, The linear storage structure is an array storage structure.

6. A computer-readable storage medium, on which computer program code is stored, and when the computer program code is executed by a computer, the method according to any one of claims 1-3 above is executed.

Images