A task scheduling method suitable for securities transaction monitoring business

Abstract

The application relates to the technical field of database task scheduling, and provides a task scheduling method suitable for securities transaction monitoring business, which comprises the following steps of S1, identifying a peak time point formed by task aggregation in a transaction day, and adjusting tasks of the peak time point to adjacent subsequent time points in the order of original business timing time in a smooth interval in which the number of tasks decreases with time; S2, setting a technical start time and a scheduling start time for each monitoring algorithm task; S3, dynamically advancing or delaying the scheduling start time of a subsequent task in a task execution process; and S4, judging whether a task can be scheduled and executed based on the business timing time of the task, a current data time and the dynamically adjusted scheduling start time. The application solves the problems of high peak period database load and long task waiting time caused by uneven data time distribution in a securities transaction monitoring system, and has the effects of load balancing and delay reduction.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of database task scheduling, and particularly relates to a task scheduling method suitable for securities transaction monitoring business. BACKGROUND

[0002] The securities transaction monitoring algorithm is a core component of the transaction monitoring system, which plays a decisive role in ensuring fair trading and market stability and preventing financial risks. For different types of irregular trading behaviors, corresponding monitoring algorithms are designed according to their data characteristics and adjustable monitoring object algorithm instances are configured. The transaction monitoring system generates scheduling tasks corresponding to the algorithm instances at different time points according to the configuration of different algorithm instances. The system obtains real-time transaction data from the upstream platform, and after the data at a specific time point is ready, the specific algorithm instance is scheduled to analyze it to find possible irregular trading behaviors. The scheduling of different tasks at the same time point is equal.

[0003] With the continuous expansion of the market size and the gradual improvement of the system, the number of algorithms that the transaction monitoring system needs to execute also increases day by day, which is reflected in the database level that the number of query tasks that need to be executed increases day by day. Due to the uneven time distribution of securities transaction data, the data volume at some time points is significantly higher than that at other time points, and the execution frequency of the corresponding monitoring algorithms at these time points is also significantly higher. For example, at the end of the collective bidding, the beginning and end of the continuous bidding, the data volume of securities transactions will have a significant peak, and the number of tasks that need to be executed at these times is also relatively large, resulting in a significantly higher load of the database at some peak time points.

[0004] In the transaction monitoring system, the existing scheduling method will schedule specific monitoring algorithm instances according to the task start time specified by the business after the data is ready. At some peak time points, the number of tasks scheduled is significantly greater than the number of tasks that the system can execute, and a considerable number of algorithm instance tasks cannot immediately obtain the database resources required for execution and enter a delay waiting state, inevitably resulting in a relatively high business delay. A reasonable task scheduling mechanism can effectively reduce the load of the system at peak time points, reduce the waiting time and technical delay of task startup. SUMMARY

[0005] To help solve the above technical problems, the present application provides a task scheduling method suitable for securities transaction monitoring business, which adopts the following technical solution:

[0006] A task scheduling method suitable for securities transaction monitoring business, comprising:

[0007] S1: identifying a peak time point formed by task aggregation in a transaction day, and in a smooth interval in which the number of tasks decreases over time, adjusting the tasks at the peak time point to adjacent subsequent time points in the order of original business timing time, so that the number of tasks at each time point after adjustment tends to be average and the original time sequence between tasks is maintained;

[0008] S2: setting a technical start time and a scheduling start time for each monitoring algorithm task; the technical start time is used to estimate the time when the task actually starts to execute in the database resource queue; the scheduling start time is calculated based on the technical start time and a fixed scheduling frequency, and is used to trigger the scheduling of the task;

[0009] S3: during the execution of the task, dynamically advancing or delaying the scheduling start time of the subsequent task according to the actual load of the database;

[0010] S4: judging whether the task can be scheduled and executed based on the business timing time of the task, the current data time, and the dynamically adjusted scheduling start time.

[0011] Preferably, the S1 comprises:

[0012] aggregating all tasks of the day according to the business timing time, and counting the number of tasks in units of minutes;

[0013] identifying a peak time point at which the number of tasks exceeds the average value, and determining the time interval between one peak and the next peak as the smooth interval, wherein the number of tasks in the smooth interval decreases over time;

[0014] for each minute in the smooth interval, if the number of tasks in the minute is greater than the number of tasks that the minute can accommodate, then the tasks exceeding the minute are adjusted to the next minute in the order of their original business timing time.

[0015] Preferably, the S1 comprises:

[0016] initializing the technical start time of the algorithm task as the value of the business timing time, aggregating all tasks of the day according to the technical start time, and recording the number of tasks t i at the i

[0017] traversing the number of tasks of each minute from the front to the back, if there is t i > t i-1 , then the number of tasks at the i

[0018] In each smoothing interval, let the wave crest of the smoothing interval be i, the length be s, and the total number of tasks in the interval be The number of tasks that can be accommodated per minute is:

[0019] For the jth minute, if

[0020] then the excess tasks are adjusted to the j+1 minute, where , is the number of tasks adjusted from the j-1 minute to the j minute, and when j is the starting minute of the smoothing interval is 0.

[0021] Preferably, the adjustment of the excess tasks to the j+1 minute includes:

[0022] adjusting the timing time of tasks to the j+1 minute, let the original number of tasks at the j+1 minute be n j , then for the mth task of the jth minute the timing time T m , if , then T m =j, if , then T m =j+1.

[0023] Preferably, in S2, the manner of setting the technical start time includes:

[0024] initializing the technical start time of tasks in the same minute;

[0025] based on the capacity of the database resource queue and the estimated time consumption of the tasks, simulating the execution process of the tasks in the queue, and dynamically calculating the technical start time of each task, wherein the technical start time of the ith task is calculated based on the technical start time of the ith-X task and the estimated time consumption, and X is the capacity of the database resource queue.

[0026] Preferably, the initialization of the technical start time of tasks in the same minute includes:

[0027] for tasks with priority 0 and 1, the technical start time is initialized to the first second of the minute;

[0028] for tasks with priority 2, the technical start time is initialized to the technical start time of the priority 0 task corresponding to it plus the estimated time consumption.

[0029] Preferably, in S2, the manner of setting the technical start time includes:

[0030] Based on the capacity X of the database resource queue and the estimated time consumption of the task, the execution process of the task in the queue is simulated, and the technical start time of the ith task is calculated according to the following formula:

[0031] The technical start time of the ith task is If , then If , then Where t'0 is the service timing time of the task after smoothing optimization, and the estimated time consumption of the ith-X task.

[0032] Preferably, the S3 comprises:

[0033] The adjustment coefficient n is initialized as 0;

[0034] If no task is acquired in one scheduling but there is a task that should be scheduled, then n=n+1 is executed, that is, the scheduling start time of all subsequent tasks is advanced by one scheduling period S';

[0035] If the maximum number X of tasks is acquired in one scheduling and there is still a schedulable task that is not acquired, then n=n-1 is executed, that is, the scheduling start time of all subsequent tasks is delayed by one scheduling period S';

[0036] The actual scheduling start time T of the task is calculated according to the formula:

[0037] Where T0 is the original service timing time of the task, T s is the technical start time of the task, and T s %S' represents the remainder obtained by dividing T s by S'.

[0038] Preferably, in the S4, the condition for judging whether a task can be scheduled is that the following conditions are met simultaneously:

[0039] The state of the task is waiting for execution, and the service start time of the task is less than or equal to the current data time;

[0040] And the current system time is less than the current data time or the current system time is greater than or equal to the scheduling start time of the task.

[0041] In summary, compared with the prior art, the present application has the following advantages:

[0042] 1. Considering the characteristics of the database of the securities transaction monitoring system that part of the peak time load is high, the monitoring algorithm task whose business start time is the same moment is redistributed to different moments for execution, which can theoretically reduce the load of the database at the peak time point, and reduce the randomness of task scheduling.

[0043] 2. The existing system is equal to all tasks at the same time point when scheduling tasks, the application adds the design of scheduling priority, which refers to the business importance and technical influence of the task when setting the priority, and is used for task sorting. In combination with the priority and the past running situation of the monitoring algorithm, the estimated time consumption and start time are set for the corresponding task, which takes into account the accuracy and flexibility of system scheduling tasks.

[0044] 3. The application does not simply move the tasks at the peak time point to the time point with lower load, but divides the interval with decreasing number of tasks over time according to the characteristics of securities transaction monitoring business. The timing time of the task can only be adjusted from front to back within a fixed interval, and the order of the tasks before and after the adjustment is unchanged according to the business timing time. This adjustment can effectively reduce the technical delay time (i.e. the waiting time from being scheduled to actually starting execution) of the task and reduce the waste of service resources.

[0045] 4. The application has a core invention point: on the basis of the existing business timing time, the scheduling start time and the technical start time are additionally set for the task as the technical reference for task scheduling and execution. Without affecting the business, the scheduling start time of the task is dynamically adjusted, which ensures that the system will not be overloaded under high load, and reduces the idle running of the database under low load. BRIEF DESCRIPTION OF DRAWINGS

[0046] Figure 1 The overall flowchart of the method of the application is shown in the figure;

[0047] Figure 2 The task timing time smoothing optimization diagram of the application is shown in the figure;

[0048] Figure 3 The technical start time setting diagram of the application is shown in the figure. DETAILED DESCRIPTION

[0049] The structure and principle of the application will be further described below in combination with the drawings, which are very clear to those skilled in the art. It should be understood that the specific embodiments described herein are only used to explain the application and not to limit the application.

[0050] The overall flowchart of the method is shown in the figure Figure 1 The execution can be divided into the following four steps:

[0051] S1. Task timing time smoothing optimization

[0052] Due to the uneven distribution of the timing of algorithm tasks of the transaction monitoring system, the number of tasks at some time points is relatively large, causing high system load. Therefore, the timing of some tasks at peak time points is adjusted to time points with fewer tasks, as shown in Figure 2 . The specific process is as follows:

[0053] A1: Initialize the technical start time of all algorithm tasks to the value of the business timing time, and then aggregate all tasks on the day according to the technical start time, recording the number of tasks ti at the i minute, i ∈ [09:30, 11:30] ∪ [13:00, 15:30].

[0054] A2: In the two trading time periods [09:30, 11:30] and [13:00, 15:30], respectively, traverse the number of tasks at each minute from the front to the back. If there exists t i > t i-1 , then the i minute is taken as a peak, and the time interval from one peak to the next peak (excluding) is taken as a smoothing interval. Such division ensures that the number of tasks per minute in a smoothing interval decreases over time.

[0055] A3: In each smoothing interval, let the peak of the smoothing interval be i, the length be s, and the total number of tasks in the interval be , and the number of tasks that can be accommodated per minute be:

[0056] For the j minute, if

[0057] , then the excess tasks are adjusted to the j+1 minute, where , is the number of tasks adjusted from the j-1 minute to the j minute, and when j is the starting minute of the smoothing interval is 0.

[0058] Adjusting the excess tasks to the j+1 minute includes:

[0059] Adjusting the timing time of tasks to the j+1 minute, and let the original number of tasks at the j+1 minute be n j . For the mth task at the j minute , the timing time T m , if , then T m = j, and if , then T m = j+1.

[0060] Compared with other similar technical solutions, the scheme is applicable to the related business of securities transaction monitoring. Limited by its characteristics, the task timing time adjustment in the scheme has limitations. First, the task timing time can only be adjusted backward within an interval, and cannot be adjusted forward. The interval must be divided so that the number of tasks per minute inside it decreases sequentially. Second, the time sequence before and after adjustment cannot be changed, that is, the precedence relationship of the task timing time cannot be changed. The tasks at the peak point cannot be simply moved to the trough. Instead, starting from the peak time point, the tasks are sorted according to the original timing time. Then, the tasks that exceed the accommodated number at each time point are sequentially moved to the next adjacent time point until the number of tasks at each time point in the interval does not exceed the accommodated number. This adjustment can reduce the load of the peak time database, reduce the randomness of task scheduling, and reduce the technical delay time (i.e., the waiting time from being scheduled to actually starting execution), and reduce the waste of service resources.

[0061] S2. Set the estimated time consumption of the task

[0062] For a task with a priority of 0, the median of the time consumption of the tasks with a priority of 0 under the algorithm at the time point in the past month is calculated. For a task with a priority of 1, the median of the execution time consumption of the task at the time point in the past month is taken. For a task with a priority of 2, its estimated time consumption is set to 1 second. For a task with an online time of less than 1 month, the existing date can be taken. For a newly added task, the scheduling interval time is taken. For the i'th task, its estimated time consumption is denoted as cost i .

[0063] S3. Set the technical start time of the algorithm task

[0064] Considering the characteristics of the database on which the embodiment is based, the execution of the monitoring algorithm task is limited by the database resource queue. The database system sets a resource queue for each user, and the number of tasks executed in parallel by each user does not exceed the capacity of the resource queue. On this basis, the actual running resource queue is simulated, and the corresponding database resource queue capacity of the monitoring system is set to X. The technical start time is set for each running of each monitoring algorithm instance. The principle is as shown in Figure 3 .

[0065] C1: initialization, the tasks with the same minute timing are divided into a group. For tasks with a priority of 0 and 1, the technical start time is initialized to the first second of the minute. For a task with a priority of 2, the technical start time is initialized to the technical start time of the priority 0 task corresponding to it plus the estimated time consumption. And sort the tasks in each minute according to the initial value of the technical start time, the priority, and the estimated time consumption.

[0066] C2: Simulate the running situation in the disk, set the technical start time of the algorithm task. For the database system based on the system, up to X tasks can be supported for parallel execution. A first-in-first-out queue is used to simulate the running situation of the resource queue in the disk, and the capacity is set to X. For each minute task, the first X tasks are placed in the queue after sorting, and each time the queue is full, all tasks in the queue are sorted according to the estimated time consumption, and the first task in the queue is dequeued, and the next task is enqueued. The technical start time of the enqueued task is the technical start time of the dequeued task plus the estimated time consumption. That is, the technical start time of the ith task , if , then , if , then , wherein t'0 is the business timing time of the task after smoothing optimization, is the estimated time consumption of the (i-X)th task.

[0067] S4. Dynamically set the scheduling start time of the algorithm task

[0068] The scheduling start time of the algorithm task is dynamically set in combination with the scheduling frequency and the actual situation. In the existing scheduling system, the frequency at which the system obtains the to-be-executed task (that is, the time S' between each scheduling) is fixed. After the task is scheduled, it cannot be executed immediately, and there is a certain waiting time. If the technical start time T S is used to determine whether the task can be scheduled, the actual time when the task is scheduled will be later than T S , which will cause empty running when there are database resources. Therefore, for a task with a technical start time of T S , the task needs to be scheduled before T S . Therefore, the time at which the last scheduling system takes the scheduled task before setting the technical start time of the task is set as the scheduling start time T of the task, which is initialized as , T s %S represents the remainder obtained by dividing T s by S.

[0069] At the same time, the design of the method is mainly based on the case that the database load is high. When the daily transaction data volume is small and the database load is low, the database will run empty due to the postponing of the timing time of part of the tasks in this case, causing resource waste. Therefore, at each scheduling, the scheduling start time of the subsequent task should be adjusted according to the idle situation of the database resources.

[0070] The period of the system database scheduling to-be-executed tasks is fixed as S', and the maximum number of tasks acquired each time is X, that is, the system acquires X to-be-executed tasks every S' seconds. The scheduling start time needs to be dynamically adjusted according to the situation of the system acquiring scheduling tasks each time, and the adjustment coefficient is n, which is initialized as 0. If the system does not acquire tasks at a certain time of scheduling, but there are tasks whose original time is less than the current data time (that is, the tasks can be scheduled on the business, but are executed later due to the set scheduling start time), the scheduling start time of all subsequent scheduling tasks is reduced by S' seconds, that is, n = n + 1. Correspondingly, if the system acquires X tasks, and the number of tasks that can be scheduled according to the scheduling start time but are not acquired by the system is greater than X, the scheduling start time of all subsequent scheduling tasks is increased by S' seconds, that is, n = n - 1. The scheduling start time is where T0 is the original business time of the task, n is calculated according to each scheduling before the task starts, and T s S' indicates T s The remainder obtained after division by S'.

[0071] The scheduling logic of the system mainly refers to the scheduling start time. The condition for a task to be scheduled is that the business start time is less than or equal to the current data time, and the state of the task is waiting for execution. On this basis, the task can be scheduled if the system time is less than the current data time or the system time is greater than or equal to the scheduling start time.

[0072] That is, the condition for judging whether a task can be scheduled is that the following conditions are met:

[0073] The state of the task is waiting for execution, and the business start time of the task is less than or equal to the current data time;

[0074] And the current system time is less than the current data time or the current system time is greater than or equal to the scheduling start time of the task.

[0075] It should be noted that the "data time" refers to the update timestamp after the upstream data source data is ready, which represents the time corresponding to the latest transaction data that can be processed by the monitoring business. The "system time" refers to the current clock time of the server where the task scheduling system is located.

[0076] In the technical solution of the present application, the above scheduling method is applied through the following database and modules:

[0077] A database is configured to store a to-be-executed task scheduling table, and the scheduling table at least stores the following fields: algorithm code, instance code, transaction date, business start time, task state, estimated time consumption, technical start time, scheduling start time, and priority;

[0078] A smoothing optimization module is configured to perform step S1.

[0079] a time calculation module configured to perform step S2;

[0080] a dynamic adjustment module configured to perform step S3;

[0081] a scheduling execution module configured to perform step S4.

[0082] Database storage design:

[0083] A to-be-executed task scheduling table (hereinafter referred to as a scheduling table):

[0084]

[0085] The transaction supervision system writes the algorithm instance tasks that need to be executed on each trading day according to the configuration information of the algorithm into the table before the market opens. The table stores the business start time of the algorithm task, the task state, the technical start time, the scheduling start time, and related fields reflecting the actual execution of the task. The transaction supervision system will schedule the corresponding algorithm task according to the table during the market.

[0086] The present application performs the initialization process of the algorithm task estimated time consumption, the scheduling start time, and the technical start time in the process of the to-be-executed task scheduling table. The estimated time consumption of the task is initialized to 0, and the scheduling start time and the technical start time of the task are initialized to the start time specified by the task.

Claims

1. A task scheduling method applicable to securities trading monitoring operations, characterized in that, include: S1: Identify peak time points within a trading day caused by task aggregation. Within a smooth interval where the number of tasks decreases over time, adjust the tasks at peak time points sequentially to adjacent subsequent time points according to their original business time schedules. This ensures that the number of tasks at each time point is more evenly distributed after adjustment while maintaining the original time order between tasks. Initialize the technical start time of the algorithm tasks to the value of the business time schedule. Aggregate all tasks for the day according to their technical start time and record the number of tasks t at minute i. i ; Iterate through the number of tasks for each minute from beginning to end. If there exists t... i >t i-1 Then, the number of tasks in the i-th minute is taken as a peak, and the time interval between one peak and the next peak is defined as a smooth interval, so that the number of tasks per minute in a smooth interval decreases with time. Within each smoothing interval, let the peak of the smoothing interval be i, and the length be s. The total number of tasks within this interval is... The number of tasks that can be accommodated per minute is: For the j-th minute, if The excess tasks will be adjusted to the (j+1)th minute, where , To adjust the number of tasks from minute j-1 to minute j, when j is the starting minute of the smoothing interval... =0; Adjusting excess tasks to minute j+1 includes: Will The timer for each task is adjusted to the (j+1)th minute. Let the original number of tasks at the (j+1)th minute be n. j For the m-th task at minute j, Time T m ,like Then T m =j, if Then T m =j+1; S2: Set the technology start time and scheduling start time for each monitoring algorithm task; the technology start time is used to estimate the actual start time of the task in the database resource queue; the scheduling start time is calculated based on the technology start time and a fixed scheduling frequency, and is used to trigger the scheduling of the task. S3: During task execution, dynamically advance or postpone the start time of subsequent tasks based on the actual load of the database; S4: Determine whether a task can be scheduled for execution based on the task's business timeout, current data time, and dynamically adjusted scheduling start time.

2. The task scheduling method for securities trading monitoring business according to claim 1, characterized in that, S1 includes: Aggregate all tasks for the day according to business time slots and count the number of tasks in minutes. Identify the peak time points where the number of tasks exceeds the average value, and define the time interval between one peak and the next peak as a smooth interval, in which the number of tasks decreases over time within the smooth interval. For each minute within the smoothing interval, if the number of tasks in that minute exceeds the number of tasks that minute can accommodate, the tasks exceeding the limit in that minute will be adjusted to the next minute according to their original business timing.

3. The task scheduling method for securities trading monitoring business according to claim 1, characterized in that, In S2, the methods for setting the technology start time include: Initialize the technical start time of tasks within the same minute; Based on the capacity of the database resource queue and the estimated time of the task, the execution process of the task in the queue is simulated, and the technical start time of each task is dynamically calculated. The technical start time of the i-th task is calculated based on the technical start time and estimated time of the iX-th task, where X is the capacity of the database resource queue.

4. The task scheduling method for securities trading monitoring business according to claim 3, characterized in that, The technical start time for initializing tasks within the same minute includes: For tasks with priorities of 0 and 1, their technology start time is initialized to the first second of that minute; For a task with priority 2, the technology start time is initialized to the technology start time of its corresponding task with priority 0 plus the estimated time.

5. The task scheduling method for securities trading monitoring business according to claim 1, characterized in that, In S2, the methods for setting the technology start time include: Based on the capacity X of the database resource queue and the estimated execution time of the task, the execution process of the task in the queue is simulated, and the technical start time of the i-th task is calculated according to the following formula: The technical start time of the i′-th task ,like ,but ,like ,but Where t′0 is the business timing time of the task after smoothing optimization. This is the estimated time for the i′-Xth task.

6. The task scheduling method for securities trading monitoring business according to claim 1, characterized in that, S3 includes: Initialize the adjustment coefficient n=0; If no task is obtained in a scheduling process but there is a task that should be scheduled, then n=n+1 is executed, that is, the scheduling start time of all subsequent tasks is advanced by one scheduling period S′. If the maximum number of tasks X are acquired in a single scheduling operation and there are still schedulable tasks that have not been acquired, then n=n-1 is executed, which means that the scheduling start time of all subsequent tasks is postponed by one scheduling period S′. The formula for calculating the actual scheduling start time T of a task is: Where T0 is the original scheduled business time of the task, T s T is the technical start time of the mission. s %S′ represents T s The remainder obtained after dividing by S′.

7. The task scheduling method for securities trading monitoring business according to claim 1, characterized in that, In S4, the condition for determining whether a task can be scheduled is that it must simultaneously meet the following conditions: The task status is either "waiting to execute" or the task's business start time is less than or equal to the current data time. And it must satisfy the condition that the current system time is less than the current data time or the current system time is greater than or equal to the scheduling start time of the task.

8. A task scheduling system suitable for securities trading monitoring operations, characterized in that, The task scheduling method applicable to securities trading monitoring business as described in any one of claims 1 to 7 includes: The database is used to store the task scheduling table to be executed. The scheduling table must store at least the following fields: algorithm code, instance code, transaction date, business start time, task status, estimated time, technology start time, scheduling start time and priority. The smoothing optimization module is used to perform step S1; The time calculation module is used to execute step S2; The dynamic adjustment module is used to execute step S3; The scheduling and execution module is used to execute step S4.

Images