WebWorker cooperative interrupt scheduling processing method and device for high-frequency time sequence monitoring data

Abstract

The invention discloses a WebWorker cooperative interrupt scheduling processing method and device for high-frequency time sequence monitoring data. The method comprises the following steps: splitting original high-frequency time sequence monitoring data into a plurality of tasks according to a statistical dimension in a main thread, and endowing the tasks with priorities; utilizing the minimum heap maintenance task queue to split each task into fragmentation units which can be independently executed; dynamically creating a Web Worker pool according to the core number of the central processing unit, and executing a current fragmentation unit in any Worker in the Web Worker pool through a scheduling controller; if the task of which the priority is higher than that of the current fragmentation unit is detected, interrupting the current fragmentation unit, storing the execution state of the current fragmentation unit, and preempting the fragmentation unit for executing the high-priority task by the thread; and increasing the weight of the low-priority task along with time through a starvation detection mechanism until the low-priority task is executed, and transmitting a task processing result to the main thread for visual rendering.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data visualization, and more particularly to a WebWorker cooperative interruption scheduling processing method and device for high-frequency time series monitoring data. BACKGROUND

[0002] The current conventional solution for computationally intensive tasks is to create a sub-thread for processing by means of a Web Worker. To optimize performance, the task is usually split and distributed to multiple worker threads for parallel computing.

[0003] Although the use of Web Worker to create a sub-thread to process tasks can avoid blocking the main thread, it has three key limitations: first, the sub-thread adopts a non-preemptive execution model, and the task cannot be interrupted until it is completed; second, there is a lack of native communication mechanism between threads, which makes it impossible for tasks to exchange data and perform collaborative computing; third, when using a data sharding strategy, the sharding granularity completely depends on manual preset, and there is a lack of quantitative evaluation capability for the optimal sharding size, which makes it difficult to guarantee the result merging efficiency. SUMMARY

[0004] In view of at least one defect or improvement demand of the prior art, the present application provides a WebWorker cooperative interruption scheduling processing method and device for high-frequency time series monitoring data, which solves the problems of low execution efficiency caused by the inability to interrupt the Worker execution and task accumulation, and improves the response efficiency for new tasks and the execution efficiency for tasks.

[0005] To achieve the above-mentioned purpose, according to a first aspect of the present application, a WebWorker cooperative interruption scheduling processing method for high-frequency time series monitoring data is provided, which comprises: splitting original high-frequency time series monitoring data into multiple tasks according to statistical dimensions and assigning priorities in the main thread; maintaining a task queue by using a minimum heap, and splitting each task into a sharding unit that can be independently executed; dynamically creating a WebWorker pool according to the number of central processor cores, and executing the current sharding unit in any Worker in the Web Worker pool through a scheduling controller; if a task with a higher priority than the current sharding unit is detected, interrupting the current sharding unit and saving the execution state of the current sharding unit, and preempting the execution of the sharding unit of the high-priority task; improving the weight of the low-priority task over time through a starvation detection mechanism until the low-priority task is executed, and transmitting the task processing result to the main thread for visual rendering.

[0006] In an optional example, the maintaining the task queue by using a minimum heap includes: sorting the plurality of tasks according to the priority of the tasks submitted by the main thread; when the main thread submits a new task, inserting the task into the heap and triggering scheduling; when a task is completed, deleting the task from the heap and notifying the main thread of the execution result of the task.

[0007] In an optional example, the re-splitting each task into independently executable slice units includes: performing coarse slicing on the task according to a preset initial slice size at the first slicing; after each slice unit is executed, the corresponding worker returns the actual CPU time consumption of the slice to the scheduling controller; adjusting the size of the next slice unit according to the comparison between the performance index and the preset time consumption interval; repeating the above steps until the task is processed, so that the execution time consumption of any single slice unit is always within the preset time consumption interval.

[0008] In an optional example, the increasing the weight of the low-priority task over time to the low-priority task being executed by the starvation detection mechanism includes: maintaining a starvation counter for each task that has not been removed from the heap; triggering starvation detection at a preset time interval, causing the starvation counter to increment; calculating the final priority of the task according to the starvation counter and a preset weight coefficient; when the final priority is lower than the system minimum priority, increasing the task to above the system minimum priority to ensure that it is selected for execution in the next scheduling period.

[0009] In an optional example, after the current slice unit is executed by any worker in the Web Worker pool by the scheduling controller, the method further includes: when scheduling a slice unit task using the scheduling controller, if there is a higher priority task, checking whether there is an idle worker; if there is an idle worker, assigning the idle worker to execute the high priority task; if there is no idle worker, saving the state of the task and suspending the task after the current slice unit is executed, and continuing to execute the slice unit of the high priority task.

[0010] In an optional example, the interrupting the current slice unit and saving the execution state of the current slice unit, and preempting the thread to execute the slice unit of the high-priority task includes: sending an interrupt signal to the target worker by the scheduling controller, and saving the execution state of the current task, wherein the execution state includes the current iteration position, the accumulated statistical value, and the offset of the processed data; returning the saved execution state to the scheduling controller as a state object, and pausing the execution of the current task.

[0011] In an optional example, after the execution of the current task is paused, the method further comprises: associating the received state object with an identifier of the corresponding task and storing the state object in a suspended task table; when the task regains a Worker, re-deploying the state object to the Worker as an initial parameter and resuming the execution of the task according to the state object.

[0012] According to a second aspect of the present application, a WebWorker cooperative interruption scheduling processing device for high-frequency timing monitoring data is also provided, which comprises: a splitting unit configured to split original high-frequency timing monitoring data into multiple tasks according to statistical dimensions and assign priorities in a main thread; a minimum heap maintaining unit configured to maintain a task queue by using a minimum heap, and split each task into a slice unit that can be independently executed; a creating and executing unit configured to dynamically create a Web Worker pool according to the number of central processor cores, and execute a current slice unit in any Worker in the Web Worker pool through a scheduling controller; an interruption saving unit configured to interrupt the current slice unit and save the execution state of the current slice unit, and preempt a slice unit of a high-priority task if a task with a higher priority than the current slice unit is detected; and a starvation detection unit configured to improve the weight of a low-priority task over time to the execution of the low-priority task through a starvation detection mechanism, and transmit a task processing result to the main thread for visual rendering.

[0013] According to a third aspect of the present application, a computer readable storage medium is also provided, which stores a computer program, wherein the computer program is configured to execute the above-mentioned WebWorker cooperative interruption scheduling processing method for high-frequency timing monitoring data when running.

[0014] According to a fourth aspect of the present application, an electronic device is also provided, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the above-mentioned WebWorker cooperative interruption scheduling processing method for high-frequency timing monitoring data through the computer program.

[0015] Overall, the above technical solutions conceived by the present application can achieve the following beneficial effects compared with the prior art:

[0016] (1) The present application provides a WebWorker cooperative interruption scheduling processing method for high-frequency timing monitoring data, which maintains a task queue by using a minimum heap, and can realize efficient insertion, removal and adjustment of task priorities compared with a conventional ordinary queue, so that the time complexity is greatly reduced, and the performance is improved more obviously as the number of tasks increases.

[0017] (2) Using the task fragmentation method, the large task can be split into multiple small tasks by providing a fragmentation method by the user or using an intelligent built-in fragmentation method. During the execution, the task can be paused by not executing the fragmentation of the task. The task can be resumed by saving the state of the task during the pause. By adjusting the priority of the task, the task can be preempted, paused and resumed without awareness by the task priority during the execution of the worker. The problem that the worker cannot be interrupted during the execution is solved, and the response efficiency for the new task is improved.

[0018] (3) The priority of the low-priority task is adjusted over time by using the starvation detection, so that the low-priority task cannot be occupied by the high-priority task for a long time and cannot be executed, and the low-priority task can be executed within a certain time, thereby improving the task execution efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required to be used in the embodiments will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0020] Figure 1 A flowchart of an optional WebWorker cooperative interruption scheduling processing method for high-frequency time sequence monitoring data provided by the embodiments of the present application is shown.

[0021] Figure 2 A schematic diagram of the architecture of an optional WebWorker cooperative interruption scheduling processing system for high-frequency time sequence monitoring data provided by the embodiments of the present application is shown.

[0022] Figure 3 An optional task scheduling and interruption processing flowchart provided by the embodiments of the present application is shown.

[0023] Figure 4 A structure diagram of an optional WebWorker cooperative interruption scheduling processing device for high-frequency time sequence monitoring data provided by the embodiments of the present application is shown.

[0024] Figure 5 An optional structure diagram of an electronic device provided by the embodiments of the present application is shown. DETAILED DESCRIPTION

[0025] In order to make the objectives, technical solutions and advantages of the present application clearer, further detailed description will be made to the present application in combination with the accompanying drawings and examples. It should be understood that the specific examples described herein are only used to explain the present application and should not be used to limit the present application. In addition, the technical features involved in each embodiment of the present application described below can be combined with each other as long as they do not conflict with each other.

[0026] The terms "first", "second", "third" and the like in the specification and claims of the present application and the above-described accompanying drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but can optionally include steps or units not listed or can optionally include other steps or units inherent to the process, method, product or device.

[0027] According to an aspect of an embodiment of the present application, a WebWorker cooperative interruption scheduling processing method for high-frequency time sequence monitoring data is provided. The following will be described in combination with Figure 1 The WebWorker cooperative interruption scheduling processing method for high-frequency time sequence monitoring data provided by the embodiment of the present application is described.

[0028] Figure 1 is a flowchart of an optional WebWorker cooperative interruption scheduling processing method for high-frequency time sequence monitoring data provided by the embodiment of the present application, as Figure 1 shown, the flow of the method can include the following steps:

[0029] S102, in the main thread, the original high-frequency time sequence monitoring data is split into multiple tasks according to the statistical dimension and is given a priority;

[0030] S104, using a minimum heap to maintain a task queue, each task is further split into a slice unit that can be independently executed;

[0031] S106, dynamically creating a WebWorker pool according to the number of central processor cores, and executing the current slice unit in any Worker in the WebWorker pool through a scheduling controller;

[0032] S108, if a task with a higher priority than the current slice unit is detected, the current slice unit is interrupted, the execution state of the current slice unit is saved, and the thread is preempted to execute the slice unit of the high-priority task;

[0033] S110, the weight of the low priority task is raised over time by a starvation detection mechanism to the low priority task being executed, and the task processing result is transmitted to the main thread for visual rendering.

[0034] The embodiment of the application provides a WebWorker cooperative interruption scheduling processing method for high-frequency time sequence monitoring data. The high-frequency time sequence monitoring data can be monitoring data continuously collected at a high frequency (such as a second level or a millisecond level) in a short time, and is usually used for real-time monitoring and analysis. Exemplarily, in a water conservancy data visualization scene, a large amount of data set formed by high-frequency time sequence monitoring data (such as second-level flow and water level) generated by facilities such as pump stations and gate stations needs to be executed by the front end to generate derivative data of multiple statistical granularities (hour / daily / month level) and multiple dimensions (spatial distribution, equipment type and the like) for multi-dimensional analysis, so as to support multi-scale visualization rendering. It should be noted that Web Worker is a multi-threading technology provided by HTML5, which allows JavaScript code to be executed in an independent thread, avoids blocking of the main thread, and thus improves the performance and response speed of the Web application. The core functions include background task processing, parallel computing and message communication mechanism with the main thread, and are suitable for computing-intensive operations and big data processing scenes.

[0035] Optionally, the original data is split into independent computing units, and parallelization of tasks is realized by relying on dynamic thread pool scheduling; a task priority mechanism is introduced to assign different weights to different statistical tasks (for example, the first screen rendering data is set as the highest priority); and based on the interruption type cooperation, the high-priority task can preempt the computing resources of the low-priority thread. The scheme makes the data processing time significantly reduced by 60%-80%, while ensuring that the key statistical task is completed within 50ms, realizing the real-time interactive experience of zero freezing.

[0036] Specifically, in view of the multi-dimensional statistical demand of massive heterogeneous data in the water conservancy field, the characteristics that the statistical dimensions dynamically switch and need to generate multiple groups of derivative data sets in parallel, a task division strategy is designed to decompose the data analysis process into independent computing units; at the same time, through the task priority mechanism, the default dimension and the high-priority statistical task after the user switches can obtain instant computing resources, realizing zero-delay visualization of key data.

[0037] As Figure 2As shown, the minimum heap is used to manage the tasks submitted by the main thread, ensuring that high-priority tasks are at the top of the heap and are executed first. By processing the tasks in slices, a single task is broken down into multiple executable slice units. When using the scheduler to schedule the slice unit tasks, if there is a higher priority task, it is checked whether there is an idle worker, and if there is, an idle worker is assigned to execute the high-priority task, and if there is not, the state of the task is saved and the task is suspended after the current slice unit is executed, and then the slice unit of the high-priority task is executed, realizing task preemption, suspension, and recovery. At the same time, in order to prevent the slice from being too large to block the execution of the next slice, the performance of the task execution is detected to dynamically adjust the size of the slice. In order to ensure that low-priority tasks are not always preempted by high-priority tasks, the patent uses starvation detection to allow low-priority tasks to increase in priority over time, ensuring that low-priority tasks will also be executed. By calculating the CPU resources, multiple Web Workers are opened in parallel to process tasks, maximizing the execution efficiency of the tasks.

[0038] Through the above steps S102 to S110, the original high-frequency time sequence monitoring data is split into multiple tasks according to the statistical dimensions and is given a priority in the main thread; the minimum heap is used to maintain the task queue, and each task is further split into a slice unit that can be independently executed; a Web Worker pool is dynamically created according to the number of central processor cores, and a scheduling controller is used to execute the current slice unit in any Worker in the Web Worker pool; if a task with a higher priority than the current slice unit is detected, the current slice unit is interrupted and the execution state of the current slice unit is saved, and the thread is preempted to execute the slice unit of the high-priority task; through the starvation detection mechanism, the weight of the low-priority task is increased over time until the low-priority task is executed, and the task processing result is transmitted to the main thread for visualization rendering, solving the problem of low execution efficiency caused by the inability to interrupt the Worker and the accumulation of tasks, and improving the response efficiency of new tasks and the execution efficiency of tasks.

[0039] In one exemplary embodiment, the use of a minimum heap to maintain a task queue includes:

[0040] S11, sorting a plurality of tasks according to the priority of the tasks submitted by the main thread;

[0041] S12, when a new task is submitted by the main thread, inserting the task into the heap and triggering scheduling;

[0042] S13, when a task is completed, deleting the task from the heap and notifying the main thread of the execution result of the task.

[0043] In the embodiments of the present application, as Figure 2As shown, the main thread is mainly responsible for submitting tasks and receiving task results, sending data to be processed and receiving processed data, using Transferable object transmission or IndexedDb storage transmission.

[0044] The scheduling controller: 1) maintains a task heap, which is sorted according to the priority of the tasks submitted by the main thread, inserts the new task into the heap and triggers scheduling when the main thread submits a new task, and deletes the task from the heap and notifies the main thread of the execution result of the task when the task has been completed. 2) Maintains a starvation detection map, which raises the priority of low priority tasks over time. 3) Task slicing, for stream data, intelligently slice by byte number, for object data, provide slicing method by user for slicing,

[0045] The workers pool is used to create Worker threads matching the number of CPU cores to handle task slices in parallel. The worker is responsible for executing tasks or task slices.

[0046] In a multi-task processing system, the priority of a task determines the execution order of the task. When the main thread submits a task, it assigns a priority value to each task. The scheduler sorts the tasks according to these priority values to ensure that high-priority tasks are executed first.

[0047] Optionally, the scheduler uses a Min-Heap to manage the task queue. The Min-Heap has the property that the top element (root node) always has the smallest priority value (i.e. the highest priority). When the main thread submits a new task, the scheduler inserts the task into the heap and adjusts the structure of the heap according to its priority to maintain the properties of the heap. After the insertion operation is completed, the scheduler triggers the scheduling process to check whether there is a higher priority task that needs to be executed immediately. When a task is completed, the scheduler needs to delete the task from the heap to release resources and update the state of the task queue. At the same time, the scheduler will notify the main thread of the execution result of the task, so that the main thread can further process, such as updating the user interface or triggering subsequent tasks.

[0048] Through this embodiment, the scheduler can efficiently manage the task queue, ensure that high-priority tasks are executed first, and timely feedback the execution result of the task to the main thread, significantly improving the response speed and processing efficiency of the system.

[0049] In one exemplary embodiment, the step of splitting each task into independently executable slice units comprises:

[0050] S21, when first slicing, according to the preset initial slicing size, coarsely slicing the task;

[0051] S22, after each slice unit is executed, the corresponding worker returns the actual CPU time consumption of the slice to the scheduling controller;

[0052] S23, adjusting the size of the next slice unit according to the comparison between the performance index and the preset time consumption interval;

[0053] S24, repeating the above steps until the task is completed, so that the execution time consumption of any single slice unit is always within the preset time consumption interval.

[0054] In the embodiments of the present application, in the task processing process, in order to realize parallel computing, a large task needs to be split into multiple small task units (slices). When the first slicing is performed, the scheduler will preliminarily split the task according to the preset initial slice size. This initial slice size is an empirical value, which is used to determine the approximate data amount or calculation amount of each slice. After each Worker completes the execution of a slice unit, it will return the actual CPU time consumption of the slice to the scheduling controller. This time consumption information is an important basis for dynamically adjusting the slice size. By collecting the actual execution time of each slice, the scheduler can evaluate whether the current slice size is reasonable.

[0055] After the scheduler receives the CPU time consumption returned by the Worker, it will compare this time consumption with the preset time consumption interval. The preset time consumption interval is a reasonable execution time range, which is used to ensure that the execution time of each slice is neither too long (leading to low task processing efficiency) nor too short (leading to excessive scheduling overhead). According to the comparison result, the scheduler will dynamically adjust the size of the next slice. The scheduler will continuously monitor the execution time consumption of each slice and dynamically adjust the slice size according to the actual time consumption. This process will be repeated until the entire task is completed. In this way, the scheduler can ensure that the execution time of each slice is always within the preset reasonable interval, thereby optimizing the efficiency and resource utilization of task processing.

[0056] In one example embodiment, the increasing the weight of the low priority task over time until the low priority task is executed by the starvation detection mechanism includes:

[0057] S31, maintaining a starvation counter for each task that has not been out of the stack;

[0058] S32, triggering starvation detection at a preset time interval, and incrementing the starvation counter;

[0059] S33, calculating the final priority of the task according to the starvation counter and a preset weight coefficient;

[0060] S34, when the final priority is lower than the system minimum priority, the task is promoted to above the system minimum priority, ensuring that it is selected for execution in the next scheduling cycle.

[0061] In the embodiments of the present application, in task scheduling, in order to ensure that all tasks can obtain reasonable execution opportunities and avoid low-priority tasks being unable to execute for a long time (i.e. the "starvation" phenomenon), a "starvation counter" is maintained for each unfinished task by the scheduler. This counter is used to record the time or number of times that the task waits for execution, thereby reflecting the "starvation degree" of the task. The scheduler triggers "starvation detection" once per preset time interval (e.g. every 100 milliseconds). In each detection, the scheduler checks all unfinished tasks and increments their starvation counters.

[0062] In order to ensure that low-priority tasks are not permanently ignored, the scheduler adjusts the final priority of a task according to the value of the starvation counter and a preset weight coefficient. Specifically, the final priority of a task increases with the increase of the starvation counter, so that a task that has not been executed for a long time can obtain a higher priority. At the same time, the scheduler sets a system minimum priority. If the final priority of a task is lower than this minimum priority, the scheduler promotes the priority of the task to above the minimum priority, thereby ensuring that the task can be selected for execution in the next scheduling cycle.

[0063] Through the embodiments, the scheduler can effectively avoid the "starvation" phenomenon of low-priority tasks and ensure that all tasks can be executed within a reasonable time.

[0064] In an example embodiment, after the current shard unit is executed by any worker in the Web Worker pool through the scheduling controller, the method further comprises:

[0065] S41, when scheduling the shard unit task using the scheduling controller, if there is a higher priority task, it is checked whether there is an idle worker;

[0066] S42, if there is an idle worker, the idle worker is assigned to execute the high-priority task;

[0067] S43, if there is no idle worker, after the current shard unit is executed, the state of the task is saved and the task is suspended, and the execution of the shard unit of the high-priority task is continued.

[0068] In the embodiments of the present application, in combination with Figure 2 and Figure 3As shown, in the task scheduling process, the scheduling controller is responsible for managing the execution order of tasks and resource allocation. When the scheduling controller is ready to schedule a task of a slice unit, it will first check whether there is a higher priority task in the task queue. If there is a higher priority task, the scheduling controller needs to decide whether to interrupt the task currently being executed to prioritize the high-priority task. To this end, the scheduling controller checks whether there is a free Worker thread available for allocation. If the scheduling controller finds a free Worker thread, it immediately allocates the high-priority task to this free Worker thread. In this way, the current task being executed can be avoided to be interrupted, thereby improving the efficiency and response speed of the system.

[0069] If all Worker threads are busy, the scheduling controller needs to interrupt a task currently being executed to free up resources for the high-priority task. In order to ensure that the interrupted task can continue to be executed in the future, the scheduling controller saves the current state of the task (for example, the current processing position, intermediate results, etc.), and then suspends the task. The suspended task will be resumed for execution at an appropriate time. The scheduling controller then allocates the slice unit of the high-priority task to a Worker thread that is executing a task.

[0070] Through the embodiment, the scheduling controller can flexibly handle the insertion of high-priority tasks, while ensuring that the interrupted task can continue to be executed in the future.

[0071] In an example embodiment, the interrupting the current slice unit and saving the execution state of the current slice unit, and preempting the thread to execute the slice unit of the high-priority task comprises:

[0072] S51, sending an interrupt signal to the target Worker through the scheduling controller, and saving the execution state of the current task, wherein the execution state includes the current iteration position, the accumulated statistical value, and the offset of the processed data;

[0073] S52, returning the saved execution state to the scheduling controller as a state object, and suspending the execution of the current task.

[0074] In the embodiment of the application, when it is necessary to interrupt a task being executed, the scheduling controller sends an interrupt signal to the target Worker. After receiving the interrupt signal, the Worker needs to save the execution state of the current task, so as to resume the execution of the task in the future.

[0075] Worker needs to return the state object to the scheduler controller after saving the execution state. The scheduler controller receives the state object and stores it for resuming the task execution at an appropriate time. Meanwhile, the Worker suspends the execution of the current task and waits for further instructions from the scheduler controller.

[0076] Through this embodiment, the scheduler controller can effectively interrupt an ongoing task and save its execution state. This allows the task to continue execution from the last interruption point in the future instead of starting over, thereby improving the efficiency and resource utilization of the system.

[0077] In an example embodiment, after suspending the execution of the current task, the method further comprises:

[0078] S61, associate the received state object with the identifier of the corresponding task and store it in the suspended task table;

[0079] S62, when the task regains the Worker, re-deploy the state object to the Worker as an initial parameter and resume the execution of the task according to the state object.

[0080] In the embodiments of the present application, when the Worker saves the execution state of the task and returns it to the scheduler controller, the scheduler controller needs to associate the state object with the identifier (such as task ID) of the corresponding task, so as to accurately resume the execution of the task in the future. The scheduler controller will store this information in a special data structure, the suspended task table. The suspended task table records the state of all interrupted tasks, ensuring that these tasks can be resumed for execution at an appropriate time.

[0081] When the scheduler controller decides to resume an interrupted task, it retrieves the state object of the task from the suspended task table and re-deploys the state object to the Worker as an initial parameter. After receiving the state object, the Worker resumes the execution of the task according to the information in the state object, ensuring that the task can continue execution from the last interruption point, improving the efficiency and resource utilization of the system.

[0082] According to another aspect of the embodiments of the present application, a design device for implementing the above-mentioned WebWorker cooperative interruption scheduling processing method for high-frequency time sequence monitoring data is also provided. Figure 4 is a structural schematic diagram of an optional WebWorker cooperative interruption scheduling processing device for high-frequency time sequence monitoring data according to an embodiment of the present application, as Figure 4 shown, the device can include:

[0083] The splitting unit 402 is configured to split original high-frequency timing monitoring data into multiple tasks according to statistical dimensions and assign priorities in a main thread.

[0084] The minimum heap maintaining unit 404 is configured to maintain a task queue by using a minimum heap, and split each task into a slice unit that can be independently executed.

[0085] The creating and executing unit 406 is configured to dynamically create a Web Worker pool according to the number of central processor cores, and execute a current slice unit in any Worker in the Web Worker pool by using a scheduling controller.

[0086] The interruption saving unit 408 is configured to interrupt a current slice unit and save an execution state of the current slice unit, and preempt a slice unit of a high-priority task if a task with a priority higher than that of the current slice unit is detected.

[0087] The starvation detection unit 410 is configured to improve the weight of a low-priority task over time to the low-priority task by using a starvation detection mechanism, and transmit a task processing result to the main thread for visual rendering.

[0088] It should be noted that the splitting unit 402 in the embodiment can be configured to perform the step S102, the minimum heap maintaining unit 404 in the embodiment can be configured to perform the step S104, the creating and executing unit 406 in the embodiment can be configured to perform the step S106, the interruption saving unit 408 in the embodiment can be configured to perform the step S108, and the starvation detection unit 410 in the embodiment can be configured to perform the step S110.

[0089] By using the above modules, the original high-frequency timing monitoring data is split into multiple tasks according to statistical dimensions and assigned priorities in a main thread, each task is split into a slice unit that can be independently executed by using a minimum heap to maintain a task queue, a Web Worker pool is dynamically created according to the number of central processor cores, a current slice unit is executed in any Worker in the Web Worker pool by using a scheduling controller, a current slice unit is interrupted and an execution state of the current slice unit is saved, and a slice unit of a high-priority task is preempted if a task with a priority higher than that of the current slice unit is detected, the weight of a low-priority task is improved over time to the low-priority task by using a starvation detection mechanism, and a task processing result is transmitted to the main thread for visual rendering, thereby solving the problem of low execution efficiency caused by the inability to interrupt a Worker and task accumulation, and improving the response efficiency to a new task and the execution efficiency of the task.

[0090] In one example embodiment, the minimum heap maintaining unit comprises:

[0091] a sorting module configured to sort the plurality of tasks according to priorities of the tasks submitted by the main thread;

[0092] an insertion module configured to insert a new task into the heap and trigger scheduling when the new task is submitted by the main thread;

[0093] a deletion module configured to delete a task from the heap and notify the main thread of an execution result of the task when the task is completed.

[0094] In an example embodiment, the minimum heap maintaining unit comprises:

[0095] a fragmentation module configured to perform coarse fragmentation on a task according to a preset initial fragmentation size when the task is fragmented for the first time;

[0096] a feedback module configured to return, by a corresponding worker, an actual CPU time consumption of a fragment to the scheduling controller after the execution of each fragment unit is completed;

[0097] an adjustment module configured to adjust a size of a next fragment unit according to a comparison between a performance index and a preset time consumption interval;

[0098] a repeated execution module configured to repeatedly execute the above steps until the task is processed, so that the execution time consumption of any single fragment unit is always within the preset time consumption interval.

[0099] In an example embodiment, the starvation detection unit comprises:

[0100] a maintaining module configured to maintain a starvation counter for each task that has not been removed from the heap;

[0101] a triggering module configured to trigger starvation detection at a preset time interval and make the starvation counter increase;

[0102] a calculation module configured to calculate a final priority of a task according to the starvation counter and a preset weight coefficient;

[0103] a promotion module configured to promote the task to above a system minimum priority when the final priority is lower than the system minimum priority, so as to ensure that the task is selected for execution in a next scheduling period.

[0104] In an example embodiment, the apparatus further comprises:

[0105] a checking unit configured to check whether there is an idle worker if there is a task with a higher priority when the scheduling controller schedules a fragment unit task;

[0106] an idle execution unit configured to assign an idle worker to execute a high-priority task if there is an idle worker.

[0107] non-idle execution unit, configured to, if there is no idle worker, save the state of the task and suspend the task after the current shard unit execution is completed, and continue to execute the shard unit of the high-priority task.

[0108] In an example embodiment, the state saving unit comprises:

[0109] a saving module configured to send an interrupt signal to a target worker through a scheduling controller, and save the execution state of the current task, wherein the execution state comprises a current iteration position, accumulated statistical values, and an offset of processed data;

[0110] a pausing module configured to return the saved execution state to the scheduling controller as a state object, and pause the execution of the current task.

[0111] In an example embodiment, the apparatus further comprises:

[0112] an association unit configured to associate the received state object with an identifier of the corresponding task, and store the state object in a suspended task table;

[0113] a resuming execution unit configured to, when the task reacquires a worker, re-deliver the state object to the worker as an initial parameter, and resume the execution of the task according to the state object.

[0114] It should be noted that the above modules and the examples and scenarios realized by the corresponding steps are the same, but are not limited to the content disclosed in the above embodiments. It should be noted that the above modules, as part of the apparatus, can run in a hardware environment, can be implemented by software, or can be implemented by hardware, wherein the hardware environment includes a network environment.

[0115] According to another aspect of the embodiments of the present application, a storage medium is also provided. Optionally, in the present embodiment, the storage medium can be used to execute the program code of any of the above-mentioned WebWorker cooperation interrupt scheduling processing methods for high-frequency timing monitoring data in the embodiments of the present application.

[0116] Optionally, in the present embodiment, the storage medium is configured to store program code for executing the following steps:

[0117] S1, in a main thread, splitting original high-frequency timing monitoring data into multiple tasks according to statistical dimensions and assigning priorities to the tasks;

[0118] S2, using a minimum heap to maintain a task queue, and splitting each task into independently executable shard units;

[0119] S3, dynamically creating a WebWorker pool according to the number of central processor cores, and executing the current split unit in any worker in the WebWorker pool through a scheduling controller;

[0120] S4, if a task with a priority higher than the current split unit is detected, interrupting the current split unit, saving the execution state of the current split unit, and preempting the thread to execute the split unit of the high-priority task;

[0121] S5, improving the weight of the low-priority task over time through a starvation detection mechanism until the low-priority task is executed, and transmitting the task processing result to the main thread for visual rendering.

[0122] Optionally, specific examples in the embodiment can refer to the examples described in the above embodiments, and details are not described herein.

[0123] The computer readable storage medium can include, but is not limited to, any type of disk, including floppy disks, optical disks, DVDs, CD-ROMs, micro-drives, and magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory devices, magnetic or optical cards, nanosystems (including molecular memory ICs), or any type of medium or device suitable for storing instructions and / or data.

[0124] According to another aspect of the embodiment of the present application, an electronic device for implementing the above-mentioned WebWorker cooperative interruption scheduling processing method for high-frequency timing monitoring data is also provided, which can be a server, a terminal, or a combination thereof.

[0125] Figure 5 is a structural schematic diagram of an optional electronic device according to an embodiment of the present application, as shown in Figure 5 including a processor 502, a communication interface 504, a memory 506, and a communication bus 508, wherein the processor 502, the communication interface 504, and the memory 506 complete mutual communication through the communication bus 508, wherein,

[0126] The memory 506 is configured to store a computer program.

[0127] The processor 502 is configured to execute the computer program stored in the memory 506, and implement the following steps:

[0128] S1, splitting original high-frequency timing monitoring data into multiple tasks according to statistical dimensions and assigning priorities in a main thread;

[0129] S2, maintaining a task queue by using a minimum heap, and splitting each task into a split unit that can be independently executed;

[0130] S3, dynamically creating a WebWorker pool according to the number of central processor cores, and executing the current split unit in any Worker in the Web Worker pool through a scheduling controller;

[0131] S4, if a task with a priority higher than the current split unit is detected, interrupting the current split unit and saving the execution state of the current split unit, and preempting the thread to execute the split unit of the high-priority task;

[0132] S5, through a starvation detection mechanism, increasing the weight of the low-priority task over time to the execution of the low-priority task, and transmitting the task processing result to the main thread for visual rendering.

[0133] Optionally, the communication bus can be a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one thick line is used, but it does not mean that there is only one bus or one type of bus. The communication interface is used for communication between the above-mentioned electronic device and other devices.

[0134] The memory can include a RAM, and can also include a non-volatile memory, for example, at least one disk memory. Optionally, the memory can also be at least one storage device located away from the aforementioned processor.

[0135] As an example, the above-mentioned memory 506 can include but is not limited to the split unit 402, the minimum heap maintenance unit 404, the creation execution unit 406, the interruption saving unit 408, and the starvation detection unit 410 in the above-mentioned WebWorker cooperative interruption scheduling processing device for high-frequency time sequence monitoring data. In addition, other module units in the above-mentioned WebWorker cooperative interruption scheduling processing device for high-frequency time sequence monitoring data can also be included, but not limited to, which will not be described in detail in this example.

[0136] The processor can be a general processor, which can include but is not limited to: CPU (Central Processing Unit), NP (Network Processor), etc.; can also be DSP (Digital Signal Processing), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array) or other programmable logic device, discrete gate or transistor logic device, discrete hardware component.

[0137] Optionally, the specific examples in the embodiment can refer to the examples described in the above embodiments, and the embodiment will not be described here.

[0138] It should be noted that, for the foregoing method embodiments, in order to simply describe, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited to the action sequence described, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily necessary for the present application.

[0139] In the above embodiments, the description of each embodiment has its own emphasis, and the parts not described in detail in a certain embodiment can refer to the related description of other embodiments.

[0140] In several embodiments provided by the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only schematic. The division of the units is only a logical function division. There can be another division manner in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between different parts can be indirect couplings or communication connections through some services, devices or units, and can be electrical or other forms.

[0141] The units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. According to actual needs, part or all of the units can be selected to achieve the purpose of the embodiment scheme.

[0142] In addition, each of the function units in the embodiments of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software function unit.

[0143] When the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer readable memory. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a memory and includes several instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the methods described in the embodiments of the present application. The aforementioned memory includes: a U disk, a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0144] Those of ordinary skill in the art can understand that all or part of the steps of the various methods of the above embodiments can be completed by a program instructing relevant hardware, and the program can be stored in a computer readable memory, which can include a flash disk, a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a magnetic disk or an optical disk, etc.

[0145] The above is only exemplary embodiments of the present disclosure, and cannot limit the scope of the present disclosure. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. Those skilled in the art will easily think of embodiments of the present disclosure after considering the specification and practicing the disclosure herein. The present application is intended to cover any variations, uses or adaptive changes of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or conventional technical means in the technical field not described in the present disclosure. The specification and examples are only considered as exemplary, and the scope and spirit of the present disclosure are defined by the claims.

[0146] Each of the technical features of the above embodiments can be combined arbitrarily. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combinations of the technical features do not exist contradictory, they should be considered as the scope of the present disclosure.

[0147] Those skilled in the art can easily understand that the above description is only the preferred embodiment of the present application, and is not intended to limit the present application. Any modification, equivalent replacement and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A WebWorker collaborative interruption scheduling and handling method for high-frequency time-series monitoring data, characterized in that, include: In the main thread, the raw high-frequency time series monitoring data is split into multiple tasks according to statistical dimensions and assigned priorities; The task queue is maintained using a min-heap, and each task is further divided into independently executable fragments. A WebWorker pool is dynamically created based on the number of CPU cores, and the current shard unit is executed by any Worker in the WebWorker pool through the scheduling controller; If a task with a higher priority than the current shard unit is detected, the current shard unit is interrupted and its execution state is saved. The thread then preempts the shard unit from executing the high-priority task. The weight of low-priority tasks is increased over time through a starvation detection mechanism until the low-priority tasks are executed, and the task processing results are transmitted to the main thread for visualization rendering.

2. The WebWorker collaborative interruption scheduling and processing method for high-frequency time-series monitoring data as described in claim 1, characterized in that, The method of maintaining the task queue using a min-heap includes: Sort multiple tasks according to the priority of the tasks submitted by the main thread; When the main thread submits a new task, the task is inserted into the heap and scheduling is triggered. When a task is completed, it is removed from the heap and the main thread is notified of the task's execution result.

3. The WebWorker collaborative interruption scheduling and processing method for high-frequency time-series monitoring data as described in claim 1, characterized in that, The step of further splitting each task into independently executable fragment units includes: During the initial sharding, the task is coarsely sharded based on the preset initial shard size; After each slice unit is completed, the corresponding Worker sends back the actual CPU time of that slice to the scheduling controller. The size of the next slice unit is adjusted after comparing the performance indicators with the preset time interval. Repeat the above steps until the task is completed, so that the execution time of any single slice unit is always within the preset time range.

4. The WebWorker collaborative interruption scheduling and processing method for high-frequency time-series monitoring data as described in claim 1, characterized in that, The step of increasing the weight of low-priority tasks over time through a starvation detection mechanism until the low-priority tasks are executed includes: Maintain a starvation counter for each task that has not yet left the heap; Hunger detection is triggered at preset time intervals, causing the hunger counter to increment. The final priority of the task is calculated based on the hunger counter and the preset weight coefficient. When the final priority is lower than the system minimum priority, the task is promoted to a level above the system minimum priority to ensure that it is selected for execution in the next scheduling cycle.

5. The WebWorker collaborative interruption scheduling and processing method for high-frequency time-series monitoring data as described in claim 1, characterized in that, After the current sharding unit is executed by any Worker in the Web Worker pool via the scheduling controller, the method further includes: When scheduling a sharded unit task using the scheduling controller, if a higher-priority task exists, check if there is an idle worker. If there are idle workers, then assign the idle worker to execute the high-priority task; If no idle worker exists, the task's state is saved and suspended after the current sharding unit completes execution, and the sharding unit for the high-priority task continues to execute.

6. The WebWorker collaborative interruption scheduling and processing method for high-frequency time-series monitoring data as described in claim 1, characterized in that, The process of interrupting the current slice unit, saving its execution state, and preempting the thread from executing the high-priority task includes: The scheduling controller sends an interrupt signal to the target worker to save the execution status of the current task. The execution status includes the current iteration position, the accumulated statistical value, and the offset of the processed data. The saved execution status is sent back to the scheduling controller as a status object, pausing the execution of the current task.

7. The WebWorker collaborative interruption scheduling and processing method for high-frequency time-series monitoring data as described in claim 6, characterized in that, After pausing the execution of the current task, the method further includes: The received status object is associated with the identifier of the corresponding task and stored in the suspended task table. When the task regains access to the Worker, the state object is resubmitted to the Worker as an initial parameter, and the execution of the task is resumed based on the state object.

8. A WebWorker collaborative interruption scheduling and processing device for high-frequency time-series monitoring data, characterized in that, include: The splitting unit is used in the main thread to split the raw high-frequency time series monitoring data into multiple tasks according to statistical dimensions and assign priorities. The min-heap maintenance unit is used to maintain the task queue using a min-heap, and each task is further divided into independently executable fragments. Create an execution unit, which is used to dynamically create a Web Worker pool based on the number of CPU cores, and execute the current shard unit in any Worker in the Web Worker pool through the scheduling controller; An interruption and saving unit is used to interrupt the current segment unit and save its execution state if a task with a higher priority than the current segment unit is detected, and to preempt the thread from executing the segment unit of the high-priority task. The hunger detection unit is used to increase the weight of low-priority tasks over time through a hunger detection mechanism until the low-priority tasks are executed, and then transmit the task processing results to the main thread for visualization rendering.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein the program, when executed, performs the method of any one of claims 1 to 7.

10. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to execute the method of any one of claims 1 to 7 through the computer program.

Images