Automated scheduling dual-thread burying point processing method and system, electronic equipment and storage medium

By predicting CPU and Memory usage and using weight values ​​to automatically schedule, the browser's buried-point tasks are distributed to different threads, solving the performance problems of the browser's single-thread processing of intensive data, and achieving more efficient page rendering and data Report.

CN119046032APending Publication Date: 2024-11-29PING AN TECH (SHENZHEN) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411248824.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-06
Publication Date
2024-11-29

AI Technical Summary

Technical Problem

The single-threaded rendering feature of existing browsers results in high CPU and memory overhead when processing intensive data, which may cause page stuttering or crashing, and will not be able to take full advantage of the advantages of multi-core processors.

Method used

By predicting CPU and Memory usage, use weight values ​​to automatically dispatch buried points to different threads, alleviating the load of the first trigger thread, and using a two-thread processing mode to share the task.

Benefits of technology

It effectively avoids page lag, improves page performance and response speed, makes full use of the advantages of multi-core processors, and achieves faster and more accurate data reporting.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119046032A_ABST
    Figure CN119046032A_ABST
Patent Text Reader

Abstract

The invention provides an automatic scheduling double-thread burying point processing method and system, electronic equipment and a storage medium, and the method comprises the steps: monitoring newly added burying point tasks in a first trigger thread, configuring a mark for each burying point task, and adding the burying point tasks into a cache task queue; traversing the point burying tasks in the cache task queue in sequence, and automatically redistributing the point burying tasks to different task queue types according to corresponding task weight values; wherein the task weight value is related to the numerical values of the CPU utilization rate and the Memory utilization rate of the task; and respectively sending the burying point tasks in different task queue types to different threads for triggering. When intensive data exposure is processed, a large number of tasks of the first trigger thread can be shared, the situation that a page is blocked or stuck is avoided, the performance and response speed of the page are improved, the advantages of a multi-core processor can be brought into full play through double-thread opening, and the operation efficiency of a computer is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of Internet technology, and in particular to an automated scheduling dual-threaded point-of-care processing method, system, electronic device and storage medium. Background Art

[0002] At present, the mainstream tracking method on the web side is to directly track and report through the rendering of the first trigger thread. Based on the characteristics of single-threaded rendering of browsers, the mainstream tracking solutions have the following shortcomings: 1. The underlying layer of the browser is a relatively mature multi-threaded architecture. If data and user behavior can only rely on the browser rendering single thread for analysis, calculation and reporting, it is impossible to fully mobilize the advantages of the computer's multi-core processor; 2. When a large number of nodes appear in the user interface, and all of them need to be processed for data exposure and reporting, there may be a large amount of intensive data calculation, collection and reporting, and the computer CPU and memory overhead will increase greatly, which will inevitably block the rendering of the interface, increase the performance overhead of the web page, and may even cause serious problems such as page crashes. Summary of the invention

[0003] The embodiments of the present invention provide a method, system, electronic device and storage medium for automatically scheduling dual-threaded point-of-use processing. By predicting the CPU usage and memory usage of each task and automatically scheduling the first trigger thread and other threads through weight values, a large amount of intensive data collection, calculation, and reporting behaviors are reasonably distributed to different threads, which can save the overhead of rendering the first trigger thread and improve page rendering efficiency.

[0004] The technical solution of the present invention is:

[0005] In a first aspect, the present application provides an automated scheduling dual-threaded tracking processing method, characterized in that it includes:

[0006] Listen to the newly added tracking tasks in the first trigger thread, each tracking task is configured with a tag, and the tracking task is added to the cache task queue;

[0007] The executor traverses the tracking tasks in the cache task queue in sequence and reallocates the tracking tasks to different task queue types according to the corresponding task weight values. The task weight value is positively correlated with the CPU usage and / or memory usage value of the task.

[0008] When the task weight value is higher than the first threshold, the burying task is sent to the first task queue and triggered by the first trigger thread; otherwise, the burying task is sent to the second task queue and triggered by the second trigger thread.

[0009] Furthermore, the method further comprises:

[0010] After the second triggering thread completes the execution of the tracking task, it feeds the result back to the first triggering thread.

[0011] Furthermore, after the tracking task is executed, the method in which the second triggering thread feeds back the result to the first triggering thread includes:

[0012] The first triggering thread starts the task first, the second triggering thread starts monitoring and is in an idle state;

[0013] When the number of tracking tasks cached in the task queue of the first trigger thread is greater than the second threshold, the tracking tasks with task weight values ​​higher than the first threshold are sent to the first task queue to be executed by the first trigger thread; the tracking tasks with task weight values ​​equal to or lower than the first threshold are sent to the second task queue to be executed by the second trigger thread. After the second trigger thread completes the execution, it feeds back the execution result to the first trigger thread.

[0014] Furthermore, the method further comprises:

[0015] The second trigger thread clears the completed task after the task is executed.

[0016] Furthermore, the second triggering thread clears the completed task after the task is executed, including:

[0017] The second trigger thread listens to and receives the tracking task, and then executes the currently distributed tracking task. At the same time, the queue task in the first trigger thread is released, and the released free first trigger thread is used to execute other tracking services.

[0018] The second triggering thread notifies the first triggering thread after the tracking task is executed, and the first triggering thread monitors and receives the result execution information of the second triggering thread;

[0019] The first triggering thread receives the result execution information of the second triggering thread, marks the current embedding task as completed, and clears the completed tasks.

[0020] In a second aspect, the present application also provides an automated scheduling dual-thread tracking system, which is characterized by including:

[0021] A monitoring unit, used for monitoring newly added tracking tasks in the first triggering thread, each tracking task is configured with a tag, and the tracking task is added to the cache task queue;

[0022] The task weight value calculation unit determines the weight value of each task according to the CPU usage and memory usage values ​​of the task;

[0023] The execution unit traverses the tracking tasks in the cache task queue in turn, and automatically reallocates the tracking tasks to different task queue types according to the corresponding task weight values;

[0024] The trigger unit sends the tracking task to the first task queue and is triggered by the first trigger thread when the task weight value is higher than the first threshold value; otherwise, the tracking task is sent to the second task queue and is triggered by the second trigger thread.

[0025] Furthermore, the system also includes a feedback unit, which is used to feed back the result to the first triggering thread after the second triggering thread's tracking task is executed.

[0026] Furthermore, the system further comprises a clearing unit, which is used to clear the completed tasks after the second triggering thread completes the execution of the tasks.

[0027] In a third aspect, the present application also provides a computer storage medium, characterized in that: the computer storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the above method steps.

[0028] In a fourth aspect, the present application further provides an electronic device, characterized in that it includes a processor and a memory, the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the above method steps.

[0029] The beneficial effects brought about by the technical solutions provided by some embodiments of the present application include at least:

[0030] When processing-intensive data is exposed, a large number of tasks of the first trigger thread can be shared, avoiding page freezes or freezes, and improving page performance and response speed.

[0031] The activation of dual threads can give full play to the advantages of multi-core processors, improve computer operating efficiency, and report data information more quickly and accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0033] Figure 1 is a flow chart of a method according to an embodiment of the present invention;

[0034] Figure 2 is a system block diagram of an embodiment of the present invention;

[0035] Figure 3 A schematic diagram of a device after system packaging according to an embodiment of the present invention;

[0036] Figure 4 It is a schematic diagram of feeding back the execution result to the first triggering thread after the WebWorker thread task is executed in an embodiment of the present invention;

[0037] Figure 5 The figure is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention.

[0038] The specific implementation modes of the present invention are further described in detail below with reference to the accompanying drawings. DETAILED DESCRIPTION

[0039] The technical solution provided by the present invention is described in more detail below in conjunction with the accompanying drawings. The description of the exemplary implementation force is merely illustrative and is by no means intended to limit the present disclosure and its application or use. The present disclosure can be implemented in many different forms and is not limited to the embodiments described herein. These embodiments are provided to make the present disclosure thorough and complete and to fully express the scope of the present disclosure to those skilled in the art. It should be noted that unless otherwise specifically stated, the relative arrangement of the parts and steps, and the composition of the materials described in these embodiments. Numerical expressions and numerical values ​​should be interpreted as being merely exemplary and not as limitations.

[0040] The serial numbers assigned to the components in the present disclosure, such as "first", "second", etc., are only used to distinguish the objects described and do not have any order or technical meaning. The "connection" and "coupling" mentioned in the present invention, unless otherwise specified, include direct and indirect. In the description of the present invention, it should be understood that the directions or positional relationships indicated by the terms "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", etc. are based on the directions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation to the invention.

[0041] In the present disclosure, when a specific device is described as being located between a first device and a second device, there may or may not be an intermediate device between the specific device and the first device or the second device. When a specific device is described as being connected to other devices, the specific device may be directly connected to the other devices without an intermediate device, or may not be directly connected to the other devices but have an intermediate device.

[0042] All terms (including technical terms or scientific terms) used in the present disclosure have the same meanings as those understood by ordinary technicians in the field to which the present disclosure belongs, unless otherwise specifically defined. It should also be understood that terms defined in general dictionaries should be interpreted as having meanings consistent with their meanings in the context of the relevant technology, and should not be interpreted in an idealized or extremely formal sense, unless explicitly defined herein.

[0043] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered part of the specification.

[0044] The dimensions of the various parts shown in the drawings are not drawn according to the actual proportional relationship. In the drawings, the same figure marks are attached to common structural elements or structural elements of the same type, and their repeated descriptions are appropriately omitted.

[0045] The first trigger thread can create a new thread environment independent of the browser rendering first trigger thread by opening WebWorker in the browser of a multi-core processor. The two threads do not interfere with each other but can communicate and exchange data through postMessage. In order to improve the performance problems of existing mainstream tracking solutions when reporting intensive data, the present invention designs an automatic asynchronous scheduler (Scheduler) model in combination with the multi-threaded characteristics of the browser, which can start and control the browser rendering the first trigger thread and the WebWorker thread to simultaneously perform dual-thread tracking data reporting. The scheduler (Scheduler) is the hub of task execution. It intelligently predicts CPU and Memory usage through an automated monitoring algorithm model, and automatically schedules the first trigger thread and the WebWorker thread through weight values, so that a large amount of intensive data collection, calculation, and reporting behaviors are reasonably distributed to the two threads, which can save the overhead of rendering the first trigger thread and improve page rendering efficiency. Therefore, it is necessary to make improvements. Specifically:

[0046] like Figure 1 As shown, this embodiment provides an automatic scheduling dual-thread tracking processing method, which is summarized as follows, including:

[0047] Monitor the newly added tracking tasks in the first trigger thread. Each tracking task is configured with a tag id. At the same time, cache each monitored tracking task in the cache task queue;

[0048] The executor traverses the tasks in the cache task queue in sequence and automatically reallocates the tasks to different task queue types according to the corresponding task weights. The task weight value is related to the CPU usage and memory usage values ​​of the task.

[0049] Send tasks in different task queue types to different threads for triggering.

[0050] In some embodiments, when the task weight is higher than a first threshold, the task is sent to a first task queue and triggered by a first trigger thread; otherwise, the task is sent to a second task queue and triggered by a second trigger thread.

[0051] In some embodiments, the method further includes the second triggering thread feeding back the result to the first triggering thread after the task is executed.

[0052] After the task is completed, the second trigger thread will feed back the result to the first trigger thread. The advantage of this is that it can make full use of CPU resources and improve the execution efficiency of the program. For example, in a Web server, when the first trigger thread receives a request, it can create a new thread to process the request according to the actual situation, while it continues to wait for other requests. When the new thread processes the request, it will feed back the result to the first trigger thread, and the first trigger thread will return the result to the client.

[0053] The practice of requiring the second triggering thread to feed back the results to the first triggering thread after completing the task can ensure the integrity of the data and the execution order of the program. Especially when these subsequent operations depend on the previous execution results. The benefits of this practice can be:

[0054] Ensure data consistency

[0055] Avoid data contention: In a multitasking environment, if multiple tasks access and modify the same data at the same time, data contention may occur. This can be avoided by ensuring that threads execute in sequence, thereby ensuring data consistency.

[0056] Controlling the flow of execution: By waiting for the result of the second triggering thread, the first triggering thread can control the execution flow of the program and ensure that all necessary tasks have been completed before continuing.

[0057] Dependency management: In some application scenarios, the subsequent operation of the first triggering thread may depend on the execution result of the second thread. This dependency requirement can be met by ensuring that the second thread completes first.

[0058] Optimize resource utilization: Reduce resource waste: If the first triggering thread continues to execute without waiting for the second thread to complete, resources may be released prematurely or allocated repeatedly, resulting in resource waste.

[0059] Unified exception management: If an error occurs in the second thread, by waiting for its result, the exception can be captured and handled uniformly in the first triggering thread, making error management more centralized and convenient.

[0060] Timely feedback of problems: Once the second thread fails to execute, the first triggering thread can get feedback immediately and take measures, instead of continuing to execute subsequent operations without knowing it, causing the problem to expand.

[0061] Easy to debug: When a problem occurs, the ordered thread execution process makes it easier to locate the problem and reduce the difficulty of debugging.

[0062] In summary, requiring the second triggering thread to feed back the result to the first triggering thread after completing the task not only helps to ensure data consistency and synchronize thread execution order, but also optimizes resource utilization, simplifies error handling and other benefits.

[0063] In some embodiments, the method further includes the second triggering thread clearing the completed task after the task is executed.

[0064] In the dual-thread processing process, when the second thread completes the task, the completed task is cleared to release system resources and prepare for the execution of new tasks.

[0065] In some embodiments, this goal can be achieved through the following steps and considerations:

[0066] Task status tracking: Each task should have an associated status indicator, such as "not started", "in progress", "completed". When the second thread starts executing the task, its status is marked as "in progress". After the task is completed, the status is updated to "completed".

[0067] Resource cleanup: After a task is completed, in addition to updating the status, it is also necessary to clean up the resources associated with the task, such as releasing memory, closing files, disconnecting network connections, etc. The cleanup work is usually completed within the task's processing logic to ensure that all resources are properly handled.

[0068] Error handling: Errors may be encountered during task execution, and these errors need to be caught and handled appropriately. Depending on the severity of the error, the task may need to be re-executed or the error may need to be logged for subsequent analysis.

[0069] In the specific execution process, in dual-threaded programming and concurrent processing, effective management of completed tasks is crucial for the release of system resources and the scheduling of subsequent tasks. This operation can not only optimize system performance, but also prevent potential resource waste and memory leaks. The following will introduce in detail several methods for clearing completed tasks after the task is executed:

[0070] Use the isTerminated() method to determine whether all tasks are completed

[0071] Applicable scenarios: Applicable to situations where the first triggering thread needs to wait for all child threads to complete before continuing execution.

[0072] Implementation: Shut down the thread pool by calling the shutdown() method of the thread pool, and then loop to check the return value of the isTerminated() method. When all tasks are completed, the method returns true, and you can safely clean up resources or perform subsequent operations.

[0073] Advantages: Easy to use, no need to know the number of tasks in advance.

[0074] Disadvantage: The first triggering thread will be blocked until all tasks are executed.

[0075] Use CountDownLatch for task synchronization.

[0076] Applicable scenarios: Applicable to scenarios that require strict sequence control and execution can only continue after all tasks are completed.

[0077] Implementation method: Initialize a CountDownLatch counter and set it to the number of tasks to be executed. After each task is completed, call the countDown() method. The first triggering thread uses the await() method to wait for the counter to return to zero before continuing to execute.

[0078] Advantages: It enables fine-grained control and ensures that specific tasks are executed in order.

[0079] Disadvantages: The total number of tasks needs to be known in advance, and the change of the number of tasks is not flexible.

[0080] Using Future objects to track task status

[0081] Applicable scenarios: Applicable to scenarios where the results of each task need to be monitored and processed.

[0082] Implementation method: Submit the task through ExecutorService to obtain the Future object, use the future.isDone() method to check whether the task is completed. After completion, you can call future.cancel() to delete the completed task.

[0083] Advantages: The status of each task can be monitored in real time, which facilitates error handling and result collection.

[0084] Disadvantages: When managing a large number of tasks, a large number of Future objects need to be maintained, which increases the complexity of the code.

[0085] Manually manage tasks using task queues

[0086] Applicable scenarios: Applicable to flexible management requirements of dynamically adding and removing tasks.

[0087] Implementation: Create a task queue and remove the completed tasks from the queue. The first triggering thread can determine whether all tasks are completed by checking whether the queue is empty.

[0088] Advantages: High flexibility, tasks can be added or removed at any time.

[0089] Disadvantages: The task queue needs to be managed manually, which increases the complexity of coding.

[0090] In summary, clearing completed tasks after they are completed is an important management strategy in dual-threaded programming. This can not only improve the utilization efficiency of system resources, but also ensure the stable operation of the program.

[0091] The present invention is illustrated below by an automatic scheduling dual-threaded point-of-sale processing method.

[0092] Usually, a complete tracking task includes data collection, calculation and reporting (the exposure of tracking also needs to calculate whether the web page DOM is in the exposure area). When high-density data reporting occurs, the implementation method is as follows:

[0093] 1. When a new tracking task is added to the first trigger thread, each task is monitored and an id is marked for each task. At the same time, each monitored task is cached in the Scheduler task queue.

[0094] In some embodiments, the following points should be noted regarding the generation and management of task IDs:

[0095] Format design: Task IDs should be designed to be unique and sortable. For example, a timestamp plus a random string can be used to generate IDs to ensure that there will be no duplication even under high concurrency conditions.

[0096] Management method: Use a global variable or database to record the last assigned ID, so that each newly generated ID is unique.

[0097] When it comes to task queue management, you need to pay attention to queue size management: in order to prevent memory overflow, the queue size should be configured according to the system's available memory. In addition, you can set a certain upper limit, and tasks that exceed this upper limit can be processed by a rejection policy, such as AbortPolicy, DiscardPolicy, etc.

[0098] Task monitoring: The monitoring mechanism can be designed through the observer mode. When the state in the first trigger thread changes (such as user operation, data change, etc.), the embedded task listener is notified, and the change is immediately recorded as a task and added to the task queue.

[0099] Task storage: Persistent storage can be used. At the same time, in order to persist the data, the data of the tracking task also needs to be stored in the database. Batch insertion can be used to improve the efficiency of database operations.

[0100] Exception handling and logging: Each task should contain exception handling logic. When an error occurs during task execution, you can retry or log the error information into the log system.

[0101] Logging: Log the execution status of each task, including success, failure, exception, etc., through log components (such as Logback and Log4j). This will facilitate subsequent troubleshooting and performance analysis.

[0102] In summary, when new tracking tasks are added to the first trigger thread, these tasks can be efficiently monitored and managed through the above methods. These methods can not only ensure the effective execution of tasks, but also improve the maintainability and scalability of the system.

[0103] 2. When the Scheduler task queue caches tasks, it traverses the tasks in the cache queue in turn. Automatically reallocate the task queue type according to the corresponding task weight: when the task starts to execute, monitor the CPU and Memory usage values, and then predict and calculate the CPU and Memory usage of all pre-executed tasks. The higher the usage, the higher the weight, the higher the occupancy rate of service resources, and the more inclined to assign the task to the delayed task queue timerQueue; the lower the usage, the lower the weight, the lower the occupancy rate of service resources, and the more inclined to assign the task to the normal queue taskQueue.

[0104] When traversing and executing cache tasks, it should be efficient and ensure correct processing of tasks. The following are several common methods to achieve this requirement:

[0105] 1. Use iterator to traverse

[0106] Security: By using thread-safe collections such as ConcurrentLinkedQueue or ConcurrentLinkedDeque, you can safely traverse the task queue in a multi-threaded environment and avoid concurrent modification exceptions.

[0107] Efficiency: The iterator pattern can provide better traversal efficiency, especially when the task queue is long, and each task can be accessed quickly without copying data.

[0108] 2. Polling inspection

[0109] Real-time: By setting a background thread to regularly check the task queue, newly added tasks can be discovered and processed in a timely manner. This method is suitable for systems that have strict requirements on task processing time.

[0110] Resource consumption: Although the polling method can ensure timely processing of tasks, it may lead to waste of resources (such as CPU) if it is not used properly, so it is necessary to set the check interval reasonably.

[0111] 3. Blocking traversal

[0112] Waiting / notification mechanism: Using a blocking queue (such as BlockingQueue), the processing thread can be made to wait when there is no task until a new task arrives to automatically wake up the processing thread.

[0113] Reducing invalid operations: This mechanism can avoid unnecessary polling and traverse only when there are tasks to process, thereby effectively reducing resource consumption.

[0114] 4. Use the Executor framework

[0115] Simplify task management: The ExecutorService framework provided by Java can effectively manage thread resources and automatically execute tasks in the task queue without manual traversal.

[0116] Flexible configuration: ExecutorService provides a variety of configuration options, such as fixed thread pool, cached thread pool, etc., and you can choose the most appropriate execution strategy according to actual needs.

[0117] 5. Asynchronous event-driven

[0118] Decoupling: Through the event monitoring and publishing mechanism, asynchronous processing of tasks can be achieved, so that the generation and processing logic of tasks are separated, and the response speed of the system is improved.

[0119] Scalability: Event-driven models can be more easily expanded and maintained, especially in large-scale systems, and can better cope with complex and changing business needs.

[0120] 6. Combine with database transactions

[0121] Persistence processing: For tasks that require persistence, they can be combined with database transactions during the traversal process to ensure the atomicity and consistency of task processing.

[0122] Error recovery: Through database transaction management, even if an error occurs when processing a task, the transaction rollback mechanism can be used to restore to a consistent state and reprocess the task.

[0123] 6. Priority Scheduling

[0124] Differentiate the urgency of tasks: Assign different priorities to tasks and process them according to their priority level during traversal to ensure that urgent tasks are handled first.

[0125] Dynamic adjustment: Dynamically adjust the priority of tasks based on the current system load and the importance of the tasks to achieve optimal allocation of resources.

[0126] In summary, when the Scheduler task queue caches tasks, traversing the tasks in the cache queue in sequence is a key step to ensure that the tasks are executed in the predetermined order and rules. According to the specific scenario requirements and system characteristics, choose the appropriate traversal and execution strategy to ensure the efficiency and stability of the system.

[0127] In addition, in some embodiments, monitoring CPU and memory usage is key to ensuring stable system operation and timely response. This process involves multiple steps, from real-time data collection to data analysis and early warning, each step is crucial. The following are the key steps to achieve effective monitoring:

[0128] Real-time data collection

[0129] Collection tool selection: You can use the tools provided by the operating system, such as top and vmstat in Linux, or Performance Monitor in Windows, to monitor the CPU and memory usage in real time. For scenarios where data needs to be stored for a long time, you can consider using Syslog or similar services to collect and store this data.

[0130] Custom scripts: Write scripts (such as Shell scripts or PowerShell scripts) to periodically obtain CPU and memory usage and send the data to a log server or database for subsequent analysis.

[0131] Performance data storage

[0132] Time series database: Consider using a time series database, such as InfluxDB, which optimizes the storage and query of time series data and is suitable for storing monitoring data.

[0133] Structured storage: Storing monitoring data in traditional relational databases, such as MySQL or PostgreSQL, facilitates complex queries and correlation analysis.

[0134] Data analysis and presentation

[0135] Real-time monitoring dashboard: Use visualization tools such as Grafana and Kibana to build real-time monitoring dashboards. These tools can be seamlessly connected to the time series database to provide real-time data charts and trend lines.

[0136] Historical data analysis: Use data analysis tools to conduct in-depth analysis of collected historical data to identify usage patterns, trends, and anomalies.

[0137] Early warning mechanism

[0138] Threshold setting: Set warning lines and thresholds for CPU and memory usage. Once the monitoring data exceeds these preset values, the system will trigger an alarm.

[0139] Notification method: Alerts can be sent through a variety of channels, including but not limited to email, SMS, instant messaging tools (such as Slack or WeChat), and mobile phone push, ensuring that relevant personnel can receive and take action in a timely manner.

[0140] Automated processing

[0141] Automatic capacity expansion: When it is detected that resource usage reaches the threshold, new instances are automatically started or resources are added to share the load and prevent system overload.

[0142] Task scheduling optimization: Automatically adjust the execution priority and allocation of tasks based on monitoring data to reduce contention for resources.

[0143] Reporting and Optimization

[0144] Regular reports: Generate resource usage reports regularly, evaluate resource usage efficiency, and make optimization suggestions.

[0145] Continuous improvement: Based on monitoring data and analysis reports, continuously adjust system configuration and application deployment to optimize resource usage.

[0146] In summary, monitoring CPU and memory usage is a cyclic process, including data collection, storage, analysis, early warning and optimization. Through effective monitoring, not only can the stable operation of the system be guaranteed, but also resource allocation can be optimized to improve overall efficiency.

[0147] It should also be noted that in some embodiments, the following method can be used to predict and calculate the CPU and memory usage of all pre-execution tasks:

[0148] Historical data: If there is enough historical data, you can use machine learning or deep learning models to predict future CPU and memory usage. These models can predict future usage based on past usage patterns.

[0149] Task analysis method can be used: the characteristics of each task can be analyzed, such as task type, task size, task complexity, etc., and then the CPU and memory usage can be predicted based on these characteristics.

[0150] You can also use system monitoring: You can use system monitoring tools to monitor CPU and Memory usage in real time, and then calculate future usage based on this information.

[0151] You can also perform simulation tests: before actually executing a task, you can first perform simulation tests in a similar environment and under similar conditions, and then predict and calculate the CPU and Memory usage based on the test results.

[0152] 3. After taskQueue and timerQueue task queues are allocated according to the weight values, the tasks in taskQueue are directly triggered by the first trigger thread, and the delayed tasks in timerQueue are distributed to the WebWorker thread and triggered. Under the scheduling function of Scheduler, the concurrent processing mode of the first trigger thread and WebWorker thread is efficiently utilized, which greatly saves the memory overhead of the first trigger thread. When the task is executed, the completed tasks in the task queue will be cleared.

[0153] 4. After all WebWorker thread tasks are executed, the execution results need to be fed back to the first trigger thread. The specific implementation process of the communication between the two-thread tasks under the scheduling of the Scheduler is as follows:

[0154] (1) In general, the first trigger thread will start the task first and the WebWorker thread will start listening through onMessage. At this time, the WebWorker thread is in an idle state.

[0155] (2) When the task queue of the first trigger thread caches a large number of tasks to be executed, it is necessary to assign the taskQueue queue tasks to the first trigger thread for execution, and distribute the timerQueue queue tasks to the WebWorker thread by calling the postMessage method. The WebWorker thread listens and receives tasks through onMessage, and then starts to execute the currently distributed tasks. At this time, the timerQueue queue tasks in the first trigger thread are released, and the released free first trigger thread can execute other businesses. After the WebWorker thread task is executed, the first trigger thread will be notified through the postMessage method, and the first trigger thread will listen and receive the result execution information of the WebWorker thread through onMessage.

[0156] (3) The first trigger thread receives the result execution information of the WebWorker thread and marks the current task as completed, at which time the completed task is cleared.

[0157] When processing intensive data exposure, the method of this embodiment can share a large number of tasks of the first trigger thread, avoid page freeze or freeze, and improve page performance and response speed.

[0158] The activation of dual threads can give full play to the advantages of multi-core processors, improve computer operating efficiency, and report data information more quickly and accurately.

[0159] like Figure 2 and Figure 3 As shown, this embodiment also provides an automatic scheduling dual-thread tracking processing system, including:

[0160] The monitoring unit is used to monitor the newly added tracking tasks in the first triggering thread, each tracking task is configured with a tag id; and at the same time, each monitored tracking task is cached in the cache task queue;

[0161] The task weight value calculation unit determines the weight value of each task according to the CPU usage and memory usage values ​​of the task;

[0162] The execution unit traverses the tasks in the cache task queue in sequence and automatically reallocates the tasks to different task queue types according to the corresponding task weights;

[0163] The trigger unit is used to send tasks in different task queue types to different threads for triggering.

[0164] In some embodiments, the task weight value calculation unit can be implemented by the following software modules:

[0165] Custom algorithm module: According to actual needs, write a custom algorithm to process the calculation of task weight values. It can be implemented using programming languages ​​such as Python and Java.

[0166] Machine learning framework: Use machine learning frameworks (such as TensorFlow, PyTorch, etc.) to implement the task weight value calculation module. Suitable for scenarios that require a large amount of data training and optimization.

[0167] Database management system: If the task weight calculation involves a large amount of data query and processing, you can use a database management system (such as MySQL, PostgreSQL, etc.) to implement it. By writing SQL statements to query and process data, calculate the task weight value.

[0168] Data processing tools: Use data processing tools (such as Excel, Pandas, etc.) to implement the task weight value calculation module. This method is suitable for scenarios with small data volumes and no need for complex calculations.

[0169] Cloud computing platform: Use services provided by cloud computing platforms (such as AWS, Azure, etc.) to implement the task weight value calculation module. These platforms usually provide rich APIs and SDKs to easily implement various computing tasks.

[0170] In some embodiments, the execution unit may be implemented by the following software modules:

[0171] Operating system: The operating system is the interface between computer hardware and application programs. It is responsible for managing and controlling computer hardware resources and providing an operating environment for application programs.

[0172] Driver: The driver is the bridge between the operating system and computer hardware devices. It is responsible for converting the operating system's instructions into signals that the hardware devices can recognize, thereby achieving control over the hardware devices.

[0173] Compiler and Interpreter: Compilers and interpreters are responsible for converting source code written in high-level programming languages ​​into machine code that can be directly executed by the computer.

[0174] Virtual machine: A virtual machine is a software that simulates computer hardware and can run multiple operating systems and applications simultaneously on a physical computer to achieve resource isolation and sharing.

[0175] Container: Container is a lightweight virtualization technology that can package an application and its dependent environment into an independent running unit to achieve rapid deployment and operation of the application.

[0176] Middleware: Middleware is a software layer located between the operating system and application software, responsible for providing common services and functions to applications, such as database management, messaging, transaction processing, etc.

[0177] Application: An application is software used directly by users and is responsible for implementing specific functions and tasks, such as text editors, image processing software, games, etc.

[0178] In some embodiments, when the task weight is higher than a first threshold, the trigger unit sends the task to a first task queue, which is triggered by a first trigger thread; when the first threshold ≥ task weight ≥ second threshold, the trigger unit sends the task to a second task queue, which is triggered by a second trigger thread.

[0179] In some embodiments, the system further includes a feedback unit, and the feedback unit is used to feed back the result to the first trigger thread after the second trigger thread or other trigger threads complete the execution of the task.

[0180] In some embodiments, the system further includes a clearing unit, which is used to clear completed tasks after the second trigger thread or other trigger threads complete the execution of the tasks.

[0181] In the tracking system of this embodiment, the function of the monitoring unit can be encapsulated into a listener, and the functions of the task weight value calculation unit, the execution unit and the trigger unit can be encapsulated into an automated executor, such as a Scheduler executor, as illustrated below.

[0182] Before creating a tracking task, start a Scheduler task scheduler. Scheduler is an automated task flow monitoring and control model. The scheduler has two modules: listener and automated executor. The listener contains the Scheduler task queue cache. When a new tracking task is added to the first trigger thread, the Scheduler listener will listen to each task and mark each task with an id. At the same time, each monitored task will be cached in the Scheduler task queue of the listener.

[0183] When a task is cached in the Scheduler task queue, the Scheduler executor starts to traverse the tasks in the cache queue in sequence. When the Scheduler executor processes the tasks in the queue, it will automatically reallocate the task queue type according to the corresponding task weight: when the task starts to execute, the Scheduler executor automatically monitors the CPU and Memory usage values, and then predicts and calculates the CPU and Memory usage of all pre-executed tasks. The higher the usage rate, the higher the weight, the higher the occupancy rate of service resources, and the more inclined to assign the task to the delayed task queue timerQueue; the lower the usage rate, the lower the weight, the lower the occupancy rate of service resources, and the more inclined to assign the task to the normal queue taskQueue.

[0184] After the Scheduler executor allocates the taskQueue and timerQueue task queues according to the weight values, the tasks in the taskQueue are directly triggered by the first trigger thread, and the delayed tasks in the timerQueue are distributed to the WebWorker thread and triggered. Under the scheduling function of the Scheduler, the concurrent processing mode of the first trigger thread and the WebWorker thread is efficiently utilized, greatly saving the memory overhead of the first trigger thread. When the task is executed, the Scheduler executor will clear the completed tasks in the task queue.

[0185] After all WebWorker thread tasks are completed, the execution results need to be fed back to the first triggering thread, such as Figure 4 As shown in the figure, the specific implementation process of communication between dual-threaded tasks under the scheduling of Scheduler is as follows:

[0186] (1) In general, the first trigger thread will start the task first and the WebWorker thread will start listening through onMessage. At this time, the WebWorker thread is in an idle state.

[0187] (2) When the task queue of the first trigger thread caches a large number of tasks to be executed, the Scheduler starts working. The taskQueue queue tasks are assigned to the first trigger thread for execution, and the timerQueue queue tasks are distributed to the WebWorker thread by calling the postMessage method. The WebWorker thread listens and receives tasks through onMessage, and then starts to execute the currently distributed tasks. At this time, the timerQueue queue tasks in the first trigger thread are released, and the released free first trigger thread can execute other businesses. After the WebWorker thread task is executed, the first trigger thread will be notified through the postMessage method. The first trigger thread listens and receives the result execution information of the WebWorker thread through onMessage.

[0188] (3) The first trigger thread receives the result execution information of the WebWorker thread and marks the current task as completed. At this time, the Scheduler executor clears the completed task.

[0189] For example, in JavaScript, the first trigger thread can receive the result execution information of the WebWorker thread by listening to the onmessage event. When the WebWorker thread completes the calculation task, it sends the result back to the first trigger thread through the postMessage method. After receiving the result, the first trigger thread can mark the current task as completed.

[0190] In addition, to clear completed tasks through the Scheduler executor, you can use the following method:

[0191] First, make sure you have the rufus-scheduler gem installed.

[0192] Next, in the Ruby code, create a Scheduler object and define a task.

[0193] Again, in order to be able to clear the completed tasks, you need to store the tasks in a variable. You can use the scheduler.schedules method to get all scheduled tasks, and then clear the completed tasks as needed.

[0194] Finally, when you need to clear the completed tasks, call the clear_completed_tasks method to clear them.

[0195] When processing intensive data exposure, the embodiment of the present invention can share the tasks of a large number of first trigger threads, avoid page freeze or freezing, and improve page performance and response speed.

[0196] The activation of dual threads can give full play to the advantages of multi-core processors, improve computer operating efficiency, and report data information more quickly and accurately.

[0197] The present application also provides a computer storage medium, which stores multiple instructions, which are suitable for being loaded by a processor and executing the above method steps. Computer storage medium is a physical device for storing data, such as a hard disk, an optical disk, a USB flash drive, etc. These storage media can store various types of data, including text, images, audio, video, and program code. The storage medium stores some programs or scripts, which contain a series of instructions suitable for being loaded by a processor and executing the above method steps, and these instructions can be read, parsed, and executed by the computer's processor.

[0198] For example, an executable file (such as an .exe file) stored on a USB flash drive contains a series of instructions. When this file is copied to a computer and run, the computer's processor will operate according to the instructions in the file to achieve a specific function or task. The specific execution process can be found in the detailed description of the system and method, which will not be repeated here.

[0199] The present application also provides an electronic device, including a processor and a memory, wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the above method steps.

[0200] See also Figure 5 , is a schematic diagram of the main hardware structure of the electronic device provided in an embodiment of the present application.

[0201] The customer voice demand classification device 300 may include: at least one processor 301 , at least one network interface 304 , a user interface 303 , a memory 305 , and at least one communication bus 302 .

[0202] The communication bus 302 is used to realize the connection and communication between these components.

[0203] The user interface 303 may include a touch screen and a camera.

[0204] The network interface 304 may optionally include but is not limited to a standard wired interface, a wireless interface (such as a WI-FI interface, Bluetooth, a traffic card, etc.).

[0205] Among them, the processor 301 may include one or more processing cores. The processor 301 uses various interfaces and lines to connect the various parts of the entire multi-tenant mode data acquisition system 300, and executes various functions and processes data of the multi-tenant mode data acquisition system 300 by running or executing instructions, programs, code sets or instruction sets stored in the memory 305, and calling data stored in the memory 305. Optionally, the processor 301 can be implemented in at least one hardware form of digital signal processing (DSP), field programmable gate array (FPGA), and programmable logic array (PLA). The processor 301 can integrate one or more combinations of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; and the modem is used to handle wireless communication. It is understandable that the above-mentioned modem may not be integrated into the processor 301, but may be implemented by a separate chip.

[0206] Among them, the memory 305 may include a random access memory (Random Access Memory, RAM) and may also include a read-only memory (Read-Only Memory). Optionally, the memory 305 includes a non-transitory computer-readable storage medium. The memory 305 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 305 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 305 may optionally be at least one storage device located away from the aforementioned processor 301. As a computer storage medium, the memory 305 may include an operating system, a network communication module, a user interface module and an application. In Figure 5 In the system 300 shown, the user interface 303 is mainly used to provide an input interface for the user and obtain the data input by the user; and the processor 301 can be used to call the application stored in the memory 305 and specifically execute the following steps: Figure 1 The specific process can be referred to Figure 1 As shown, no further description is given here.

[0207] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium along with the main application program. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be cloud storage, a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0208] In summary, the embodiments of the present invention combine the multi-threaded characteristics of the browser to design an automated asynchronous scheduler (Scheduler) model, which can start and control the browser to render the first trigger thread and the WebWorker thread and simultaneously perform dual-threaded point data reporting. The scheduler (Scheduler) is the core of task execution. It uses an automated monitoring algorithm model to intelligently predict CPU and Memory usage, and automatically schedules the first trigger thread and the WebWorker thread through weight values, so that a large amount of intensive data collection, calculation, and reporting behaviors are reasonably distributed to the two threads, which can save the overhead of rendering the first trigger thread and improve page rendering efficiency. Therefore, it is necessary to make improvements.

[0209] The features or feature combinations mentioned above in the specification, drawings and claims, as long as they are meaningful within the scope of the present invention and do not contradict each other, can be used in any combination or alone. The advantages and features described for the system provided by the present invention are applicable to the method and computer-readable storage medium provided by the present invention in a corresponding manner, and vice versa.

[0210] It should also be noted that the terms "comprises", "includes" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, product or apparatus that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, product or apparatus. In the absence of further restrictions, the elements defined by the sentence "comprises one..." do not exclude the presence of other identical elements in the process, method, product or apparatus that includes the elements.

[0211] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An automated scheduling dual-threaded point-of-use processing method, characterized in that: include: Listen to the newly added tracking tasks in the first trigger thread, each tracking task is configured with a tag, and the tracking task is added to the cache task queue; Traverse the tracking tasks in the cache task queue in turn, and reallocate the tracking tasks to different task queue types according to the corresponding task weight values; the task weight value is positively correlated with the CPU usage and / or memory usage value of the task; When the task weight value is higher than the first threshold, the burying task is sent to the first task queue and triggered by the first trigger thread; otherwise, the burying task is sent to the second task queue and triggered by the second trigger thread.

2. The automatic scheduling dual-threaded point-of-use processing method according to claim 1, characterized in that: Also includes: After the second trigger thread completes the tracking task, it feeds the result back to the first trigger thread.

3. The automatic scheduling dual-threaded point-of-use processing method according to claim 2, characterized in that: After the second triggering thread completes the execution of the tracking task, the method for feeding back the result to the first triggering thread includes: The first triggering thread starts the task first, the second triggering thread starts monitoring and is in an idle state; When the number of tracking tasks cached in the task queue of the first trigger thread is greater than the second threshold, the tracking tasks with task weight values ​​higher than the first threshold are sent to the first task queue to be executed by the first trigger thread; the tracking tasks with task weight values ​​equal to or lower than the first threshold are sent to the second task queue to be executed by the second trigger thread. After the second trigger thread completes the execution, it feeds back the execution result to the first trigger thread.

4. The automatic scheduling dual-threaded tracking processing method according to claim 2, characterized in that: Also includes: The second trigger thread clears the completed task after the task is executed.

5. The automatic scheduling dual-threaded point-of-use processing method according to claim 4, characterized in that: The second trigger thread clears the completed tasks after the tasks are executed, including: The second trigger thread listens to and receives the tracking task, and then executes the currently distributed tracking task. At the same time, the queue task in the first trigger thread is released, and the released free first trigger thread is used to execute other tracking services. The second triggering thread notifies the first triggering thread after the tracking task is executed, and the first triggering thread monitors and receives the result execution information of the second triggering thread; The first triggering thread receives the result execution information of the second triggering thread, marks the current embedding task as completed, and clears the completed tasks.

6. An automated scheduling dual-threaded point-of-use processing system, characterized in that: include: A monitoring unit, used for monitoring newly added tracking tasks in the first triggering thread, each tracking task is configured with a tag, and the tracking task is added to the cache task queue; The task weight value calculation unit determines the weight value of each task according to the CPU usage and memory usage values ​​of the task; The execution unit traverses the tracking tasks in the cache task queue in turn, and automatically reallocates the tracking tasks to different task queue types according to the corresponding task weight values; The trigger unit sends the tracking task to the first task queue and is triggered by the first trigger thread when the task weight value is higher than the first threshold value; otherwise, the tracking task is sent to the second task queue and is triggered by the second trigger thread.

7. The automatic scheduling dual-threaded point-of-use processing system according to claim 6, characterized in that: The system also includes a feedback unit, which is used to feed back the result to the first trigger thread after the second trigger thread's tracking task is executed.

8. The automatic scheduling dual-threaded point-of-sale processing system according to claim 6, characterized in that: The system further comprises a clearing unit, which is used for clearing the completed tasks after the second triggering thread completes the execution of the tasks.

9. A computer storage medium, characterized in that: The computer storage medium stores a plurality of instructions, which are suitable for being loaded by a processor and executing the method steps according to any one of claims 1 to 5.

10. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the method steps according to any one of claims 1 to 5.