Efficient tagging verification task processing system and method

Abstract

The invention relates to computer task processing, in particular to an efficient tagging check task processing system and method, a task decomposition and distribution module is used for decomposing a complex service request into a plurality of sub-tasks and intelligently distributing the sub-tasks; the concurrent execution engine is used for selecting a proper concurrent strategy according to task types and requirements, and performing multi-task concurrent execution based on the concurrent strategy through a concurrent execution model; the asynchronous task management module supports creation, scheduling, execution and result callback of asynchronous tasks, and simplifies asynchronous logic processing by introducing an asynchronous programming mode; the task scheduler is used for realizing asynchronous transmission and queue management of tasks by using the message queue and dynamically scheduling the tasks in the queue; according to the technical scheme provided by the invention, the defects of unreasonable resource allocation, poor task scheduling flexibility and low task execution efficiency in the prior art can be effectively overcome.

Description

Technical Field

[0001] The present invention relates to computer task processing, and in particular to a high-efficiency coding verification task processing system and method. Background Art

[0002] With the rapid development of the Internet, software systems are facing increasing demands for concurrent access and data processing. Traditional synchronous processing modes often encounter problems such as resource bottlenecks and performance degradation in high-concurrency scenarios. In order to solve the above problems, the industry has proposed a variety of concurrent and asynchronous task processing methods, such as multi-threaded programming, event-driven programming, message queues, etc. However, these methods still have significant shortcomings in practical applications, such as complex thread management, fierce resource competition, and inflexible task scheduling. Summary of the invention

[0003] 1. Technical issues to be solved

[0004] In view of the above-mentioned shortcomings of the prior art, the present invention provides an efficient coding verification task processing system and method, which can effectively overcome the defects of the prior art such as unreasonable resource allocation, poor task scheduling flexibility and low task execution efficiency.

[0005] (II) Technical solution

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0007] An efficient coding verification task processing system, including a task decomposition and allocation module, a concurrent execution engine, an asynchronous task management module and a task scheduler;

[0008] The task decomposition and allocation module decomposes complex business requests into multiple subtasks and allocates them intelligently;

[0009] The concurrent execution engine selects the appropriate concurrency strategy according to the task type and requirements, and performs multi-task concurrent execution based on the concurrency strategy through the concurrent execution model;

[0010] Asynchronous task management module, which supports the creation, scheduling, execution and result callback of asynchronous tasks, and simplifies asynchronous logic processing by introducing asynchronous programming mode;

[0011] The task scheduler uses message queues to implement asynchronous task transmission and queue management, and dynamically schedules tasks in the queue.

[0012] Preferably, the task decomposition and allocation module decomposes a complex business request into multiple subtasks and performs intelligent allocation, including:

[0013] After receiving a business request, it is decomposed into multiple subtasks according to the business logic and request type;

[0014] Based on system resources and task priorities, a load balancing algorithm is used to intelligently allocate subtasks to ensure that subtasks are evenly distributed among multiple processing units to avoid overloading a single node.

[0015] Preferably, the concurrent execution engine selects a suitable concurrent strategy according to the task type and requirements, and performs concurrent execution of multiple tasks based on the concurrent strategy through the concurrent execution model, including:

[0016] Use multiple processing units to build a concurrent execution model, select appropriate concurrency strategies according to task types and requirements, and perform multi-task concurrent execution based on the concurrent strategy through the concurrent execution model;

[0017] Among them, concurrent execution models include multi-threading, multi-process and coroutines.

[0018] Preferably, the performing multi-task concurrent execution based on a concurrent strategy through a concurrent execution model includes:

[0019] Use concurrency control technology to reduce resource competition and deadlock risks, improve concurrent execution efficiency, and ensure fast task execution and effective use of resources;

[0020] Among them, concurrency control technologies include thread pool management, lock mechanism and semaphore.

[0021] Preferably, the asynchronous task management module supports the creation, scheduling, execution and result callback of asynchronous tasks, and simplifies asynchronous logic processing by introducing an asynchronous programming model, including:

[0022] Simplify asynchronous logic processing by introducing asynchronous programming patterns, and improve code readability and maintainability;

[0023] At the same time, it provides task priority adjustment mechanism, timeout control mechanism and exception handling mechanism to ensure the stable execution of asynchronous tasks;

[0024] Among them, asynchronous programming modes include Promise and async / await.

[0025] Preferably, the task scheduler uses a message queue to implement asynchronous transmission of tasks and queue management, and dynamically schedules tasks in the queue, including:

[0026] Use message queues to implement asynchronous transmission of tasks and queue management, and dynamically schedule tasks in the queue according to preset strategies to relieve system pressure, improve task processing efficiency, and ensure efficient execution of tasks;

[0027] Among them, the preset strategies include polling and priority queue.

[0028] Preferably, the system also includes a cache and database optimization module, wherein the database optimization module uses cache technology to reduce the number of database accesses, improve data reading speed, and optimize the database to improve the database's read and write performance and concurrent processing capabilities;

[0029] Among them, optimization operations include index optimization, table and database sharding, and connection pool management.

[0030] Preferably, the system further comprises a monitoring and performance tuning module, which uses real-time monitoring and performance analysis tools to monitor and analyze the system operation status, collects key performance indicators to timely discover and handle performance bottlenecks, and realizes dynamic tuning and continuous optimization of the system;

[0031] Among them, key performance indicators include CPU usage, memory usage and response time.

[0032] An efficient coding verification task processing method includes the following steps:

[0033] S1. After receiving a business request, the task decomposition and allocation module decomposes the business request into multiple subtasks according to the business logic and request type, and uses a load balancing algorithm to intelligently allocate the subtasks based on system resources and task priorities to ensure that the subtasks are evenly distributed among multiple processing units;

[0034] S2, the concurrent execution engine selects the appropriate concurrent strategy according to the task type and requirements, and performs multi-task concurrent execution based on the concurrent strategy through the concurrent execution model;

[0035] S3, the asynchronous task management module creates, schedules, executes and results callbacks asynchronous tasks, simplifies asynchronous logic processing by introducing asynchronous programming mode, and provides task priority adjustment mechanism, timeout control mechanism and exception handling mechanism to ensure stable execution of asynchronous tasks;

[0036] S4, the task scheduler uses the message queue to realize the asynchronous transmission and queue management of tasks, and dynamically schedules the tasks in the queue according to the preset strategy to ensure the efficient execution of tasks;

[0037] Among them, during the task processing:

[0038] The cache and database optimization module uses cache technology to reduce the number of database accesses and increase data reading speed. It also optimizes the database to improve the database's read and write performance and concurrent processing capabilities.

[0039] The cache and database optimization module uses real-time monitoring and performance analysis tools to monitor and analyze the system operation status. By collecting key performance indicators, it promptly discovers and handles performance bottlenecks, thereby achieving dynamic tuning and continuous optimization of the system.

[0040] Preferably, before receiving the service request in S1, the following steps are included:

[0041] Adopting microservice architecture, the system is divided into multiple independent service modules. Each module is responsible for a specific business function. The lightweight communication protocol between services is used to achieve decoupling and collaborative work between modules.

[0042] Among them, the lightweight communication protocol between services includes RESTful API.

[0043] (III) Beneficial effects

[0044] Compared with the prior art, the efficient coding verification task processing system and method provided by the present invention realizes rapid scheduling, efficient execution and result feedback of tasks by integrating multiple concurrent control technologies and asynchronous task management mechanisms. While improving the processing capability and response speed of the system, it reduces resource competition and operation and maintenance costs. It has broad application prospects and promotion value, and is particularly suitable for application scenarios with high concurrency and large data volumes. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0046] Figure 1 It is a schematic diagram of the system of the present invention;

[0047] Figure 2 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] An efficient coding verification task processing system, such as Figure 1As shown, it includes a task decomposition and allocation module, a concurrent execution engine, an asynchronous task management module and a task scheduler;

[0050] The task decomposition and allocation module decomposes complex business requests into multiple subtasks and allocates them intelligently;

[0051] The concurrent execution engine selects the appropriate concurrency strategy according to the task type and requirements, and performs multi-task concurrent execution based on the concurrency strategy through the concurrent execution model;

[0052] Asynchronous task management module, which supports the creation, scheduling, execution and result callback of asynchronous tasks, and simplifies asynchronous logic processing by introducing asynchronous programming mode;

[0053] The task scheduler uses message queues to implement asynchronous task transmission and queue management, and dynamically schedules tasks in the queue.

[0054] ①The task decomposition and allocation module decomposes complex business requests into multiple subtasks and allocates them intelligently, including:

[0055] After receiving a business request, it is decomposed into multiple subtasks according to the business logic and request type;

[0056] Based on system resources and task priorities, a load balancing algorithm is used to intelligently allocate subtasks to ensure that subtasks are evenly distributed among multiple processing units to avoid overloading a single node.

[0057] ② The concurrent execution engine selects the appropriate concurrent strategy according to the task type and requirements, and performs multi-task concurrent execution based on the concurrent strategy through the concurrent execution model, including:

[0058] Use multiple processing units to build a concurrent execution model, select appropriate concurrency strategies according to task types and requirements, and perform multi-task concurrent execution based on the concurrent strategy through the concurrent execution model;

[0059] Among them, concurrent execution models include multi-threading, multi-process and coroutines.

[0060] Specifically, the concurrent execution model is used to perform multi-task concurrent execution based on the concurrent strategy, including:

[0061] Use concurrency control technology to reduce resource competition and deadlock risks, improve concurrent execution efficiency, and ensure fast task execution and effective use of resources;

[0062] Among them, concurrency control technologies include thread pool management, lock mechanism and semaphore.

[0063] ③The asynchronous task management module supports the creation, scheduling, execution and result callback of asynchronous tasks, and simplifies asynchronous logic processing by introducing asynchronous programming mode, including:

[0064] Simplify asynchronous logic processing by introducing asynchronous programming patterns, and improve code readability and maintainability;

[0065] At the same time, it provides task priority adjustment mechanism, timeout control mechanism and exception handling mechanism to ensure the stable execution of asynchronous tasks;

[0066] Among them, asynchronous programming modes include Promise and async / await.

[0067] ④The task scheduler uses the message queue to implement asynchronous transmission of tasks and queue management, and dynamically schedules tasks in the queue, including:

[0068] Use message queues to implement asynchronous transmission of tasks and queue management, and dynamically schedule tasks in the queue according to preset strategies to relieve system pressure, improve task processing efficiency, and ensure efficient execution of tasks;

[0069] Among them, the preset strategies include polling and priority queue.

[0070] ⑤ In the technical solution of this application, the coding verification task processing system also includes a cache and database optimization module. The database optimization module uses cache technology to reduce the number of database accesses and improve the data reading speed. At the same time, it optimizes the database to improve the database's read and write performance and concurrent processing capabilities;

[0071] Among them, optimization operations include index optimization, table and database sharding, and connection pool management.

[0072] ⑥ In the technical solution of this application, the coding verification task processing system also includes a monitoring and performance tuning module, which uses real-time monitoring and performance analysis tools to monitor and analyze the system operation status, collects key performance indicators to timely discover and handle performance bottlenecks, and realizes dynamic tuning and continuous optimization of the system;

[0073] Among them, key performance indicators include CPU usage, memory usage and response time.

[0074] Based on the above coding verification task processing system, the technical solution of this application also discloses an efficient coding verification task processing method, such as Figure 2 As shown, the following steps are included:

[0075] S1. After receiving a business request, the task decomposition and allocation module decomposes the business request into multiple subtasks according to the business logic and request type, and uses a load balancing algorithm to intelligently allocate the subtasks based on system resources and task priorities to ensure that the subtasks are evenly distributed among multiple processing units;

[0076] S2, the concurrent execution engine selects the appropriate concurrent strategy according to the task type and requirements, and performs multi-task concurrent execution based on the concurrent strategy through the concurrent execution model;

[0077] S3, the asynchronous task management module creates, schedules, executes and results callbacks asynchronous tasks, simplifies asynchronous logic processing by introducing asynchronous programming mode, and provides task priority adjustment mechanism, timeout control mechanism and exception handling mechanism to ensure stable execution of asynchronous tasks;

[0078] S4, the task scheduler uses the message queue to realize the asynchronous transmission and queue management of tasks, and dynamically schedules the tasks in the queue according to the preset strategy to ensure the efficient execution of tasks;

[0079] Among them, during the task processing:

[0080] The cache and database optimization module uses cache technology to reduce the number of database accesses and increase data reading speed. It also optimizes the database to improve the database's read and write performance and concurrent processing capabilities.

[0081] The cache and database optimization module uses real-time monitoring and performance analysis tools to monitor and analyze the system operation status. By collecting key performance indicators, it promptly discovers and handles performance bottlenecks, thereby achieving dynamic tuning and continuous optimization of the system.

[0082] Specifically, before receiving the service request in S1, the following steps are performed:

[0083] Adopting microservice architecture, the system is divided into multiple independent service modules. Each module is responsible for a specific business function. The lightweight communication protocol between services is used to achieve decoupling and collaborative work between modules.

[0084] Among them, the lightweight communication protocol between services includes RESTful API.

[0085] In the technical solution of this application, by integrating multiple concurrent control technologies and asynchronous task management mechanisms, rapid scheduling, efficient execution and result feedback of tasks are achieved. While improving the processing power and response speed of the system, it reduces resource competition and operation and maintenance costs. It has broad application prospects and promotion value, and is particularly suitable for application scenarios with high concurrency and large data volumes.

[0086] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements will not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An efficient coding verification task processing system, Features: It includes task decomposition and allocation module, concurrent execution engine, asynchronous task management module and task scheduler; The task decomposition and allocation module decomposes complex business requests into multiple subtasks and allocates them intelligently; The concurrent execution engine selects the appropriate concurrency strategy according to the task type and requirements, and performs multi-task concurrent execution based on the concurrency strategy through the concurrent execution model; Asynchronous task management module, which supports the creation, scheduling, execution and result callback of asynchronous tasks, and simplifies asynchronous logic processing by introducing asynchronous programming mode; The task scheduler uses message queues to implement asynchronous task transmission and queue management, and dynamically schedules tasks in the queue.

2. According to the efficient coding verification task processing system of claim 1, Features: The task decomposition and allocation module decomposes complex business requests into multiple subtasks and allocates them intelligently, including: After receiving a business request, it is decomposed into multiple subtasks according to the business logic and request type; Based on system resources and task priorities, a load balancing algorithm is used to intelligently allocate subtasks to ensure that subtasks are evenly distributed among multiple processing units to avoid overloading a single node.

3. According to the efficient coding verification task processing system of claim 1, Features: The concurrent execution engine selects a suitable concurrent strategy according to the task type and requirements, and performs concurrent execution of multiple tasks based on the concurrent strategy through the concurrent execution model, including: Use multiple processing units to build a concurrent execution model, select appropriate concurrency strategies according to task types and requirements, and perform multi-task concurrent execution based on the concurrent strategy through the concurrent execution model; Among them, concurrent execution models include multi-threading, multi-process and coroutines.

4. The efficient coding verification task processing system according to claim 3, Features: The concurrent execution of multiple tasks based on the concurrent strategy through the concurrent execution model includes: Use concurrency control technology to reduce resource competition and deadlock risks, improve concurrent execution efficiency, and ensure fast task execution and effective use of resources; Among them, concurrency control technologies include thread pool management, lock mechanism and semaphore.

5. The efficient coding verification task processing system according to claim 1, Features: The asynchronous task management module supports the creation, scheduling, execution and result callback of asynchronous tasks, and simplifies asynchronous logic processing by introducing asynchronous programming mode, including: Simplify asynchronous logic processing by introducing asynchronous programming patterns, and improve code readability and maintainability; At the same time, it provides task priority adjustment mechanism, timeout control mechanism and exception handling mechanism to ensure the stable execution of asynchronous tasks; Among them, asynchronous programming modes include Promise and async / await.

6. The efficient coding verification task processing system according to claim 1, Features: The task scheduler uses the message queue to implement asynchronous transmission of tasks and queue management, and dynamically schedules tasks in the queue, including: Use message queues to implement asynchronous transmission of tasks and queue management, and dynamically schedule tasks in the queue according to preset strategies to relieve system pressure, improve task processing efficiency, and ensure efficient execution of tasks; Among them, the preset strategies include polling and priority queue.

7. The efficient coding verification task processing system according to claim 1, Features: The system also includes a cache and database optimization module, which uses cache technology to reduce the number of database accesses and increase data reading speed, while optimizing the database to improve the database's read and write performance and concurrent processing capabilities; Among them, optimization operations include index optimization, table and database sharding, and connection pool management.

8. The efficient coding verification task processing system according to claim 7, Features: The system also includes a monitoring and performance tuning module, which uses real-time monitoring and performance analysis tools to monitor and analyze the system operation status, collects key performance indicators to timely discover and handle performance bottlenecks, and realizes dynamic tuning and continuous optimization of the system; Among them, key performance indicators include CPU usage, memory usage and response time.

9. An efficient coding verification task processing method, applied to the efficient coding verification task processing system of claim 8, Features: The following steps are involved: S1. After receiving a business request, the task decomposition and allocation module decomposes the business request into multiple subtasks according to the business logic and request type, and uses a load balancing algorithm to intelligently allocate the subtasks based on system resources and task priorities to ensure that the subtasks are evenly distributed among multiple processing units; S2, the concurrent execution engine selects the appropriate concurrent strategy according to the task type and requirements, and performs multi-task concurrent execution based on the concurrent strategy through the concurrent execution model; S3, the asynchronous task management module creates, schedules, executes and results callbacks asynchronous tasks, simplifies asynchronous logic processing by introducing asynchronous programming mode, and provides task priority adjustment mechanism, timeout control mechanism and exception handling mechanism to ensure stable execution of asynchronous tasks; S4, the task scheduler uses the message queue to realize the asynchronous transmission and queue management of tasks, and dynamically schedules the tasks in the queue according to the preset strategy to ensure the efficient execution of tasks; Among them, during the task processing: The cache and database optimization module uses cache technology to reduce the number of database accesses and increase data reading speed. It also optimizes the database to improve the database's read and write performance and concurrent processing capabilities. The cache and database optimization module uses real-time monitoring and performance analysis tools to monitor and analyze the system operation status. By collecting key performance indicators, it promptly discovers and handles performance bottlenecks, thereby achieving dynamic tuning and continuous optimization of the system.

10. The efficient coding verification task processing method according to claim 9, Features: Before receiving the service request in S1, it includes: Adopting microservice architecture, the system is divided into multiple independent service modules. Each module is responsible for a specific business function. The lightweight communication protocol between services is used to achieve decoupling and collaborative work between modules. Among them, the lightweight communication protocol between services includes RESTful API.

Images