Asynchronous task processing method and device

By generating and configuring local configuration files, automating the creation and startup of asynchronous tasks, the problem of inefficient processing of Celery asynchronous tasks is solved, simplifying the configuration process, and improving development efficiency and system performance.

CN118885271APending Publication Date: 2024-11-01HUNAN HAPPLY SUNSHINE INTERACTIVE ENTERTAINMENT MEDIA CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410951926.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-16
Publication Date
2024-11-01

AI Technical Summary

Technical Problem

When using Celery to implement asynchronous tasks, users need to perform cumbersome configuration and startup processes, resulting in low usage efficiency. Especially for users who are not familiar with Celery, the learning curve is steep.

Method used

By generating local configuration files, including asynchronous managed functions to be executed, middleware information and ending functions, and configuring them into the asynchronous task queue package, the creation and startup process of asynchronous tasks are automated, users only need to be in the same Python file. Write business logic without additional manual configuration and startup.

Benefits of technology

Simplify the configuration process, reduce configuration errors and time costs, reduce learning thresholds, improve the efficiency and system performance of asynchronous task processing, and improve the response speed.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118885271A_ABST
    Figure CN118885271A_ABST
Patent Text Reader

Abstract

The invention discloses an asynchronous task processing method and device, when a user demand is received, a corresponding local configuration file is generated according to the user demand, the local configuration file at least comprises a to-be-executed asynchronous hosting function, middleware information and a binding function, the local configuration file is configured to an asynchronous task queue package, and the asynchronous hosting function is executed according to the local configuration file. And if a current execution command in the automatic asynchronous task is a consumer command for starting the asynchronous task queue, starting the asynchronous task queue through a starting code in the automatic asynchronous task to realize the process of converting the synchronous task into the asynchronous task.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computer software technology, and more specifically, to an asynchronous task processing method and device. Background Art

[0002] A cross-platform computer programming language (Python) asynchronous task queue (Celery) framework for processing asynchronous tasks.

[0003] When using Celery to implement asynchronous tasks, users need to manually perform tedious configuration and startup processes. For users who are not familiar with Celery, the learning curve is steep and the use is relatively complicated, resulting in low processing efficiency of asynchronous tasks using Celery.

[0004] Therefore, how to improve the processing efficiency of asynchronous tasks using Celery is an urgent problem to be solved in this application. Summary of the invention

[0005] In view of this, the present application discloses an asynchronous task processing method and device, aiming to improve the processing efficiency of asynchronous tasks using an asynchronous task queue.

[0006] In order to achieve the above purpose, the disclosed technical solution is as follows:

[0007] The first aspect of the present application discloses an asynchronous task processing method, the method comprising:

[0008] When receiving a user demand, generating a corresponding local configuration file according to the user demand; wherein the local configuration file at least includes an asynchronous managed function to be executed, middleware information and a conclusion function;

[0009] Configuring the local configuration file to the asynchronous task queue package to create an automated asynchronous task;

[0010] If the current execution command in the automated asynchronous task is a command to start an asynchronous task queue consumer, the asynchronous task queue is started through the startup code in the automated asynchronous task to implement the process of converting the synchronous task to the asynchronous task.

[0011] Preferably, configuring the local configuration file to an asynchronous task queue package to create an automated asynchronous task includes:

[0012] Generate the queue name, function path, function name and related configuration information corresponding to the asynchronous managed function to be executed;

[0013] Get the middleware information according to the middleware address;

[0014] Configure the queue name, the function path, the function name, the relevant configuration information, the middleware information and the ending function to the asynchronous task queue package;

[0015] Create automated asynchronous tasks through the asynchronous task queue package.

[0016] Preferably, configuring the queue name, the function path, the function name, the relevant configuration information, the middleware information and the ending function to the asynchronous task queue package includes:

[0017] Instantiate the asynchronous managed function to be executed, middleware information, conclusion function and default configuration information to the application of the asynchronous task queue, associate the application of the asynchronous task queue with the local configuration file, and complete the configuration process of the asynchronous task queue package.

[0018] Preferably, if the current execution command in the automated asynchronous task is a command to start an asynchronous task queue consumer, the asynchronous task queue is started by the startup code in the automated asynchronous task to implement the process of converting the synchronous task into the asynchronous task, including:

[0019] Parse the currently executed command in the automated asynchronous task and obtain the parsing result;

[0020] If the startup parameters in the parsing result contain parameters for running asynchronous task queues, and the startup parameters are commands for starting asynchronous task queues, the consumer is started;

[0021] In the process of starting the consumer, the local configuration files and environment variables in the automated asynchronous task are determined as the startup code;

[0022] The asynchronous task queue is started according to the startup code to realize the process of converting the synchronous task into the asynchronous task.

[0023] Preferably, it also includes:

[0024] If there is no start asynchronous task queue command in the start parameters in the parsing result, it is determined that the current execution command in the automated asynchronous task is a non-start asynchronous task queue consumer command, a new automated asynchronous task is created, and the instruction type of the current execution command of the new automated asynchronous task is determined.

[0025] The second aspect of the present application discloses an asynchronous task processing device, the device comprising:

[0026] A generating unit, configured to generate a corresponding local configuration file according to the user demand when receiving the user demand; wherein the local configuration file at least includes an asynchronous managed function to be executed, middleware information and a conclusion function;

[0027] A configuration unit, configured to configure the local configuration file to an asynchronous task queue package to create an automated asynchronous task;

[0028] The starting unit is used to start the asynchronous task queue through the starting code in the automated asynchronous task if the current execution command in the automated asynchronous task is a start asynchronous task queue consumer command, so as to realize the process of converting the synchronous task into the asynchronous task.

[0029] Preferably, configuring the local configuration file to an asynchronous task queue package to create an automated asynchronous task includes:

[0030] A generation module is used to generate the queue name, function path, function name and related configuration information corresponding to the asynchronous managed function to be executed;

[0031] The acquisition module is used to obtain the middleware information according to the middleware address;

[0032] A configuration module, used to configure the queue name, the function path, the function name, the relevant configuration information, the middleware information and the ending function to the asynchronous task queue package;

[0033] A creation module is used to create an automated asynchronous task through the asynchronous task queue package.

[0034] Preferably, the configuration module is specifically used to:

[0035] Instantiate the asynchronous managed function to be executed, middleware information, conclusion function and default configuration information to the application of the asynchronous task queue, associate the application of the asynchronous task queue with the local configuration file, and complete the configuration process of the asynchronous task queue package.

[0036] Preferably, if the currently executed command in the automated asynchronous task is a command to start an asynchronous task queue, the asynchronous task queue is started by a startup code in the automated asynchronous task to implement the process of converting the synchronous task into the asynchronous task, including:

[0037] The parsing module is used to parse the current execution command in the automated asynchronous task and obtain the parsing result;

[0038] A first startup module, used for starting the consumer if there is a running asynchronous task queue parameter in the startup parameter in the parsing result, and the startup parameter is a start asynchronous task queue command;

[0039] A determination module is used to determine the local configuration file and environment variables in the automated asynchronous task as the startup code during the process of starting the consumer;

[0040] The first startup module is used to start the asynchronous task queue according to the startup code to realize the process of converting the synchronous task into the asynchronous task.

[0041] Preferably, it also includes:

[0042] The determination unit is used to determine that the current execution command in the automated asynchronous task is a non-start asynchronous task queue consumer command if there is no start asynchronous task queue command in the start parameters in the parsing result, create a new automated asynchronous task, and determine the instruction type of the current execution command of the new automated asynchronous task.

[0043] It can be known from the above technical scheme that the present application discloses an asynchronous task processing method and device. When receiving user requirements, a corresponding local configuration file is generated according to the user requirements, wherein the local configuration file at least includes the asynchronous managed function to be executed, the middleware information and the conclusion function, and the local configuration file is configured to the asynchronous task queue package to create an automated asynchronous task. If the current execution command in the automated asynchronous task is a command to start the asynchronous task queue consumer, the asynchronous task queue is started by the startup code in the automated asynchronous task to realize the process of converting the synchronous task to the asynchronous task. Through the above scheme, a corresponding local configuration file is generated according to user requirements, and the local configuration file is configured to the asynchronous task queue package and the function is instantiated to complete the configuration and startup of the asynchronous task queue. The asynchronous task queue package encapsulates the conventional configuration process internally, and automatically generates the corresponding local configuration file and startup command according to the user's needs. Users only need to write business logic in the same Python file, without the need for additional tedious manual configuration and startup processes, to convert the original synchronous tasks into asynchronous tasks of the asynchronous task queue, reducing configuration errors and time costs. This solution uses asynchronous task processing to more efficiently utilize resources, improve system performance and response speed, and thus improve the processing efficiency of asynchronous tasks using asynchronous task queues. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0045] Figure 1 This is a schematic diagram of an asynchronous processing system based on the Celery framework disclosed in an embodiment of the present application;

[0046] Figure 2 A flowchart of an asynchronous task processing method disclosed in an embodiment of the present application;

[0047] Figure 3 A code diagram of the relevant functions in TaskOP disclosed in the embodiment of the present application;

[0048] Figure 4 A schematic diagram of the structure of an asynchronous task processing device disclosed in an embodiment of the present application;

[0049] Figure 5 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0051] In this application, the terms "comprises", "comprising" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element.

[0052] As can be seen from the background technology, when using Celery to implement asynchronous tasks, the use of Celery requires users to manually perform cumbersome configuration and startup processes. For users who are not familiar with Celery, the learning curve is steep and the use is relatively complicated, resulting in low processing efficiency of using Celery to implement asynchronous tasks. Therefore, how to improve the processing efficiency of using Celery to implement asynchronous tasks is a problem that needs to be solved urgently in this application.

[0053] In order to solve the above problems, the present application discloses an asynchronous task processing method and device, which generates a corresponding local configuration file according to user needs, configures the local configuration file to the asynchronous task queue package and instantiates the function to complete the configuration and startup of the asynchronous task queue. The asynchronous task queue package encapsulates the conventional configuration process, and automatically generates the corresponding local configuration file and startup command according to the user's needs. The user only needs to write the business logic in the same Python file, without the need to perform additional cumbersome manual configuration and startup processes, to convert the original synchronous task into an asynchronous task of the asynchronous task queue, thereby reducing configuration errors and time costs. This solution can use resources more efficiently through asynchronous task processing, improve system performance and response speed, and thus improve the processing efficiency of asynchronous tasks using asynchronous task queues. The specific implementation method is specifically described through the following embodiments.

[0054] refer to Figure 1 , which is a schematic diagram of an asynchronous processing system based on the celery framework disclosed in an embodiment of the present application. The asynchronous processing system based on the celery framework includes an initialization module (Init), a task operation module (TaskOP), a local configuration file operation module (ConfigOP), a task real-time generation module (TaskGen) and an exit execution module (ExitRun).

[0055] The functions of Init are as follows:

[0056] A. Use ConfigOP to read information from the local configuration file and the default configuration module, and use this information to instantiate an asynchronous task queue application (celery app).

[0057] Among them, the default configuration information of the default configuration module includes commonly used Celery configuration information, such as heartbeat, time zone, result saving format, etc.

[0058] B. Instantiate TaskOP and associate it with celery app and ConfigOP.

[0059] The functionality of TaskOP is as follows:

[0060] A. Create automated asynchronous tasks and generate corresponding local configuration files based on user needs; the local configuration files at least include the asynchronous managed function to be executed, middleware information and the conclusion function, and configure the local configuration files to the asynchronous task queue package to create automated asynchronous tasks. Specifically, input the specified function to be asynchronously managed (i.e. the asynchronous managed function to be executed) / class and related configuration information (such as timeout information, etc.), automatically generate the queue name based on the function name, solidify the function's path, name, queue, timeout and other parameters into the local configuration file, generate the route for the task in the celery app, and return the route information; the route information is returned when the task is created, and the route information is an address in the form of a string / text.

[0061] It should be noted that Celery is an asynchronous framework that can put the tasks that need to be executed in a queue for execution. The asynchronous managed functions / classes here are these tasks, which are generally time-consuming programs.

[0062] Celery has the following core features, making it a commonly used task queue framework in Python development:

[0063] Asynchronous execution: The execution of tasks is asynchronous, that is, after a task is released, other operations can be continued without waiting for the task to be completed;

[0064] Distributed processing: Celery supports distributing tasks to multiple worker processes or machines for parallel processing, improving the system's processing power and performance;

[0065] Reliability: Celery uses a message broker as a task queue to ensure reliable delivery and execution of tasks;

[0066] Scalability: By starting multiple worker processes, the task processing capabilities can be flexibly expanded according to demand.

[0067] B. Update middleware information: Enter the middleware address, and the function will solidify the middleware information into the local configuration file to complete the update of the middleware information;

[0068] C. Task sending function: Input the routing information and task parameters of the task, the function will send the task information to the task queue to wait for the consumer to execute, and return the task id;

[0069] D. Result query: Enter the task ID, and the function will return the task execution status and execution result. The task execution status includes waiting for PEDDING, executing STARTING, successful SUCCESS, failed FAILED, etc. The execution result varies depending on the task.

[0070] E. Configure the ending function: According to the function entered during program execution, determine whether the current task is to execute business logic or start consumers. If it is to execute business logic, proceed to the next step, create a new automated asynchronous task, and determine the instruction type of the current execution command of the new automated asynchronous task. Call ExitRun to determine whether it is necessary to start consumers. If it is necessary to start consumers, exit the current process. Send tasks through other business functions or call TaskOP's task sending function.

[0071] The codes of the relevant functions in TaskOP are as follows Figure 3 shown.

[0072] Instantiate the asynchronous managed function to be executed, middleware information, conclusion function and default configuration information to the application of the asynchronous task queue, associate the application of the asynchronous task queue with the local configuration file, and complete the configuration process of the asynchronous task queue package.

[0073] The functions of ConfigOP are as follows:

[0074] A local configuration file will be created in the local default path (such as the .celery folder), and the celery configuration will be saved and read according to the rules, where the rules are used to save and read information. This module also contains some commonly used configuration information, such as common celery configuration information, such as heartbeat, time zone, result saving format, etc.

[0075] The features of TaskGen are as follows:

[0076] According to the information in the local configuration file and the queue name, the package is automatically imported and the Celery consumer function is automatically created, without the need for additional code to create asynchronous functions.

[0077] The function of ExitRun is as follows:

[0078] The module is triggered when the program exits. ExitRun will determine whether the consumer is currently started. If the consumer needs to be started, a celery consumer startup command will be automatically generated and executed according to the configuration file and environment variables.

[0079] If the current execution command in the automated asynchronous task is a command to start an asynchronous task queue consumer, the asynchronous task queue (celery) is started through the startup code in the automated asynchronous task, that is, the consumer is started to realize the process of converting the synchronous task into an asynchronous task. Specifically, the current execution command in the automated asynchronous task is parsed to obtain the parsing result. If the startup parameter in the parsing result contains the run asynchronous task queue parameter (run_celery), and the startup parameter is a command to start an asynchronous task queue, the consumer is started.

[0080] In the process of starting the consumer, the local configuration file and environment variables in the automated asynchronous task are determined as the startup code, and the asynchronous task queue is started according to the startup code to realize the process of converting the synchronous task into the asynchronous task.

[0081] The above functional parts of this solution have been made into a package on a local git server. When you need to use it, you only need to install the package.

[0082] The specific usage code of this solution: (demo.py)

[0083] Line0: Import the functions that need to be managed;

[0084] Line1: Import TaskOP instantiated in Init of this package;

[0085] Line2: Call TaskOP's update middleware information function;

[0086] Line3: Call TaskOP to create an automated task and specify the function or class that needs to be hosted (if there are multiple tasks that need to be hosted, just perform line3 several times);

[0087] Line4: Call the configuration ending function of TaskOP to declare the task configuration is finished;

[0088] Line5: other business functions, or calling TaskOP's task sending function to send tasks;

[0089] In the above description, Line1-Line4 are the only operations that need to be performed and no other configuration is required.

[0090] The specific deployment method of this solution has two parts. One part is the business function responsible for sending tasks, and the other part is the consumer function responsible for consuming tasks. The program entry of these two parts is the above demo.py, which is distinguished by the execution parameters:

[0091] A. Business functions: python demo.py;

[0092] B. Consumer function: python demo.py --run_celery -c 1-l info-Pprefork.--run_celery indicates that the celery consumer needs to be executed. When demo.py executes to the configuration ending function, it will determine whether there is this parameter (python demo.py --run_celery -c 1-l info-Pprefork.--run_celery). If there is this parameter, the terminal executes demo.py, exits the program, and calls ExitRun to execute the consumer task.

[0093] The specific execution process of the asynchronous processing system based on the Celery framework is as follows:

[0094] Before starting the execution, you need to prepare:

[0095] a. Functions or classes that need to be executed asynchronously (want to use celery-hosted tasks);

[0096] b. The communication middleware required by Celery (such as redis, a tool used for communication between Celery);

[0097] Install this package locally;

[0098] Execution steps:

[0099] 1. Import the TaskOP instance in this package:

[0100] The following processes will be performed in this step:

[0101] A. Determine the type of command currently being executed based on the parameters (execute business function, configure Celery, start Celery);

[0102] B. Read the default general configuration and local configuration in the package and create a celery asynchronous instance;

[0103] 2. Use TaskOP to specify the middleware prepared in b above:

[0104] In this step, the middleware information will be stored in the local configuration file;

[0105] 3. Use TaskOP to specify the function or class prepared in a above:

[0106] In this step, the function information will be stored in a local file, and the routing information of the function in the Celery task queue will be returned;

[0107] 4. Use TaskOP to configure the ending:

[0108] In this step, the program will determine the type of command in A (determine the type of command currently executed based on the parameters) and then perform different operations based on the different types.

[0109] A. Execute business functions: When the first parameter is not celery and there is no "--run_celery" in the parameter, only the current code is executed, and task creation only obtains routing information;

[0110] For example, if you want to run a python script pythonmain.py, or if you want to run celery, celery-Acelery4ai2mg.tasks worker-Q aimg_demo-c 1-l info-P solo, the first parameter refers to the first parameter python or celery;

[0111] B. Configure celery: When the first parameter is not celery and the parameter contains "--run_celery", the current code will be stopped, the program will exit and the celery command will be executed (this command is automatically generated according to the current program, which is the case C below);

[0112] C. Start celery: When the first parameter is celery, start the consumer.

[0113] This solution aims to provide an improved celery package, which integrates the conventional configuration process into the package, so that users can use the celery framework to implement asynchronous tasks more easily. This improvement aims to reduce the learning cost of users when using Celery and improve the efficiency of algorithm teams in developing asynchronous tasks.

[0114] The innovation of this solution is to encapsulate the conventional configuration process into an independent package. Users only need to import the package and instantiate the function to automatically complete the configuration process of Celery and generate the corresponding Celery startup command. Users only need to write business logic in the same Python file, without having to learn the complicated configuration process, and can convert the original synchronous task into Celery's asynchronous task.

[0115] Through the improved celery package provided by this solution, users can significantly reduce the time and energy cost of configuring and starting celery. The algorithm team can focus more on the development of business logic, improve development efficiency and overall team productivity. In addition, since celery can be used to process tasks asynchronously, the system's performance and response speed can be further improved, thereby providing a better user experience.

[0116] This solution provides an improved celery package. Users only need to import the package and instantiate the function to complete the configuration and startup of celery. The package encapsulates the conventional configuration process and automatically generates the corresponding configuration files and startup commands according to user needs. Users only need to write business logic in the same Python file, without having to learn the complicated configuration process, and can convert the original synchronous tasks into celery asynchronous tasks.

[0117] In the embodiment of the present application, a corresponding local configuration file is generated according to user needs, and the local configuration file is configured to the asynchronous task queue package and the function is instantiated to complete the configuration and startup of the asynchronous task queue. The asynchronous task queue package encapsulates the conventional configuration process, and automatically generates the corresponding local configuration file and startup command according to the user's needs. The user only needs to write business logic in the same Python file, without additional manual cumbersome configuration and startup process, and the original synchronous task can be converted into an asynchronous task of the asynchronous task queue, reducing configuration errors and time costs. This solution can use resources more efficiently through asynchronous task processing, improve the performance and response speed of the system, thereby improving the processing efficiency of asynchronous tasks using asynchronous task queues.

[0118] refer to Figure 2 As shown, an asynchronous task processing method disclosed in an embodiment of the present application is applicable to the above-mentioned embodiment. Figure 1 The disclosed asynchronous processing system based on the Celery framework, the asynchronous task processing method mainly includes the following steps:

[0119] S201: When receiving a user demand, generating a corresponding local configuration file according to the user demand; wherein the local configuration file at least includes an asynchronous managed function to be executed, middleware information and a conclusion function.

[0120] User demand refers to the user's demand to use the Celery framework to implement asynchronous tasks.

[0121] S202: Configuring the local configuration file to the asynchronous task queue package to create an automated asynchronous task.

[0122] Specifically, the local configuration file is configured to the asynchronous task queue package to create an automated asynchronous task process, as shown in A1-A4.

[0123] A1: Generate the queue name, function path, function name and related configuration information corresponding to the asynchronous managed function to be executed.

[0124] Related configuration information includes but is not limited to timeout information.

[0125] A2: Get the middleware information based on the middleware address.

[0126] A3: Configure the queue name, function path, function name, related configuration information, middleware information and conclusion function to the asynchronous task queue package.

[0127] The asynchronous task queue package also includes default configuration information, which includes commonly used Celery configuration information, such as heartbeat, time zone, result saving format, etc.

[0128] In A3, the asynchronous managed function to be executed, the middleware information, the ending function and the default configuration information are instantiated to the application of the asynchronous task queue, so that the application of the asynchronous task queue is associated with the local configuration file to complete the configuration process of the asynchronous task queue package.

[0129] A4: Create automated asynchronous tasks through the asynchronous task queue package.

[0130] The specific execution process and execution principle of A1-A4 are the same as those in the above embodiment. Figure 1 The execution process and principle of TaskOP are the same, which can be referred to and will not be described here.

[0131] S203: If the current execution command in the automated asynchronous task is a command to start an asynchronous task queue consumer, Celery is started through the startup code in the automated asynchronous task to implement the process of converting the synchronous task into the asynchronous task.

[0132] Specifically, the asynchronous task queue is started by automating the startup code in the asynchronous task to realize the process of converting the synchronous task into the asynchronous task, as shown in B1-B4.

[0133] B1: Parse the current execution command in the automated asynchronous task and obtain the parsing result.

[0134] B2: If the startup parameters in the parsed result contain the run asynchronous task queue parameter (run_celery), and the startup parameter is the start asynchronous task queue command, the consumer is started.

[0135] B3: In the process of starting the consumer, the local configuration file and environment variables in the automated asynchronous task are determined as the startup code.

[0136] B4: Start the asynchronous task queue according to the startup code to realize the process of converting synchronous tasks into asynchronous tasks.

[0137] If there is no start asynchronous task queue command in the start parameters in the parsing result, determine that the current execution command in the automated asynchronous task is a non-start asynchronous task queue consumer command, create a new automated asynchronous task, and determine the instruction type of the current execution command of the new automated asynchronous task.

[0138] Determine whether the currently executed command in the automated asynchronous task is a command to start the asynchronous task queue consumer, that is, determine whether the automated asynchronous task starts the consumer. In order to facilitate understanding of the process of whether the automated asynchronous task starts the consumer, an example is given here to illustrate:

[0139] There are two criteria, both in the startup command (for example: pythonmain.py or python main.py --run_celery -c 1 -l info -P prefork, or celery-Atasks -c 1 -l info -Pprefork). One criterion is whether the first command started is "celery"; the other criterion is whether there is "--run_celery" in the parameter;

[0140] A. When the first parameter is not celery and there is no "--run_celery" in the parameter, the current code is executed and task creation only obtains routing information;

[0141] B. When the first parameter is not celery and the parameter contains "--run_celery", the current code will be stopped, the program will exit and the celery command will be executed (this command is automatically generated according to the current program, which is the case C below);

[0142] C. When the first parameter is celery, the consumer is started. That is, when the parameter is run_celery and the startup parameter is the celery command, the consumer will be started.

[0143] The technical solution of the present invention has the following beneficial effects:

[0144] Simplified configuration process: Users do not need to manually perform tedious Celery configuration, reducing configuration errors and time costs;

[0145] Lower learning threshold: Users only need to import the improved Celery package and instantiate functions, without having to deeply understand the underlying principles and complex concepts of Celery;

[0146] Improve development efficiency: The algorithm team can quickly convert the original synchronous tasks into Celery asynchronous tasks, improving task processing efficiency and overall productivity;

[0147] Improve system performance: Through asynchronous task processing, the system can use resources more efficiently and improve system performance and response speed.

[0148] In summary, the improved Celery package provided by the present invention can simplify the configuration process, lower the learning threshold, improve development efficiency and improve system performance, and has broad application prospects and economic benefits.

[0149] This solution only needs to add three lines of code (i.e. asynchronous managed function, middleware information and conclusion function) where the function is called to realize automatic configuration extraction and automatic task creation. When calling a task, it will not create the task, but only send the task to the route, reducing memory usage.

[0150] In the embodiment of the present application, a corresponding local configuration file is generated according to user needs, and the local configuration file is configured to the asynchronous task queue package and the function is instantiated to complete the configuration and startup of the asynchronous task queue. The asynchronous task queue package encapsulates the conventional configuration process, and automatically generates the corresponding local configuration file and startup command according to the user's needs. The user only needs to write business logic in the same Python file, without additional manual cumbersome configuration and startup process, and the original synchronous task can be converted into an asynchronous task of the asynchronous task queue, reducing configuration errors and time costs. This solution can use resources more efficiently through asynchronous task processing, improve the performance and response speed of the system, thereby improving the processing efficiency of asynchronous tasks using asynchronous task queues.

[0151] Based on the above embodiments Figure 2 The disclosed asynchronous task processing method, the embodiment of the present application also discloses an asynchronous task processing device, such as Figure 4 As shown, the asynchronous task processing device includes:

[0152] The generating unit 401 is used to generate a corresponding local configuration file according to the user demand when receiving the user demand; wherein the local configuration file at least includes the asynchronous managed function to be executed, the middleware information and the ending function;

[0153] The configuration unit 402 is used to configure the local configuration file to the asynchronous task queue package to create an automated asynchronous task;

[0154] The starting unit 403 is used to start the asynchronous task queue through the starting code in the automated asynchronous task if the current execution command in the automated asynchronous task is a start asynchronous task queue consumer command, so as to realize the process of converting the synchronous task into the asynchronous task.

[0155] Furthermore, the configuration unit 402 includes:

[0156] A generation module is used to generate the queue name, function path, function name and related configuration information corresponding to the asynchronous managed function to be executed;

[0157] The acquisition module is used to obtain the middleware information according to the middleware address;

[0158] Configuration module, used to configure the queue name, function path, function name, related configuration information, middleware information and conclusion function to the asynchronous task queue package;

[0159] Create a module for creating automated asynchronous tasks through the asynchronous task queue package.

[0160] Furthermore, the configuration module is specifically used to instantiate the asynchronous managed function to be executed, the middleware information, the conclusion function and the default configuration information to the application of the asynchronous task queue, so that the application of the asynchronous task queue is associated with the local configuration file to complete the configuration process of the asynchronous task queue package.

[0161] Furthermore, the startup unit 403 includes:

[0162] The parsing module is used to parse the current execution command in the automated asynchronous task and obtain the parsing result;

[0163] The first startup module is used to start the consumer if the startup parameters in the parsing result contain a running asynchronous task queue parameter and the startup parameter is a start asynchronous task queue command;

[0164] A determination module is used to determine the local configuration file and environment variables in the automated asynchronous task as the startup code during the process of starting the consumer;

[0165] The first startup module is used to start the asynchronous task queue according to the startup code to realize the process of converting the synchronous task into the asynchronous task.

[0166] Furthermore, the asynchronous task processing device also includes:

[0167] The determination unit is used to determine that the current execution command in the automated asynchronous task is a non-start asynchronous task queue consumer command if there is no start asynchronous task queue command in the start parameters in the parsing result, create a new automated asynchronous task, and determine the instruction type of the current execution command of the new automated asynchronous task.

[0168] In the embodiment of the present application, a corresponding local configuration file is generated according to user needs, and the local configuration file is configured to the asynchronous task queue package and the function is instantiated to complete the configuration and startup of the asynchronous task queue. The asynchronous task queue package encapsulates the conventional configuration process, and automatically generates the corresponding local configuration file and startup command according to the user's needs. The user only needs to write business logic in the same Python file, without additional manual cumbersome configuration and startup process, and the original synchronous task can be converted into an asynchronous task of the asynchronous task queue, reducing configuration errors and time costs. This solution can use resources more efficiently through asynchronous task processing, improve the performance and response speed of the system, thereby improving the processing efficiency of asynchronous tasks using asynchronous task queues.

[0169] An embodiment of the present application further provides a storage medium, which includes stored instructions, wherein when the instructions are executed, the device where the storage medium is located is controlled to execute the asynchronous task processing method as described above.

[0170] The present application also provides an electronic device, whose structural diagram is as follows: Figure 5 As shown, it specifically includes a memory 501 and one or more instructions 502, wherein the one or more instructions 502 are stored in the memory 501 and are configured to be executed by one or more processors 503 to execute the one or more instructions 502 to perform the above-mentioned asynchronous task processing method.

[0171] For the aforementioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0172] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0173] The steps in the methods of the embodiments of the present application can be adjusted in order, combined, and deleted according to actual needs.

[0174] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations.

[0175] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

[0176] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. An asynchronous task processing method, characterized in that: The method comprises: When receiving a user demand, generating a corresponding local configuration file according to the user demand; wherein the local configuration file at least includes an asynchronous managed function to be executed, middleware information and a conclusion function; Configuring the local configuration file to the asynchronous task queue package to create an automated asynchronous task; If the current execution command in the automated asynchronous task is a command to start an asynchronous task queue consumer, the asynchronous task queue is started through the startup code in the automated asynchronous task to implement the process of converting the synchronous task to the asynchronous task.

2. The method according to claim 1, characterized in that The configuring the local configuration file to the asynchronous task queue package to create an automated asynchronous task includes: Generate the queue name, function path, function name and related configuration information corresponding to the asynchronous managed function to be executed; Get the middleware information according to the middleware address; Configure the queue name, the function path, the function name, the relevant configuration information, the middleware information and the ending function to the asynchronous task queue package; Create automated asynchronous tasks through the asynchronous task queue package.

3. The method according to claim 2, characterized in that The step of configuring the queue name, the function path, the function name, the relevant configuration information, the middleware information and the ending function to the asynchronous task queue package includes: Instantiate the asynchronous managed function to be executed, middleware information, conclusion function and default configuration information to the application of the asynchronous task queue, associate the application of the asynchronous task queue with the local configuration file, and complete the configuration process of the asynchronous task queue package.

4. The method according to claim 1, characterized in that If the current execution command in the automated asynchronous task is a command to start an asynchronous task queue consumer, the asynchronous task queue is started by the start code in the automated asynchronous task to implement the process of converting the synchronous task into the asynchronous task, including: Parse the currently executed command in the automated asynchronous task and obtain the parsing result; If the startup parameters in the parsing result contain parameters for running asynchronous task queues, and the startup parameters are commands for starting asynchronous task queues, the consumer is started; In the process of starting the consumer, the local configuration files and environment variables in the automated asynchronous task are determined as the startup code; The asynchronous task queue is started according to the startup code to realize the process of converting the synchronous task into the asynchronous task.

5. The method according to claim 4, characterized in that Also includes: If there is no start asynchronous task queue command in the start parameters in the parsing result, it is determined that the current execution command in the automated asynchronous task is a non-start asynchronous task queue consumer command, a new automated asynchronous task is created, and the instruction type of the current execution command of the new automated asynchronous task is determined.

6. An asynchronous task processing device, characterized in that: The device comprises: A generating unit, configured to generate a corresponding local configuration file according to the user demand when receiving the user demand; wherein the local configuration file at least includes an asynchronous managed function to be executed, middleware information and a conclusion function; A configuration unit, configured to configure the local configuration file to an asynchronous task queue package to create an automated asynchronous task; The starting unit is used to start the asynchronous task queue through the starting code in the automated asynchronous task if the current execution command in the automated asynchronous task is a start asynchronous task queue consumer command, so as to realize the process of converting the synchronous task into the asynchronous task.

7. The device according to claim 6, characterized in that The configuring the local configuration file to the asynchronous task queue package to create an automated asynchronous task includes: A generation module is used to generate the queue name, function path, function name and related configuration information corresponding to the asynchronous managed function to be executed; The acquisition module is used to obtain the middleware information according to the middleware address; A configuration module, used to configure the queue name, the function path, the function name, the relevant configuration information, the middleware information and the ending function to the asynchronous task queue package; A creation module is used to create an automated asynchronous task through the asynchronous task queue package.

8. The device according to claim 7, characterized in that The configuration module is specifically used for: Instantiate the asynchronous managed function to be executed, middleware information, conclusion function and default configuration information to the application of the asynchronous task queue, associate the application of the asynchronous task queue with the local configuration file, and complete the configuration process of the asynchronous task queue package.

9. The device according to claim 6, characterized in that If the currently executed command in the automated asynchronous task is a command to start an asynchronous task queue, the asynchronous task queue is started by the start code in the automated asynchronous task to implement the process of converting the synchronous task into the asynchronous task, including: The parsing module is used to parse the current execution command in the automated asynchronous task and obtain the parsing result; A first startup module, used for starting the consumer if there is a running asynchronous task queue parameter in the startup parameter in the parsing result, and the startup parameter is a start asynchronous task queue command; A determination module is used to determine the local configuration file and environment variables in the automated asynchronous task as the startup code during the process of starting the consumer; The first startup module is used to start the asynchronous task queue according to the startup code to realize the process of converting the synchronous task into the asynchronous task.

10. The device according to claim 9, characterized in that Also includes: The determination unit is used to determine that the current execution command in the automated asynchronous task is a non-start asynchronous task queue consumer command if there is no start asynchronous task queue command in the startup parameters in the parsing result, create a new automated asynchronous task, and determine the instruction type of the current execution command of the new automated asynchronous task.