Timing task setting method and device, electronic equipment and storage medium

Abstract

The application provides a timing task setting method and device, electronic equipment and a storage medium. The method is used for setting an AsyncIOScheduler scheduler in a timing task framework apscheduler, and the method comprises the following steps: obtaining a port triggering function of a target port of a flask server, the port triggering function carrying a task parameter; and setting a timing task according to the task parameter carried by the port triggering function, so that the timing task can be set by the method.

Description

Technical Field

[0001] This application relates to the field of Internet technology, and more specifically, to a method, apparatus, electronic device, and storage medium for setting up timed tasks. Background Technology

[0002] With the continuous development of computer technology, various scheduled tasks are usually set up on computers to perform task processing. Therefore, how to set up scheduled tasks is crucial. Summary of the Invention

[0003] The purpose of this application is to provide a method, apparatus, electronic device, and storage medium for setting up timed tasks, in order to solve the problems in the prior art.

[0004] The first aspect of this application provides a method for setting up a scheduled task, the method being used in the AsyncIOScheduler scheduler set in the scheduled task framework apscheduler, including:

[0005] Obtain the port trigger function for the target port of the Flask server, the port trigger function carrying task parameters;

[0006] Set up a timed task based on the task parameters carried by the port trigger function.

[0007] In one embodiment, before obtaining the port trigger function for the target port of the Flask server, the method further includes:

[0008] The AsyncIOScheduler scheduler is started by triggering the port function of the target port.

[0009] In one embodiment, the method further includes:

[0010] The scheduled tasks are stored in a preset sqlite3 database for execution.

[0011] In one embodiment, the method further includes:

[0012] If the scheduled task meets the execution conditions stored in the preset sqlite3 database, the scheduled task will be executed.

[0013] In one embodiment, the method further includes:

[0014] The scheduled tasks stored in the preset sqlite3 database are monitored to see if they meet the execution conditions by iterating through the preset sqlite3 database using asyncio.get_event_loop().run_forever().

[0015] In one embodiment, the target port specifically includes a task addition port for adding new scheduled tasks, wherein the task parameters carried by the task addition port include task time parameters, process directory parameters, and task identifier parameters; and,

[0016] Based on the task parameters carried by the port trigger function, a timed task is set, specifically including:

[0017] A new scheduled task is generated based on the task time parameter, the process directory parameter, and the task identifier parameter.

[0018] In one embodiment, the target port specifically includes a task change port for deleting, pausing, or resuming existing scheduled tasks, wherein the task parameters carried by the task change port include a task identifier parameter; and,

[0019] Based on the task parameters carried by the port trigger function, a timed task is set, specifically including:

[0020] Based on the task identifier parameter, the existing scheduled tasks identified by the task identifier parameter are modified to delete, pause, or resume.

[0021] A second aspect of this application provides a timed task setting apparatus, the apparatus being used for the AsyncIOScheduler scheduler set in the timed task framework apscheduler, comprising:

[0022] The acquisition unit is used to acquire the port triggering function of the target port of the Flask server, wherein the port triggering function carries task parameters;

[0023] The setting unit is used to set a timed task based on the task parameters carried by the port trigger function.

[0024] A third aspect of this application provides an electronic device, including:

[0025] Memory, used to store computer programs;

[0026] A processor for executing the method described in any one of the embodiments of the present application.

[0027] A fourth aspect of this application provides a storage medium comprising: a program, which, when run on an electronic device, enables the electronic device to execute any method described in any of the method embodiments of this application.

[0028] The web content crawling method provided in this application includes a main crawler, a task distribution platform, and multiple sub-crawlers for handling different task types. The method involves the main crawler sending the target task type of the web content crawling task to the task distribution platform. The target task type is determined based on the task information carried in the task processing request of the web content crawling task. Then, the task distribution platform allocates the web content crawling task to the corresponding sub-crawler for processing according to the target task type. Because the web content crawling task is categorized by task type and processed using corresponding sub-crawlers during the crawling process, the possibility of blocking can be reduced during large-scale web content crawling. Attached Figure Description

[0029] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 A schematic diagram of the specific structure of an electronic device provided in an embodiment of this application;

[0031] Figure 2 A schematic diagram illustrating the interaction between an electronic device and a terminal, provided as an embodiment of this application;

[0032] Figure 3 A schematic flowchart illustrating the timed task setting method provided in one embodiment of this application;

[0033] Figure 4 This is a schematic diagram of the specific structure of a timed task setting device provided in an embodiment of this application. Detailed Implementation

[0034] The technical solutions of the embodiments of this application will now be described with reference to the accompanying drawings. In the description of this application, terms such as "first," "second," and "third" are used only for distinguishing descriptions and should not be construed as indicating or implying relative importance or order.

[0035] As mentioned earlier, tasks are usually handled by setting up various scheduled tasks on a computer. Therefore, it is crucial to know how to set up scheduled tasks, including how to add, delete, pause, and resume them.

[0036] Based on this, embodiments of this application provide a method, apparatus, electronic device, and storage medium for setting timed tasks, which can be used to set timed tasks. For example... Figure 1The image shown is an electronic device 1 provided in this embodiment. The electronic device 1 includes at least one processor 11 and a memory 12. Figure 1 Taking a processor as an example, the processor 11 and the memory 12 can be connected via a bus 10. The memory 12 stores instructions that can be executed by the processor 11. The instructions are executed by the processor 11 to enable the electronic device 1 to perform all or part of the process of the method in the following embodiments.

[0037] In practical applications, the electronic device 1 can be a mobile phone, laptop, desktop computer, or a large server or server cluster composed of them. For example, in the application scenario of this application embodiment, when setting up scheduled tasks, the scheduled task framework apscheduler can be deployed on the electronic device, thereby using the electronic device 1 as an apscheduler server or apscheduler server cluster to execute the methods provided in this application embodiment. The scheduled task framework apscheduler, short for Advanced Python Scheduler, is a lightweight Python scheduled task framework. The apscheduler framework also includes an AsyncIOScheduler scheduler, which can be used to execute the methods provided in this application embodiment.

[0038] like Figure 2 As shown, electronic device 1 can also connect to one or more terminals 2, and electronic device 1 can communicate with each terminal 2 separately. Terminal 2 can be a server deployed with Flask (referred to as a Flask server), where Flask is a lightweight web application framework written in Python. Therefore, in one embodiment, when a user needs to add a new scheduled task, or needs to delete, pause, and resume an existing scheduled task, they can perform operations such as clicking, dragging, touching, and writing on the graphical user interface of terminal 2, thereby enabling electronic device 1 to obtain the task parameters carried by the port trigger function in the Flask server of terminal 2, and then execute the method provided in this embodiment.

[0039] This application provides a method for setting up scheduled tasks, which can be used to set up scheduled tasks. The method can be... Figure 1 The method is executed using the electronic device 1 shown. Here, we will use this electronic device as an apscheduler server as an example to illustrate the method. Figure 3 The diagram shows the specific flow of this method, which includes the following steps:

[0040] Step S31: Obtain the port trigger function for the target port of the Flask server, which carries the task parameters.

[0041] The apscheduler server deploys a scheduled task framework apscheduler, which in turn has an AsyncIOScheduler scheduler. As mentioned above, the method in this embodiment can be executed through the AsyncIOScheduler scheduler.

[0042] It's important to note that in a Flask server, the ports used for setting up scheduled tasks typically include ports for adding new scheduled tasks and ports for modifying existing scheduled tasks. The port for adding new scheduled tasks can be called the task addition port, and the port for modifying existing scheduled tasks can be called the task modification port. Modifying existing scheduled tasks includes deleting, pausing, or resuming them. Therefore, in this Flask server, the task addition port is used to add new scheduled tasks, and the task modification port is used to delete, pause, or resume existing scheduled tasks.

[0043] In step S31, the target port can be either the task addition port or the task change port. Both the task addition port and the task change port include a port trigger function, and both of these port trigger functions carry task parameters. However, the task parameters carried by the port trigger functions of the two different ports are not the same.

[0044] The task parameters carried in the port trigger function for adding a port include task time parameters, process directory parameters, and task identifier parameters. The task time parameter is used to indicate the execution time and end time of the newly generated scheduled task. The process directory parameter is used to indicate the execution flow of the newly generated scheduled task, including the steps to be executed by the new scheduled task and the order of these steps. The task identifier parameter is used to indicate the task identifier of the newly generated scheduled task, which can be the task name, number, etc.

[0045] The task parameters carried in the port trigger function of the task change port include the task identifier parameter, which is used to indicate the task changes such as deletion, pause or resumption of the existing scheduled task identified by the task identifier parameter.

[0046] In practical applications, when users need to set up scheduled tasks, including adding new scheduled tasks or deleting, pausing, and resuming existing scheduled tasks, they can first select a target port in the Flask server. For example, if a user needs to create a new scheduled task, they can select the task addition port as the target port; or if a user needs to delete, pause, or resume an existing scheduled task, they can select the task modification port as the target port. After selecting the target port, users can perform operations such as clicking, dragging, touching, and writing on the Flask server's graphical user interface to set the port trigger function in the target port to carry task parameters. In step S31, the AsyncIOScheduler can obtain the port trigger function of the target port. For example, the AsyncIOScheduler can periodically scan the target port to obtain its port trigger function. The scanning period can be 1 minute, 5 minutes, or other times. Alternatively, the target port can be monitored. When a new port trigger function is detected, step S31 is executed to obtain the new port trigger function generated by the target port.

[0047] Step S32: Set up a timed task based on the task parameters carried by the port trigger function of the target port.

[0048] As mentioned above, the target port can be either a task addition port or a task change port. The task parameters carried by the port triggering functions for task addition ports and task change ports are different, which can be explained in detail here.

[0049] When the target port is specifically a task addition port, the task parameters carried by the port trigger function of the target port include task time parameters, process directory parameters, and task identifier parameters. Accordingly, step S32 can specifically include generating a new scheduled task based on the task time parameters, process directory parameters, and task identifier parameters carried by the port trigger function of the target port. For example, according to the task time parameter, the task execution time and end time of the new scheduled task are set; according to the process directory parameter, the execution flow of the new scheduled task is set, including the steps to be executed by the new scheduled task and the order of the steps; and according to the task identifier parameter, the task identifier of the new scheduled task is set, thereby finally generating the new scheduled task.

[0050] When the target port is specifically a task change port, the task parameters carried by the port trigger function of the target port specifically include task identifier parameters; accordingly, step S32 may specifically include, according to the task identifier parameters carried by the port trigger function of the target port, deleting, pausing or resuming the existing timed task identified by the task identifier parameters.

[0051] It should be further clarified that, in actual application, this task change port can further include a task deletion port, a task pause port, and a task resumption port. Specifically, the task deletion port carries a task identifier parameter, which indicates whether to delete the existing scheduled task identified by the task identifier parameter; the task pause port also carries a task identifier parameter, which indicates whether to pause the existing scheduled task identified by the task identifier parameter; and the task resumption port also carries a task identifier parameter, which indicates whether to resume the existing scheduled task identified by the task identifier parameter.

[0052] Therefore, in practical applications, the target port can be a port for adding tasks, or it can be a task deletion port, task pause port, or task resumption port in the task change port. In this way, through the task parameters carried by the port trigger function of the target port, it is possible to generate new scheduled tasks, as well as to delete, pause, and resume existing scheduled tasks, thereby setting up scheduled tasks.

[0053] The scheduled task setting method provided in this application includes first obtaining the port trigger function of the target port of the Flask server, which carries task parameters, and then setting the scheduled task according to the task parameters carried by the port trigger function of the target port. Therefore, it is possible to set the scheduled task, thereby solving the problems in the prior art.

[0054] It should be further explained that while new scheduled tasks can typically be generated through other methods, modifying existing scheduled tasks, including deleting, pausing, and resuming them, is usually difficult. The method provided in this application combines the Flask server and the AsyncIOScheduler scheduler in the Apscheduler scheduled task framework. By passing parameters, the AsyncIOScheduler scheduler obtains the port trigger function of the target port on the Flask server. Then, based on the task parameters carried by the port trigger function, the scheduled task is set. At this point, according to actual needs, such as generating new scheduled tasks or deleting, pausing, and resuming existing scheduled tasks, the appropriate port is selected as the target port, and the corresponding task parameters are set. This enables the deletion, pausing, and resuming of existing scheduled tasks, as well as the generation of new scheduled tasks. This method of setting scheduled tasks through parameter passing between the Flask server and the AsyncIOScheduler scheduler is generally highly efficient and user-friendly.

[0055] In practical applications, the scheduled task framework apscheduler typically has multiple different schedulers, which may not be running to save resources. Therefore, before step S31, the method of this embodiment may further include: starting the AsyncIOScheduler scheduler through a port trigger function of the target port. For example, the port trigger function can start the AsyncIOScheduler scheduler by calling `from apscheduler.schedulers.asyncio import AsyncIOScheduler`, and then, after the AsyncIOScheduler scheduler starts, step S31 can be executed.

[0056] In addition, after setting the scheduled task through the above step S32, the method may further include: storing the set scheduled task in a preset sqlite3 database for execution of the scheduled task.

[0057] For example, a newly generated scheduled task can be stored in a preset sqlite3 database; a deleted existing scheduled task can be deleted from the preset sqlite3 database; a paused existing scheduled task can be set to a paused state in the preset sqlite3 database; and a resumed existing scheduled task can be set to a resumed state in the preset sqlite3 database.

[0058] In addition, regarding the execution method of scheduled tasks, the scheduled tasks stored in the preset sqlite3 database can usually be monitored. For example, it can be monitored whether the scheduled tasks stored in the preset sqlite3 database meet the execution conditions, and if the scheduled tasks stored in the preset sqlite3 database meet the execution conditions, the scheduled task can be executed.

[0059] One specific way to monitor the scheduled tasks stored in the preset sqlite3 database is to use asyncio.get_event_loop().run_forever() to iterate through the preset sqlite3 database to monitor whether the scheduled tasks stored in the preset sqlite3 database meet the execution conditions.

[0060] Based on the same inventive concept as the scheduled task setting method provided in the embodiments of this application, the embodiments of this application also provide a scheduled task setting device. For any unclear aspects of this device embodiment, please refer to the corresponding content of the method embodiment. This device is used in the AsyncIOScheduler scheduler set in the scheduled task framework apscheduler. Figure 4 As shown, the device 40 includes an acquisition unit 401 and a setting unit 402, wherein:

[0061] Acquisition unit 401 is used to acquire the port trigger function of the target port of the Flask server, wherein the port trigger function carries task parameters;

[0062] Setting unit 402 is used to set a timed task according to the task parameters carried by the port trigger function.

[0063] The apparatus 40 provided in the embodiments of this application adopts the same inventive concept as the method provided in the embodiments of this application. Under the premise that the method can solve the technical problem, the apparatus 40 can also solve the technical problem. This will not be elaborated here.

[0064] In addition, in practical applications, the technical effects achieved by combining the device 40 with specific hardware and software, cloud technology, etc., are also within the scope of protection of this application. For example, using a distributed cluster approach to deploy different units in the device 40 in different nodes of the distributed cluster can improve efficiency.

[0065] Prior to the acquisition unit 401 of the device 40, the device 40 may also include a startup unit for starting the AsyncIOScheduler scheduler through the port trigger function of the target port.

[0066] Following the setting unit 402 of the device 40, the device 40 may further include a storage unit for storing the set timed tasks to a preset sqlite3 database for use in the execution of the timed tasks.

[0067] The device 40 may further include an execution unit for executing the scheduled task when the scheduled task stored in the preset sqlite3 database meets the execution conditions.

[0068] The device 40 may also include a monitoring unit for monitoring whether the scheduled tasks stored in the preset sqlite3 database meet the execution conditions by iterating through the preset sqlite3 database using asyncio.get_event_loop().run_forever().

[0069] The target port specifically includes a task addition port for adding new scheduled tasks, wherein the task parameters carried by the task addition port include task time parameters, process directory parameters, and task identifier parameters; and the setting unit 402 may specifically include a first setting subunit for generating new scheduled tasks based on the task time parameters, the process directory parameters, and the task identifier parameters.

[0070] The target port specifically includes a task change port for deleting, pausing, or resuming existing scheduled tasks, wherein the task change port carries task parameters including task identifier parameters; and the setting unit 402 may specifically include a second setting subunit for deleting, pausing, or resuming existing scheduled tasks identified by the task identifier parameters according to the task identifier parameters.

[0071] This invention also provides a storage medium, comprising: a program, which, when run on an electronic device, enables the electronic device to execute all or part of the processes of the methods described in the above embodiments. The storage medium may be a disk, optical disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk drive (HDD), or solid-state drive (SSD), etc. The storage medium may also include combinations of the above types of memory.

[0072] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for setting up a scheduled task, characterized in that, The method is used in the AsyncIOScheduler scheduler set in the timed task framework apscheduler, and includes: A port trigger function is used to obtain the target port of the Flask server. The port trigger function carries task parameters. The task parameters are obtained by the user on the graphical user interface of the Flask server after selecting the target port. The target port is the port selected by the user on the Flask server when a scheduled task needs to be set. Set a timed task based on the task parameters carried by the port trigger function; Before the port trigger function that retrieves the target port of the Flask server, the method further includes: The AsyncIOScheduler scheduler is started by calling the function from apscheduler.schedulers.asyncio to import AsyncIOScheduler via the port. The method further includes: The system monitors whether the scheduled tasks stored in the preset sqlite3 database meet the execution conditions by iterating through the preset sqlite3 database using asyncio.get_event_loop().run_forever(). The preset sqlite3 database is used to store the scheduled tasks set by the user. If the scheduled task meets the execution conditions stored in the preset sqlite3 database, the scheduled task will be executed.

2. The method according to claim 1, characterized in that, The target port specifically includes a task addition port for adding new scheduled tasks, wherein the task parameters carried by the task addition port include task time parameters, process directory parameters, and task identifier parameters; and, Based on the task parameters carried by the port trigger function, a timed task is set, specifically including: A new scheduled task is generated based on the task time parameter, the process directory parameter, and the task identifier parameter.

3. The method according to claim 1, characterized in that, The target port specifically includes a task modification port for deleting, pausing, or resuming existing scheduled tasks, wherein the task parameters carried by the task modification port include a task identifier parameter; and... Based on the task parameters carried by the port trigger function, a timed task is set, specifically including: Based on the task identifier parameter, the existing scheduled tasks identified by the task identifier parameter are modified to delete, pause, or resume.

4. A timed task setting device, characterized in that, The device is used in the AsyncIOScheduler scheduler set in the timed task framework apscheduler, and includes: The acquisition unit is used to acquire the port trigger function of the target port of the Flask server. The port trigger function carries task parameters. The task parameters are obtained based on the operation performed by the user on the graphical user interface of the Flask server after selecting the target port in the Flask server. The target port is the port selected by the user on the Flask server when a scheduled task needs to be set. The setting unit is used to set a timed task according to the task parameters carried by the port trigger function; The startup unit is used to start the AsyncIOScheduler scheduler by calling the port trigger function through the port trigger function to obtain the target port of the Flask server; The monitoring unit is used to monitor whether the scheduled tasks stored in the preset sqlite3 database meet the execution conditions by iterating through the preset sqlite3 database in a loop using asyncio.get_event_loop().run_forever(). The preset sqlite3 database is used to store the scheduled tasks set by the user. An execution unit is used to execute the scheduled task when the scheduled task stored in the preset sqlite3 database meets the execution conditions.

5. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for performing the method as described in any one of claims 1 to 3.

6. A storage medium, characterized in that, Includes: a program, when run on an electronic device, that enables the electronic device to perform the method as described in any one of claims 1 to 3.

Images