Task allocation processing method and system and storage medium

Abstract

The invention provides a task allocation processing method and system and a storage medium, which are applied to a task unified query service in a task allocation processing system, and the task processing system comprises a plurality of task execution programs and a preset task unified query service. The task unified query service is provided to uniformly distribute and manage a plurality of task execution programs, and part of the task execution programs are pre-started after the unified query service is started, so that when the task execution programs are distributed, the task execution programs do not need to be restarted, the task execution program starting time is saved, and the task execution program starting efficiency is improved. Therefore, the task allocation processing efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of big data distributed databases, and specifically to a task allocation processing method, system and storage medium. Background Art

[0002] As the scale of different business application systems within enterprises continues to expand, the amount of data is showing an explosive growth trend. The existing task processing methods occupy a large amount of resources and have low processing efficiency. Summary of the invention

[0003] The present application provides a task allocation processing method, system and storage medium, aiming to solve the problem of low efficiency of task processing methods in the prior art.

[0004] on the one hand,

[0005] On the other hand, the present application provides a task allocation processing device, the device comprising:

[0006] On the other hand, the present application also provides a server, the server comprising:

[0007] one or more processors;

[0008] Memory; and

[0009] One or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the processor to implement the task allocation processing method.

[0010] On the other hand, the present application also provides a computer-readable storage medium on which a computer program is stored, and the computer program is loaded by a processor to execute the steps in the task allocation processing method.

[0011] The present application provides a task allocation processing method, system and storage medium, which are applied to a unified task query service in a task allocation processing system. The task processing system includes multiple task execution programs and a preset unified task query service. The method includes: determining whether the unified task query service is started; if the unified task query service is started, starting a preset first number of task execution programs to obtain a pre-started task execution program queue; receiving a task request for a task to be executed, and determining whether there is an idle target task execution program in the task execution program queue; if there is a target task execution program, directly assigning the task to be executed to the target task execution program to execute the task to be executed. The present application provides a unified task query service to uniformly allocate and manage multiple task execution programs, and pre-starts some task execution programs after the unified query service is started, so that when assigning task execution programs, there is no need to restart the task execution programs, saving the time of starting the task execution programs, thereby improving the efficiency of task allocation processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0013] Figure 1 A schematic diagram of a task allocation and processing system according to an embodiment of the present application;

[0014] Figure 2 A flowchart of an embodiment of a task allocation processing method in an embodiment of the present application;

[0015] Figure 3 A schematic diagram of an embodiment of a process flow for determining a target task execution procedure provided in an embodiment of the present application;

[0016] Figure 4 A schematic diagram of a flow chart of an embodiment of allocating tasks to be performed provided in an embodiment of the present application;

[0017] Figure 5 A flowchart of an embodiment of adjusting the number of task execution programs provided in an embodiment of the present application;

[0018] Figure 6 A flowchart of an embodiment of a task allocation processing method provided in an embodiment of the present application;

[0019] Figure 7 A timing diagram of an embodiment of a task allocation processing method provided in an embodiment of the present application;

[0020] Figure 8A schematic diagram of the structure of a server involved in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0022] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, which are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first" and "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0023] In this application, the word "exemplary" is used to mean "serving as an example, illustration, or illustration." Any embodiment described in this application as "exemplary" is not necessarily to be construed as being preferred or advantageous over other embodiments. The following description is given to enable any person skilled in the art to implement and use the invention. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the invention can be implemented without using these specific details. In other instances, well-known structures and processes are not elaborated in detail to avoid obscuring the description of the invention with unnecessary details. Therefore, the present invention is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in this application.

[0024] The embodiments of the present application provide a task allocation processing method, system and storage medium, which are described in detail below.

[0025] like Figure 1As shown, it is a scene diagram of the task allocation processing system provided by the embodiment of the present application. The task allocation processing system may include multiple hosts 100 and servers 200. The hosts 100 and the servers 200 are connected to a network. The server 200 is integrated with a task allocation processing device, such as Figure 1 In the server, host 100 can access server 200.

[0026] In the embodiment of the present invention, the server 200 is mainly used to determine whether the unified task query service is started; if the unified task query service is started, a preset first number of task execution programs are started to obtain a pre-started task execution program queue; a task request for a task to be executed is received, and it is determined whether there is an idle target task execution program in the task execution program queue; if there is a target task execution program, the task to be executed is directly assigned to the target task execution program to execute the task to be executed.

[0027] In the embodiment of the present invention, the server 200 may be an independent server, or a server network or server cluster composed of servers. For example, the server 200 described in the embodiment of the present invention includes but is not limited to a computer, a network host, a single network server, a plurality of network server sets or a cloud server composed of a plurality of servers. Among them, the cloud server is composed of a large number of computers or network servers based on cloud computing. In the embodiment of the present invention, communication between the server and the host can be achieved through any communication method, including but not limited to mobile communications based on the 3rd Generation Partnership Project (3GPP), Long Term Evolution (LTE), Worldwide Interoperability for Microwave Access (WiMAX), or computer network communications based on TCP / IP Protocol Suite (TCP / IP), User Datagram Protocol (UDP), etc.

[0028] It is understandable that the host 100 used in the embodiment of the present invention may be a device including both receiving and transmitting hardware, that is, a device having receiving and transmitting hardware capable of performing two-way communication on a two-way communication link. Such a host may include: a cellular or other communication device having a single-line display or a multi-line display or a cellular or other communication device without a multi-line display. The specific host 100 may be a desktop terminal or a mobile terminal, and the host 100 may also be a mobile phone, a tablet computer, a laptop computer, etc.

[0029] Those skilled in the art will understand that Figure 1 The application environment shown in the figure is only one application scenario of the present application solution and does not constitute a limitation on the application scenario of the present application solution. Other application environments may also include Figure 1 More or fewer servers, or server network connections, as shown in Figure 1 Only one server and two hosts are shown. It can be understood that the task allocation processing system can also include one or more other servers, and / or one or more hosts connected to the server network, which is not specifically limited here.

[0030] In addition, if Figure 1 As shown, the task allocation processing system may further include a storage unit for storing data, such as data related to the tasks to be processed.

[0031] It should be noted that Figure 1 The scenario diagram of the task allocation processing system shown is only an example. The task allocation processing system and scenario described in the embodiment of the present invention are intended to more clearly illustrate the technical solution of the embodiment of the present invention, and do not constitute a limitation on the technical solution provided by the embodiment of the present invention. Ordinary technicians in this field can know that with the evolution of the task allocation processing system and the emergence of new business scenarios, the technical solution provided by the embodiment of the present invention is also applicable to similar technical problems.

[0032] like Figure 2 FIG. 1 is a flow chart of an embodiment of a task allocation processing method in an embodiment of the present application, and the task allocation processing method includes:

[0033] 21. Determine whether the unified query service for tasks is started.

[0034] 22. If the unified task query service is started, a preset first number of task execution programs are started to obtain a pre-started task execution program queue.

[0035] The task allocation processing method provided in the embodiment of the present application is applied to the task unified query service in the task allocation processing system, and the task processing system includes not only the preset task unified query service, but also multiple task execution programs. The present application manages all tasks through the task unified query service, including allocating task execution programs and monitoring task processing processes. Therefore, it is first necessary to determine whether the task unified query service is started. Only when the task unified query service is started can the task execution program be managed.

[0036] If the unified task query service is started, the unified task query service can be controlled to start the first number of task execution programs to obtain a pre-started task execution program queue. That is, if the unified task query service is started, a certain number of task execution programs are pre-started according to the number of preset parameters, and these pre-started task execution programs do not execute specific task requests at this time; when there is a task to be executed, the unified task query service will allocate a task execution program from the pre-started multiple task execution programs to execute the task. This can avoid restarting the task execution program after receiving the task to be executed, which wastes time.

[0037] 23. Receive a task request for a task to be executed, and determine whether there is an idle target task executor in the task executor queue.

[0038] 24. If there is a target task execution program, the task to be executed is directly assigned to the target task execution program to execute the task to be executed.

[0039] If there are tasks to be executed, a task request corresponding to the tasks to be executed will be generated and sent to the unified task query service, which will determine whether there is an idle target task executor in the task executor queue, that is, a target task executor that is not currently occupied and can process the task. If there is a target task executor set, the unified task query service will directly assign the tasks to be executed to the target task executor so that the target task executor can execute the tasks to be executed.

[0040] The task allocation method provided in the embodiment of the present application is applied to the unified task query entry in the task allocation processing system. The present application provides a unified task query service to uniformly allocate and manage multiple task execution programs, and pre-start some task execution programs after the unified query service is started, so that when the task execution program is allocated, there is no need to restart the task execution program, which saves the time of starting the task execution program, thereby improving the efficiency of task allocation processing.

[0041] like Figure 3 As shown, a flowchart of an embodiment of a procedure for determining a target task execution provided by an embodiment of the present application may include:

[0042] 31. Determine the number of tasks to be performed.

[0043] 32. Determine the number of task executors in the task executor queue.

[0044] 33. Determine whether the number of tasks is not greater than the number of programs.

[0045] 34. If the number of tasks is not greater than the number of programs, determine that there is an idle target task execution program in the task execution program queue.

[0046] In the embodiment of the present application, a task execution program can usually only process one task to be executed, and the unified task query service will assign a task execution program to each task to be executed. And the multiple task execution programs in the present application are usually configured to process a variety of different tasks, that is, the unified task query service is used in the present application to randomly assign task execution programs, and a certain task to be executed does not have a unique task execution program corresponding to it.

[0047] At the same time, multiple pre-started task executors will be stored in the task executor queue. When there are tasks to be executed, the unified task query service will allocate them from the multiple pre-started task executors in the task executor queue to save the startup time of the task executor. The already allocated task executors will be removed from the task executor queue, so the number of task executors in the task executor queue is constantly changing. Therefore, when judging whether there are idle target task executors in the task executor queue, it is only necessary to determine the number of tasks to be executed and the number of programs of the task executors in the current task executor queue, and further judge whether the number of tasks is not greater than the number of programs. If the number of tasks is not greater than the number of programs, it can be determined that there are idle target task executors in the task executor queue; that is, the unified task query service can select unallocated task executors from the task executors and assign them to the current task to be executed.

[0048] It should be noted that the target task execution program is any task execution program randomly selected from the current task execution program queue. After determining that the target task execution program exists, the task to be executed needs to be assigned to the target task execution program. Figure 4 , the process of allocating tasks to be performed in this application may include:

[0049] 41. Parse the task request parameters to determine the connection string identifier corresponding to the task to be executed.

[0050] 42. Send the connection string identifier to the target task execution program.

[0051] In an embodiment of the present application, each task to be executed corresponds to a task request parameter, which may include what type of task the task to be processed is, what data needs to be called, etc. When the task unified query service receives the task to be processed, it is necessary to parse the task request parameters corresponding to the task to be processed to determine the connection string identifier corresponding to the task to be executed. Among them, each task to be processed corresponds to a unique connection string identifier, that is, each task request corresponds to a unique connection string identifier; according to the connection string identifier, it can be determined which specific task to be processed. After determining the connection string identifier, the task unified query service will send the connection string identifier to the target task execution program to establish a connection with the target task execution program and control the target task execution program to execute the task to be executed. At the same time, the target task execution program will reject the connection string identifiers of other tasks to be executed.

[0052] 43. Determine whether the heartbeat information corresponding to the target task execution program is received within the first preset time period.

[0053] 44. If the heartbeat information corresponding to the target task execution program is received, the task to be executed is sent to the target task execution program so that the target task execution program is used to execute the task to be executed.

[0054] After sending the connection string identifier to the target task execution program, it is also necessary to determine whether the heartbeat information corresponding to the target task execution program is received to determine whether the target task execution program is in a normal state where it can process tasks. Among them, the heartbeat information is generated by the task execution program and is mainly used to determine whether the task execution is in a normal state where it can process tasks; and the heartbeat information will be generated regardless of whether the task execution program is processing tasks, not only during the process of processing tasks.

[0055] In the above embodiment, it is necessary not only to determine whether the heartbeat information of the target task execution program can be received, but also to determine whether the heartbeat information is received within the first preset time period. If the heartbeat information is not received, or if the heartbeat information is not received within the first preset time period, the target task execution program is determined to be abnormal, and a new target task execution program needs to be determined. The present application can monitor the running status of the task execution program in real time through the heartbeat information, and clean up the abnormal task execution program after the task execution program is abnormal or the connection with the task execution program is lost.

[0056] After the target task execution program is determined to be in a normal state according to the heartbeat information, the task to be executed can be sent to the target task execution program so that the target task execution program can be used to execute the task. When actually sending the task to be executed, it is actually necessary to determine the task execution program configuration parameters that match the task to be executed in the task unified query service, and send the task to be executed and the task execution program parameters to the target task execution program together.

[0057] Among them, the task executor configuration parameters are used to configure the task executor so that the configured task executor can execute various different tasks. The task executor configuration parameters corresponding to different tasks are usually different; the task executor configuration parameters corresponding to different tasks are usually stored in the task unified query service, and when it is necessary to send the task to be executed to the task executor, the task unified query service determines and sends it together with the task to be executed. Therefore, in this application, the task executor in the initial state cannot execute various tasks, but different task executor configuration parameters need to be sent to the task executor so as to configure the task executor using the task executor configuration parameters. After the task execution is completed, the task executor will usually be restored to the initial state, that is, the original configuration parameters are eliminated, so that new parameters can be configured later to handle different tasks.

[0058] After assigning the task to be executed to the target task executor, the unified task query service will also move the target task executor out of the task executor queue. Therefore, in the task executor queue in the present application, there will be no task executors that are processing tasks, only task executors that are to be assigned and not configured. After moving the target task executor out of the task executor queue, the target task executor will also be moved into the preset resource job queue; wherein, the resource job queue includes the resources required to execute the task, which can be called directly. Therefore, the target task executor can be controlled to call the resources in the resource job to execute the task to be executed; and after the task execution is completed, the target task executor will return the task execution result to the unified service query queue.

[0059] It should be noted that during the process of the target task executor executing the task, the unified task query service will continuously monitor the working status of the target task executor; therefore, after the target task executor is removed from the task executor queue, the unified task query service will send the task countdown to the target task executor. At the same time, in this application, regardless of whether the task executor is processing the task, the task executor will regularly generate heartbeat information and periodically send it to the unified task query service, so that the unified task query service can determine whether the task executor is in a normal state. Therefore, after the unified task query service sends the task countdown to the target task executor, the unified task query service can periodically reset the task countdown by periodically receiving the heartbeat information of the target task executor.

[0060] In a specific embodiment, it is assumed that the task countdown is one hour, and the heartbeat information will be sent every 30 minutes. After the first target task program receives the task countdown, if the first target task program is in a state of normal task processing, the heartbeat information will be sent to the task unified query service 30 minutes after receiving the task countdown. After the task unified query service receives the heartbeat information, it will resend the task countdown to the target task execution program, and the original task countdown in the target task execution program will terminate, and a new task countdown will be executed, that is, the task countdown is reset. Therefore, in the present application, the duration of the task countdown needs to be greater than the cycle of the heartbeat information to ensure that the countdown reset can be achieved before the task execution ends.

[0061] In other embodiments, if a request is received to actively close the connection with the target task executor, or a request is received to end the currently processed task, or the target task executor does not execute the task within the time period; then the unified query service will send a configuration reset request including the task executor identifier corresponding to the target task executor to the target task executor that matches the task executor identifier. The configuration reset request is mainly used to reset the configuration parameters of the target task executor so that the target task executor is reset to the initial state. At the same time, the reset target task executor will also continue to send heartbeat information to the unified query service, and the unified query service can re-add the target task executor with the updated configuration to the pre-started task executor queue.

[0062] The above embodiments describe the situation where there is only one task to be executed, while in other embodiments, there may be a situation where multiple tasks to be executed appear at the same time. At this time, if the number of tasks to be executed is greater than the number of multiple task execution programs in the current task execution program queue, the unified task query service will directly start other task execution programs that are not in the task execution program queue. At the same time, the unified task query service will shrink or expand the task execution program queue according to the number of multiple tasks to be executed, that is, adjust the number of task execution programs in the task execution program queue.

[0063] Therefore, after obtaining the pre-started task executor queue, the unified task query service will also generate a queue idleness determination thread, and use the idleness determination thread to monitor the queue idleness of the task executor queue at all times. Among them, the main purpose is to determine the allocated number of task executors that have been allocated in the task executor queue. However, it should be noted that since the task executor will be removed from the task executor queue after being allocated, the allocated number determined in this application actually includes the number of task executors that have been removed from the task executor queue.

[0064] like Figure 5 As shown, a flowchart of an embodiment of adjusting the number of task execution programs provided in an embodiment of the present application may include:

[0065] 51. Obtain a second number of task executors in the current task executor queue.

[0066] Since the number of task execution programs in the task execution program queue in the present application is constantly changing, the number of task execution programs in the current task execution program queue is usually used to determine whether capacity expansion or contraction is required. The present application needs to obtain the second number of task execution programs in the current task execution program queue.

[0067] 52. Determine whether the allocated quantity is not less than a preset third quantity, where the third quantity is the product of the preset first ratio and the second quantity.

[0068] 53. If the allocated quantity is not less than the preset third quantity, the task executor queue is gradually expanded according to the preset step expansion parameter to increase the number of task executors in the task executor queue.

[0069] In an embodiment of the present application, if the number of task execution programs that have been allocated in the task execution program queue is not less than the preset third number, it means that most of the task execution programs in the current task execution program queue have been allocated. If there are multiple tasks to be executed at the same time, the unallocated task execution programs in the current task execution program queue may not be enough to be allocated, and the task execution queue will be allocated from the task execution programs that have not been started, so it is necessary to restart the task execution program, resulting in a waste of a certain amount of time. Therefore, it is necessary to expand the capacity of the task execution program queue, that is, to increase the number of task execution programs in the task execution program queue.

[0070] In the present application, the task execution queue program can be gradually expanded according to the preset step expansion parameters, rather than directly increasing the number of task execution programs in the task execution program queue to the maximum number of task execution programs. The pre-step expansion parameters can be set according to actual needs, for example, ten task execution programs can be added to the task execution program queue each time. Since the judgment basis in the present application is the number of task execution programs in the current task execution program queue, the third number can be the product of the preset first ratio and the second number. In a specific embodiment, the preset first ratio can be 70%, that is, to judge whether the number of currently allocated task execution programs occupies 70% of the current task execution program queue.

[0071] It should be noted that since a task execution program can be assigned multiple times, when determining the allocated number of assigned task execution programs, if the task execution of the assigned task execution program is completed, the allocated number will be subtracted by 1 to avoid repeated calculation of the allocated number.

[0072] 54. Determine a fourth number of task execution programs corresponding to the task execution programs that have completed task execution among the assigned task execution programs.

[0073] Since in the process of allocating new task execution programs, some task execution programs are executing tasks, and some task execution programs have completed task execution; and after the task execution is completed, such task execution programs can be reallocated. Therefore, this application not only needs to determine the number of task execution programs that have been allocated, but also needs to determine the fourth number of task execution programs that have completed task execution among the allocated task execution programs.

[0074] 55. Determine whether the fourth quantity is not less than a preset fifth quantity, and the second quantity is greater than the first quantity; the preset fifth quantity is the product of the preset second ratio and the preset first quantity.

[0075] 56. If the fourth number is not less than the preset fifth number, and the second number is greater than the first number, the expanded task execution program queue is reduced to reduce the number of task execution programs in the task execution program queue.

[0076] In this application, the expansion and contraction of the task execution program queue is a dynamic process, which will not be endlessly expanded or contracted. In the process of continuous expansion, the number of task execution programs in the task execution program queue will continue to grow, and when it grows to a certain extent, it will trigger contraction.

[0077] Specifically, it can be determined whether the fourth number of task execution programs after the task execution is completed is not less than the preset fifth number; if not, it means that most of the task execution programs are in an idle state and can be rejoined to the task execution program queue. At the same time, it is also necessary to determine whether the second number of task execution programs in the expanded task execution program queue is greater than the first number of initially pre-started task execution programs; if greater, it means that the number of task execution programs in the current task execution program queue has exceeded the first number determined based on empirical values. If there are no multiple tasks to be executed at the same time in the future, it will cause a waste of resources; therefore, the current task execution program needs to be scaled down.

[0078] In the above embodiment, the preset fifth number may be the product of the preset second ratio and the preset first number, that is, the difference between the number of task execution programs that have completed task execution and the number of task execution programs that were initially started. In a specific embodiment, the preset second ratio may be 50%, that is, it is determined whether the number of some task execution programs that have completed task execution in the assigned task execution programs has reached 50% of the number of task execution programs that were initially started; and at the same time, it is determined whether the number of task execution programs in the current task execution program queue is greater than the number of task execution programs that were initially started.

[0079] like Figure 6 FIG. 1 is a flow chart of an embodiment of a task allocation processing method provided in an embodiment of the present application. Figure 6 In the task unified query service, tasks are allocated. When the task unified query service is started, a program management thread Spark Runner Manager will be started inside the task unified query service, and a batch of task execution programs will be pre-started using this thread, for example, 50 Spark Runners will be pre-started, and the started task execution programs will be moved into the task execution program queue. At the same time, the task unified query service will also generate a queue idleness determination thread supervisor to monitor the idleness of the task execution programs in the task execution program queue at all times; and the task execution program queue can be expanded or reduced at any time according to the idleness of the task execution programs.

[0080] After the unified task query service establishes a connection with the task to be executed, the unified task query service will determine whether there is an idle and assignable task executor in the task executor queue; if so, it will select a task executor from the task executor queue and assign it to the task to be executed. If not, a new task executor will be selected and assigned from other task executors that are not started and are not in the task executor queue. Regardless of whether the task executor is assigned or is executing a task, it will periodically send heartbeat information to the program management thread Spark Runner Manager in the unified task query service. The program management thread will determine the status of the task executor based on the heartbeat information corresponding to each task executor. During the process of executing a task, if the task executor actively ends the execution process of the task or the task executor has not executed a task for a period of time, the configuration parameters of the task executor will be reset.

[0081] like Figure 7 FIG. 1 is a timing diagram of an embodiment of a task allocation processing method provided in an embodiment of the present application. Figure 7 In the example, FederalDriver is the client that generates pending tasks, and RunnerServer is the task executor, which is used to execute pending tasks generated by the client; and the task unified query service includes the data center FederalServerFederalMeta, the connection center FederalConnection, and the program management thread RunnerManager. When the client generates pending tasks, it needs to establish a connection (newConnection) with the task unified query service. The client will send the task request with the connection string identifier to the data center in the task unified query service. The data center will then send the received task request to the connection center. The connection center parses the connection string identifier (spark-submit) in the task request and sends the connection string identifier to the program management thread. The program management thread assigns the pending tasks to the task executor RunnerServer, and the task executor executes the pending tasks, where OpenConnection means opening the link.

[0082] In this application, the task execution program will periodically send heartbeat information (heart) to the data center in the unified task query service. The data center will control the program management thread and use the program management thread to control the task execution program to control the task execution program to execute the pending tasks. If the task execution is completed, the task execution program will end its own processing process (kill craete); the program management thread will also end the control of the task execution program at the same time (sync).

[0083] When the user manually operates (Manal ConnectionHandle) on the client to close the connection (closeConnection) between the client and the unified task query service, the connection center will generate a release signal (releaseRunner) to release the task executor and send it to the program management thread. The program management thread will send the release signal to the task executor (resetConf) to disconnect the connection with the task executor. In order to determine which task executor is to be released, the release signal will include the task executor identifier (conID=SPEC); and the task executor identifier can be determined when the connection is established.

[0084] During this process, the task execution program will still periodically send heartbeat information to the data center; when the task execution program needs to be released, the heartbeat information sent to the data center includes the task execution program identifier to determine which task execution program needs to be released. The data center will determine (ask) the task execution program that needs to be released from the program management thread based on the task execution program identifier; the program management identifier will release the corresponding task execution program based on the task execution program identifier.

[0085] When the task executor is abnormal, the heartbeat information sent by the task executor is abnormal; if the data center receives the abnormal heartbeat information, the data center will control the program management thread to check the abnormal task executor (checkConFall) and remove the abnormal task executor according to the task executor identifier corresponding to the abnormal task executor (killJob reset).

[0086] The embodiment of the present application also provides a task allocation processing system, the task processing system includes multiple task execution programs and a preset task unified query service; wherein the task unified query service is configured to execute the steps in any of the above-mentioned task allocation processing methods. The task allocation processing system also includes a server, such as Figure 8 As shown, it shows a schematic diagram of the structure of the server involved in the embodiment of the present application, specifically:

[0087] The server may include one or more processing core processors 801, one or more computer-readable storage media memories 802, a power supply 803, an input unit 804 and other components. Those skilled in the art will appreciate that Figure 8 The server structure shown in the figure does not constitute a limitation on the server, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0088] in:

[0089] The processor 801 is the control center of the server, and uses various interfaces and lines to connect various parts of the entire server. By running or executing software programs and / or modules stored in the memory 802, and calling data stored in the memory 802, the processor 801 performs various functions of the server and processes data, thereby monitoring the server as a whole. Optionally, the processor 801 may include one or more processing cores; preferably, the processor 801 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 801.

[0090] The memory 802 can be used to store software programs and modules. The processor 801 executes various functional applications and data processing by running the software programs and modules stored in the memory 802. The memory 802 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the server, etc. In addition, the memory 802 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 802 may also include a memory controller to provide the processor 801 with access to the memory 802.

[0091] The server also includes a power supply 803 for supplying power to each component. Preferably, the power supply 803 can be logically connected to the processor 801 through a power management system, so as to manage charging, discharging, power consumption and other functions through the power management system. The power supply 803 can also include one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, power status indicators and other arbitrary components.

[0092] The server may further include an input unit 804, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0093] Although not shown, the server may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 801 in the server will load the executable files corresponding to the processes of one or more application programs into the memory 802 according to the following instructions, and the processor 801 will run the application programs stored in the memory 802, thereby realizing various functions, as follows:

[0094] Determine whether the unified task query service is started; if the unified task query service is started, start a preset first number of task execution programs to obtain a pre-started task execution program queue; receive a task request for a task to be executed, and determine whether there is an idle target task execution program in the task execution program queue; if there is a target task execution program, directly assign the task to be executed to the target task execution program to execute the task to be executed.

[0095] The present application also provides a computer-readable storage medium, which may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc. The storage medium stores a computer program, which is loaded by a processor to execute the steps in any of the task allocation processing methods provided in the embodiments of the present application. For example, the computer program may be loaded by a processor to execute the following steps:

[0096] Determine whether the unified task query service is started; if the unified task query service is started, start a preset first number of task execution programs to obtain a pre-started task execution program queue; receive a task request for a task to be executed, and determine whether there is an idle target task execution program in the task execution program queue; if there is a target task execution program, directly assign the task to be executed to the target task execution program to execute the task to be executed.

[0097] It should be noted that since the method of the embodiment of the present application is executed in an electronic device, the processing objects of each electronic device exist in the form of data or information. For example, time is actually time information. It can be understood that if size, quantity, position, etc. are mentioned in subsequent embodiments, they are all corresponding data for processing by the electronic device. The details will not be elaborated here.

[0098] The above is a detailed introduction to a task allocation processing method, system and storage medium provided in an embodiment of the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A task allocation processing method, characterized in that: A unified task query service is applied to a task allocation processing system, wherein the task processing system includes a plurality of task execution programs and a preset unified task query service, and the method includes: Determine whether the task unified query service is started; If the unified task query service is started, a preset first number of task execution programs are started to obtain a pre-started task execution program queue; Receive a task request for a task to be executed, and determine whether there is an idle target task execution program in the task execution program queue; If the target task execution program exists, the task to be executed is directly assigned to the target task execution program to execute the task to be executed.

2. The task allocation processing method according to claim 1, characterized in that: The receiving of the task request for the task to be executed and determining whether there is an idle target task execution program in the task execution program queue includes: Determine the number of tasks to be performed; Determining the number of task execution programs in the task execution program queue; Determining whether the number of tasks is not greater than the number of programs; If the number of tasks is not greater than the number of programs, it is determined that there is an idle target task execution program in the task execution program queue.

3. The task allocation processing method according to claim 1, characterized in that: The task request includes a task request parameter; if the target task execution program exists, directly assigning the task to be executed to the target task execution program to execute the task to be executed includes: Parsing the task request parameters to determine the connection string identifier corresponding to the task to be executed; Sending the connection string identifier to the target task execution program; Determining whether a heartbeat message corresponding to the target task execution program is received within a first preset time period; If the heartbeat information corresponding to the target task execution program is received, the task to be executed is sent to the target task execution program, so that the target task execution program is used to execute the task to be executed.

4. The task allocation processing method according to claim 3, characterized in that: If the heartbeat information corresponding to the target task execution program is received, the task to be executed is sent to the target task execution program so as to execute the task to be executed by using the target task execution program, including: Determine task execution program configuration parameters that match the task to be executed; The task to be executed and the task execution program configuration parameters are sent to the target task execution program, so as to update the target task execution program using the task execution program configuration parameters, and execute the task to be executed using the updated target task execution program.

5. The task allocation processing method according to claim 3, characterized in that: If the target task execution program exists, after directly allocating the task to be executed to the target task execution program to execute the task to be executed, the method further includes: removing the target task execution program from the task execution program queue; Moving the target task execution program into a preset resource job queue; The target task execution program is controlled to call the resources in the resource job queue to execute the task to be executed.

6. The task allocation processing method according to claim 5, characterized in that: After the target task execution program is removed from the task execution program queue, the method further includes: Sending a task countdown to the target task execution program; The task countdown is reset periodically according to the heartbeat information.

7. The task allocation processing method according to claim 1, characterized in that: If the unified task query service is started, a preset number of task execution programs are started, and a pre-started task execution program queue is obtained, the method further includes: The task unified query service generates a queue idleness determination thread; Determine the allocated number of the task execution programs that have been allocated to the task execution program queue by using the queue idleness determination thread; Determine whether to adjust the number of task executors in the task executor queue according to the allocated number.

8. The task allocation processing method according to claim 7, characterized in that: The determining whether to adjust the number of task execution programs in the task execution program queue according to the allocated number includes: Get a second number of task executors in the current task executor queue; Determining whether the allocated quantity is not less than a preset third quantity, where the third quantity is a product of a preset first ratio and the second quantity; If the allocated number is not less than the preset third number, gradually expanding the task execution program queue according to a preset step expansion parameter to increase the number of task execution programs in the task execution program queue; Determine a fourth number of task execution programs corresponding to the task execution programs that have been assigned and whose task execution has been completed; Determine whether the fourth number is not less than a preset fifth number, and the second number is greater than the first number; the preset fifth number is the product of a preset second ratio and the preset first number; If the fourth number is not less than the preset fifth number, and the second number is greater than the first number, the expanded task execution program queue is reduced to reduce the number of task execution programs in the task execution program queue.

9. A task allocation processing system, characterized in that: The task processing system includes a plurality of task execution programs and a preset unified task query service, and the unified task query service is configured to execute the task allocation processing method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in the task allocation processing method according to any one of claims 1 to 8.

Images