Batch task scheduling method and system, electronic equipment and storage medium

Abstract

The invention discloses a batch task scheduling method and system, electronic equipment and a storage medium, and the method comprises the steps: receiving a task request, packaging the task request into a task event, and publishing the task event to an event message queue; determining that the event message queue has a to-be-processed task event and application instance resources in an idle state exist, and obtaining the task event and the application instance resources; and processing the task event according to the application instance resource. According to the method, the task request is packaged into the task event and cached into the event message queue, so that a system does not need to continuously monitor the task request in real time, the problem of resource waste caused by monitoring the task request in real time in the prior art is solved, and the resource utilization rate is improved; the task event is stored through the event message queue, asynchronous decoupling of the task request and the processing process is achieved, and then the concurrent request response rate and the task processing efficiency are improved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of data processing, and particularly relate to a batch task scheduling method and system, electronic equipment and storage medium. BACKGROUND

[0002] In the era of explosive growth of data, the demand for large-scale batch data processing is increasingly strong. Distributed task scheduling mode has become the mainstream technical means for batch task processing because it can efficiently coordinate multi-node resources and improve processing efficiency.

[0003] At present, a typical distributed batch task scheduling system mainly consists of three core components: a batch controller, a distributed coordination center and a batch task executor. During system operation, the batch task executor needs to listen to the message queue of the distributed coordination center in real time. Once a scheduling instruction is received, the execution process of the corresponding task is started immediately, and the result after execution is fed back to the coordination center.

[0004] However, this existing mode has certain disadvantages. The batch task executor needs to listen to the job scheduling message of the distributed coordination center in real time, so during the period without batch task scheduling execution, the batch task executor still needs to be in a running ready state, waiting for the task scheduling allocation of the distributed coordination center, resulting in slow concurrent request response and low resource utilization. SUMMARY

[0005] The present application provides a batch task scheduling method, system, electronic equipment and storage medium to solve the problem of resource waste caused by real-time listening, improve resource utilization and task processing efficiency.

[0006] In one aspect of the embodiments of the present application, a batch task scheduling method is provided, comprising:

[0007] receiving a task request and encapsulating the task request as a task event to publish to an event message queue;

[0008] determining that the event message queue has a task event to be processed, and that there is idle application instance resource, obtaining the task event and the application instance resource;

[0009] processing the task event according to the application instance resource.

[0010] In one aspect of the embodiments of the present application, a task scheduling system is provided, comprising:

[0011] an event publisher, configured to receive a task request and encapsulate the task request as a task event to publish to an event message queue;

[0012] An event subscriber is configured to determine that the event message queue has a task event to be processed and that there is an application instance resource in an idle state, acquire the task event and the application instance resource.

[0013] A controller is configured to process the task event according to the application instance resource.

[0014] In another aspect of the embodiments of the present application, an electronic device is provided, comprising:

[0015] at least one processor; and

[0016] a memory in communication with the at least one processor;

[0017] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the batch task scheduling method of any of the embodiments of the present application.

[0018] In another aspect of the embodiments of the present application, a computer readable storage medium is provided, comprising computer instructions configured to enable a processor to perform the batch task scheduling method of any of the embodiments of the present application when executed by the processor.

[0019] The present application discloses a batch task scheduling method, system, electronic device and storage medium, the method comprising: receiving a task request and encapsulating the task request as a task event to publish to an event message queue; determining that the event message queue has a task event to be processed and that there is an application instance resource in an idle state, acquiring the task event and the application instance resource; and processing the task event according to the application instance resource. The present application encapsulates the task request as a task event and caches it in the event message queue, so that the system does not need to continuously and real-time monitor the task request, solves the problem of resource waste caused by real-time monitoring of the task request in the prior art, and improves the resource utilization rate. The event message queue stores the task event, realizes asynchronous decoupling of the task request and the processing process, and further improves the concurrent request response rate and the task processing efficiency.

[0020] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0022] Figure 1 is a batch task scheduling method flowchart provided according to an embodiment one of the present application;

[0023] Figure 2 is another batch task scheduling method flowchart provided according to an embodiment two of the present application;

[0024] Figure 3 is a distributed task scheduling mode diagram of a batch task provided according to an embodiment three of the present application;

[0025] Figure 4 is a common event-driven architecture diagram provided according to an embodiment three of the present application;

[0026] Figure 5 is a general architecture diagram of a batch task hosting device based on event-driven provided according to an embodiment three of the present application;

[0027] Figure 6 is a running architecture diagram of a batch task hosting device based on event-driven provided according to an embodiment three of the present application;

[0028] Figure 7 is a batch task event asynchronous execution flowchart provided according to an embodiment three of the present application;

[0029] Figure 8 is an application instance state transition diagram provided according to an embodiment three of the present application;

[0030] Figure 9 is an LSTM resource prediction model architecture diagram provided according to an embodiment three of the present application;

[0031] Figure 10 is an LSTM-based resource scaling flowchart provided according to an embodiment three of the present application;

[0032] Figure 11 is a structure schematic diagram of a batch task scheduling system provided according to an embodiment four of the present application;

[0033] Figure 12 is an electronic device block diagram for executing a batch task scheduling method provided according to an embodiment five of the present application. DETAILED DESCRIPTION

[0034] In the following, the technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all the other embodiments obtained by a person of ordinary skill in the art without creative effort should belong to the scope of protection of the present application.

[0035] It should be noted that the terms "first", "second" and the like in the description and claims of the present application and the above drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or electronic device including a series of steps or units does not necessarily have to include all the steps or units clearly listed, but can include other steps or units not clearly listed or inherent to the process, method, product or electronic device.

[0036] Figure 1 A flowchart of a batch task scheduling method is provided for the embodiments of the present application. The embodiments of the present application can be applied to batch task scenarios that require high concurrency processing capability, especially scenarios involving large-scale data processing (such as batch data import, export, conversion, analysis, report generation, etc.), high resource utilization rate and flexible response to system changes. The method can be executed by a batch task scheduling system, which can be realized in the form of hardware and / or software, and can be configured in an electronic device. As shown in the figure, the method comprises: Figure 1

[0037] S110, receiving a task request and encapsulating the task request as a task event to publish to an event message queue.

[0038] The task request is a pre-defined task instruction containing a series of task attribute information, and can be initiated by a user or an external system. The task request can contain task attribute information such as the identification of the task to be processed or the task command for processing data. The task command can include data batch import or data batch conversion commands. During batch task scheduling, the batch task scheduling system can perform grouping, sorting, resource allocation and execution of the task to be processed according to the task attribute information in the task request after receiving the task request.

[0039] ​The task event refers to a data object with specific attribute information after encapsulating the task request. For example, the task event can include event type, event source, event identifier, event time, and event data load. The event type is used to describe the type and nature of the specific event, and can be used to identify the start, completion, or failure of the to-be-processed task. The event source refers to the source of the event, which is used to indicate the context in which the task event is generated, and can be the unique identifier of the service or system that initiates the to-be-processed task. The event identifier is a unique identifier of the task event, which ensures the traceability of the task event. The event time records the time when the task event is generated. The data load of the event includes the to-be-processed data related to the to-be-processed task. Optionally, in the embodiment of the present application, the task request can be encapsulated according to the preset event specification to obtain the task event. The preset event specification can be defined according to the key task state and key task operation of the to-be-processed task corresponding to the historical task request.

[0040] The event message queue can be understood as a first-in, first-out data structure for storing and transmitting task events. As an intermediary between event publishers and event consumers, the event message queue can realize asynchronous communication between event publishers and event consumers by temporarily storing task events. For example, the types of event message queues can include Kafka event message queues and RabbitMQ event message queues. Different types of event message queues can be selected according to business needs. For example, if high-throughput data channels, persistent storage of events, and stream processing are required to adapt to batch task high concurrency and large-scale data processing scenarios, Kafka event message queues can be selected. If a more flexible message routing mechanism and multiple message transmission modes are required to cope with complex event interaction logic, RabbitMQ event message queues can be selected.

[0041] Specifically, a set of preset event specifications is defined in advance according to the key task state and key task operation of the to-be-processed task corresponding to the historical task request. The preset specification standard can be stored in the buffer of the control batch task scheduling system or the external system, or in the configuration file of the control batch task scheduling system. The preset event specification can be obtained by scanning the buffer or the configuration file. The control batch task scheduling system receives the receiving request transmitted by the user or the external system by calling the application programming interface or the hypertext transfer protocol interface. The task request is converted into event type, event source, event identifier, event time, and event data load by using the preset event specification defined in advance. A series of attribute information is integrated to obtain the task event. The task event is published to the event message queue through the event bus, or broadcast to the specified event message queue through the message broker.

[0042] S120, determine that the event message queue has a task event to be processed, and there is an application instance resource in an idle state, obtain the task event and the application instance resource.

[0043] The application instance resource refers to actual computing resources required for processing the task event, and is used for actually executing the task event. For example, the application instance resource can include event processing logic code, CPU or memory, and the like. A group of application instance resources corresponds to an application instance, each application instance runs in an independent resource environment, and the state of the application instance can include a start state, a running state, a running success state, a running failure state, an idle state, and a deletion state, and the state of the application instance resource is kept synchronous with the state of the application instance. It is worth noting that the application instance resource can be dynamically adjusted according to the number of task requests, and the resource usage at a future time node can be predicted based on historical data of the task event, and the dynamic adjustment of the application instance resource is realized according to the predicted resource usage.

[0044] Specifically, by querying the length of the event message queue, if the length is greater than zero, it indicates that there is a task event to be processed in the queue, or the specific number of task events to be processed in the event message queue is directly queried, and when the number exceeds a set reference value, it can be confirmed that there is a task event to be processed. The application instance resource is dynamically monitored by using a monitoring mechanism, so as to determine whether there is idle resource or to accurately determine by means of a resource usage rate index, which specifically includes a CPU usage rate and a memory usage rate, and when the resource usage rate index is lower than a preset threshold, it can be determined that there is an application instance resource in an idle state. After it is determined that the event message queue has a task event to be processed, and there is an application instance resource in an idle state, the task event is extracted from the event message queue, and a group of application instance resources in an idle state are obtained.

[0045] S130, process the task event according to the application instance resource.

[0046] Specifically, the application instance resource in an idle state is obtained, and the task event to be processed is extracted from the event message queue in a first-in first-out order, and the application instance resource in an idle state is used when the task event to be processed is processed.

[0047] The application discloses a batch task scheduling method and system, electronic equipment and a storage medium, and the method is as follows: receiving a task request, encapsulating the task request into a task event, and publishing the task event to an event message queue; determining that the event message queue has a task event to be processed, and that there is an application instance resource in an idle state, obtaining the task event and the application instance resource; and processing the task event according to the application instance resource. The application encapsulates the task request into the task event and caches the task event in the event message queue, so that the system does not need to continuously and real-timely monitor the task request, the problem of resource waste caused by real-time monitoring of the task request in the prior art is solved, and the resource utilization rate is improved; the task event is stored in the event message queue, the asynchronous decoupling of the task request and the processing process is realized, and the concurrent request response rate and the task processing efficiency are further improved.

[0048] Further, the application embodiment also optimizes a batch task scheduling method, and specifically, how to manage the application instance resource is optimized, including:

[0049] A1, obtaining an execution state of the task event and an event processing result, and writing the execution state and the event processing result as an event record into a state message queue.

[0050] The event record can be understood as an execution log, and is used to describe the execution process of the task event. For example, the event record can include the execution state of the task event and the event processing result. The execution state is the state of the application instance resource processing the task event, and is used to reflect the real-time progress of the task event processing. For example, the execution state can include the states of starting, running, success and failure. The event processing result refers to result data generated after the task event is executed, and is used to reflect the effectiveness of the task event processing. The event record can be stored in the state message queue to manage the application instance resource.

[0051] The state message queue can be understood as a data structure used to cache the event record, and also adopts the first-in first-out mechanism. The asynchronous storage and transmission of the event record are realized. It can be understood that the state message queue and the event message queue belong to the message queue, and are respectively named as the state message queue and the event message queue according to different functions of storing the task event and caching the event record.

[0052] Specifically, the processing process of the task event is monitored in real time. In the processing process of the task event, the execution state of the task event is collected in real time through a monitoring mechanism, for example, the states of starting, running, success and failure, and the event processing result of processing the task event is obtained, and the execution state and the event processing result are written into the state message queue as the final event record to provide data support for resource management.

[0053] A2, manage the application instance resources based on the resource prediction engine and the event records in the state message queue.

[0054] The resource prediction engine can be understood as a system component for predicting future resource requirements, and the resource prediction engine at least includes a long short-term memory network prediction model, which is trained and generated based on historical event records and historical application instance resources and other historical data.

[0055] Specifically, a resource prediction engine for predicting future resource requirements is obtained, and the resource prediction engine at least includes a long short-term memory network prediction model, which is trained and generated based on historical event records and historical application instance resources and other historical data.

[0056] Further, the embodiment of the present application also details step A2, specifically, how to manage the application instance resources based on the resource prediction engine and the event records in the state message queue, including:

[0057] A21, read at least one event record in the state message queue at a fixed time interval, and extract the pre-processing state, event time sequence and task sequence of each event record as the to-be-processed data.

[0058] The to-be-processed data can be understood as a series of feature sets directly associated with the event records, which are used to predict the application instance resources required by the application instance in a specific period in the future, and at least include the pre-processing state of the task event, the event time sequence of the task event and the task sequence of the task event.

[0059] Specifically, at least one event record in the state message queue is read at a fixed time interval, for example, every 1 hour or every 30 minutes, and at least the pre-processing state, event time sequence and task sequence are extracted from these event records.

[0060] A22, time series feature extraction is performed on each to-be-processed data to obtain a time series feature vector.

[0061] The time series feature vector can be understood as a numerical vector with a fixed length. For example, the dimensions of the vector elements in the time series feature vector can include a basic statistical feature dimension, a time series dynamic feature dimension, and a morphological pattern feature dimension. The vector elements in the basic statistical feature dimension can include a mean value and a standard deviation of the to-be-processed data. The vector elements in the time series dynamic feature dimension can include a rolling window variance and a linear trend slope. The vector elements in the morphological pattern feature dimension can include a peak duration and a valley flatness. The time series feature vector can simultaneously include vector elements of the three dimensions.

[0062] Specifically, to-be-processed data of an event record is obtained, and time series feature extraction is performed on each to-be-processed data. For example, by analyzing the to-be-processed data, a peak duration, a valley flatness, a mean value, a standard deviation, a rolling window variance, and a linear trend slope of the to-be-processed data are obtained. Each feature is taken as a vector element, and is arranged in order to form a time series feature vector that can be processed by a resource prediction engine.

[0063] A23, the resource prediction engine is called to process the time series feature vector to obtain an application instance resource prediction result.

[0064] The application instance resource prediction result is result data output by the resource prediction engine, and reflects application instance resources required by the application instance in a future specific period. The application instance resource prediction result provides a basis for adjustment of the application instance resources, and can realize elastic scaling of the application instance resources. Meanwhile, the specific range of the future period can be pre-set according to actual business needs.

[0065] Specifically, the resource prediction engine is called, and the time series feature vector is taken as an input of the resource prediction engine and is input into the resource prediction engine. The resource prediction engine processes the time series feature vector to obtain the application instance resource prediction result.

[0066] A24, resource adjustment is performed on the application instance resources according to the application instance resource prediction result.

[0067] Specifically, resource adjustment is performed on the application instance resources according to the application instance resource prediction result. For example, when the predicted application instance resource quantity shown in the application instance resource prediction result exceeds the current actual application instance resource quantity, a resource expansion mechanism is triggered, and the actual application instance resources are increased to meet possible future task processing needs. Conversely, if the predicted application instance resource quantity is less than or equal to the current actual application instance resource quantity, a resource reduction process is started, and the actual application instance resources are correspondingly reduced to avoid idling and waste of resources.

[0068] Further, the embodiment of the present application also optimizes a batch task scheduling method, specifically, how to initialize application instance resources is optimized, including:

[0069] Obtaining the application instance resource prediction result of the resource prediction engine, and initializing the application instance resources according to the application instance resource prediction result.

[0070] Specifically, the application instance resource prediction result output by the resource prediction engine is obtained, and the application instance resources are initialized according to the result. The initialization operation can include: creating a corresponding number of application instances according to the number specified in the resource application instance resource prediction result, and / or creating application instances meeting specific configuration requirements according to the information about application instance configuration in the prediction result, such as memory limit and concurrency number, etc. The application instance resources are initialized according to the application instance resource prediction result to ensure the stable operation of the system.

[0071] Embodiment two

[0072] Figure 2 A flowchart of another batch task scheduling method is provided for the second embodiment of the present application. The embodiment of the present application is a refinement of the above-mentioned embodiment, specifically, the specific steps of how to encapsulate the task request into a task event and the specific steps of how to process the task event according to the application instance resources are refined.

[0073] As shown in Figure 2 , another batch task scheduling method can include the following specific steps:

[0074] S210, receiving a task request of at least one type of batch task, and splitting and encapsulating each task request into at least one task event.

[0075] Specifically, the task type of the batch task can include batch data conversion and batch data import, etc. The task request of at least one type of batch task can be split into multiple sub-task requests according to the type of the batch task, and each sub-task request can be encapsulated into a standardized task event containing event type and event source, etc.

[0076] S220, sending the task event to the event message queue for caching.

[0077] Specifically, the task event is sent to the event message queue, and the queue performs persistent caching on the event.

[0078] S230, determining that the event message queue has a task event to be processed, and there is idle application instance resource, obtaining the task event and the application instance resource.

[0079] S240, initialize and start the application instance resource, trigger the application instance resource processing task event.

[0080] Specifically, the application instance resource is initialized, for example, the initialization operation can include: configuring the dependent library, allocating CPU or memory and other running environment, etc. By loading the processing code, starting the application instance resource, passing the task event to the application instance corresponding to the application instance resource, triggering the application instance to execute the task event, and using the application instance resource when the application instance executes the task event.

[0081] S250, determine that all task events are subscribed and consumed by the application instance resource, obtain the execution state of the task event and the event processing result.

[0082] Specifically, by comparing the total number of task events in the event message queue with the consumed number, it is determined that all task events are subscribed and processed by the application instance resource, and the execution state of each task event is obtained from the state message queue, for example, running successfully or running unsuccessfully, and the corresponding task event processing completion event processing result.

[0083] S260, determine that the application instance resource completes the processing of the task event, and mark the application instance resource as idle.

[0084] Specifically, after receiving the signal returned by the application instance resource that the task event processing is completed, the state field of the application instance resource is updated to idle, and the currently identified application instance resource can be recorded to the state message queue for subsequent other task event processing.

[0085] The embodiment of the present application receives task requests of at least one batch task, splits and encapsulates each task request into at least one task event, sends the task event to an event message queue for caching, determines that the event message queue has a task event to be processed and that there is an application instance resource in an idle state, acquires the task event and the application instance resource, initializes and starts the application instance resource, triggers the application instance resource to process the task event, determines that the task event is completely subscribed and consumed by the application instance resource, acquires an execution state and an event processing result of the task event, determines that the application instance resource completes processing of the task event, and identifies the application instance resource as an idle state. The embodiment of the present application can realize fine management of tasks by splitting the request of a batch task into multiple sub-task requests, realizes asynchronous decoupling of task requests and execution by converting the task request into a standardized event and caching, avoids the disadvantages of real-time continuous monitoring of task requests in the prior art, reduces invalid resource occupation, ensures resource on-demand allocation by starting a processing flow when there is a task and there is an idle resource, avoids resource waste caused by long-term waiting of an executor in the prior art, improves resource utilization, ensures stable system operation by initializing the application instance resource, ensures completeness and traceability of batch task processing by tracking the execution state and the event processing result of the task event, avoids data inconsistency problems caused by multi-node cooperation in the prior art, provides a reliable basis for subsequent resource adjustment, and identifies the application instance that completes processing as an idle state, so that the application instance is in a reusable state after task completion, avoids idle waste of resources, and improves resource reuse rate and concurrent processing capability of the system.

[0086] Further, the embodiment of the present application further refines step S250, specifically, how to acquire the execution state and the event processing result of the task event, including:

[0087] If it is determined that each task event has been completely subscribed and consumed by the application instance resource, the event processing result of the task event is acquired within a threshold time; if the event processing result is accurately acquired, it is determined that the execution state is running successfully; if the event processing result is not accurately acquired or the event processing result is not correctly acquired, it is determined that the execution state is running unsuccessfully.

[0088] The threshold time can be understood as a preset time limit for determining whether the task event processing is timed out.

[0089] Specifically, a threshold time for judging whether the task event processing is timed out is acquired, and after confirming that all task events have been subscribed and consumed by the application instance, the event processing result of the task event needs to be acquired, wherein if the event processing result is acquired within the threshold time, the execution state of the task event is marked as running successfully, and if the event processing result is not acquired within the threshold time, the execution state of the task event is marked as running failure.

[0090] Embodiment three

[0091] Figure 3 A distributed task scheduling mode diagram of batch tasks provided for the embodiment three of the application; Figure 4 A common event-driven architecture diagram provided for the embodiment three of the application; Figure 5 A batch task hosting device overall architecture diagram based on event driving provided for the embodiment three of the application; Figure 6 A batch task hosting device running architecture diagram based on event driving provided for the embodiment three of the application; Figure 7 A batch task event asynchronous execution flow diagram provided for the embodiment three of the application; Figure 8 An application instance state migration diagram provided for the embodiment three of the application; Figure 9 An LSTM resource prediction model architecture diagram provided for the embodiment three of the application; Figure 10 An LSTM-based resource expansion and contraction capacity flow diagram provided for the embodiment three of the application; the embodiment of the application is an optimization of the above-mentioned embodiments, specifically, the technical background of the batch task scheduling (hosting) method, the overall architecture and running architecture of the batch task hosting device, the task event asynchronous execution flow, the application instance state migration flow, the LSTM resource prediction model architecture, and the LSTM-based resource expansion and contraction capacity flow are supplemented.

[0092] At present, batch tasks generally adopt a distributed task scheduling mode to realize the processing of large-scale batch data, and the mechanism is usually composed of components such as a batch controller, a distributed coordination center, and a batch task executor. Figure 3 As shown in the figure, in the mechanism, the batch scheduler is used for task scheduling and publishing, the distributed scheduling coordination center is responsible for scheduling and coordinating batch tasks in a distributed cluster, and the batch executor starts a batch task running instance and synchronously updates the execution result and state of the batch task after receiving the task sent by the distributed coordination center. Since the batch task executor needs to synchronously listen to the job scheduling message of the distributed coordination center in real time, the batch task executor still needs to be in a running ready state during the batch task scheduling and execution period, waiting for the task scheduling and distribution of the distributed coordination center, which leads to slow concurrent request response and low resource utilization.

[0093] The embodiments of the present application aim to realize an event-driven batch task hosting method and device. The core of the method and device is to solve the problem of slow concurrent request response and low system resource utilization caused by synchronous real-time waiting between components in the batch task processing. At present, the industry mainly has the following solutions to the low resource utilization of batch task processing: reserving application instances, and fixed-time resource expansion and contraction. Unlike traditional solutions, the embodiments of the present application mainly adopt an event-driven mode combined with a resource expansion and contraction model based on a long short-term memory (LSTM) to manage batch task application requests and running resources, thereby providing a more efficient and more accurate resource control batch task hosting solution. Compared with the traditional distributed scheduling mode, the strategy combines the event-driven mode with resources, defines batch task requests as standardized events, and designs an event-driven batch task processing architecture, complete event state migration, and event processing flow to realize asynchronous decoupling of batch task scheduling and execution. The strategy not only enables a fast response to batch task requests, but also enables elastic scaling of running resources according to the number of request events, thereby avoiding long-term invalid waiting of resources and effectively improving the utilization of resources. Compared with the traditional resource quota expansion and batch task instance reservation method, the resource expansion and contraction model based on LSTM can efficiently process long-range historical resource usage time series, and predict resource requirements corresponding to different requests at future time nodes according to continuous time series, thereby realizing more accurate and efficient resource rapid expansion and contraction. The event-driven mode connects various components of the system through a series of relationships such as event processing relationship and sequence, so that the system can flexibly respond to external events. Unlike the traditional process control architecture, the event-driven architecture focuses on the generation, detection, consumption, and response of events in the system. The behavior of the system is completely controlled by the event flow, rather than the sequential program flow execution control. It allows system components to trigger communication and interaction when events occur, rather than continuously polling or passively waiting for requests, and has the characteristics of asynchrony, loose coupling, etc. For example Figure 4As shown in the commonly used event-driven architecture, four main components are mainly included, namely, event, event source, event consumer, and event bus, wherein the event (Event) : is a basic component unit in the event-driven architecture, which represents a specific behavior or state change in the system. The event can be an external behavior of the system (such as a user operation such as a key), or an internal behavior of the system (such as a timer interrupt), which usually includes a timestamp, type, data content and the like to describe what happens, when it happens and related details. The event source (Source) : the event source is an entity or component object that generates events. It can be a user who performs an operation, an application system that changes a state, external data input, etc. The event source publishes events through the event bus for the event consumer to process. The event consumer (Consumer) : the event consumer is an entity or component that processes events. The event consumer subscribes to the events of interest through the event bus. When the event source publishes events, the event consumer will receive a notification and execute the response processing logic for the specific event. For example, a processing program that responds to a user operation, a business logic that processes external input data. The event bus (Broker) : the event bus is a data structure for storing events. For example, a queue, an array, etc. The event bus serves as a communication intermediary between the event source and the event consumer, first receives the published events from the event source, and delivers these events to the event subscriber (i.e., the event consumer). For example Figure 4 As shown, the processing flow of the event-driven architecture is described as follows: event generation: the event source encapsulates the message content, event and type information of the specific state change or operation according to the business logic of the event source, generates the event defined by the system; event publishing: the event generator publishes the encapsulated event to the event bus, waiting for the event consumer to process; event delivery: the event is delivered from the event bus to one or more event consumers through the event channel, ensuring that the event can be safely and reliably delivered, and ensuring that loss does not occur; event processing: the event consumer receives the event, triggers the corresponding business logic to execute, saves the processing result of the event after completing the processing of the event, and returns the state to complete the response of the event.

[0094] The overall structure of the batch task hosting device based on event driving proposed in the embodiment of the application is as follows Figure 5As shown, the device mainly includes the following modules: an event source module, an event bus module, and an event consumption module. The event source module is responsible for obtaining batch task requests and encapsulating the obtained batch task requests to generate corresponding events, and then packaging and publishing the event messages to the corresponding event bus module. The event bus module mainly includes two types of message queues: an event message queue and a state message queue. The event message queue is responsible for caching event messages of batch tasks, and the state message queue is responsible for caching and recording the execution states of batch tasks and their related application instances. The event consumption module mainly includes four sub-modules: an event receiving module, an event processing module, a controller, and an LSTM resource prediction engine. The event receiving module is responsible for obtaining events from the event bus module, triggering the event processing module to complete corresponding batch task event processing by the application instance, and cooperating with the controller to complete the scaling of application instance resources according to event traffic. The event processing module is responsible for running the application instance to complete batch task request response. The controller is responsible for monitoring the application instance and the event state, and controlling the elastic scaling of the application instance resource according to the prediction return result of the LSTM resource prediction engine. The LSTM resource prediction engine is responsible for predicting the resource usage at future time nodes based on historical data of events and resources.

[0095] In the embodiment of the application, in order to ensure the consistency of events between different components, a corresponding batch task standardized event is defined according to the key states and operations in the common batch task, which mainly includes the following core parts: an event type, an event source, an event identifier, and an event data load. The event type is used to describe the type and nature of the specific event, and can be used to identify the start, completion or failure of the batch task, so that the system components can judge the batch task state according to the event type. The event source is the source of the event, which is used to represent the context of the event generation, and can be the unique identifier of the service or system that initiates the batch task. The event identifier serves as a unique identifier for the event to ensure traceability of the event. The event time records the time when the batch task event is generated. The data load of the event contains specific information and data related to the event. The batch task standardized event can also include an event sub-type, which is used to describe the sub-type of the event. For batch tasks, it can be a specific task name or other identifier. The event extension attribute is used to describe additional attributes related to the batch task, such as the priority of the task.

[0096] Figure 6The running architecture of the batch task hosting device based on event driving proposed for the embodiments of the present application is as follows: in the running architecture, the task request module in the event source module is responsible for obtaining the requests of various specific batch tasks, and sends the task requests to the event publisher for processing. After receiving the task requests, the event publisher is responsible for converting the task requests into events conforming to the system specification standards, decomposing and scheduling events of different tasks, and then sending the events to the Kafka message queue for caching, so as to enable the subsequent modules to complete the consumption of the corresponding task events. The event subscriber in the event consumption module is set for different batch task applications, and its functions mainly include: being responsible for obtaining the corresponding events from the message queue and triggering the corresponding application instances to process the events; obtaining the corresponding application running instance resources by cooperating with the controller; being the core of realizing the elastic scaling feature of the application instance resources based on event traffic, and realizing the elastic scaling of the application instance resources by cooperating with the controller. The application instance mainly includes event processing logic code and code base (including dependent libraries and running environment), and is the module for actually consuming and processing the events and responding to the batch task requests in the architecture. The controller is the control subject of the event consumption module, and its functions include: completing the update and management of the task state queue according to the running state of the application instance; and managing the resource state queue and the application instance resources based on the prediction result of the LSTM resource prediction engine according to the event execution and the state of the application instance resources. The LSTM resource prediction engine provides the main basis for the system resource elastic scaling decision, is responsible for predicting the resource usage at future time nodes based on the historical data of events and resources, and returns the prediction result to the controller for resource elastic scaling decision. Due to the stability of the batch task, the engine adopts a periodic offline training mode, and includes three sub-engines, namely a data processing engine, a training engine and an inference engine. The data engine is responsible for reading the function request event data and the resource usage data, and pre-processing the data, such as normalization and serialization. The training engine is responsible for retraining and updating the model weight parameters based on new data. The inference engine is responsible for using the trained lightweight LSTM model to infer and predict the resource demand at future time points, and saving the prediction result for the control. The event bus module mainly includes a Kafka event message queue and a state message queue for processing and saving batch task event information and batch task application instance state information, etc. The Kafka event message queue is responsible for caching the event messages of different batch tasks, providing a high-throughput data channel, supporting event persistent storage and stream processing, ensuring the consistency of batch task events, and realizing the decoupling of event scheduling and event execution. The Kafka state message queue is responsible for caching the batch task request state and the batch task application instance resource state, etc. for the controller and the LSTM resource prediction engine.

[0097] The event request processing procedure of the above-mentioned event-driven batch task hosting device can include the following steps:

[0098] Batch task request acquisition: the task request module acquires requests of various specific batch tasks, and sends the task requests to the event publisher for processing.

[0099] Event encapsulation and publishing: the event publisher is responsible for encapsulating the task requests into events conforming to the standard, decomposing and scheduling events of different tasks, and then publishing the events to the Kafka event message queue.

[0100] Event acquisition: the event subscriber of the batch task application acquires corresponding events from the Kafka event message queue.

[0101] Application instance resource acquisition: the event subscriber sends an application instance resource application to the controller.

[0102] Event triggered execution: after the event subscriber acquires the application instance resource allocated by the controller, the application instance is triggered to run, and corresponding event processing and request response are performed.

[0103] State return: the application instance returns the execution state and event processing result of the response, which is recorded by the controller and written back to the Kafka state message queue.

[0104] State recording: the controller records the application instance execution state and event processing result, and writes back to the Kafka state message queue, and releases the corresponding application instance resource.

[0105] Event and resource historical data acquisition: the LSTM resource prediction engine acquires the historical data of events and resources, pre-processes the historical data, and then predicts the resource usage at future time nodes.

[0106] Prediction result return: the LSTM resource prediction engine returns the prediction result to the controller, and the controller makes a decision on resource scaling.

[0107] In the batch task hosting device of the embodiment of the application, the event publisher, the event message queue and the event subscriber are used to complete the decoupling and asynchronous processing of batch task event publishing and consumption. The asynchronous execution flow is as follows Figure 7As shown, the specific process can be divided into three stages of application instance initialization, event consumption and application instance recycling. In the resource initialization stage, the controller makes decisions on resource usage according to the prediction results of the LSTM resource prediction engine, and creates corresponding application instance resources. In the event consumption stage: the event publisher receives the task request, encapsulates the task request as an event and publishes it to the event message queue. In this event scheduling stage, the publisher does not continuously wait for the processing result of a task event to return before continuing to publish other task events, but asynchronously schedules and publishes other task events to the event message queue to realize the asynchrony of event publishing and consumption. In the event subscriber detects that there are events to be processed in the event message queue, and there are idle application instance resources in the cluster that can be used, obtains the event from the event message queue, and then tries to apply for resources to trigger the application instance to execute. In this stage, the controller continuously monitors the event execution state and the application instance running state, and when the application instance execution ends, the application instance is placed in an idle state, and the corresponding event state is recorded in the state message queue. After all the event processing of the task is completed, the controller will ensure that all the application instances are recycled and deleted, and the information in the Kafka state message queue is updated. The LSTM resource prediction engine predicts the resource scaling at future time nodes based on the current running data, and returns the prediction results for the controller to make decisions.

[0108] As shown in Figure 8 To achieve fine management of the execution state of batch task events and improve system stability and support application instance resource scalability, the application instance state of the event is divided into six states: start state, running state, running success state, running failure state, idle state and delete state. Among them, the start state: after the task application instance is initialized successfully, the application instance enters the start state. The running state: the event subscriber triggers the application instance to execute by using the corresponding event, and completes the event consumption processing logic. Running success: when all the events corresponding to the task are subscribed and consumed, and the application instance returns the correct state, it is determined as running success. Running failure: after the events corresponding to the task are subscribed and consumed, the corresponding application instance does not return correctly or the state is timeout lost, which is determined as running failure. Idle state: after the application instance of the task runs successfully or fails, it marks the end of event processing and runs into the idle state. Delete state: if the event subscriber corresponding to the task event does not obtain the task event again within a timeout, the application instance will enter the delete state, and all application instances of the task in this state will be deleted and recycled, thereby realizing the complete on-demand use of application instance resources.

[0109] The embodiment of the application also provides a resource expansion and contraction model based on LSTM, which is based on batch task requests and application instance resource running historical data (such as CPU, memory, etc.) to predict future application instance resource requirements, and make resource allocation strategies, realize accurate resource expansion and contraction, effectively improve the computing resource utilization rate, and support high-concurrency batch task request response.

[0110] The architecture of the LSTM resource prediction model is shown in Figure 9 The LSTM prediction model mainly includes the following steps: data preprocessing: the data engine collects and preprocesses resource state, event time series, batch task sequence and other historical data, extracts and processes time series features, processes missing values, normalizes time series data to construct a feature vector to improve training speed and accuracy. Data input: the preprocessed data is input into the inference engine. Inference prediction: the inference engine predicts future time resource usage data based on the offline trained model. Result return: the inference engine generates a prediction result and sends the prediction result to the controller. Resource scaling: the controller makes resource elasticity scaling decisions based on the prediction result and adjusts the application instance resources. The execution process of the LSTM resource prediction model is shown in Figure 10 and described in detail as follows:

[0111] Historical data preprocessing: the historical running index data of resources, events and tasks collected by the controller and the Kafka state message queue are preprocessed (such as CPU, memory), mainly including extracting and processing time series features, processing missing values in time series, and normalizing original resource time series data to improve training speed and accuracy.

[0112] Constructing a data vector sequence: the preprocessed historical data is vector-constructed and converted into a data model suitable for LSTM model training and inference.

[0113] LSTM model offline training: set reasonable model hyperparameters such as learning rate, batch size and training period, then input the vector sequence corresponding to the resource data into the LSTM model in batches for iterative training, continuously update the corresponding weight parameters in reverse, and save the LSTM model after training.

[0114] LSTM model inference: based on the trained LSTM model, the application instance resource capacity and other index data at future time nodes are predicted, and the results are returned to the controller for resource scaling decisions.

[0115] The embodiment of the application mainly aims at a solution to high synchronization coupling and low resource utilization in batch task scheduling execution, and is intended to realize asynchronous decoupling of batch task scheduling and execution based on event driving, improve batch task request response efficiency, and improve system resource utilization; compared with distributed task scheduling mode, hardware scheduling optimization and other high-cost solutions for improving batch task request response efficiency in the industry, the application has lower implementation complexity.

[0116] The embodiment of the application adopts an LSTM model to process long-time sequence historical data of batch task resource requests, and performs early prediction on resource demand based on the trained LSTM model, and cooperates with a batch task scheduler to complete automatic expansion and contraction of capacity. Compared with traditional statistical machine learning and fixed quota expansion and contraction strategies, the application can not only better process and remember long-time sequence data, improve future resource expansion and contraction accuracy, but also reduce the number of reserved running resources and reduce resource waste.

[0117] The method combining the event driving mode with the resource capacity prediction based on the LSTM proposed in the embodiment of the application can not only reduce resource utilization caused by synchronization waiting, but also improve resource expansion and contraction efficiency with small model training cost, reduce idle waiting of reserved resources, avoid waste of reserved resources, and improve overall resource utilization.

[0118] Embodiment four

[0119] Figure 11 A structural schematic diagram of a batch task scheduling system provided by the embodiment five of the application is provided. As shown in the figure, Figure 11 The system includes an event publisher 310, an event subscriber 320, and a controller 330.

[0120] The event publisher 310 is used to receive a task request, and encapsulate the task request as a task event and publish the task event to an event message queue.

[0121] The event subscriber 320 is used to determine that the event message queue has a task event to be processed, and that there is an idle application instance resource, obtain the task event and the application instance resource.

[0122] The controller 330 is used to process the task event according to the application instance resource.

[0123] Optionally, the batch task scheduling system further comprises: a record writing module, configured to obtain an execution state of a task event and an event processing result, and write the execution state and the event processing result as an event record into a state message queue; and a resource management module, configured to manage application instance resources based on a resource prediction engine and the event record in the state message queue, wherein the resource prediction engine at least comprises a long short-term memory network prediction model trained and generated based on historical event records and historical application instance resources.

[0124] Optionally, the resource management module is specifically configured to read at least one event record in the state message queue at a time, and extract a preprocessing state, an event time sequence and a task sequence of each event record as to-be-processed data; perform time sequence feature extraction on the to-be-processed data to obtain a time sequence feature vector; call the resource prediction engine to process the time sequence feature vector to obtain an application instance resource prediction result; and perform resource adjustment on the application instance resources according to the application instance resource prediction result.

[0125] Optionally, the batch task scheduling system further comprises a resource initialization module, configured to obtain an application instance resource prediction result of the resource prediction engine, and initialize the application instance resources according to the application instance resource prediction result.

[0126] Optionally, the event publisher 310 further comprises: an event encapsulation unit, configured to receive a task request of at least one type of batch task, and split and encapsulate each task request into at least one task event; and an event caching unit, configured to send the task event to an event message queue for caching.

[0127] Optionally, the controller 330 further comprises: an event triggering unit, configured to initialize and start the application instance resources, and trigger the application instance resources to process the task event; a result obtaining unit, configured to determine that the task event is completely subscribed and consumed by the application instance resources, and obtain an execution state of the task event and an event processing result; and a state determining unit, configured to determine that the application instance resources complete processing of the task event, and identify the application instance resources as an idle state.

[0128] Optionally, the result obtaining unit is specifically configured to determine that each task event has been completely subscribed and consumed by the application instance resources, and obtain the event processing result of the task event within a threshold time; if the event processing result is accurately obtained, it is determined that the execution state is running successfully; if the event processing result is not accurately obtained, or the event processing result is not correctly obtained, it is determined that the execution state is running unsuccessfully.

[0129] Embodiment five

[0130] Embodiment five of the present application provides an electronic device for executing a batch task scheduling method, a computer readable storage medium and a computer program product.

[0131] Figure 12 A structural diagram of an electronic device that can be used to implement the batch task scheduling method of any of the present embodiments is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile systems such as personal digital processors, cellular telephones, smart phones, wearable electronic devices (e.g., headsets, glasses, watches, etc.), and other similar computing systems. The components shown in the electronic device of the present embodiments, their connections, and their functions, as well as the software implemented by the electronic device, are meant only to be examples and are not intended to limit the present embodiments described and / or claimed in this document.

[0132] As shown in Figure 12 The electronic device includes at least one processor 11 and a memory, such as a Read-Only Memory (ROM) 12, a Random Access Memory (RAM) 13, etc., connected to the at least one processor 11 in communication, where the memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer programs stored in the ROM 12 or loaded into the RAM 13 from the storage unit 18. In the RAM 13, various programs and data required for the operation of the electronic device can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An Input / Output (I / O) interface 15 is also connected to the bus 14.

[0133] A plurality of components in the electronic device are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc., an output unit 17, such as various types of displays, a speaker, etc., a storage unit 18, such as a magnetic disk, an optical disk, etc., and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device to exchange information / data with other electronic devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0134] The processor 11 can be various general and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit, a graphics processing unit, various special-purpose artificial intelligence computing chips, various processors running machine learning model algorithms, a digital signal processor, and any appropriate processor, controller, microcontroller, etc. The processor 11 performs various methods and processes described above, such as the batch task scheduling method.

[0135] In some embodiments, the batch task scheduling method can be implemented as a computer program tangibly embodied in a computer readable storage medium, e.g., storage unit 18. In some embodiments, parts or all of the computer program can be loaded and / or installed onto the electronic device via, e.g., ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the batch task scheduling method can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform the batch task scheduling method by any other suitable means, e.g., with the aid of firmware.

[0136] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, specially designed application specific integrated circuits, application specific standard products, chips, microprocessors, computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input system, and at least one output system.

[0137] Computer programs implementing methods of embodiments of the application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing system to produce a machine, such that the computer program, when executed, enables the system to implement the functions / acts specified in the flowcharts and / or block diagrams. The computer program can be executed entirely on a machine, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0138] In the context of embodiments of the present application, a computer- readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, system, or electronic device. A computer-readable storage medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, system, or electronic device, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of a machine-readable storage medium will include one or more lines of electronic communication, portable computer disks, hard disk drives, RAM, ROM, erasable programmable read-only memory (EPROM or Flash memory), optical fibers, compact disc read-only memories (CD-ROMs), optical storage electronic devices, magnetic storage electronic devices, or any suitable combination of the foregoing.

[0139] To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having a display system (e.g., a cathode ray tube or a liquid crystal display monitor) for displaying information to the user and a keyboard and a pointing system (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of systems can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0140] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0141] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system, to solve the defects of large management difficulty and weak business scalability in traditional physical host and virtual private server service.

[0142] It should be understood that the various forms of flow shown above can be reordered, added to, or have steps deleted. For example, the steps described in the present application can be performed in parallel, in series, or in a different order, as long as the desired results of the technical solutions of the present application can be achieved, and this is not limited herein.

[0143] The above detailed description does not constitute a limitation on the protection scope of the embodiments of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A batch task scheduling method, characterized in that, The method comprises: receiving a task request and encapsulating the task request as a task event to publish to an event message queue; determining that the event message queue has the task event to be processed and that there is an application instance resource in an idle state, obtaining the task event and the application instance resource; processing the task event according to the application instance resource.

2. The method of claim 1, wherein, Further comprising: obtaining an execution state and an event processing result of the task event and writing the execution state and the event processing result as an event record to a state message queue; managing application instance resources based on a resource prediction engine and the event record in the state message queue, wherein the resource prediction engine at least comprises a long short-term memory network prediction model trained and generated based on historical event records and historical application instance resources.

3. The method of claim 2, wherein, The management of the application instance resources based on the resource prediction engine and the event record in the state message queue comprises: timely reading at least one event record in the state message queue and extracting a preprocessing state, an event time sequence and a task sequence of each event record as processing data; performing time sequence feature extraction on each processing data to obtain a time sequence feature vector; calling the resource prediction engine to process the time sequence feature vector to obtain an application instance resource prediction result; performing resource adjustment on the application instance resource according to the application instance resource prediction result.

4. The method of claim 1, wherein, The receiving of the task request and the encapsulation of the task request as a task event to publish to an event message queue comprises: receiving the task request of at least one batch task and splitting and encapsulating each task request as at least one task event; sending the task event to the event message queue for caching.

5. The method of claim 1, wherein, The processing of the task event according to the application instance resource comprises: initializing and starting the application instance resource to trigger the application instance resource to process the task event; determining that the task event is completely subscribed and consumed by the application instance resource, obtaining an execution state and an event processing result of the task event; determining that the application instance resource completes the processing of the task event and marking the application instance resource as an idle state.

6. The method of claim 5, wherein, The determination that the task event is completely subscribed and consumed by the application instance resource and the obtaining of the execution state and the event processing result of the task event comprise: determining that each task event has been completely subscribed and consumed by the application instance resource, and obtaining the event processing result of the task event within a threshold time; if the event processing result is accurately obtained, determining that the execution state is running successfully; if the event processing result is not accurately obtained or the event processing result is not correctly obtained, determining that the execution state is running unsuccessfully.

7. The method of claim 2, wherein, Further comprising: obtaining an application instance resource prediction result of the resource prediction engine and initializing the application instance resource according to the application instance resource prediction result.

8. A batch task scheduling system, characterized by, The system comprises: an event publisher configured to receive a task request and encapsulate the task request as a task event to publish to an event message queue; An event subscriber is configured to determine that the event message queue has the task event to be processed, and there is an application instance resource in an idle state, acquire the task event and the application instance resource; A controller is configured to process the task event according to the application instance resource.

9. An electronic device, comprising: The electronic device comprises: at least one processor; and a memory connected with the at least one processor in communication; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the batch task scheduling method in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions for enabling the processor to execute the batch task scheduling method in any one of claims 1-7 when executed.

Images