Event-driven big data substrate task scheduling method and device

Abstract

This invention discloses an event-driven big data infrastructure task scheduling method and apparatus, relating to the field of task scheduling or other related technical fields. The method includes: obtaining the execution status of a target big data infrastructure task from a task execution engine; encapsulating the execution status of the target big data infrastructure task into an event object through an event producer embedded in the task execution engine, and publishing the event object to an event bus; determining the subscription information of the target big data infrastructure task through the event bus, and routing the event object to a target event consumer based on the subscription information; and scheduling downstream tasks corresponding to the target big data infrastructure task by the target event consumer based on the event object. This invention solves the technical problem in related technologies where scheduling is performed by polling and checking the task execution status and then scheduling the task accordingly, resulting in scheduling delays and low scheduling efficiency.

Description

Technical Field

[0001] This invention relates to the field of task scheduling or other related technical fields, and more specifically, to an event-driven big data-based task scheduling method and apparatus. Background Technology

[0002] In the field of big data processing, the automated orchestration and efficient scheduling of data processing tasks are core elements to ensure data timeliness, system stability, and resource utilization. Currently, mainstream big data infrastructure platforms generally adopt task scheduling systems based on directed acyclic graphs (DAGs) to manage thousands of periodic or event-triggered data processing tasks such as data processing, data modeling, and report generation.

[0003] In related technologies, the core architecture of scheduling systems generally adopts a centralized scheduler, metadata storage, and static DAG orchestration model. Users define the dependencies between tasks through a graphical interface or configuration files, forming a static DAG structure. The scheduler, as the sole control center of the system, periodically polls the metadata database to check whether all upstream dependencies of each task have been completed. Once it finds that all parent tasks of a task are in a "successful" state, the scheduler marks the task as "schedulable" and allocates resources for execution. This mechanism relies on database polling operations at fixed time intervals (such as 30 seconds or 1 minute) to achieve "passive discovery" and "delayed triggering" of task status.

[0004] This static, passive scheduling mechanism reveals several significant technical flaws in large-scale production environments. First, because task triggering relies entirely on the scheduler's polling cycle, the initiation of downstream tasks inevitably involves inherent scheduling delays, leading to low scheduling efficiency and impacting subsequent business decisions. For example, in a data processing pipeline with a three-level chain of dependencies, if the polling interval is 1 minute, even if the upstream task completes within 5 seconds, the downstream task will still need to wait at least 25 to 55 seconds before being detected and started. In scenarios with extremely high data timeliness requirements, such as real-time analytics, financial risk control, and advertising, this delay will directly lead to delayed business decisions.

[0005] Secondly, as the scale of tasks grows (thousands or even tens of thousands of tasks running concurrently), the scheduler needs to perform high-frequency, full-scale status scans of the metadata database, leading to a sharp increase in database load. Each polling operation requires multi-table joins (such as the task status table, dependency table, and task definition table), generating significant I / O and CPU consumption. During peak periods, this polling behavior can lead to database connection pool exhaustion and slow query backlog, thus creating a systemic bottleneck and forcing the operations team to expand the database cluster, resulting in unnecessary hardware and maintenance costs.

[0006] There is currently no effective solution to the above problems. Summary of the Invention

[0007] This invention provides an event-driven big data infrastructure task scheduling method and apparatus to at least solve the technical problem in related technologies where scheduling is delayed and results in low scheduling efficiency due to polling the task execution status and scheduling based on that status.

[0008] According to one aspect of the present invention, an event-driven big data infrastructure task scheduling method is provided, comprising: obtaining the execution status of a target big data infrastructure task from a task execution engine; encapsulating the execution status of the target big data infrastructure task into an event object through an event producer embedded in the task execution engine, and publishing the event object to an event bus; determining the subscription information of the target big data infrastructure task through the event bus, and routing the event object to a target event consumer based on the subscription information; and having the target event consumer schedule downstream tasks corresponding to the target big data infrastructure task based on the event object.

[0009] Furthermore, before obtaining the execution status of the target big data foundation task from the task execution engine, the method further includes: identifying the dependencies between the big data foundation tasks; and registering the dependencies between the big data foundation tasks to the task orchestration center.

[0010] Furthermore, the step of scheduling downstream tasks corresponding to the target big data foundation task by the target event consumer based on the event object includes: the target event consumer parsing the event object to obtain a parsing result; if the parsing result indicates that the execution status of the target big data foundation task is successful, querying the task orchestration center based on the target big data foundation task to determine the downstream tasks that have a dependency relationship with the target big data foundation task; the target event consumer generating an execution instance of the downstream task and submitting the execution instance to the task execution engine.

[0011] Further, the step of generating an execution instance of the downstream task from the target event consumption includes: obtaining the execution information of the target big data foundation task, wherein the execution information includes at least one of the following: output path, processed data volume, metadata identifier, and execution log summary; obtaining the execution logic, resource requirement information, and parameter template of the downstream task, and generating the execution parameters of the downstream task in combination with the execution information; and generating an execution instance of the downstream task from the target event consumption based on the execution parameters.

[0012] Furthermore, after obtaining the parsing result, the method further includes: if the parsing result indicates that the execution status of the target big data foundation task is a failure state, invoking a preset processing strategy based on the priority of the target big data foundation task; and having the event consumer process the target big data foundation task in the failure state based on the preset processing strategy.

[0013] Furthermore, before obtaining the execution status of the target big data foundation task from the task execution engine, the method further includes: embedding the event producer into the task execution engine in the form of a software development kit; and creating an access interface for the event producer, wherein the access interface is used to capture the execution status of the big data foundation task when the big data foundation task is completed.

[0014] Furthermore, the event-driven big data infrastructure task scheduling also includes: calculating the hash value of the event object corresponding to each big data infrastructure task; and using the hash value of the event object to partition the event bus.

[0015] According to another aspect of the present invention, an event-driven big data infrastructure task scheduling device is also provided, comprising: an acquisition unit, configured to acquire the execution status of a target big data infrastructure task from a task execution engine; a publishing unit, configured to encapsulate the execution status of the target big data infrastructure task into an event object through an event producer embedded in the task execution engine, and publish the event object to an event bus; a routing unit, configured to determine the subscription information of the target big data infrastructure task through the event bus, and route the event object to a target event consumer based on the subscription information; and a scheduling unit, configured to have the target event consumer schedule downstream tasks corresponding to the target big data infrastructure task based on the event object.

[0016] Furthermore, the event-driven big data infrastructure task scheduling also includes: a first identification module for identifying the dependencies between the big data infrastructure tasks; and a first registration module for registering the dependencies between the big data infrastructure tasks to the task orchestration center.

[0017] Furthermore, the scheduling unit includes: a first parsing module, used by the target event consumer to parse the event object and obtain a parsing result; a first determining module, used to query the task orchestration center based on the target big data base task and determine downstream tasks that have a dependency relationship with the target big data base task when the parsing result indicates that the execution status of the target big data base task is successful; and a first generating module, used by the target event consumer to generate an execution instance of the downstream task and submit the execution instance to the task execution engine.

[0018] Further, the first generation module includes: a first acquisition submodule, used to acquire the execution information of the target big data foundation task, wherein the execution information includes at least one of the following: output path, processed data volume, metadata identifier, and execution log summary; a first generation submodule, used to acquire the execution logic, resource requirement information, and parameter template of the downstream task, and generate the execution parameters of the downstream task in combination with the execution information; and a second generation submodule, used by the target event consumer to generate an execution instance of the downstream task based on the execution parameters.

[0019] Furthermore, the event-driven big data infrastructure task scheduling also includes: a first invocation module, used to invoke a preset processing strategy based on the priority of the target big data infrastructure task when the parsing result indicates that the execution status of the target big data infrastructure task is a failure state; and a first processing module, used by the event consumer to process the target big data infrastructure task in the failure state based on the preset processing strategy.

[0020] Furthermore, the event-driven big data infrastructure task scheduling also includes: a first embedding module, used to embed the event producer into the task execution engine in the form of a software development kit; and a first creation module, used to create an access interface for the event producer, wherein the access interface is used to capture the execution status of the big data infrastructure task when the big data infrastructure task is completed.

[0021] Furthermore, the event-driven big data infrastructure task scheduling also includes: a first calculation module, used to calculate the hash value of the event object corresponding to each big data infrastructure task; and a first partitioning module, used to partition the event bus using the hash value of the event object.

[0022] According to another aspect of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to execute any of the above-described event-driven big data infrastructure task scheduling methods.

[0023] According to another aspect of the present invention, an electronic device is also provided, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement any of the above-described event-driven big data infrastructure task scheduling methods.

[0024] In this application, the following steps are taken: the execution status of the target big data foundation task is obtained from the task execution engine; the execution status of the target big data foundation task is encapsulated into an event object by the event producer embedded in the task execution engine; the event object is published to the event bus; the subscription information of the target big data foundation task is determined by the event bus; and the event object is routed to the target event consumer based on the subscription information; and the target event consumer schedules the downstream tasks corresponding to the target big data foundation task based on the event object.

[0025] In this application, an event producer is embedded within the task execution engine for state awareness. Within the millisecond-level time window of task instance completion, an event object is actively generated and published to the event bus for push. Downstream tasks no longer passively wait for the scheduler's periodic checks, but are activated the moment the upstream task's state changes. The execution of downstream tasks is automatically scheduled based on the event consumer's subscription to the event object, avoiding passive scheduling, reducing scheduling latency, and improving scheduling efficiency. This solves the technical problem in related technologies where scheduling latency exists and results in low scheduling efficiency due to the method of polling and checking the task execution status by the scheduler and scheduling tasks accordingly. Attached Figure Description

[0026] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0027] Figure 1 This is a flowchart of an optional event-driven big data infrastructure task scheduling method according to an embodiment of the present invention;

[0028] Figure 2 This is an optional event-driven big data-based task scheduling system architecture diagram according to an embodiment of the present invention;

[0029] Figure 3 This is a schematic diagram of an optional event-driven big data infrastructure task scheduling process according to an embodiment of the present invention;

[0030] Figure 4 This is a schematic diagram of an optional event-driven big data infrastructure task scheduling device according to an embodiment of the present invention;

[0031] Figure 5 This is a hardware structure block diagram of an electronic device (or mobile device) that executes an event-driven big data infrastructure task scheduling method according to an embodiment of the present invention. Detailed Implementation

[0032] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0033] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0034] It should be noted that the event-driven big data platform task scheduling method and apparatus in this application can be used in the field of task scheduling when upstream and downstream tasks are scheduled based on event-driven dependencies, and can also be used in any field other than the field of blockchain technology when upstream and downstream tasks are scheduled based on event-driven dependencies. This application does not limit the application field of the event-driven big data platform task scheduling method and apparatus.

[0035] It should be noted that the relevant information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, and displayed data) involved in this application are information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of such data all comply with the relevant laws, regulations, and standards of the relevant regions, necessary confidentiality measures have been taken, and they do not violate public order and good morals. Corresponding operation entry points are provided for users to choose to authorize or refuse. For example, this system has interfaces with relevant users or organizations. Before obtaining relevant information, a request to obtain the information needs to be sent to the aforementioned user or organization through the interface, and the relevant information is obtained only after receiving consent from the aforementioned user or organization.

[0036] It should be noted that in this application, when collecting and analyzing customer information, users are provided with corresponding operation entry points to choose whether to agree to or reject the automated decision-making results; if the user chooses to reject, the process will proceed to the expert decision-making process.

[0037] The following embodiments of the present invention can be applied to various event-driven big data-based task scheduling systems / applications / devices. The present invention embeds a state-aware component within the task execution engine, proactively generating and publishing current state change information within a millisecond-level time window after a task instance completes. Downstream tasks no longer passively wait for periodic checks by the scheduler, but are activated the moment the upstream task's state changes.

[0038] This invention eliminates the need for polling the metadata database by transferring the responsibility of status querying from a central scheduler to an event publish / subscribe mechanism. Changes in task status are no longer written to a query-intensive database as a "signal source," but are instead pushed directly through a lightweight message channel.

[0039] This invention introduces an independent event distribution middleware to physically and logically decouple the three major functional modules: "state awareness" (task execution engine), "scheduling decision" (event consumer), and "metadata management" (task orchestration center). This breaks single-point dependencies and enhances system availability and elastic scalability.

[0040] The present invention will now be described in detail with reference to various embodiments.

[0041] Example 1

[0042] According to an embodiment of the present invention, an embodiment of an event-driven big data infrastructure task scheduling method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0043] Figure 1 This is a flowchart of an optional event-driven big data infrastructure task scheduling method according to an embodiment of the present invention, such as... Figure 1 As shown, the method includes the following steps:

[0044] Furthermore, before obtaining the execution status of the target big data foundation task from the task execution engine, the process also includes: identifying the dependencies between big data foundation tasks; and registering the dependencies between big data foundation tasks to the task orchestration center.

[0045] Specifically, after receiving the directed acyclic graph (DAG) definition submitted by the user through the graphical interface or API of the task orchestration center, the DAG resolver module within the task orchestration center automatically resolves the pre- and post-order constraints between task nodes. For example, if the user defines that task B can only start after task A is completed, the system identifies "task A → task B" as a dependency edge. This dependency relationship belongs to the task-level control flow constraint, rather than the data flow dependency.

[0046] After identifying dependencies, the subscription relationship manager in the task orchestration center converts the dependency into an "event subscription rule" and persists it in its internal data storage layer. Specifically, the system writes a record to the event subscription table, whose fields may include: subscriber_task_id: pointing to the downstream task (e.g., "Task B"); event_topic: pointing to the "success" event topic of the upstream task (e.g., "Task A: SUCCESS"); dag_id: identifying the workflow to which it belongs; is_active: marking the subscription relationship as enabled (value 1).

[0047] The aforementioned registration mechanism explicitly abstracts the control logic of "task B is triggered after task A is completed" in the traditional DAG into an event-driven mechanism of "task B subscribes to the SUCCESS event of task A." This transforms the dependencies originally implicit in the DAG structure into structured metadata that can be identified and queried by event consumers. The dependency registration mechanism establishes a queryable mapping relationship between the driving logic of tasks in advance, ensuring that after subsequent task execution, event consumers can accurately locate which downstream tasks should be activated based on this registration information. This avoids complex and inefficient dependency recalculations or full table scans during event triggering. This operation occurs before any task is actually scheduled and executed, belonging to the initialization configuration phase of the scheduling process, and its result provides basic data support for the subsequent event-driven triggering mechanism.

[0048] Furthermore, before obtaining the execution status of the target big data foundation task from the task execution engine, the process also includes: embedding the event producer into the task execution engine in the form of a software development kit; and creating an access interface for the event producer, wherein the access interface is used to capture the execution status of the big data foundation task when the big data foundation task is completed.

[0049] Specifically, the event producer is a perception component built into the task execution engine. Its responsibility is to automatically generate and output standardized state change information when key lifecycle nodes (such as successful task completion or execution failure) occur in the target big data infrastructure task. The event producer is encapsulated as a lightweight, cross-language compatible software development kit (SDK) and embedded into the task execution engine. This SDK does not depend on a specific task computing framework and can be integrated into the execution code of various big data infrastructure tasks.

[0050] Subsequently, a set of standardized calling methods is defined for this SDK, serving as the sole functional entry point exposed by the event producer. The access interface includes, but is not limited to: `emitSuccessEvent(taskId:String,payload:Map)`<String,Object> `emitFailedEvent(taskId: String, errorMessage:String)`: Used to report the status when a task is successfully completed; `emitFailedEvent(taskId: String, errorMessage:String)`: Used to report failure information when a task terminates abnormally; `emitStartedEvent(taskId:String)`: Used to report the execution start signal when a task starts.

[0051] By establishing a standardized and unified status reporting capability within the task execution engine, any big data infrastructure task running within this engine can automatically, reliably, and with low latency send its execution status as an event upon completion by calling a unified access interface. This provides accurate and timely trigger signals for subsequent event-driven scheduling. This mechanism transforms status acquisition from external polling to proactive internal reporting, fundamentally improving the real-time performance and reliability of status collection.

[0052] Step S101: Obtain the execution status of the target big data foundation task from the task execution engine.

[0053] In step S101 above, a big data infrastructure task refers to a schedulable, monitorable, and reliable computing unit deployed on the big data infrastructure platform to complete the core data processing workflow. Its execution involves core data processing operations such as extracting, transforming, loading, aggregating, modeling, analyzing, or outputting massive amounts of data. A target big data infrastructure task refers to any big data infrastructure task within the data processing workflow.

[0054] The task execution engine refers to the underlying computing framework agent system responsible for the actual execution of the target big data infrastructure tasks. This engine receives execution instructions from the scheduling module, allocates computing resources (such as CPU and memory), starts task containers or processes, and monitors the lifecycle state of the task during execution. The task execution engine embeds an event producer, possessing native awareness of the task execution state. Its internal state machine can accurately identify different stages of the task, such as "running," "successfully completed," "failed to terminate," or "cancelled."

[0055] During task execution, the system actively retrieves the latest running status information of the target task through the status query interface provided by the event producer. This interface is a standard API exposed by the task execution engine, which supports precise queries by task instance ID (such as task_instance_20250405_001). The returned content may include: status value (such as SUCCESS, FAILED, RUNNING), end timestamp, exit code, exception information (such as error stack, log summary), and resource consumption indicators (such as CPU utilization, peak memory usage).

[0056] Step S102: The execution status of the target big data foundation task is encapsulated into an event object by the event producer embedded in the task execution engine, and the event object is published to the event bus.

[0057] In step S102 above, when the target big data foundation task finishes execution, the task execution engine calls its embedded event producer to encapsulate the execution status and associated metadata obtained in the previous step into a complete event object according to the above structure. Subsequently, the event producer sends the event object to the specified topic of the event bus through the configured communication channel.

[0058] Through the above steps, the discrete state changes of task execution are transformed into a standardized, asynchronously transmittable, and system-subscribing communication signal. By encapsulating "state" as "event," the proactive reporting of task execution results is achieved, rather than relying on external system polling, thus eliminating the latency and resource overhead of state acquisition. This mechanism is the core component for implementing "event-driven scheduling," providing a structured, traceable, and consumable data carrier for subsequent event bus routing and automatic triggering of downstream tasks.

[0059] Furthermore, the event-driven big data infrastructure task scheduling also includes: calculating the hash value of the event object corresponding to each big data infrastructure task; and using the hash value of the event object to partition the event bus.

[0060] Specifically, an event object refers to a standardized structured data packet generated by the event producer embedded in the task execution engine, used to characterize the execution status of the target big data infrastructure task. Its content includes fields such as eventId, topic, sourceTaskId, timestamp, and payload, used to transmit lifecycle information such as task completion and failure in the event bus. Based on the key field in the event object, the system strives to generate a fixed-length numerical digest using a deterministic hash algorithm (such as MurmurHash3 or SHA-256) for the sourceTaskId (i.e., the unique identifier of the target big data infrastructure task that generated the event). For example, calculating the hash value for sourceTaskId = "task_user_profile_agg" yields 0x1a3f8c2b. This hash value is only related to the task identifier and is independent of the event time, payload content, and status type, ensuring that all events of the same task always map to the same value in the hash operation.

[0061] An event bus is a high-throughput, persistent message middleware that supports a publish / subscribe model. Internally, it uses a partitioning mechanism to ensure parallel processing and message ordering. Each topic can be divided into multiple partitions, and messages within each partition are strictly ordered according to their write order. When an event producer publishes an event object to the event bus, the system no longer uses a random or round-robin strategy to allocate partitions. Instead, it uses the calculated sourceTaskId hash value modulo the total number of partitions on the event bus to determine the specific partition where the event object should be written. This mechanism ensures that the same event object can be routed to the same partition.

[0062] In event-driven scheduling, a task may generate multiple events (e.g., FAILED → RETRYING → SUCCESS) due to retries, exception recovery, or asynchronous callbacks. If these events are distributed across different partitions, event consumers will not be able to process them in chronological order, potentially leading to misjudgments of task status or triggering incorrect dependencies. By using hash partitioning based on sourceTaskId, the global ordering of the single-task event stream is ensured, providing a reliable input sequence for subsequent state verification, dependency determination, and retry control logic of the event consumer.

[0063] Step S103: Determine the subscription information of the target big data foundation task through the event bus, and route the event object to the target event consumer based on the subscription information.

[0064] In step S103 above, subscription information refers to the event-task dependency mapping relationship that is pre-registered and persisted in the task orchestration and registry center. This information explicitly records which event consumers (e.g., task_aggregate_user) subscribe to the event topic (e.g., task_ingest_log: SUCCESS). The subscription information is dynamically loaded and cached in memory by the event bus during deployment or runtime to implement event routing decisions. The target event consumer refers to the resident service component responsible for listening to and processing specific events. It registers its subscription interest with the event bus when the system starts. Each event consumer instance is typically bound to one or more event topics to trigger the scheduling and execution of corresponding downstream tasks.

[0065] Step S104: The target event consumer schedules the downstream tasks corresponding to the target big data foundation task based on the event object.

[0066] In step S104 above, a downstream task refers to a subsequent task in a directed acyclic graph (DAG) that is activated due to its dependence on the completion of the target big data foundation task. A target big data foundation task may correspond to multiple downstream tasks, or it may have only one. After receiving an event object, the target event consumer first parses its topic field to identify the target event consumer corresponding to the event. Then, based on the system's preset task dependency model, the target event consumer searches for all downstream tasks defined in the workflow for the target big data foundation task. If the target big data foundation task is task_aggregate_user, it may have two downstream tasks configured in the DAG definition: task_report_generation and task_data_quality_alert. The target event consumer obtains the metadata of these downstream tasks, including task ID, execution type, resource requirements, and execution command, through a local cache or a lightweight configuration library (not a real-time query registry). Subsequently, the target event consumer creates a scheduling request instance for each downstream task, thereby transforming the event object into a task scheduling start instruction.

[0067] Furthermore, the steps for the target event consumer to schedule downstream tasks corresponding to the target big data foundation task based on the event object include: the target event consumer parses the event object to obtain the parsing result; if the parsing result indicates that the execution status of the target big data foundation task is successful, the target event consumer queries the task orchestration center based on the target big data foundation task to determine the downstream tasks that have a dependency relationship with the target big data foundation task; the target event consumer generates execution instances of the downstream tasks and submits the execution instances to the task execution engine.

[0068] Specifically, after receiving the event object, the target event consumer first performs a parsing operation: deserializing the JSON format of the event object into an in-memory object, extracting the topic field, separating the sourceTaskId and status, and obtaining the parsing result. If the value of status in the parsing result is "SUCCESS", the subsequent scheduling conditions are met; if it is "FAILED" or "CANCELLED", this step is skipped, and downstream tasks are not triggered. Under the premise of meeting the "success status" condition, the target event consumer uses sourceTaskId as the query key to send a query request to the task orchestration center to obtain a list of all downstream task IDs that have subscribed to the "SUCCESS" event of this task. For example, querying task_ingest_log:SUCCESS returns the result ["task_aggregate_user", "task_data_quality_check"]. Subsequently, the target event consumer generates an execution instance for each downstream task in the list. Finally, the target event consumer submits the generated execution instances to the task execution engine in batches by calling the standardized API (such as REST interface or RPC service) provided by the task execution engine, completing the transmission of scheduling instructions. After receiving the request, the task execution engine will launch the corresponding task process according to the request of the execution instance.

[0069] Furthermore, the steps for generating execution instances of downstream tasks from target event consumption include: obtaining execution information of the target big data foundation task, wherein the execution information includes at least one of the following: output path, amount of data processed, metadata identifier, and execution log summary; obtaining the execution logic, resource requirement information, and parameter template of the downstream task, and generating execution parameters of the downstream task in combination with the execution information; and generating execution instances of the downstream task from the target event consumption based on the execution parameters.

[0070] Specifically, after confirming that the target big data foundation task execution status is "successful," the target event consumer first extracts execution information from the payload field of the event object, including fields such as output path, processed data volume, metadata identifier, and execution log summary. Then, based on the unique identifier of the downstream task, the target event consumer obtains its execution logic, resource requirements (e.g., 4 CPUs, 8GB memory), and parameter template from the task orchestration center. Next, the target event consumer dynamically replaces the values ​​in the extracted execution information according to the keys defined in the template to generate complete execution parameters. For example, {output_path} is replaced with the actual output path / data / raw / log / 20250405, {min_record_threshold} is replaced with the processed data volume 895200, and {metadata_id} is replaced with the metadata identifier v20250405-001. Finally, the target event consumer packages the generated execution parameters, execution logic, resource requirements, and metadata such as task ID and trigger event ID to construct a complete downstream task execution instance, which can then be submitted to the task execution engine.

[0071] By carrying execution information through event objects and combining them with parameter templates, a seamless, dynamic, and non-intrusive binding from upstream output to downstream input is achieved, enabling the task flow to be adaptive and context-aware.

[0072] Furthermore, after obtaining the parsing result, the process also includes: if the parsing result indicates that the execution status of the target big data base task is in a failed state, invoking a preset processing strategy based on the priority of the target big data base task; and having the event consumer process the failed target big data base task based on the preset processing strategy.

[0073] Specifically, when the target event consumer parses the event object, if the status in the parsing result is "FAILED", the failure handling process is triggered. At this time, the event consumer first queries the task orchestration center to obtain the priority attribute of the target big data infrastructure task. Subsequently, based on this priority, the event consumer looks up the corresponding set of processing strategies in the preset strategy mapping table. For example, for a task with a priority of "high", the preset processing strategy is: automatically retry twice, with an interval of 5 minutes between each retry; if the retry still fails, an alarm notification is triggered to the operation and maintenance system, and the subsequent task scheduling of the entire DAG is blocked.

[0074] The event consumer then executes the following strategy sequence: First, it stores the task identifier and retry counter in its local state cache; if this is the first failure, it schedules a retry after 5 minutes (triggered by an internal timer or message queue); if this is the second failure, it sends a structured message containing task_id, error_code, and log_snippet to the alarm system; at the same time, it sends a "stop DAG" instruction to the task orchestration center, marking the workflow as a "failed final state".

[0075] If the target big data infrastructure task has a "low" priority, then only the "logging" strategy will be executed, without sending alarms, retrying, or affecting other tasks.

[0076] Through the above steps, differentiated and intelligent responses to failed tasks are achieved, avoiding a one-size-fits-all approach to all failed tasks. By binding task priorities with preset strategies, the system can automatically select the most appropriate response based on business importance, ensuring the stability of critical links while reducing the interference of non-critical task failures on the overall system.

[0077] Through the above steps, the execution status of the target big data foundation task is obtained from the task execution engine. The execution status of the target big data foundation task is encapsulated into an event object by the event producer embedded in the task execution engine, and the event object is published to the event bus. The subscription information of the target big data foundation task is determined by the event bus, and the event object is routed to the target event consumer based on the subscription information. The target event consumer then schedules the downstream tasks corresponding to the target big data foundation task based on the event object.

[0078] In this embodiment, an event producer is embedded within the task execution engine for state awareness. Within the millisecond-level time window of task instance completion, an event object is actively generated and published to the event bus for push. Downstream tasks no longer passively wait for the scheduler's periodic checks, but are activated the moment the upstream task's state changes. The execution of downstream tasks is automatically scheduled based on the event consumer's subscription to the event object, avoiding passive scheduling, reducing scheduling latency, and improving scheduling efficiency. This solves the technical problem in related technologies where scheduling latency exists and scheduling efficiency is low due to the scheduler polling and checking the task execution status.

[0079] The following describes in detail another optional implementation method.

[0080] Figure 2 This is an optional event-driven big data-based task scheduling system architecture diagram according to an embodiment of the present invention, such as... Figure 2As shown, the event-driven big data-based task scheduling system includes: a task orchestration and registration center, event producers, an event bus, event consumers, and a task execution engine. Specifically,

[0081] The Task Orchestration and Registry Center serves as the interface and repository for users to define tasks. Users orchestrate task flows using a Directed Acyclic Graph (DAG), defining the execution logic and event dependencies for each task. The Task Orchestration and Registry Center is not merely a traditional task manager, but also a registry for event listening relationships. When a downstream task (such as Task B) is defined as dependent on an upstream task (such as Task A), the system automatically records a rule in the registry center: "Task B subscribes to Task A's 'SUCCESS' event."

[0082] Event producers are functional components embedded in the task execution engine. They are used to generate corresponding events at key nodes in the task lifecycle. When a task instance in the task execution engine reaches a specific state (such as "start", "success", "failure"), the event producer captures this state, encapsulates it into a standardized event object, and then publishes it to the event bus. For example, when task A completes successfully, its embedded event producer will publish an event with the topic TASK_A:SUCCESS.

[0083] The event bus is a highly available message communication middleware that supports a publish / subscribe model. It receives event objects from all event producers and, based on predefined subscription relationships in the task orchestration and registry center, precisely pushes these objects to interested subscribers (i.e., event consumers). This ensures reliable event delivery and system decoupling. The event bus employs a hash partitioning strategy, partitioning the event bus based on the hash value of the sourceTaskId. This guarantees that events emitted by the same upstream task are always sent to the same partition in order, thus being processed by the same event consumer instance, maintaining event ordering.

[0084] Event consumers exist as one or more resident services, pre-subscribing to the event types they are interested in from the event bus. When an event consumer receives an event pushed by the event bus (e.g., TASK_A: SUCCESS), it is immediately activated. It then queries the task orchestration and registry center to find all tasks (e.g., task B) that have subscribed to this event. Finally, it issues an instruction to the task execution engine to execute task B. This process is triggered instantaneously, without polling.

[0085] For example, when an event consumer receives an event related to `user_click_ingest: SUCCESS`, its internal workflow is as follows: Deserialization and validation: The JSON message is parsed into an in-memory Event object; Subscription relationship query: A query is sent to the registry center: "Which tasks have subscribed to `user_click_ingest: SUCCESS`?" The registry center returns a list: ["user_click_aggregate"]; Task definition retrieval: The complete definition of the task `user_click_aggregate` is retrieved; Dynamic parameter injection: The event consumer dynamically sets `properties.outputPath` (i.e., the output path of the upstream task) from the event as a parameter named `input_path` to the `user_click_aggregate` task. This eliminates the need for hard-coding the input path for the aggregation task, achieving data transfer and decoupling between tasks. Task submission: The event consumer submits the enhanced task definition to the task execution engine for execution by calling the Apache Livy REST API.

[0086] The task execution engine is responsible for executing specific task instances. It receives execution commands from event consumers, allocates computing resources (such as CPU and memory), launches task containers, and executes them. Meanwhile, its embedded event producers provide real-time feedback on the task execution status, forming a closed loop.

[0087] Figure 3 This is a schematic diagram of an optional event-driven big data infrastructure task scheduling process according to an embodiment of the present invention, such as... Figure 3 As shown, the event-driven big data infrastructure task scheduling process includes:

[0088] Step 1: Users orchestrate task DAGs and register event subscription relationships;

[0089] Users define a DAG workflow through the task orchestration and registry center's UI or API. For example, task B might depend on the successful completion of task A. The system translates this dependency into an event subscription rule: "Task B subscribes to the event with the topic TASK_A:SUCCESS," persists this rule to the registry center, and notifies the event bus.

[0090] Step two, the upstream task is scheduled for execution;

[0091] The system starts or is triggered by an external timer to initiate a task in the DAG (a task without a parent node, such as task A). The event consumer (or the initial scheduler) submits task A to the task execution engine.

[0092] Step 3: Obtain the execution status;

[0093] Step four: The embedded event producer generates standardized event objects;

[0094] The task execution engine begins running task A. When task A finishes running (whether successfully or not), its embedded event producer immediately takes action, obtains the task execution status, encapsulates the event, and thus creates a standard event object.

[0095] Step 5: Publish the event object to the event bus;

[0096] The event bus receives a TASK_A:SUCCESS event from task A. Based on pre-established subscription relationships, it discovers that the event consumer has subscribed to this topic. Therefore, the event bus immediately pushes the event to the corresponding event consumer.

[0097] Step six: The event consumer receives the event object;

[0098] Step 7: Determine the downstream tasks based on the event object;

[0099] Step 8: Downstream task scheduling;

[0100] Upon receiving the event, the event consumer is immediately activated. By parsing the event content, it learns that "Task A has succeeded." It then queries the task orchestration and registry center: "Which tasks have subscribed to the TASK_A:SUCCESS event?", and receives the downstream task query results returned by the registry center. The event consumer then generates execution instances of the downstream tasks and submits them to the task execution engine.

[0101] If task A fails, its event producer will publish a TASK_A:FAILED event. Upon receiving this event object, the event consumer can process it according to a predefined strategy, such as: triggering a specified error handling process task; directly marking the entire DAG as failed and notifying the user; or retrying (in conjunction with retry counter logic).

[0102] Step nine: The execution engine performs the task.

[0103] The task execution engine receives and executes task B. After task B completes, its embedded event producer will also generate and publish new events (e.g., TASK_B: SUCCESS), thereby continuing to drive task C, task D, and so on, until the entire DAG is completed.

[0104] This invention embeds a state-aware component within the task execution engine, proactively generating and publishing current state change information within a millisecond-level time window after a task instance completes. Downstream tasks no longer passively wait for periodic checks by the scheduler, but are activated the moment an upstream task's state changes.

[0105] This invention eliminates the need for polling the metadata database by transferring the responsibility of status querying from the central scheduler to an event publish / subscribe mechanism. Changes in task status are no longer written to a query-intensive database as a "signal source," but are instead pushed directly through a lightweight message channel.

[0106] This invention introduces an independent event distribution middleware to physically and logically decouple the three major functional modules: "state awareness" (task execution engine), "scheduling decision" (event consumer), and "metadata management" (task orchestration center). This breaks single-point dependencies and enhances system availability and elastic scalability.

[0107] The following is a detailed description with reference to another embodiment.

[0108] Example 2

[0109] The event-driven big data infrastructure task scheduling device provided in this embodiment includes multiple implementation units, each of which corresponds to a specific implementation step in the above embodiment one. The specific implementation method and beneficial effects can be referred to the foregoing method embodiment, and will not be repeated here.

[0110] Figure 4 This is a schematic diagram of an optional event-driven big data infrastructure task scheduling device according to an embodiment of the present invention, such as... Figure 4 As shown, the event-driven big data infrastructure task scheduling device may include: an acquisition unit 41, a publishing unit 42, a routing unit 43, and a scheduling unit 44, wherein,

[0111] The acquisition unit 41 is used to obtain the execution status of the target big data foundation task from the task execution engine;

[0112] The publishing unit 42 is used to encapsulate the execution status of the target big data base task into an event object through the event producer embedded in the task execution engine, and publish the event object to the event bus;

[0113] The routing unit 43 is used to determine the subscription information of the target big data foundation task through the event bus, and to route the event object to the target event consumer based on the subscription information;

[0114] The scheduling unit 44 is used by the target event consumer to schedule the downstream tasks corresponding to the target big data foundation task based on the event object.

[0115] The aforementioned event-driven big data infrastructure task scheduling device obtains the execution status of the target big data infrastructure task from the task execution engine through the acquisition unit 41; the publishing unit 42 encapsulates the execution status of the target big data infrastructure task into an event object through the event producer embedded in the task execution engine, and publishes the event object to the event bus; the routing unit 43 determines the subscription information of the target big data infrastructure task through the event bus, and routes the event object to the target event consumer based on the subscription information; and the scheduling unit 44 schedules the downstream tasks corresponding to the target big data infrastructure task based on the event object by the target event consumer.

[0116] In this embodiment, an event producer is embedded within the task execution engine for state awareness. Within the millisecond-level time window of task instance completion, an event object is actively generated and published to the event bus for push. Downstream tasks no longer passively wait for the scheduler's periodic checks, but are activated the moment the upstream task's state changes. The execution of downstream tasks is automatically scheduled based on the event consumer's subscription to the event object, avoiding passive scheduling, reducing scheduling latency, and improving scheduling efficiency. This solves the technical problem in related technologies where scheduling latency exists and scheduling efficiency is low due to the scheduler polling and checking the task execution status.

[0117] Furthermore, the event-driven big data infrastructure task scheduling also includes: a first identification module for identifying the dependencies between big data infrastructure tasks; and a first registration module for registering the dependencies between big data infrastructure tasks to the task orchestration center.

[0118] Furthermore, the scheduling unit includes: a first parsing module, used by the target event consumer to parse the event object and obtain the parsing result; a first determining module, used to query the task orchestration center based on the target big data base task and determine the downstream tasks that have a dependency relationship with the target big data base task when the parsing result indicates that the execution status of the target big data base task is successful; and a first generating module, used by the target event consumer to generate execution instances of the downstream tasks and submit the execution instances to the task execution engine.

[0119] Furthermore, the first generation module includes: a first acquisition submodule, used to acquire the execution information of the target big data foundation task, wherein the execution information includes at least one of the following: output path, amount of data processed, metadata identifier, and execution log summary; a first generation submodule, used to acquire the execution logic, resource requirement information, and parameter template of the downstream task, and generate the execution parameters of the downstream task in combination with the execution information; and a second generation submodule, used to generate the execution instance of the downstream task by the target event consumer based on the execution parameters.

[0120] Furthermore, the event-driven big data infrastructure task scheduling also includes: a first invocation module, used to invoke a preset processing strategy based on the priority of the target big data infrastructure task when the parsing result indicates that the execution status of the target big data infrastructure task is a failure state; and a first processing module, used by the event consumer to process the target big data infrastructure task in the failure state based on the preset processing strategy.

[0121] Furthermore, the event-driven big data infrastructure task scheduling also includes: a first embedding module, used to embed event producers into the task execution engine in the form of a software development kit; and a first creation module, used to create access interfaces for event producers, wherein the access interfaces are used to capture the execution status of the big data infrastructure task when the big data infrastructure task is completed.

[0122] Furthermore, the event-driven big data infrastructure task scheduling also includes: a first computing module, used to calculate the hash value of the event object corresponding to each big data infrastructure task; and a first partitioning module, used to partition the event bus using the hash value of the event object.

[0123] The aforementioned event-driven big data infrastructure task scheduling device may also include a processor and a memory. The aforementioned acquisition unit 41, publishing unit 42, routing unit 43, scheduling unit 44, etc., are all stored in the memory as program units, and the processor executes the aforementioned program units stored in the memory to realize the corresponding functions.

[0124] The aforementioned processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured, and the scheduling of big data infrastructure tasks can be achieved by adjusting kernel parameters.

[0125] The aforementioned memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0126] According to another aspect of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored computer program, wherein, when the computer program is running, it controls the device where the computer-readable storage medium is located to execute any of the above-described event-driven big data infrastructure task scheduling methods.

[0127] According to another aspect of the present invention, an electronic device is also provided, including one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement any of the above-described event-driven big data infrastructure task scheduling methods.

[0128] According to another aspect of the present invention, a computer program product is also provided, the computer program product including a computer program, wherein the computer program, when executed by a processor, implements any of the above-described event-driven big data infrastructure task scheduling methods.

[0129] This application also provides a computer program product, which, when executed on a data processing device, is suitable for executing an initialization program with the following method steps: obtaining the execution status of a target big data infrastructure task from a task execution engine; encapsulating the execution status of the target big data infrastructure task into an event object through an event producer embedded in the task execution engine, and publishing the event object to an event bus; determining the subscription information of the target big data infrastructure task through the event bus, and routing the event object to a target event consumer based on the subscription information; and having the target event consumer schedule downstream tasks corresponding to the target big data infrastructure task based on the event object.

[0130] This application also provides a computer program product that, when executed on a data processing device, is suitable for executing an initialization program with the following method steps: identifying dependencies between big data infrastructure tasks; and registering the dependencies between big data infrastructure tasks to a task orchestration center.

[0131] This application also provides a computer program product that, when executed on a data processing device, is suitable for executing an initialization program with the following method steps: the target event consumer parses the event object to obtain the parsing result; if the parsing result indicates that the execution status of the target big data base task is successful, the target big data base task queries the task orchestration center to determine the downstream tasks that have a dependency relationship with the target big data base task; the target event consumer generates an execution instance of the downstream task and submits the execution instance to the task execution engine.

[0132] This application also provides a computer program product that, when executed on a data processing device, is suitable for executing an initialization program with the following method steps: obtaining execution information of a target big data infrastructure task, wherein the execution information includes at least one of the following: output path, amount of data processed, metadata identifier, and execution log summary; obtaining the execution logic, resource requirement information, and parameter template of a downstream task, and generating execution parameters for the downstream task in combination with the execution information; and generating an execution instance of the downstream task by a target event consumer based on the execution parameters.

[0133] This application also provides a computer program product, which, when executed on a data processing device, is suitable for executing an initialization program with the following method steps: when the parsing result indicates that the execution status of the target big data base task is in a failed state, a preset processing strategy is invoked based on the priority of the target big data base task; and an event consumer processes the failed target big data base task based on the preset processing strategy.

[0134] This application also provides a computer program product that, when executed on a data processing device, is suitable for executing an initialization program with the following method steps: embedding an event producer into a task execution engine in the form of a software development kit; creating an access interface for the event producer, wherein the access interface is used to capture the execution status of the big data base task when the big data base task is completed.

[0135] This application also provides a computer program product that, when executed on a data processing device, is suitable for executing an initialization program with the following method steps: calculating hash values ​​for event objects corresponding to each big data base task; and partitioning the event bus using the hash values ​​of the event objects.

[0136] Figure 5 This is a hardware structure block diagram of an electronic device (or mobile device) that executes an event-driven big data infrastructure task scheduling method according to an embodiment of the present invention. Figure 5 As shown, an electronic device may include one or more processors ( Figure 5 The processor, denoted by 502a, 502b, ..., 502n, can include, but is not limited to, a processing device such as a microprocessor (MCU) or a programmable logic device (FPGA), and a memory 504 for storing data. In addition, it may include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports in the I / O interface), a network interface, a keyboard, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 5 The structure shown is for illustrative purposes only and does not limit the structure of the electronic device described above. For example, the electronic device may also include components that are more... Figure 5 The more or fewer components shown, or having the same Figure 5 The different configurations shown.

[0137] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0138] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0139] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0140] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0141] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0142] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0143] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A task scheduling method for a big data infrastructure based on event-driven methods, characterized in that, include: Obtain the execution status of the target big data foundation task from the task execution engine; The execution status of the target big data foundation task is encapsulated into an event object by the event producer embedded in the task execution engine, and the event object is published to the event bus. The event bus determines the subscription information of the target big data foundation task, and the event object is routed to the target event consumer based on the subscription information. The target event consumer schedules downstream tasks corresponding to the target big data foundation task based on the event object.

2. The method according to claim 1, characterized in that, Before obtaining the execution status of the target big data foundation task from the task execution engine, the following is also included: Identify the dependencies between the big data infrastructure tasks; Register the dependencies between the big data foundation tasks to the task orchestration center.

3. The method according to claim 2, characterized in that, The steps for the target event consumer to schedule downstream tasks corresponding to the target big data foundation task based on the event object include: The target event consumer parses the event object to obtain the parsing result; If the parsing result indicates that the execution status of the target big data foundation task is successful, the task orchestration center is queried based on the target big data foundation task to determine the downstream tasks that have a dependency relationship with the target big data foundation task; The downstream task execution instance is generated by consuming the target event, and the execution instance is submitted to the task execution engine.

4. The method according to claim 3, characterized in that, The steps for generating execution instances of the downstream tasks from the consumption of the target event include: Obtain the execution information of the target big data foundation task, wherein the execution information includes at least one of the following: output path, amount of data processed, metadata identifier, and execution log summary; Obtain the execution logic, resource requirement information, and parameter template of the downstream task, and generate the execution parameters of the downstream task in combination with the execution information; The target event consumer generates an execution instance of the downstream task based on the execution parameters.

5. The method according to claim 3, characterized in that, After obtaining the parsing results, the following is also included: If the parsing result indicates that the execution status of the target big data infrastructure task is a failure, a preset processing strategy is invoked based on the priority of the target big data infrastructure task. The event consumer processes the failed target big data infrastructure task based on the preset processing strategy.

6. The method according to claim 1, characterized in that, Before obtaining the execution status of the target big data foundation task from the task execution engine, the following is also included: The event producer is embedded into the task execution engine in the form of a software development kit; An access interface is created for the event producer, wherein the access interface is used to capture the execution status of the big data base task when the big data base task is completed.

7. The method according to claim 1, characterized in that, Also includes: Calculate the hash value for the event object corresponding to each of the aforementioned big data foundation tasks; The event bus is partitioned using the hash value of the event object.

8. An event-driven big data infrastructure task scheduling device, characterized in that, include: The acquisition unit is used to obtain the execution status of the target big data foundation task from the task execution engine; The publishing unit is used to encapsulate the execution status of the target big data foundation task into an event object through the event producer embedded in the task execution engine, and publish the event object to the event bus; The routing unit is used to determine the subscription information of the target big data foundation task through the event bus, and to route the event object to the target event consumer based on the subscription information; The scheduling unit is used by the target event consumer to schedule the downstream tasks corresponding to the target big data foundation task based on the event object.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the event-driven big data infrastructure task scheduling method according to any one of claims 1 to 7.

10. An electronic device, characterized in that, It includes one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the event-driven big data infrastructure task scheduling method according to any one of claims 1 to 7.

Images