A device, method and system for asynchronously submitting SQL tasks to obtain results
By submitting SQL tasks asynchronously, the problem of SQL tasks being unable to be cancelled and thread pool exhaustion is solved, asynchronous execution and cancellation of tasks is realized, high concurrency is supported, the pressure on middle-end services is reduced, and engineering code is simplified.
Patent Information
- Application Number
- CN202411173238.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-26
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2044-08-26
AI Technical Summary
The existing technology cannot support mid-way cancellation when executing SQL tasks, resulting in thread blocking, requiring restart of middle-end applications, affecting usage, and frequent sending of execution requests leads to exhausting the thread pool and blocking the request.
Using the method of asynchronously submitting SQL tasks to obtain results, generate SQL task objects through the task operation interface, and submit them to the task queue to be executed by the executor manager, supports task status saving and cancellation, uses thread pool to execute and store results, and provides Obtain task status and results from external interfaces.
It realizes asynchronous execution and cancellation of SQL tasks, avoids thread pool exhaustion and blockage, supports high concurrent execution, reduces the pressure of middle-end services, simplifies engineering code, and improves system stability and maintenance.
Smart Images

Figure CN118689616B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of Internet technology, and in particular to a device, method and system for asynchronously submitting an SQL task to obtain results. Background Art
[0002] The data development page is the basic sql development page of the data middle platform. By writing sql, submitting and connecting to the data source, the results are obtained. However, common middle platform services often directly connect to the data source to execute sql, and need to wait for the interface to return to obtain the results. After refreshing the page, the results of the last request are often not obtained and need to be rerun. In addition, if execution requests are sent frequently, the thread pool of the middle platform service application will be quickly exhausted, resulting in a large number of requests being blocked.
[0003] The data middle platform often needs to support the SQL execution capabilities of dozens or hundreds of data sources. If this logic is directly coupled with the middle platform, it will increase the pressure on the back-end services of the data middle platform, make the engineering code more coupled, and greatly increase the difficulty of maintenance.
[0004] Existing technologies often do not support midway cancellation when executing SQL, which will cause a problem. If there is a problem with a data source, the thread executing SQL will be blocked, and the processing method often requires restarting the middle-end application, which affects actual use. Summary of the invention
[0005] The present invention aims at at least one disadvantage of the prior art and provides a device for asynchronously submitting SQL tasks to obtain results. In order to solve the above technical problems, the present invention is solved by the following technical solutions:
[0006] (I) A device for asynchronously submitting SQL tasks to obtain results, comprising the following structure:
[0007] External task operation interface, used to generate SQL task objects according to data source connection information configuration and task parameters, and to be called to implement task operation methods; each SQL task object includes a SQL identifier and a SQL task identifier;
[0008] The task run executor is used to execute SQL tasks. The task run executor includes the SQL task queue and SQL execution thread pool, and maintains the SQL task identifier and data source type identifier;
[0009] The run executor manager is used to obtain or initialize the task run executor of the corresponding data source type according to the data source connection configuration and task parameters;
[0010] The sql task object is submitted to the sql task queue of the corresponding task running executor, and the sql tasks consuming the sql task queue are executed in the sql execution thread pool, and each sql result is stored in the execution result cache of the associated sql identifier; at different stages of the task, the task status is saved in the task status cache of the sql task object;
[0011] The external task operation interface is used to be called to obtain the real-time task status according to the SQL task identifier;
[0012] When the SQL task is completed, the external task operation interface is called to obtain the task running results and clear the task running result cache.
[0013] Preferably, the external task operation interface is called to implement the task operation method, including a method for determining whether the queue is full, to determine whether the queue of the current data source is full. If not, the SQL task object is submitted to the SQL task queue of the corresponding task running executor. If so, the submission is rejected.
[0014] Preferably, the sql task object includes a log collector, which is initialized to periodically collect logs during the execution process.
[0015] Preferably, the task running executor maintains a reference relationship between the sql task identifier and the current execution thread. When the task needs to be canceled, the current execution thread of the corresponding sql task object is interrupted through the sql task identifier to cancel the execution process.
[0016] The present invention also provides a method for asynchronously submitting an SQL task to obtain a result, comprising the following steps:
[0017] (ii) the external task operation interface is called, and the data source connection configuration and task parameters received and encapsulated by the external service are passed in; a SQL task object is generated according to the data source connection configuration and task parameters; each SQL task object includes a SQL identifier and a SQL task identifier;
[0018] Call the run executor manager according to the data source connection configuration and task parameters to obtain or initialize the task run executor of the corresponding data source type. The task run executor maintains the corresponding SQL task identifier and data source type identifier.
[0019] Submit the SQL task object to the SQL task queue of the corresponding task running executor, consume the SQL task in the SQL task queue to the execution thread for execution, and store each SQL result in the execution result cache associated with the SQL identifier; at the same time, update the task status according to the execution status of the SQL task;
[0020] The external task operation interface is called to obtain the real-time task status according to the SQL task identifier;
[0021] When the SQL task is completed, the external task operation interface is called to obtain the task running results and clear the task running result cache.
[0022] Preferably, the method of submitting the SQL task object to the SQL task queue of the task execution device includes: when the external task operation interface is called, it is determined whether the SQL task queue is full, if not, the SQL task object is submitted to the SQL task queue, and if so, the submission is rejected. The queue is used as a buffer, which has an effective solution to the problem of frequent execution request blocking.
[0023] Preferably, a log collection method is also included, including:
[0024] The sql task object includes a log collector, which initializes the log collector and collects logs during execution at a fixed time.
[0025] The external task operation interface is called to obtain log information.
[0026] Preferably, an interruption method is also included:
[0027] The current execution thread of the corresponding SQL task object is interrupted through the SQL task identifier, entering the resource release phase, and each SQL result is stored in the execution result cache associated with the SQL unique identifier.
[0028] (III) The present invention also proposes a system for asynchronously submitting SQL tasks and obtaining results, comprising a middle platform service unit and a SQL execution service unit, wherein the SQL execution service unit comprises the device for asynchronously submitting SQL tasks and obtaining results proposed above; wherein,
[0029] The middle-office service unit is used to receive the SQL tasks submitted by users, and encapsulate the SQL task information, data source connection information and task parameters in the layer data source connection configuration and task parameters, and call the external task operation interface to submit the task to the SQL execution service for execution;
[0030] The sql execution service unit includes an external task operation interface, an execution manager, and a task execution executor.
[0031] Preferably, the middle platform service unit includes a task scheduling unit, and the task scheduling unit is connected to the external task operation interface.
[0032] The task scheduling unit is used to initialize the external task operation interface implementation class, encapsulate the data source connection configuration and task parameters, call the submit task method of the external task operation interface, and submit the task to the SQL execution service for execution.
[0033] Beneficial Effects
[0034] 1) This application provides a method for asynchronously submitting SQL tasks to obtain results. By setting the implementation class of the external task interface, the data source SQL execution logic is decoupled from the business system, thereby achieving the purpose of lightweight.
[0035] 2) Obtain or initialize the task executor of the corresponding data source type according to the data source connection configuration and task parameters, maintain the data source type identifier and SQL task identifier, so that the submitted SQL tasks can be isolated according to the resources of different data source types. Each data source is configured with a set of queues and thread pools related to itself.
[0036] 3) The results of the SQL execution will be temporarily stored in the memory for subsequent asynchronous result acquisition, which solves the problem that the execution request cannot be obtained asynchronously, and supports high-concurrency execution solutions by submitting tasks to the queue. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0038] Figure 1 It is a flow chart of a method for asynchronously submitting SQL tasks to obtain results according to an embodiment disclosed in the present invention.
[0039] Figure 2 A system for asynchronously submitting SQL tasks and obtaining results according to an embodiment of the present invention;
[0040] Figure 3 It is a timing diagram of a method for asynchronously submitting SQL tasks to obtain results according to an embodiment disclosed in the present invention. DETAILED DESCRIPTION
[0041] The present invention is further described in detail below in conjunction with embodiments. The following embodiments are for explanation of the present invention but the present invention is not limited to the following embodiments.
[0042] (I) A device for asynchronously submitting SQL tasks and obtaining results, also known as a SQL execution service unit, includes the following structure:
[0043] External task operation interface, used to generate SQL task objects according to data source connection information configuration and task parameters, and to be called to implement task operation methods; each SQL task object includes a SQL identifier and a SQL task identifier;
[0044] The task run executor is used to execute SQL tasks. The task run executor includes the SQL task queue and SQL execution thread pool, and maintains the SQL task identifier and data source type identifier;
[0045] The run executor manager is used to obtain or initialize the task run executor of the corresponding data source type according to the data source connection configuration and task parameters;
[0046] The sql task object is submitted to the sql task queue of the corresponding task running executor, and the sql tasks consuming the sql task queue are executed in the sql execution thread pool, and each sql result is stored in the execution result cache of the associated sql identifier; at different stages of the task, the task status is saved in the task status cache of the sql task object;
[0047] The external task operation interface is used to be called to obtain the real-time task status according to the SQL task identifier;
[0048] When the SQL task is completed, the external task operation interface is called to obtain the task running results and clear the task running result cache.
[0049] Wherein, the implementation class of the external task operation interface obtains the task operation executor of the corresponding data source from the data source operation execution manager before implementing any task operation method;
[0050] Through the relationship between the data source type identifier and the task submission executor, the corresponding task running executor is called to submit the sql task object into the sql task queue.
[0051] Furthermore, the external task operation interface is called to implement the task operation method, including a method for determining whether the queue is full, to determine whether the queue of the current data source is full. If not, the SQL task object is submitted to the SQL task queue of the corresponding task running executor. If so, the submission is rejected.
[0052] Furthermore, the SQL task object includes a log collector, which is initialized to collect logs during execution at regular intervals. Before the task ends, the log information can be called by an external service in real time by calling a corresponding method of the external task operation interface.
[0053] Furthermore, the task running executor maintains a reference relationship between multiple SQL task identifiers and SQL tasks (SQL tasks belonging to the same data source type). The SQL task object internally maintains a reference to the thread executing the current SQL task. When the task needs to be canceled, the SQL task identifier is used to interrupt the current execution thread of the corresponding SQL task object and cancel the execution process.
[0054] The specific process of canceling a task is as follows: call the cancel task method of the external task interface, and pass in the data source connection configuration and task parameters, obtain the specified data source type and the corresponding SQL task identifier that needs to be canceled, obtain the task run executor of the specified data source from the task run executor manager, then call the cancel task method of the task run executor and pass in the corresponding SQL task identifier obtained, then obtain the running SQL task object through the SQL task unique identifier of the SQL task that needs to be canceled, and finally call the cancel task method of the SQL task object, and call the thread's interrupt method inside the cancel task method to interrupt execution. This method effectively solves the problem that the SQL execution thread cannot be interrupted.
[0055] (ii) According to the apparatus for asynchronously submitting SQL tasks and obtaining results disclosed above, it can be known that the external task operation interface, the task running executor, and the running executor manager are configured as a plug-in service, which is connected with the external service to implement the method of asynchronously submitting SQL tasks and obtaining results.
[0056] (1) Specifically, the external task operation interface is used to be called to implement the following methods. The external task operation interface mentioned above is interpreted as the corresponding method of the external task operation interface being called to implement the corresponding function:
[0057] The task submission method is called by the data middle-end service unit to submit the SQL task, which includes: passing in the data source connection configuration and task parameters, and putting the task into the queue when the queue is idle.
[0058] The method to determine whether the list is full (or whether the list is free) is called to determine whether the queue is free. If the queue of the current data source is full, a Boolean value indicating that the task cannot be submitted is returned to the caller.
[0059] That is, the method of judging whether the list is full of external task operation interface is called to judge whether the queue of the current data source is idle. If so, the SQL task object is submitted to the queue. If not, the submission is rejected.
[0060] Cancel task method. The method is called to interrupt the thread that is executing the SQL task. This cancel task method is used for external interruption operations.
[0061] Get the task status method. The method is called to query the task running status. The task running status includes task running, task completed, task canceled and task failed.
[0062] Clear the task running result cache method. The method is called to clear the cached task results.
[0063] Get the task running result method, which is called to return the data result of the completed running.
[0064] The above methods are called and implemented according to the data source connection configuration and task parameters; the above methods are called by external services, such as the middle and back-end services.
[0065] (2) Specifically, the class attributes of the task execution executor include the following:
[0066] The data source connection client is used to connect to the data source and the client object that specifically executes SQL.
[0067] The data source type identifier is used to identify a unique type of data source. Each data source has its own data source task executor.
[0068] The sql task identifier includes the maintained sql task identifier and the sql task association relationship.
[0069] The sql task queue, when there is no available thread, is temporarily placed in the sql task queue (cache queue).
[0070] The sql execution thread pool is the thread pool that executes sql tasks. The sql tasks in the sql task queue are continuously consumed and submitted to the sql execution thread pool for execution.
[0071] (3) Specifically, the operation methods implemented by the task execution executor include the following:
[0072] Submit the task to run. The method is called to submit the sql task from the sql task submission queue to the sql task to the thread pool for running.
[0073] Cancel the task method, call the cancel thread method of the sql task object to interrupt the thread and cancel the execution process.
[0074] Get task status method, which is called to query the task running status.
[0075] Get the task running log method. The method is called to get the log during the task execution process.
[0076] Get the task running result. The method is called to get the result of SQL execution.
[0077] During the execution period of the data source task, the task is continuously retrieved from the SQL task queue and submitted to the SQL execution thread pool.
[0078] (4) SQL task object, the defined class attributes include:
[0079] The SQL set to be executed stores the SQL set that needs to be executed.
[0080] sql task ID, the unique ID of the sql task;
[0081] The sql execution result is used to temporarily store the sql execution result associated with the sql unique identifier, that is, a cache mechanism for storing the sql execution result, and the sql unique identifier and its corresponding sql execution result are stored correspondingly, so as to obtain the corresponding sql execution result through the sql task unique identifier.
[0082] Log collection is used to temporarily store log collections, specifically logs during or after task execution.
[0083] Error log collection, used to store log information when exceptions occur during task execution.
[0084] The log collector is used to periodically collect logs during the execution of SQL tasks.
[0085] Whether to cancel the flag. When the cancel task method is called, the flag is true.
[0086] The above configurations all configure the corresponding memory. You can also configure other class attributes as needed without restriction.
[0087] (5) The implementation method of SQL task object configuration includes:
[0088] Constructor: constructs the task object and initializes the log collector and other information.
[0089] The start running method is called to start executing the SQL statements in the SQL collection and start the log collector to collect logs.
[0090] Cancel the task method, call the interrupt method of the thread handle to interrupt the thread execution, remove the task cache, and collect execution information.
[0091] The log collection method is called to collect logs and return log information.
[0092] Execute the sq collection method to retrieve and execute the SQL in the sql collection.
[0093] Release connection resources, stop tasks, release resources, and automatically clear log information and task running results after 10 seconds.
[0094] Based on the above device configuration, a method for asynchronously submitting SQL tasks to obtain results is further disclosed, referring to Figure 1 , including the following steps:
[0095] Step 1: The external task operation interface is called, and the data source connection configuration and task parameters received and encapsulated by the external service are passed in; a SQL task object is generated according to the data source connection configuration and task parameters; each SQL task object includes a SQL identifier and a SQL task identifier;
[0096] Step 2: Call the run executor manager to obtain or initialize the task run executor of the corresponding data source type according to the data source connection configuration and task parameters. The task run executor maintains the corresponding SQL task identifier and data source type identifier.
[0097] Step 3: Submit the SQL task object to the SQL task queue of the corresponding task execution device, consume the SQL task in the SQL task queue to the execution thread for execution, and store each SQL result in the execution result cache associated with the SQL identifier; at the same time, update the task status according to the execution status of the SQL task;
[0098] The external task operation interface is called to obtain the real-time task status according to the SQL task identifier;
[0099] Step 4: When the SQL task is completed, the external task operation interface is called to obtain the task running result and clear the task running result cache.
[0100] First, the method disclosed in this embodiment provides various operation methods of the task by exposing an external task operation interface. External services, such as the data middle-end back-end service, can call the task submission interface to submit SQL tasks, call to obtain task running results to obtain execution results, and other operations.
[0101] Secondly, through the design of unique identification, the submitted SQL tasks are isolated according to the different data source types. Each data source is configured with a set of queues and thread pools corresponding to itself. When the queue is full, the rejection strategy will be used to reject the task submission request; further, the results of the SQL execution will be temporarily stored in the memory for subsequent asynchronous results.
[0102] The method for submitting a SQL task object to a SQL task queue of a task execution device includes: when an external task operation interface is called, determining whether the SQL task queue is full, if not, submitting the SQL task object to the SQL task queue, and if so, refusing to submit.
[0103] It also includes a log collection method, including: the sql task object includes a log collector, the log collector is initialized, and the logs in the execution process are collected regularly; the external task operation interface is called to obtain log information.
[0104] Among them, all task status, log information, SQL results, etc. before the SQL running task is removed (or before the task is completed) can be called and queried at any time through the external task operation interface.
[0105] Among them, the reference relationship between the sql task identifier and the current execution thread is maintained. When the execution operation needs to be canceled, the current execution thread of the corresponding sql task object is interrupted through the sql task identifier. That is, the interrupt method of the thread handle is called according to the reference relationship of the sql task object to interrupt the thread execution. Figure 1 As shown in , even if the task is interrupted, the completed SQL results are temporarily stored and then the resources are released.
[0106] Based on the device for asynchronously submitting SQL tasks to obtain results, a system for asynchronously submitting SQL tasks to obtain results is disclosed, referring to Figure 2 As shown, it includes a middle platform service unit and a SQL execution service unit, and the SQL execution service unit includes a device for asynchronously submitting SQL tasks to obtain results; wherein,
[0107] The middle-office service unit is used to receive SQL tasks submitted by users, and encapsulate the SQL task information, data source connection information and task parameters in the layer of data source connection configuration and task parameters, and call the external task operation interface to submit the task to the SQL execution service for execution.
[0108] The sql execution service unit includes an external task operation interface, an execution manager, and a task execution executor.
[0109] The middle platform service unit includes a task scheduling unit, which is connected to the external task operation interface.
[0110] The task scheduling unit is used to initialize the external task operation interface implementation class, encapsulate the data source connection configuration and task parameters, call the submit task method of the external task operation interface, and submit the task to the SQL execution service for execution.
[0111] See Figure 2 , the device for asynchronously submitting SQL tasks to obtain results (SQL execution service unit) is named as the data source connection plug-in. The data source connection plug-in is configured as the data source plug-in package of the data stack. It can be further understood that each data source corresponds to a plug-in, and a plug-in corresponds to a data source connection client. The task scheduling unit initializes the implementation class of the external task operation interface through the service discovery mechanism, and requests to build the data source connection client and task parameters of the corresponding plug-in according to the above process, and calls the method of the external task operation interface to operate.
[0112] Based on the system of asynchronous submission of SQL tasks to obtain results, an asynchronous submission of SQL tasks to obtain results is provided. Figure 3 , including the following steps:
[0113] Step 10, the user submits the SQL to the data center backend service. The data center backend service encapsulates the SQL information, data source connection information configuration and task parameters into a SQL task object, and calls the external task operation interface to determine whether the queue is full. If it can be submitted, the submit task method is called to submit the task.
[0114] Step 20, after the external task operation interface implementation class encapsulates the SQL task object, it calls the execution manager according to the data source connection configuration to obtain or initialize the task execution executor of the specified data source type identifier.
[0115] The data center backend service periodically polls the execution status according to the SQL task identifier; calls the method to obtain log information during the execution process according to the SQL task identifier; and calls the method to obtain task execution results according to the SQL task identifier to obtain the SQL results during the execution process.
[0116] Step 30: The data source runs the executor to continuously consume SQL tasks from the SQL task queue and submit them to the thread pool for execution, and sets the task running status for subsequent acquisition of the execution status.
[0117] Step 40, the sql task object initializes the log collector, collects logs regularly, executes the sql set, and stores the execution results in the execution result cache for subsequent retrieval.
[0118] When the cancellation status flag is True during thread execution, it means that the current thread is interrupted and enters the resource release phase, and the execution ends later.
[0119] Step 50, the SQL run task is completed, the task status is set to completed, and the log collector is waited for to end.
[0120] Step 60: After the task is completed, the data center backend service calls the method of obtaining task running results of the external task operation interface to obtain the result;
[0121] Remove the corresponding task cache to avoid memory leaks. Finally, the data center backend service returns the execution result.
[0122] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed.
[0123] The units may or may not be physically separated, and the components displayed as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.
[0124] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0125] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program codes.
[0126] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.
Claims
1. A device for asynchronously submitting SQL tasks to obtain results, characterized in that: Includes the following structures: External task operation interface, used to generate SQL task objects according to data source connection information configuration and task parameters, and to be called to implement task operation methods; each SQL task object includes a SQL identifier and a SQL task identifier; The task run executor is used to execute SQL tasks. The task run executor includes the SQL task queue and SQL execution thread pool, and maintains the SQL task identifier and data source type identifier; The run executor manager is used to obtain or initialize the task run executor of the corresponding data source type according to the data source connection configuration and task parameters; The sql task object is submitted to the sql task queue of the corresponding task running executor, and the sql tasks consuming the sql task queue are executed in the sql execution thread pool, and each sql result is stored in the execution result cache of the associated sql identifier; at different stages of the task, the task status is saved in the task status cache of the sql task object; The external task operation interface is used to be called to obtain the real-time task status according to the SQL task identifier; When the SQL task is completed, the external task operation interface is called to obtain the task running results and clear the task running result cache.
2. The device for asynchronously submitting SQL tasks and obtaining results according to claim 1, characterized in that: The external task operation interface is called to implement the task operation method, including a method for determining whether the queue is full, to determine whether the queue of the current data source is full, if not, submit the SQL task object to the SQL task queue of the corresponding task running executor, if so, refuse to submit.
3. The device for asynchronously submitting SQL tasks and obtaining results according to claim 1, characterized in that: The sql task object includes a log collector, which is initialized and collects logs during the execution process at a fixed time.
4. The device for asynchronously submitting SQL tasks and obtaining results according to claim 1, characterized in that: The task running executor maintains the reference relationship between multiple SQL task identifiers and the current execution thread. When the task needs to be canceled, the current execution thread of the corresponding SQL task object is interrupted through the SQL task identifier to cancel the execution process.
5. A method for asynchronously submitting SQL tasks to obtain results, characterized in that: The following steps are involved: The external task operation interface is called, and the data source connection configuration and task parameters received and encapsulated by the external service are passed in; Generate a SQL task object based on the data source connection configuration and task parameters; Each sql task object includes a sql identifier and a sql task identifier; Call the run executor manager according to the data source connection configuration and task parameters to obtain or initialize the task run executor of the corresponding data source type. The task run executor maintains the corresponding SQL task identifier and data source type identifier. Submit the SQL task object to the SQL task queue of the corresponding task running executor, consume the SQL task in the SQL task queue to the execution thread for execution, and store each SQL result in the execution result cache associated with the SQL identifier; at the same time, update the task status according to the execution status of the SQL task; The external task operation interface is called to obtain the real-time task status according to the SQL task identifier; When the SQL task is completed, the external task operation interface is called to obtain the task running results and clear the task running result cache.
6. The method for asynchronously submitting SQL tasks to obtain results according to claim 5, characterized in that: The method of submitting the SQL task object to the SQL task queue of the task execution executor includes: The external task operation interface is called to determine whether the sql task queue is full. If not, the sql task object is submitted to the sql task queue. If so, the submission is rejected.
7. The method for asynchronously submitting SQL tasks to obtain results according to claim 5, characterized in that: Also included are log collection methods, including: The sql task object includes a log collector, which initializes the log collector and collects logs during execution at a fixed time. The external task operation interface is called to obtain log information.
8. The method for asynchronously submitting SQL tasks to obtain results according to claim 5, characterized in that: Also includes interrupt method: The current execution thread of the corresponding SQL task object is interrupted through the SQL task identifier, entering the resource release phase, and each SQL result is stored in the execution result cache associated with the SQL unique identifier.
9. A system for asynchronously submitting SQL tasks to obtain results, characterized in that: It includes a middle platform service unit and a SQL execution service unit, and the SQL execution service unit includes the device for asynchronously submitting SQL tasks and obtaining results according to any one of claims 1 to 4; wherein, The middle-office service unit is used to receive the SQL tasks submitted by users, and encapsulate the SQL task information, data source connection information and task parameters in the layer data source connection configuration and task parameters, and call the external task operation interface to submit the task to the SQL execution service for execution; The sql execution service unit includes an external task operation interface, an execution manager, and a task execution executor.
10. The system for asynchronously submitting SQL tasks and obtaining results according to claim 9, characterized in that: The middle platform service unit includes a task scheduling unit, which is connected to the external task operation interface. The task scheduling unit is used to initialize the external task operation interface implementation class, encapsulate the data source connection configuration and task parameters, call the submit task method of the external task operation interface, and submit the task to the SQL execution service for execution.
