Computing task processing method and device

Abstract

The invention provides a computing task processing method and a computing governance platform, the computing task processing method is applied to the computing governance platform, and the computing task processing method comprises the following steps: in response to a computing task processing request sent by an application system received by the computing governance platform, obtaining resource information of a target computing engine matched with the computing task processing request; according to the resource information of the target calculation engine, calling the target calculation engine to execute a target calculation task requested by the calculation task processing request to obtain an execution result of the target calculation task; and sending an execution result of the target calculation task to the application system. Therefore, unified task scheduling and resource management are performed by introducing the computing management platform, thorough decoupling of the application system and the underlying computing engine is realized, the architecture and development complexity of the application system are simplified, and the problems of difficult operation and maintenance, task congestion, complex adaptation and the like when multiple computing tasks are parallel can be avoided; and the execution efficiency and stability of the calculation task are improved.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a method and apparatus for processing computing tasks. Background Technology

[0002] In scenarios such as big data processing, real-time stream computing, and offline data warehouse construction, computational tasks typically refer to specific tasks involving cleaning, transforming, analyzing, and modeling massive amounts of data. Examples include real-time transaction data statistics for e-commerce platforms (requiring real-time calculation of order volume and average order value), historical risk control data backtracking analysis for financial institutions (requiring offline processing of months of transaction data), and user behavior profiling for internet companies (requiring integration of multi-source data and feature extraction). The efficient execution of these tasks often relies on heterogeneous third-party computing engines such as Flink, Hive, and Spark. In practice, these computational tasks require the use of client tools provided by each computing engine (such as Flink Client (a client of Apache Flink Client), Hive Beeline (a lightweight command-line client tool for the Hive database), and Spark Submit (a command-line client tool provided by Apache Spark)) to submit data computations to the corresponding computing engine as independent processes.

[0003] However, when a large number of computing tasks (such as hundreds of real-time monitoring tasks during peak hours or dozens of offline statistical tasks running in batches every morning) are executed in parallel in a business scenario, the following problems may occur: 1. Difficulty in operation and maintenance Because each computing engine relies on independent client tools, separate deployments, version updates, and environment configurations are required for Flink Client, HiveBeeline, Spark Submit, etc., resulting in high maintenance costs. For example, FlinkClient needs to match the Flink version of the cluster, and Hive Beeline needs to configure JDBC (Java Database Connectivity) connection parameters and authentication information. Furthermore, the dependency libraries of different clients (such as Java environment and Python version) may conflict, requiring separate maintenance of isolated environments.

[0004] 2. Task congestion After each computing task is submitted through an independent client, there is a lack of a unified resource management and scheduling mechanism. For example, the YARN Container resources requested by Flink tasks, the MapReduce slots occupied by Hive tasks, and the Executor resources used by Spark tasks are all allocated separately by the schedulers of each engine, without any resource coordination logic between them. When a large number of tasks are submitted in parallel, resource contention is very likely to occur: some high-priority tasks may be delayed in starting due to low-priority tasks occupying too much CPU (Central Processing Unit) and memory resources; if a task of one engine excessively occupies storage I / O (Input / Output), it will cause other engines' tasks to experience data read and write delays; even "resource fragmentation" may occur, that is, each engine has a small amount of unused resources, but cannot be integrated and allocated to tasks that require a large number of resources, resulting in resource waste and a large number of tasks queuing for resources, forming task congestion, which may cause some tasks to time out and fail in severe cases.

[0005] 3. Complex adaptation The client interfaces and data interaction protocols of different third-party computing engines differ significantly: Flink Client specifies task topology, parallelism, and other configurations via REST API or command-line parameters; Hive Beeline requires defining data query logic using HQL statements, and its parameter passing format (such as partition filtering syntax) is incompatible with other engines; Spark Submit requires specifying the JAR package path, main class name, and dependency package management method (such as the packages parameter). This means that developers need to write separate adaptation code for each engine when submitting different types of tasks. For example, the same data cleaning logic needs to be translated into Flink DataStream API code, Hive SQL statements, and Spark DataFrame code respectively, making adaptation complex.

[0006] The above problems arise because the computing tasks are strongly bound to the computing engine clients. The task submission logic, configuration parameters, and runtime dependencies are all deeply associated with specific clients. Furthermore, the clients of each computing engine run as independent processes with no information exchange or collaborative logic between them. This results in a lack of collaborative scheduling during task execution. When a large number of computing tasks are executed in parallel, problems such as operational difficulties, task congestion, and complex adaptation can easily occur. Summary of the Invention

[0007] This disclosure provides a method and apparatus for processing computational tasks, to at least partially solve one of the technical problems in the related art. The technical solution of this disclosure is as follows: According to a first aspect of the present disclosure, a computing task processing method is provided, applied to a computing governance platform, comprising: responding to the computing governance platform receiving a computing task processing request sent by an application system, obtaining resource information of a target computing engine matching the computing task processing request; based on the resource information of the target computing engine, invoking the target computing engine to execute the target computing task requested by the computing task processing request, so as to obtain the execution result of the target computing task; and sending the execution result of the target computing task to the application system.

[0008] According to a second aspect of the present disclosure, a computing task processing apparatus is provided, applied to a computing governance platform, comprising: an acquisition module, configured to acquire resource information of a target computing engine matching the computing task processing request in response to the computing governance platform receiving a computing task processing request sent by an application system; a processing module, configured to invoke the target computing engine to execute the target computing task requested by the computing task processing request based on the resource information of the target computing engine, so as to obtain the execution result of the target computing task; and a sending module, configured to send the execution result of the target computing task to the application system.

[0009] According to a third aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute the instructions to implement the computing task processing method as described in the first aspect of the present disclosure.

[0010] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided that, when instructions in the computer-readable storage medium are executed by a processor of an electronic device, enables the electronic device to perform a computing task processing method as described in the first aspect of the present disclosure.

[0011] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising: a computer program that, when executed by a processor, implements the computing task processing method as described in the first aspect of the present disclosure.

[0012] The technical solutions provided by the embodiments of this disclosure have at least the following beneficial effects: In this technical solution, by responding to a computing task processing request sent by an application system to the computing governance platform, the resource information of the target computing engine matching the computing task processing request is obtained. This allows for precise location of the resource information of the compatible target computing engine, preventing execution failures and inefficiencies caused by mismatch between the computing engine and the computing task from the source, thus ensuring the adaptability of computing task execution. Furthermore, based on the resource information of the target computing engine, the target computing engine is invoked to execute the target computing task requested in the computing task processing request to obtain the execution result of the target computing task. By directly invoking the target computing engine to execute the target computing task requested in the computing task processing request, the processing chain of the target computing task can be shortened, improving the execution efficiency of the target computing task. Finally, the execution result of the target computing task is sent to the application system, enabling timely feedback of results to the application system, reducing business process interruptions or waiting costs caused by result delays, and improving the operational stability of the application system and the user experience. Specifically, according to the configured tag processing rules, the original tags carried in the computing task processing request are preprocessed to obtain the target tags, which can filter out errors in the original tags. Redundant and erroneous information is eliminated to ensure that target labels accurately reflect the core requirements of the task. Furthermore, based on the resource information of the target computing engine, the target resources corresponding to the target labels are determined, achieving precise alignment between task requirements and the specific resources of the computing engine. Finally, the target computing engine is invoked, and the target computing task is executed based on the target resources to obtain the execution result. By executing the target computing task based on the adapted target resources, unnecessary resources of the target computing engine can be avoided, improving the execution efficiency and stability of the target computing task and helping the computing governance platform to rationally allocate the resources of the target computing engine. In addition, locking the target resources prevents other computing tasks from calling them, ensuring the stability of the target computing task execution environment from the source. Further, the target computing engine is invoked, and the target computing task is executed based on the locked target resources to obtain the execution result, ensuring the stable progress of the target computing task. Finally, the target resources are unlocked, allowing for timely release of the target resources after the target computing task ends, improving resource utilization and providing resource support for resource matching and efficient execution of other computing tasks.

[0013] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0014] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0015] Figure 1This is a flowchart illustrating the computational task processing method shown in the first embodiment of this disclosure; Figure 2 This is a flowchart illustrating the computational task processing method shown in the second embodiment of this disclosure; Figure 3 This is a flowchart illustrating the computational task processing method shown in the third embodiment of this disclosure; Figure 4 This is a schematic diagram of the structure of the computing governance platform shown in the fourth embodiment of this disclosure; Figure 5 This is a schematic diagram illustrating the relationship between the application system, the computing governance platform, and the computing engine as shown in the fifth embodiment of this disclosure; Figure 6 This is a task processing timing diagram shown in the sixth embodiment of this disclosure; Figure 7 This is a schematic diagram of the computing task processing device shown in the seventh embodiment of this disclosure; Figure 8 This is a schematic diagram of the structure of an electronic device shown in an exemplary embodiment of the present disclosure. Detailed Implementation

[0016] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0017] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data are interchangeable where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0018] It should be noted that the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved in the technical solution disclosed herein are all carried out with the consent of the user, and all comply with the provisions of relevant laws and regulations, and do not violate public order and good morals.

[0019] The computing task processing method and apparatus of this disclosure are described below with reference to the accompanying drawings.

[0020] Figure 1 This is a flowchart illustrating the computational task processing method shown in the first embodiment of this disclosure.

[0021] like Figure 1 As shown, the computational task processing method includes the following steps: Step 101: In response to the computing governance platform receiving a computing task processing request sent by the application system, the resource information of the target computing engine that matches the computing task processing request is obtained.

[0022] The computing governance platform is used for unified management of computing engines. These computing engines can be multi-source, heterogeneous third-party computing engines, such as Flink, Hive, Spark, etc., and databases.

[0023] The application system can be any application system, such as enterprise-level applications, data analysis platforms, etc.

[0024] The resource information of the target computing engine can be various resource information related to the target computing engine, such as: the maximum available resources of the target computing engine, the resources used by the target computing engine, and the locked resources of the target computing engine.

[0025] In this embodiment of the disclosure, after receiving a computing task processing request sent by the application system, the computing governance platform can determine the target computing engine that matches the computing task processing request and obtain the resource information of the target computing engine.

[0026] As an example, a computing governance platform can determine the target computing engine from multiple computing engines based on the task information carried in the computing task processing request and the status of the multiple computing engines it manages, and obtain the resource information of the target computing engine.

[0027] In some embodiments, such as Figure 2 As shown, step 101 specifically includes: Step 1011: In response to the computing governance platform receiving a computing task processing request sent by the application system, the target computing engine is determined based on the identification information of the computing engine carried in the computing task processing request.

[0028] To improve task execution efficiency and enhance business adaptability, in this embodiment, the computing task processing request carries the identification information of the computing engine. That is, the application system specifies the computing engine, and the computing governance platform can then determine the target computing engine from among the multiple computing engines it manages, based on the matching identification information of the computing engine carried in the computing task processing request. This bypasses the complex multi-computing engine matching decision logic of the computing governance platform, enabling precise and rapid routing of tasks to the target computing engine.

[0029] Step 1012: Obtain the resource information of the target computing engine.

[0030] As an example, a computing governance platform can establish a unified connector module and use this module to establish a connection with the target computing engine to exchange information and obtain the target computing engine's resource information.

[0031] As an example, after the computing governance platform establishes a connection with the target computing engine through the connector module, it can call the resource information acquisition interface to obtain the resource information of the target computing engine.

[0032] Among them, when calling the resource information acquisition interface to obtain the resource information of the target computing engine, an exception handling mechanism can also be set. For example, when calling the resource information acquisition interface to obtain the resource information of the target computing engine, a time parameter can be set. If the waiting time for obtaining the resource information exceeds the limit of the time parameter, the acquisition of the resource information will be automatically treated as a failure.

[0033] Step 102: Based on the resource information of the target computing engine, call the target computing engine to execute the target computing task requested by the computing task processing request, so as to obtain the execution result of the target computing task.

[0034] In this embodiment of the disclosure, the target resources for executing the target computing task requested by the computing task processing request can be determined based on the resource information of the target computing engine. Then, the target computing engine is invoked to execute the target computing task based on the target resources to obtain the execution result of the target computing task.

[0035] In some embodiments, such as Figure 3 As shown, step 102 specifically includes: Step 1021: Preprocess the original labels carried in the computing task processing request according to the configured label processing rules to obtain the target labels.

[0036] The configured tag processing rules may include at least one of the following: Filtering rules, such as which tags in the original tags to filter, for example, filtering tags that are not related to resource allocation (such as request submission time) or tags containing sensitive information (such as user names). Combination rules, such as which tags in the original tags to combine, for example, combining tags in the original tags that have a relationship; Transformation rules, such as which information in the original label to transform, such as transforming the format and / or meaning of the original label (e.g., the priority of the task in the original label may be described in words (e.g., "urgent", "important", "general"), which can be converted to use numbers to represent the priority (e.g., 3 represents urgent, 2 represents important, and 1 represents general)).

[0037] Accordingly, preprocessing the original labels carried in the computation task processing request may include, for example, at least one of the following: Filter the original tags carried in the computing task processing request; Combine the original tags carried in the computing task processing request; The original tags carried in the computing task processing request are converted.

[0038] In this embodiment of the disclosure, the original tags carried in the computing task processing request can be preprocessed according to the configured tag processing rules to obtain the target tags. This makes the granularity of subsequent resource determination more flexible and improves the precision and flexibility of subsequent resource determination.

[0039] The number of original tags carried in the computation task processing request can be one or more, and correspondingly, the number of target tags can also be one or more.

[0040] Step 1022: Determine the target resource corresponding to the target tag based on the resource information of the target computing engine.

[0041] The target resource is used to execute the target computing task requested by the computing task processing request.

[0042] To improve the efficiency of resource location and ensure that the target computing task can efficiently and stably acquire the required resources and execute smoothly, in this embodiment of the disclosure, the computing task processing request may carry the resource request information of the target computing task; thereby, the target tag can be locked; the target resource information corresponding to the target tag can be determined according to the resource information of the target computing engine; the target resource can be determined according to the resource request information and the target resource information of the target computing task; and the target tag can be unlocked.

[0043] The resource information of the target computing engine may be, for example, the resource record corresponding to the target computing engine. Similarly, the target resource information corresponding to the target tag may be, for example, the resource record related to the target tag.

[0044] As an example, a target tag can be locked; the resource record related to the target tag is determined from the resource record corresponding to the target computing engine; then, the target resource is determined based on the resource request information of the target computing task and the locked resource record; and finally, the previously locked target tag is unlocked.

[0045] It should be noted that locking a target tag can be understood as locking all resources related to the target tag, including resource information and actual resources.

[0046] To ensure that the target computing engine is invoked only when resources are sufficient, and to avoid task queuing and execution failures due to resource overload, in this embodiment of the disclosure, the resource request information of the target computing task may include the amount of resources requested by the target computing task; then the remaining available amount of resources indicated by the target resource information can be determined; in response to the target computing task's requested resource amount not being greater than the remaining available amount, it is determined that the target computing engine will be invoked to execute the target computing task; and the resources corresponding to the requested resource amount of the target computing task among the resources indicated by the target resource information are determined as the target resources.

[0047] It should be noted that since there can be one or more target tags, the target resource information corresponding to any target tag can be determined based on the resource information of the target computing engine. That is, the number of determined target resource information is consistent with the number of target tags. Therefore, when determining the remaining available amount of resources indicated by the target resource information, it is necessary to determine the remaining available amount of resources indicated by the target resource information corresponding to each target tag for each target tag. Then, the remaining available amount of resources indicated by the target resource information corresponding to each target tag is compared with the requested resource amount of the target computing task. When the remaining available amount of resources indicated by the target resource information corresponding to all target tags is greater than or equal to the requested resource amount of the target computing task, it is determined that the target computing engine will be invoked to execute the target computing task.

[0048] As an example, for any target tag, the remaining availability of the resource indicated by the target resource information corresponding to that target tag can be determined in the following way: 1. Query the resource information table. If the resource information table records the remaining available quantity of the resource indicated by the target resource information corresponding to the target label, then retrieve the remaining available quantity of the resource indicated by the target resource information corresponding to the target label from the resource information table. 2. If the resource information table does not record the remaining available amount of the resource indicated by the target resource information corresponding to the target label, then the remaining available amount of the resource indicated by the target resource information corresponding to the target label is calculated and determined by the formula [Remaining available resources = Maximum available resources - Used resources - Locked resources - Protected resources]. Among them, the maximum available resource is the maximum available amount of resource indicated by the target resource information corresponding to the target label; the used resource is the amount of resource already used as indicated by the target resource information corresponding to the target label; the locked resource is the amount of resource already locked as indicated by the target resource information corresponding to the target label; and the protected resource is the emergency reserve amount of resource indicated by the target resource information corresponding to the target label.

[0049] If the resource information table does not record at least one of the maximum available quantity, used quantity, locked quantity, or reserved emergency quantity of the resource indicated by the target resource information corresponding to the target label, then the configuration information is checked. If the corresponding information not recorded in the resource information table can be determined based on the configuration information, then the remaining available quantity of the resource indicated by the target resource information corresponding to the target label is calculated using the formula [Remaining available resources = Maximum available resources - Used resources - Locked resources - Protected resources]. If there is no configuration information, or the corresponding information not recorded in the resource information table cannot be determined based on the configuration information, then the resource judgment for that target label is skipped.

[0050] If there is at least one target tag, and if the remaining available resources indicated by the target resource information corresponding to that target tag are less than the requested resource amount of the target computing task, then the target computing engine is deemed to have insufficient resources. In this case, only the target tag is unlocked, and the target computing engine is not invoked to execute the target computing task. Only when the remaining available resources indicated by the target resource information corresponding to all target tags are greater than or equal to the requested resource amount of the target computing task is the target computing engine deemed to have sufficient resources, and only then is the target computing engine invoked to execute the target computing task.

[0051] Step 1023: Invoke the target computing engine to execute the target computing task based on the target resources, so as to obtain the execution result of the target computing task.

[0052] To prevent concurrent calls to the target resource by other computing tasks during the execution of the target computing task, in this embodiment of the disclosure, the target resource can be locked; the target computing engine can be invoked to execute the target computing task based on the locked target resource to obtain the execution result of the target computing task; and the target resource can be unlocked.

[0053] In order to persistently store the lock status of the target resource in a structured and traceable form, and to avoid the loss of temporary lock information due to problems such as restarts or network fluctuations, in this embodiment of the disclosure, a resource lock record corresponding to the target resource can be generated in the resource information table; wherein, the resource lock record is associated with the target tag corresponding to the target resource.

[0054] The resource lock records in the resource information table, which correspond to the target resource, are used to indicate the locked resource information of the target resource.

[0055] The resource information table can be used to record the resource information of multiple computing engines managed by the computing governance platform, or it can be used only to record the resource information of the target computing engine; there is no restriction on this.

[0056] In this embodiment of the disclosure, during the execution of the target computing task, a first broadcast message can be received; wherein the first broadcast message is used to report the usage status of target resources in the target computing engine; in response to the first broadcast message, it is determined whether there is a resource locking record corresponding to the target resource in the resource information table; if there is, the target tag associated with the resource locking record is locked; a resource usage record corresponding to the target resource is generated in the resource information table; wherein the resource usage record is used to indicate the resource usage status of the target resource, and the resource usage record is associated with the target tag corresponding to the target resource; the target tag associated with the resource locking record is unlocked.

[0057] It should be noted that the resource usage records corresponding to the target resource in the resource information table are only used to indicate the resource usage of the target resource in the target computing engine during the execution of the target computing task, and do not indicate the used resource information of the target resource.

[0058] Specifically, by receiving the first broadcast message reporting the target resource usage status during the execution of the target computing task, the dynamic usage status of the target resources (such as changes in resource occupancy, usage trends, etc.) can be captured in real time, avoiding the computing governance platform's lag in perceiving the target resource usage status. By responding to the first broadcast message, it first checks whether there is a corresponding resource lock record in the resource information table, which can pre-verify whether the target resource is in a legitimate pending management state, filtering out invalid operations on unlocked resources, and preventing record chaos caused by abnormal resource status from the source. If a resource lock record exists, the target tag associated with the resource lock record is locked, which can be done in the critical link of generating resource usage records. This section isolates the current resource data from interference by other concurrent operations (such as tag unlocking and resource status modification in other tasks), avoiding the problem of "data overlap or omission caused by multiple tasks updating resource usage records simultaneously." It ensures the safe and unique transition of resources from "locked for use" to "confirmed used." Resource usage records associated with target tags are generated in the resource information table, persistently storing the used state of target resources in a structured format. After record generation, the target tags are unlocked promptly to prevent long-term tag occupation from blocking other resource operations that depend on the tag (such as tag matching and resource release in subsequent tasks), ensuring smooth resource flow. The overall process, through a closed-loop design of "real-time perception - legal verification - locking protection - accurate recording - unlocking and releasing," not only ensures the consistency between resource usage data and actual status and the security of the update process, but also improves the efficiency and traceability of resource information management. This provides reliable data support for subsequent resource reserve judgment and scheduling decisions, further optimizing the reliability of resource management during the execution of computing tasks.

[0059] In this embodiment of the disclosure, after calling the target computing engine and executing the target computing task based on the locked target resources, a second broadcast message can be received; wherein the second broadcast message is used to instruct the reclamation of the target resources in the target computing engine; in response to the second broadcast message, it is determined whether there is a resource usage record corresponding to the target resource in the resource information table; if there is, the target tag associated with the resource lock record is locked; according to the resource usage record corresponding to the target resource, the used resource information of the target resource recorded in the resource information table is updated; and the target tag associated with the resource lock record is unlocked.

[0060] Specifically, by receiving a second broadcast message instructing the reclamation of target resources, the system can perceive the reclamation needs of resources in the target computing engine in real time, ensuring the timely response of the computing governance platform to changes in resource status and avoiding long-term idleness or misallocation of resources due to delayed reclamation information. In response to the second broadcast message, the system first checks whether a corresponding resource usage record exists in the resource information table, pre-verifying whether the resource is in a legitimate "used" reclamation state, filtering out invalid reclamation operations on unused resources, and preventing errors in updating used resource information caused by abnormal resource status from the source. If a resource usage record exists, the target tag associated with the resource usage record is locked, isolating other concurrent operations (such as other tasks' resources) during the critical stage of updating used resource information. The system prevents interference with current resource data (such as source usage record modification and tag unlocking) and avoids issues like data conflicts or distortions caused by multiple tasks simultaneously updating used resource amounts (e.g., used resources not being correctly reduced after reclamation). This ensures the accuracy and uniqueness of used resource information updates after resource reclamation. Updating used resource information based on resource usage records synchronizes the target resource's reclamation status to the data layer in real time, ensuring complete consistency between the target resource's used resource information and the actual resource release status. This provides accurate data support for subsequent resource allocation decisions, avoiding inefficient resource allocation due to data lag. Timely unlocking of target tags after updates prevents long-term tag occupation from blocking resource flow and improves resource reuse efficiency. The overall process, through a closed-loop design of "real-time perception - legal verification - locking protection - precise update - unlocking and release," ensures the accuracy and security of data updates after resource reclamation, accelerates resource reuse cycles, provides reliable support for dynamic resource scheduling and efficient management, and further optimizes the computing engine's resource utilization and overall operating efficiency.

[0061] Step 103: Send the execution result of the target computing task to the application system.

[0062] In this embodiment of the disclosure, by promptly feeding back the execution results to the application system, it can be ensured that the application system can quickly obtain task processing feedback, avoid the impact of result delays on the progress of subsequent business processes, and ensure the continuity of the application system's business logic.

[0063] In summary, by responding to computing task processing requests sent by application systems and obtaining resource information of the target computing engine matching the request from the computing governance platform, the resource information of the suitable target computing engine can be accurately located. This avoids problems such as execution failure and inefficiency caused by mismatch between computing engines and computing tasks from the source, ensuring the adaptability of computing task execution. Furthermore, based on the resource information of the target computing engine, the target computing engine is invoked to execute the target computing task requested by the computing task processing request to obtain the execution result of the target computing task. By directly invoking the target computing engine to execute the target computing task requested by the computing task processing request, the processing chain of the target computing task can be shortened, improving the execution efficiency of the target computing task. Finally, the execution result of the target computing task is sent to the application system, enabling timely feedback of results to the application system, reducing business process interruptions or waiting costs caused by result delays, and improving the operational stability of the application system and the user experience.

[0064] To clearly illustrate the computing governance platform disclosed herein, Figure 4 This is a schematic diagram of the computing governance platform shown in the fourth embodiment of this disclosure.

[0065] like Figure 4 As shown, the computing governance platform 400 may include a connector module 401, a computing governance module 402, and a public management module 403; The connector module 401 is used to establish a connection with at least one computing engine and exchange information. The computation governance module 402 is used to execute the computation task processing method provided in this disclosure; The public management module 403 is used to manage the public materials of at least one computing engine.

[0066] As an example, the connector module could be an engine connector, used to connect to the computing engine to execute computing tasks, push task information, and return results. The engine connector could include the following module capabilities:

[0067] Among them, the computing governance platform can solve the problems of difficult operation and maintenance and high resource consumption of third-party client tools in related technologies by establishing a connector module.

[0068] As an example, the public administration module includes the following capabilities: It provides unified data source capabilities: data sources are defined and managed uniformly, and application tools only need to use them by data source name, eliminating the need to maintain connection information for each data source. Furthermore, the meaning of the data source is the same across different tools. It also provides the ability to query the metadata of the corresponding data sources.

[0069] Provides public UDF capabilities: unifies the definition specifications and semantics of UDFs and small functions, so that multiple tools can use them if defined in one place.

[0070] Provides the ability to provide unified materials: Provides unified materials, supports shared access to these materials across multiple tools, and supports storing multiple file types and version control.

[0071] Provides unified configuration and variable capabilities: It provides unified configuration capabilities, supports templated configuration of different engine parameter templates, and supports custom variables, built-in commonly used system variables and time format variables, etc.

[0072] As an example, the public management module may include the following module capabilities:

[0073] Among them, the computing governance platform can manage the computing engine's public materials, such as UDFs, public variables, and resource files, in a unified manner by establishing a public management module, thereby reducing the overhead of public resource management.

[0074] As an example, the compute governance module is responsible for the unified management of all connectors and third-party compute engines, and can be managed at a granular level based on users, connectors, or other complexly tagged definitions. For example, the compute governance module can be used for: Maintain the resource information of the computing engine reported by the connector module; Process computing task processing requests sent by application systems; After the resources requested by the computing task processing request are approved, the connector module reports the resource usage of the target computing engine that matches the computing task processing request in real time throughout the lifecycle of the computing task. Provides relevant interfaces for querying current resource usage.

[0075] The connector module, as the provider of underlying resources, is responsible for registering and unregistering resources with the computing governance module. Each connector instance has a corresponding resource record in the computing governance module, which includes information such as the total resources it provides and the resources it protects, and dynamically updates the resources it has used.

[0076] The connector instance is the actual execution unit of the user job. Simultaneously, as the actual user of resources, the connector instance is responsible for reporting actual resource usage to the compute governance module. Each connector instance also has a corresponding resource record in the compute governance module: during startup, it is reflected as a resource locking record; during operation, it is reflected as a resource usage record; when the instance is terminated, the resource record is deleted.

[0077] As an example, the resource types supported by the compute governance module may include the following resource management types:

[0078] As an example, the execution flow of the computation governance module includes the following steps: Step 1: Receive the computing task processing request sent by the application system, determine the target computing engine matching the request, establish a connection with the target computing engine through the connector module, and call the resource information acquisition interface to obtain the resource information of the target computing engine. The resource information acquisition interface accepts an optional time parameter. If the waiting time for acquiring resource information exceeds the limit of this time parameter, the acquisition of resource information will automatically be treated as a failure.

[0079] Step 2: Based on the configured tag processing rules, preprocess the original tags carried in the calculation processing request to obtain the target tags.

[0080] As an example, the computation governance module can perform operations such as filtering, combining, and transforming the original tags carried in the computation processing request according to rules, so as to make the granularity of subsequent resource judgment more flexible and varied.

[0081] Step 3: Based on the resource information of the target computing engine, determine the target resource corresponding to the target tag.

[0082] As one possible implementation, the computation task processing request can carry the computation task's resource request information; the computation governance module can lock the target tag; determine the target resource information corresponding to the target tag based on the target computation engine's resource information; determine the target resource based on the computation task's resource request information and the target resource information; and unlock the target tag.

[0083] The resource request information for the computing task may include the amount of resources requested by the computing task. The process of determining the target resource based on the resource request information and the target resource information may include, for example, determining the remaining available amount of resources indicated by the target resource information; in response to the fact that the resource request amount of the computing task is not greater than the remaining available amount, determining to call the target computing engine to execute the computing task; and determining the resource corresponding to the resource request amount of the computing task from the resources indicated by the target resource information as the target resource.

[0084] As an example, the computation governance module can lock each target tag individually, ensuring that the resource information corresponding to them remains unchanged during the target resource determination process. Then, based on the resource information of the target computation engine, the target resource information corresponding to the target tag is determined, thus identifying the target resource based on the computation task's requested resource information and the target resource information. Specifically, for each target tag, the target resource is determined in the following manner: 1. Query the resource information table. If the resource information table records the remaining available amount of the resource indicated by the target resource information corresponding to the target label, directly compare whether the remaining available amount is greater than the resource amount requested by the computing task. If it is greater than or equal to, then the resource corresponding to the resource amount requested by the computing task among the resources indicated by the target resource information corresponding to the target label is determined as the target resource corresponding to the target label.

[0085] 2. If the resource information table does not record the remaining available amount of the resource indicated by the target resource information corresponding to the target label, then the remaining available amount of the resource indicated by the target resource information corresponding to the target label is calculated using the formula [Remaining Available Resources = Maximum Available Resources - Used Resources - Locked Resources - Protected Resources]. Then, the calculated remaining available amount is compared with the resource amount requested by the computing task. If it is greater than or equal to the resource amount requested by the computing task, then the resource indicated by the target resource information corresponding to the target label that corresponds to the resource amount requested by the computing task is determined as the target resource corresponding to the target label.

[0086] It should be noted that if the resource information table does not record at least one of the maximum available quantity, used quantity, locked quantity, or reserved emergency quantity of the resource indicated by the target resource information corresponding to the target label, then the configuration information is checked. If the corresponding information not recorded in the resource information table can be determined based on the configuration information, then the remaining available quantity of the resource indicated by the target resource information corresponding to the target label is calculated using the formula [Remaining Available Resources = Maximum Available Resources - Used Resources - Locked Resources - Protected Resources]. If there is no configuration information, or if the corresponding information not recorded in the resource information table cannot be determined based on the configuration information, then the resource judgment for that target label is skipped.

[0087] If there is at least one target tag, and if the remaining available resources indicated by the target resource information corresponding to that target tag are less than the requested resource amount of the computing task, then the target computing engine is considered to have insufficient resources. In this case, each target tag is unlocked one by one, and the target computing engine is not invoked to execute the computing task requested by the computing task processing request. Only when the remaining available resources indicated by the target resource information corresponding to all target tags are greater than or equal to the requested resource amount of the computing task is the target computing engine considered to have sufficient resources, and only then will the target computing engine be invoked to execute the computing task requested by the computing task processing request.

[0088] Step 4: Lock onto the target resource.

[0089] As one possible implementation, the computational governance module can generate a resource lock record corresponding to the target resource in the resource information table; wherein, the resource lock record is associated with the target tag corresponding to the target resource.

[0090] As an example, the computation governance module can generate a new record (i.e., a resource lock record) in the resource information table based on the target resource corresponding to each target label, and associate it with each target label.

[0091] It should be noted that if the resource information table records the remaining available quantity of the resource indicated by the target resource information corresponding to each target label, then for each target label, the remaining available quantity of the resource indicated by the target resource information corresponding to that target label is subtracted from the quantity of the target resource corresponding to that target label.

[0092] As one possible implementation, a timed task can be generated to check whether the locked target resource has been used after a set time. If it has not been used after the timeout, the locked target resource can be forcibly reclaimed.

[0093] Step 5: Invoke the target computing engine, and based on the locked target resources, execute the target computing task to obtain the execution result of the target computing task.

[0094] Step 6: Send the execution results of the target computing task to the application system.

[0095] Step 7: Record the usage of the target resource.

[0096] As one possible implementation, the computation governance module can receive a first broadcast message during the execution of the target computation task; wherein the first broadcast message is used to report the usage status of the target resources in the target computation engine; in response to the first broadcast message, it determines whether there is a resource locking record corresponding to the target resource in the resource information table; if there is, it locks the target tag associated with the resource locking record; it generates a resource usage record corresponding to the target resource in the resource information table; wherein the resource usage record is used to indicate the resource usage status of the target resource, and the resource usage record is associated with the target tag corresponding to the target resource; and it unlocks the target tag associated with the resource locking record.

[0097] As an example, after the connector instance connected to the target computing engine starts, it can send a first broadcast message to the computing governance module, reporting the usage status of the target resources in the target computing engine. Upon receiving the first broadcast message, the computing governance module can determine whether a resource lock record corresponding to the target resource exists in the resource information table; if not, it will report an error. If it does exist, it will lock the target tag associated with the resource lock record and convert the corresponding resource lock record into a resource usage record for each target tag. Afterwards, all target tags will be unlocked.

[0098] Step 8: Update the used resource information of the target resource.

[0099] As one possible implementation, the computation governance module can receive a second broadcast message; wherein the second broadcast message is used to instruct the reclamation of target resources in the target computation engine; in response to the second broadcast message, it determines whether there is a resource usage record corresponding to the target resource in the resource information table; if there is, it locks the target tag associated with the resource lock record; according to the resource usage record corresponding to the target resource, it updates the used resource information of the target resource recorded in the resource information table; and it unlocks the target tag associated with the resource lock record.

[0100] As an example, after the connector instance connected to the target computing engine reaches the end of its lifecycle, it can send a second broadcast message to the computing governance module, instructing that the target resources in the target computing engine be reclaimed. Upon receiving the second broadcast message, the computing governance module can determine whether a resource usage record corresponding to the target resource exists in the resource information table. If so, it locks the target tag associated with the resource lock record and, for each target tag, updates the used resource information of the corresponding target resource according to the corresponding resource usage record in the resource information table. For example, it adds the used resource information of the corresponding target resource to the resource usage quantity indicated by the corresponding resource usage record in the resource information table. Then, it unlocks all target tags.

[0101] As an example, Figure 5This is a schematic diagram illustrating the relationship between the application system, the computing governance platform, and the computing engine as shown in the fifth embodiment of this disclosure.

[0102] like Figure 5 As shown, the application system sends a computing task processing request to the computing governance platform. The computing task processing request carries the identification information of the computing engine (such as the type and version of the execution engine selected by the application system) and the parameter information related to the requested target computing task (such as operation parameters and other parameters (such as resource requirements, server requirements, special configurations, etc.)).

[0103] The application systems include APP_A, APP_B, ..., APP_N.

[0104] The computing governance platform includes a public management module, a computing governance module, and a connector module. Through these modules, the platform invokes the target computing engine that matches the computing task processing request to execute the target computing task.

[0105] The common management module manages the context, materials, metadata, and data sources of at least one computing engine; the computing governance module implements the following functions: status monitoring; connector management; concurrency control; clustering and high availability; resource relationships; retry and exception handling; the connector module implements the following functions: connector implementation; execution process log query; execution status query; execution result query; computing engine status monitoring; and other capabilities supported by the computing engine.

[0106] The computing governance platform invokes the connector module to establish a connection with the computing engine via client / API (Application Programming Interface) / HTTP (Hypertext Transfer Protocol) requests. A connector can have multiple implementations.

[0107] The computing engines include Flink, Hive, Spark, ..., and DB.

[0108] Because the computing governance platform includes a connector module, the task processing sequence becomes more ordered. Figure 6 This is a task processing timing diagram shown in the sixth embodiment of this disclosure.

[0109] like Figure 6As shown, the application system submits a task to the computing governance platform. The computing governance module in the computing governance platform receives the task submitted by the application system, performs a pre-hook operation, and then submits the task to the connector module in the computing governance platform. The connector module establishes a connection with the third-party computing engine and submits the task to the third-party computing engine. The third-party computing engine returns the task submission result (submission successful / submission failed) to the connector module, which then returns it to the computing governance module, which in turn returns the task submission result to the application system. Specifically, the application system initiates a task execution log tracking request to the computing governance platform. The computing governance module in the computing governance platform submits a log and task status query request to the connector module, which in turn submits the log and task status query request to the third-party computing engine. The third-party computing engine returns the log and task status information to the connector module, which then returns it to the computing governance module, which in turn returns the log and task status information to the application system. The application system then updates the task status and logs. Finally, the third-party computing engine returns the task execution result to the connector module, which in turn returns it to the computing governance module, which in turn returns the task execution result to the application system.

[0110] Pre-hook operations refer to actions or hooks that are performed in advance before a main task is executed. These operations are typically used to perform preparatory work or checks to ensure that the main task can proceed smoothly.

[0111] Corresponding to the computing task processing method provided in the above embodiments, this disclosure also provides a computing task processing apparatus. Since the computing task processing apparatus provided in this disclosure corresponds to the computing task processing method provided in the above embodiments, the implementation of the computing task processing method is also applicable to the computing task processing apparatus provided in this disclosure, and will not be described in detail in this disclosure.

[0112] Figure 7 This is a schematic diagram of the computing task processing device shown in the seventh embodiment of this disclosure, which is applied to a computing governance platform.

[0113] like Figure 7 As shown, the computing task processing device 700 includes: an acquisition module 701, a processing module 702, and a sending module 703.

[0114] The acquisition module 701 is used to acquire resource information of a target computing engine that matches the computing task processing request in response to the computing governance platform receiving a computing task processing request sent by the application system; the processing module 702 is used to call the target computing engine to execute the target computing task requested by the computing task processing request based on the resource information of the target computing engine, so as to obtain the execution result of the target computing task; and the sending module 703 is used to send the execution result of the target computing task to the application system.

[0115] As one possible implementation of this disclosure, the processing module 702 includes: a preprocessing unit, configured to preprocess the original tags carried in the computing task processing request according to the configured tag processing rules to obtain target tags; a determining unit, configured to determine the target resources corresponding to the target tags according to the resource information of the target computing engine; and a processing unit, configured to call the target computing engine and execute the target computing task based on the target resources to obtain the execution result of the target computing task.

[0116] As one possible implementation of this disclosure, the processing unit is specifically configured to lock the target resource; invoke the target computing engine to execute the target computing task based on the locked target resource, so as to obtain the execution result of the target computing task; and unlock the target resource.

[0117] As one possible implementation of this disclosure, the processing unit is specifically configured to generate a resource lock record corresponding to the target resource in a resource information table; wherein the resource lock record is associated with the target tag corresponding to the target resource.

[0118] As a possible implementation of this disclosure, the above apparatus further includes: a first receiving module, configured to receive a first broadcast message during the execution of the target computing task; wherein the first broadcast message is used to report the usage status of the target resource in the target computing engine; a first determining module, configured to determine, in response to the first broadcast message, whether there is a resource locking record corresponding to the target resource in the resource information table; a first locking module, configured to lock the target tag associated with the resource locking record if it exists; a generating module, configured to generate a resource usage record corresponding to the target resource in the resource information table; wherein the resource usage record is used to indicate the resource usage status of the target resource, and the resource usage record is associated with the target tag corresponding to the target resource; and a first unlocking module, configured to unlock the target tag associated with the resource locking record.

[0119] As a possible implementation of this disclosure, the above apparatus further includes: a second receiving module, configured to receive a second broadcast message; wherein the second broadcast message is used to instruct the reclamation of the target resource in the target computing engine; a second determining module, configured to determine, in response to the second broadcast message, whether there is a resource usage record corresponding to the target resource in the resource information table; a second locking module, configured to lock the target tag associated with the resource locking record if it exists; an updating module, configured to update the used resource information of the target resource recorded in the resource information table according to the resource usage record corresponding to the target resource; and a second unlocking module, configured to unlock the target tag associated with the resource locking record.

[0120] As one possible implementation of this disclosure, the computing task processing request carries the resource request information of the target computing task; the determining unit is specifically used for: locking the target tag; determining the target resource information corresponding to the target tag based on the resource information of the target computing engine; determining the target resource based on the resource request information of the target computing task and the target resource information; and unlocking the target tag.

[0121] As one possible implementation of this disclosure, the resource request information of the target computing task includes the amount of resources requested by the target computing task; the determining unit is specifically configured to: determine the remaining available amount of resources indicated by the target resource information; in response to the fact that the amount of resources requested by the target computing task is not greater than the remaining available amount, determine to call the target computing engine to execute the target computing task; and determine the resources corresponding to the amount of resources requested by the target computing task from the resources indicated by the target resource information as the target resources.

[0122] As one possible implementation of this disclosure, the computing task processing request carries identification information of the computing engine; the acquisition module 701 is further configured to: in response to the computing governance platform receiving a computing task processing request sent by the application system, determine the target computing engine based on the identification information of the computing engine carried in the computing task processing request; and acquire the resource information of the target computing engine.

[0123] The computing task processing apparatus of this disclosure, in response to a computing governance platform receiving a computing task processing request sent by an application system, obtains resource information of a target computing engine matching the computing task processing request. This allows for precise location of the resource information of the compatible target computing engine, preventing execution failures and inefficiencies caused by mismatch between the computing engine and the computing task from the source, thus ensuring the adaptability of computing task execution. Furthermore, based on the resource information of the target computing engine, the apparatus calls the target computing engine to execute the target computing task requested by the computing task processing request, thereby obtaining the execution result of the target computing task. By directly calling the target computing engine to execute the target computing task requested by the computing task processing request, the processing chain of the target computing task can be shortened, improving the execution efficiency of the target computing task. Finally, the apparatus sends the execution result of the target computing task to the application system, enabling timely feedback of results to the application system, reducing business process interruptions or waiting costs caused by result delays, and improving the operational stability of the application system and the user experience.

[0124] In an exemplary embodiment, an electronic device is also proposed.

[0125] The electronic devices include: processor; Memory used to store processor-executable instructions; The processor is configured to execute instructions to implement the computational task processing method proposed in any of the foregoing embodiments.

[0126] As an example, Figure 8 This is a schematic diagram of the structure of an electronic device 800 as shown in an exemplary embodiment of this disclosure, as follows: Figure 8 As shown, the above-mentioned electronic device 800 may further include: The present invention includes a memory 810 and a processor 820, and a bus 830 connecting different components (including the memory 810 and the processor 820). The memory 810 stores a computer program, and when the processor 820 executes the program, it implements the computing task processing method described in the embodiments of the present disclosure.

[0127] Bus 830 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0128] Electronic device 800 typically includes a variety of electronic device readable media. These media can be any available media that can be accessed by electronic device 800, including volatile and non-volatile media, removable and non-removable media.

[0129] The memory 810 may also include computer system readable media in the form of volatile memory, such as random access memory (RAM) 840 and / or cache memory 850. The server may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 860 may be used to read and write non-removable, non-volatile magnetic media (…). Figure 8 Not shown; usually referred to as a "hard drive"). Although Figure 8 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 830 via one or more data media interfaces. Memory 810 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of this disclosure.

[0130] A program / utility 880 having a set (at least one) of program modules 870 may be stored, for example, in memory 810. Such program modules 870 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 870 typically perform the functions and / or methods described in the embodiments of this disclosure.

[0131] Electronic device 800 can also communicate with one or more external devices 890 (e.g., keyboard, pointing device, display 891, etc.), and with one or more devices that enable a user to interact with electronic device 800, and / or with any device that enables electronic device 800 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via input / output (I / O) interface 892. Furthermore, electronic device 800 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 893. As shown, network adapter 893 communicates with other modules of electronic device 800 via bus 830. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 800, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0132] The processor 820 performs various functional applications and data processing by running programs stored in the memory 810.

[0133] It should be noted that the implementation process and technical principles of the electronic device in this embodiment are explained in the foregoing description of the computing task processing method of the present disclosure embodiment, and will not be repeated here.

[0134] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory including instructions, which can be executed by a processor of an electronic device to perform the computational task processing method proposed in any of the above embodiments. Optionally, the computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0135] In an exemplary embodiment, a computer program product is also provided, including a computer program / instructions that, when executed by a processor, implement the computing task processing method proposed in any of the above embodiments.

[0136] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0137] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A method for processing computational tasks, characterized in that, Applications in computing governance platforms include: In response to the computing governance platform receiving a computing task processing request sent by the application system, the resource information of the target computing engine that matches the computing task processing request is obtained. Based on the resource information of the target computing engine, the target computing engine is invoked to execute the target computing task requested by the computing task processing request, so as to obtain the execution result of the target computing task; The execution result of the target computing task is sent to the application system.

2. The method according to claim 1, characterized in that, The step of invoking the target computing engine to execute the target computing task requested by the computing task processing request based on the resource information of the target computing engine, so as to obtain the execution result of the target computing task, includes: According to the configured tag processing rules, the original tags carried in the computing task processing request are preprocessed to obtain the target tags; Based on the resource information of the target computing engine, determine the target resource corresponding to the target tag; The target computing engine is invoked to execute the target computing task based on the target resources, so as to obtain the execution result of the target computing task.

3. The method according to claim 2, characterized in that, The step of invoking the target computing engine to execute the target computing task based on the target resources, and obtaining the execution result of the target computing task, includes: Lock the target resource; The target computing engine is invoked to execute the target computing task based on the locked target resources, so as to obtain the execution result of the target computing task; Unlock the target resource.

4. The method according to claim 3, characterized in that, The locking of the target resource includes: Generate a resource lock record corresponding to the target resource in the resource information table; wherein the resource lock record is associated with the target tag corresponding to the target resource.

5. The method according to claim 4, characterized in that, The method further includes: During the execution of the target computing task, a first broadcast message is received; wherein the first broadcast message is used to report the usage status of the target resources in the target computing engine; In response to the first broadcast message, determine whether there is a resource lock record corresponding to the target resource in the resource information table; If it exists, then lock the target tag associated with the resource locking record; A resource usage record corresponding to the target resource is generated in the resource information table; wherein, the resource usage record is used to indicate the resource usage status of the target resource, and the resource usage record is associated with the target tag corresponding to the target resource; Unlock the target tag associated with the resource lock record.

6. The method according to claim 5, characterized in that, After invoking the target computing engine and executing the target computing task based on the locked target resources, the process further includes: Receive a second broadcast message; wherein the second broadcast message is used to instruct the reclamation of the target resources in the target computing engine; In response to the second broadcast message, determine whether there is a resource usage record corresponding to the target resource in the resource information table; If it exists, then lock the target tag associated with the resource locking record; Update the used resource information of the target resource recorded in the resource information table according to the resource usage record corresponding to the target resource; Unlock the target tag associated with the resource lock record.

7. The method according to claim 2, characterized in that, The computing task processing request carries the resource request information of the target computing task; determining the target resource corresponding to the target tag based on the resource information of the target computing engine includes: Lock the target label; Based on the resource information of the target computing engine, determine the target resource information corresponding to the target tag; The target resource is determined based on the resource request information of the target computing task and the target resource information; Unlock the target tag.

8. The method according to claim 7, characterized in that, The resource request information for the target computing task includes the amount of resources requested for the target computing task; determining the target resources based on the resource request information and the target resource information includes: Determine the remaining available amount of the resource indicated by the target resource information; If the requested resource amount for the target computing task is not greater than the remaining available resource amount, it is determined that the target computing engine will be invoked to execute the target computing task. The resource corresponding to the requested resource amount of the target computing task among the resources indicated by the target resource information is determined as the target resource.

9. The method according to claim 1, characterized in that, The computing task processing request carries the identification information of the computing engine; the response of the computing governance platform to receiving the computing task processing request sent by the application system and obtaining the resource information of the target computing engine matching the computing task processing request includes: In response to the computing governance platform receiving a computing task processing request sent by the application system, the target computing engine is determined based on the identification information of the computing engine carried in the computing task processing request. Obtain the resource information of the target computing engine.

10. A computing governance platform, characterized in that, include: Connector module, computing governance module, and public management module; The connector module is used to establish a connection with at least one computing engine and exchange information. The computational governance module is used to execute the method as described in any one of claims 1-9; The public management module is used to manage the public materials of the at least one computing engine.

Images