Method and apparatus for executing job in private cloud environment

Abstract

The present disclosure provides a job execution method in a private cloud environment, and relates to the technical field of artificial intelligence such as big data, cloud service, cloud computing and deep learning. The job execution method comprises the following steps: obtaining a target job through a job query interface; determining a target machine cluster in a candidate machine cluster in the private cloud environment according to the target job; obtaining node information of nodes in the target machine cluster through a node query interface, and determining a target node in the target machine cluster according to the node information and the target job; and sending target data corresponding to the target job to the target node through a job creation interface, so that the target node creates the target job and processes the target data. The present disclosure can realize the execution of the job in the non-invasive private cloud environment, and enhances the security of the private cloud and improves the efficiency of the private cloud in executing the job.

Description

Execution methods and devices for operations in a private cloud environment Technical Field

[0001] This disclosure relates to the field of data processing technology, and more particularly to the fields of big data, cloud services, cloud computing, deep learning, and other artificial intelligence technologies. It provides a method, apparatus, electronic device, and readable storage medium for executing tasks in a private cloud environment. Background Technology

[0002] With the rapid development of cloud computing services, more and more enterprises are using private clouds to meet their computing needs. However, due to the strict limitations on the use of private clouds, how to ensure job execution while maintaining the security of private clouds is a pressing technical problem that needs to be solved. Summary of the Invention

[0003] According to a first aspect of this disclosure, a method for executing a job in a private cloud environment is provided, comprising: obtaining a target job through a job query interface; determining a target machine cluster among candidate machine clusters in the private cloud environment based on the target job; obtaining node information of nodes in the target machine cluster through a node query interface, and determining a target node in the target machine cluster based on the node information and the target job; and sending target data corresponding to the target job to the target node through a job creation interface, so that the target node can create the target job and process the target data.

[0004] According to a second aspect of this disclosure, an execution apparatus for a job in a private cloud environment is provided, comprising: an acquisition unit, configured to acquire a target job through a job query interface; a first determination unit, configured to determine a target machine cluster among candidate machine clusters in a private cloud environment based on the target job; a second determination unit, configured to acquire node information of nodes in the target machine cluster through a node query interface, and determine a target node in the target machine cluster based on the node information and the target job; and an execution unit, configured to send target data corresponding to the target job to the target node through a job creation interface, so that the target node can create the target job and process the target data.

[0005] According to a third aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method as described above.

[0006] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform the method described above.

[0007] According to a fifth aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the method described above.

[0008] According to a sixth aspect of this disclosure, an execution system for operating in a private cloud environment is provided, comprising the aforementioned electronic device, a machine cluster in the private cloud environment, and a storage device.

[0009] As can be seen from the above technical solutions, this disclosure achieves the purpose of performing operations in a private cloud environment without requiring intrusive operations such as component installation on the machine cluster in the private cloud environment, thereby enhancing the security of the private cloud. Furthermore, this embodiment directly sends the target data to the determined target node, eliminating the need for the machine cluster in the private cloud environment to determine the target node, thus improving the efficiency of the private cloud in performing operations.

[0010] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0011] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0012] Figure 1 is a schematic diagram according to a first embodiment of the present disclosure;

[0013] Figure 2 is a schematic diagram according to a second embodiment of the present disclosure;

[0014] Figure 3 is a schematic diagram according to a third embodiment of the present disclosure;

[0015] Figure 4 is a schematic diagram according to the fourth embodiment of the present disclosure;

[0016] Figure 5 is a schematic diagram according to the fifth embodiment of the present disclosure;

[0017] Figure 6 is a schematic diagram according to a sixth embodiment of the present disclosure;

[0018] Figure 7 is a schematic diagram according to the seventh embodiment of the present disclosure;

[0019] Figure 8 is a block diagram of an electronic device used to implement the method for executing a job in a private cloud environment according to embodiments of the present disclosure. Detailed Implementation

[0020] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and mechanisms are omitted in the following description.

[0021] Figure 1 is a schematic diagram according to the first embodiment of this disclosure. As shown in Figure 1, the method for executing a job in a private cloud environment according to this embodiment specifically includes the following steps:

[0022] S101. Obtain the target job through the job query interface;

[0023] S102. Based on the target task, determine the target machine cluster among the candidate machine clusters in the private cloud environment;

[0024] S103. Obtain node information of nodes in the target machine cluster through the node query interface, and determine the target node in the target machine cluster based on the node information and the target job;

[0025] S104. The target data corresponding to the target job is sent to the target node through the job creation interface, so that the target node can create the target job and process the target data.

[0026] The method for executing jobs in a private cloud environment in this embodiment completes the acquisition of target jobs, the determination of target nodes, and the sending of target data through different interfaces. This achieves the purpose of executing jobs in a private cloud environment without the need for intrusive operations such as component installation on the machine cluster in the private cloud environment, thereby enhancing the security of the private cloud. Furthermore, this embodiment sends the target data directly to the target node in the target machine cluster, eliminating the need for the machine cluster in the private cloud environment to determine the target node, thus improving the efficiency of the private cloud in executing jobs.

[0027] The execution entity of the job execution method in the private cloud environment of this embodiment can be a server or terminal device connected to the private cloud through multiple interfaces; the server or terminal device can achieve the purpose of executing jobs in the private cloud without intruding on the private cloud by relying only on the query permissions of the nodes in the machine cluster in the private cloud environment.

[0028] In this embodiment, when executing S101, the target job is obtained through the job query interface. The obtained target job is the task to be executed in the machine cluster, such as model training job, speech recognition job, etc. This embodiment does not limit the job type of the target job.

[0029] In this embodiment, the server or terminal device is connected to the storage device through a job query interface. The storage device stores jobs created by the user and job data corresponding to different jobs, such as training samples corresponding to model training jobs and voice data corresponding to speech recognition jobs.

[0030] In this embodiment, the number of target jobs obtained by executing S101 can be one, that is, one target job is executed in a private cloud environment; or it can be multiple, that is, multiple target jobs are executed simultaneously in a private cloud environment. The job types of the multiple target jobs can be the same or different.

[0031] In this embodiment, after obtaining the target job through the job query interface in S101, the obtained target job can be registered. Then, based on the registration information of the target job, operations such as executing the target job, managing the target job, and synchronizing the status of the target job in the private cloud environment can be completed. Different target jobs have different registration information.

[0032] In this embodiment, when registering the target job in S101, either static registration or dynamic registration can be used. Static registration is local registration, which is performed through a configuration file located locally. Dynamic registration is performed through the registration service provided by the RPC (Remote Procedure Call Protocol) server.

[0033] In this embodiment, after the target job is registered in step S101, the target job can be considered to be effective. Then, based on the registration information of the target job, operations such as executing the target job, managing the target job, and synchronizing the status of the target job can be performed.

[0034] After obtaining the target job in S101, this embodiment can also determine the target plugin corresponding to the target job. In this embodiment, the job plugins correspond to different job types. The target plugin can be determined by the job type of the target job, thereby enabling operations such as executing the target job, managing the target job, and synchronizing the status of the target job through the target plugin, which improves the convenience when executing, managing, or synchronizing the target job.

[0035] In this embodiment, the plugins corresponding to different job types can support in-tree mode (i.e., run together with the source code corresponding to the method provided in this embodiment) or out-of-tree mode (i.e., run separately from the source code corresponding to the method provided in this embodiment and run independently).

[0036] In this embodiment, after executing S101 to obtain the target job, S102 is executed to determine the target machine cluster in the candidate machine clusters in the private cloud environment based on the target job. In this embodiment, the private cloud environment may contain multiple candidate machine clusters. Different candidate machine clusters may have the same cluster type or different cluster types, such as Kubernetes clusters, Yarn clusters, etc.

[0037] In this embodiment, when performing S102 to determine the target machine cluster in the candidate machine clusters in the private cloud environment based on the target job, the optional implementation method can be as follows: obtain the job information of the target job, which may include job type, job-specified cluster, job resource request amount, job-specified node identifier information, job-specified node tag, etc.; and take the machine cluster in the candidate machine cluster that matches the obtained job information as the target machine cluster.

[0038] In other words, this embodiment can determine the target machine cluster based on the job information of the target job, so that the determined target machine cluster corresponds to the job type or job-specified cluster of the target job, thereby improving the accuracy of the determined target machine cluster.

[0039] In this embodiment, when executing S102 to select the machine cluster in the candidate machine cluster that matches the obtained job information as the target machine cluster, the following method can also be adopted: Based on the job information, obtain the resource request amount of the target job, which is the total amount of machine resources required to execute the target job. The machine resources can be memory resources, CPU resources, GPU resources, etc.; Based on the resource request amount and the resource amount of the candidate machine cluster, determine the target machine cluster from the candidate machine cluster.

[0040] In other words, this embodiment determines the target machine cluster from the candidate machine clusters by the amount of resource requests in the job information, so that the determined target machine cluster has sufficient machine resources to execute the target job, thereby improving the success rate of the target job execution.

[0041] In this embodiment, when executing S102 to determine the target machine cluster among the candidate machine clusters based on the resource request amount and the resource amount of the candidate machine clusters, the candidate machine cluster whose resource amount can meet the resource request amount can be used as the target machine cluster; if there are multiple candidate machine clusters that can meet the resource request amount, the candidate machine cluster with the lowest resource utilization rate can be used as the target machine cluster, or a randomly selected candidate machine cluster can be used as the target machine cluster.

[0042] In this embodiment, when performing S102 to determine the target machine cluster among the candidate machine clusters in the private cloud environment based on the target job, the following may also be included: performing permission verification on the private cloud environment. In this embodiment, the user name corresponding to the private cloud environment can be used to verify whether the private cloud environment can execute the target job. This can be done by verifying whether the private cloud environment has permission to use the cloud product corresponding to the target job, or by verifying whether a neural network model corresponding to the target job exists in the private cloud environment. If the permission verification result is determined to be that the private cloud environment can execute the target job (e.g., the private cloud environment has permission to use the relevant cloud product, or a relevant neural network model exists in the private cloud environment), the operation of determining the target machine cluster among the candidate machine clusters in the private cloud environment based on the target job is performed.

[0043] In other words, this embodiment ensures that the private cloud environment can execute the target job by verifying the private cloud environment, thus avoiding the problem that the private cloud environment cannot execute the job after the target node is determined due to the lack of relevant permissions, thereby improving the success rate of job execution.

[0044] In this embodiment, after executing S102 to determine the target machine cluster, S103 is executed to obtain the node information of the nodes in the target machine cluster through the node query interface, and the target node in the target machine cluster is determined based on the node information and the target job; in this embodiment, the node is the machine located in the machine cluster.

[0045] In this embodiment, when executing S103 to obtain node information of nodes in the target machine cluster through the node query interface and determine the target node in the target machine cluster based on the node information and the target job, the optional implementation method can be as follows: obtain the job information of the target job, which may include the resource request amount of the job, the node identification information specified by the job, the node label specified by the job, etc.; take the nodes corresponding to the node information that matches the job information as candidate nodes; and determine the target node from the candidate nodes.

[0046] In other words, this embodiment obtains the node information of the target machine cluster through the node query interface connected to the private cloud environment, and then determines the target node based on the node information and the job information of the target job. This achieves the goal of relying only on the node query permissions in the cluster, without adding additional components to the private cloud environment to intrude into the private cloud environment, thereby enhancing the security of the private cloud environment.

[0047] In this embodiment, when executing S103 to select nodes corresponding to node information matching the job information as candidate nodes, the optional implementation method can be as follows: determine the resource request amount of the target job based on the job information; determine the available resource amount of the node based on the node information; and select nodes whose available resource amount is greater than the resource request amount as candidate nodes.

[0048] In other words, this embodiment determines candidate nodes by the resource request amount of the target job and the available resources of the nodes, ensuring that the determined candidate nodes have sufficient resources to process the target data corresponding to the target job, thereby improving the success rate of job execution.

[0049] In this embodiment, when executing S103 to select nodes with available resources greater than resource requests as candidate nodes, the optional implementation method can be as follows: select nodes with available resources greater than resource requests as initial nodes; determine the identification information (e.g., node name) and / or tag of the initial nodes based on the node information of the initial nodes; and select the initial nodes corresponding to the identification information and / or tags that match the job information as candidate nodes.

[0050] In other words, after selecting nodes whose available resources can meet the resource request amount as initial nodes, this embodiment can further determine candidate nodes from the initial nodes based on the node's identification information and / or tags, so that the determined candidate nodes are more compatible with the target job while being able to execute the target job.

[0051] In this embodiment, when executing S103 to select the initial node corresponding to the identification information and / or tag that matches the job information as a candidate node, the initial node corresponding to the identification information of the job specified node can be selected as a candidate node, and the initial node corresponding to the tag of the job specified node can also be selected as a candidate node.

[0052] In this embodiment, when executing S103 to determine the target node from the candidate nodes, the candidate nodes can be sorted according to their resource utilization rate or available resource quantity, and the top N candidate nodes can be used as the target nodes, where N is a positive integer greater than or equal to 1; alternatively, multiple nodes can be randomly selected from the candidate nodes as the target nodes.

[0053] In this embodiment, after determining the target node in S103, S104 is executed to send the target data corresponding to the target job to the target node through the job creation interface, so that the target node can create the target job and process the target data.

[0054] In this embodiment, when executing S104, the target data corresponding to the target job can be obtained first through the job query interface, and then the obtained target data can be sent to the target node through the job creation interface.

[0055] In this embodiment, when executing S105, the target data corresponding to the target job is sent to the target node through the job creation interface, the optional implementation method is as follows: the target job is split into multiple sub-target jobs; the sub-target data corresponding to the multiple sub-target jobs is sent to multiple target nodes through the job creation interface, so that the multiple target nodes can create sub-target jobs and process the sub-target data respectively.

[0056] In other words, this embodiment can also split the target job into sub-target jobs, and then have multiple target nodes execute the sub-target jobs respectively, so as to achieve the purpose of parallel processing of the target job and improve the execution efficiency of the target job.

[0057] In this embodiment, when executing S105, if a target plugin corresponding to the target job is determined, the target data corresponding to the target job is sent to the target node through the job creation interface and the target plugin during the execution of S105. Specifically, the target data is first sent to the target plugin through the job creation interface, and then the target plugin sends the received target data to the target node.

[0058] Figure 2 is a schematic diagram according to a second embodiment of the present disclosure. As shown in Figure 1, the method for executing a job in a private cloud environment according to this embodiment may further include the following:

[0059] S201. Obtain the operation request corresponding to the target job;

[0060] S202. The operation request is sent to the target node through the job management interface so that the target node can process the target job according to the operation request.

[0061] In other words, this embodiment interacts with the target node through the job management interface, enabling the operation of the target job being executed on the target node, thereby improving the flexibility of job execution in a private cloud environment.

[0062] In this embodiment, the operation request obtained in S201 is an operation request made by the user for the target job. The operation corresponding to the operation request can be deleting the target job, stopping the target job, updating the target job, etc. In the case of multiple target jobs, this embodiment can determine the operation request corresponding to different target jobs based on the registration information or target plugins corresponding to the target jobs.

[0063] In this embodiment, when the operation request is sent to the target node through the job management interface in S202, if a target plugin corresponding to the target job is determined, the operation request can be sent to the target node through the job management interface and the target plugin in S202. Specifically, the operation request is first sent to the target plugin through the job management interface, and then the target plugin sends the received operation request to the target node.

[0064] It is understood that the job query interface, node query interface, job creation interface, job management interface, etc. in this embodiment can be a unified interface, that is, the unified interface can realize the functions of job query, node query, job creation, job management, etc.

[0065] Figure 3 is a schematic diagram according to a third embodiment of the present disclosure. As shown in Figure 3, the method for executing a job in a private cloud environment according to this embodiment may further include the following:

[0066] S301. Obtain the job status sent by the target node through the job management interface;

[0067] S302. The job status is sent to the storage device through the job query interface so that the storage device can display the job status.

[0068] In other words, this embodiment interacts with the target node through the job query interface and the job management interface, which can synchronize the job status of the target node when executing the target job. This allows users to have a clearer understanding of the execution status of the target job and then perform corresponding operations on the target job based on the execution status, thereby improving the convenience of executing the target job in a private cloud environment.

[0069] The job status received in S301 of this embodiment can be whether the target node has completed the execution of the target job, or whether the target node has encountered a fault while executing the target job. When there are multiple target jobs, this embodiment can determine the job status corresponding to different target jobs based on the registration information or job plugins corresponding to the target jobs.

[0070] In this embodiment, after executing S302 to send the job status to the storage device through the job query interface, the storage device can display the job status to the user, and the user can perform operations such as updating, deleting, or stopping the target job based on the displayed job status.

[0071] In this embodiment, when executing S301 to obtain the job status sent by the target node through the job management interface, if a target plugin corresponding to the target job is determined, this embodiment can obtain the job status sent by the target node through the job management interface and the target plugin when executing S301. Specifically, the job status sent by the target node in the job plugin is obtained through the job management interface. That is, the target node will send the job status of the target job to the job plugin corresponding to the target job, thereby achieving the purpose of obtaining the job status from the job plugin through the job management interface.

[0072] Figure 4 is a schematic diagram according to the fourth embodiment of this disclosure. Figure 4 shows a schematic diagram of the dynamic registration of job plugins in this embodiment: the target job sends a registration request to the RPC server responsible for registration; the RPC server performs dynamic registration according to the registration request, and after the registration of the target job is completed, it determines the target plugin corresponding to the target job, thereby realizing operations such as executing the target job, managing the target job, and synchronizing the status of the target job through the target plugin.

[0073] Figure 5 is a schematic diagram according to the fifth embodiment of this disclosure. Figure 5 shows the structure of the operation of the job in a private cloud environment in this embodiment: The device for executing the job in the private cloud environment (e.g., a server or terminal device) is connected to the storage device through a job query interface to obtain the target job, obtain the target data, and synchronize the job status, etc.; The device for executing the job in the private cloud environment realizes job creation, job operation (update, delete, stop), and job status synchronization through a unified interface (including a node query interface, a job creation interface, a job management interface, etc.); If there is a job plugin corresponding to the target job, this embodiment can connect to the job plugin through the corresponding interface (job creation interface, job management interface, etc.), and then realize the interaction between the job plugin and the target machine cluster, for example, sending the target data to the job plugin through the job creation interface, and then the job plugin sending the target data to the target node, etc.

[0074] Figure 6 is a schematic diagram according to the sixth embodiment of this disclosure. Figure 6 shows a flowchart of the execution of a job in a private cloud environment according to this embodiment: After the user creates a target job in the storage device, the device executing the job in the private cloud environment starts a scheduling cycle, which is the time period during which the target data corresponding to the target job is sent to the target node for execution; the device executing the job in the private cloud environment requests the job query interface in the unified interface to obtain the created target job; then, based on the judgment result of whether the target job specifies a machine cluster, the target machine cluster is determined from the candidate machine clusters, and the target node is determined from the target machine cluster through the request node query interface; after the target node is determined, the target data corresponding to the target job is sent to the target node through the request job creation interface in the unified interface, so that the target node processes the target data after the target job is created.

[0075] Figure 7 is a schematic diagram according to the seventh embodiment of the present disclosure. As shown in Figure 7, the execution device 700 for working in a private cloud environment according to this embodiment includes:

[0076] Acquisition unit 701 is used to obtain the target job through the job query interface;

[0077] The first determining unit 702 is used to determine the target machine cluster among the candidate machine clusters in the private cloud environment based on the target job.

[0078] The second determining unit 703 is used to obtain node information of nodes in the target machine cluster through a node query interface, and determine the target node in the target machine cluster based on the node information and the target job.

[0079] The execution unit 704 is used to send target data corresponding to the target job to the target node through the job creation interface, so that the target node can create the target job and process the target data.

[0080] The acquisition unit 701 obtains the target job through the job query interface. The obtained target job is the task to be executed in the machine cluster, such as model training job, speech recognition job, etc. In this embodiment, the job type of the target job is not limited.

[0081] The number of target jobs acquired by the acquisition unit 701 can be one, that is, one target job is executed in the private cloud environment; or it can be multiple, that is, multiple target jobs are executed simultaneously in the private cloud environment. The job types of the multiple target jobs can be the same or different.

[0082] After obtaining the target job through the job query interface, the acquisition unit 701 can also register the obtained target job, and then, based on the registration information of the target job, complete operations such as executing the target job, managing the target job, and synchronizing the status of the target job in the private cloud environment; among them, different target jobs have different registration information.

[0083] When the acquisition unit 701 registers the target job, it can use either static registration or dynamic registration. Static registration is local registration, which is performed through a configuration file located locally. Dynamic registration is performed through the registration service provided by the RPC (Remote Procedure Call Protocol) server.

[0084] After the acquisition unit 701 completes the registration of the target job, the target job can be considered to be effective. Then, based on the registration information of the target job, operations such as executing the target job, managing the target job, and synchronizing the status of the target job can be implemented.

[0085] After acquiring the target job, the acquisition unit 701 can also determine the target plugin corresponding to the target job. In this embodiment, the job plugin corresponds to different job types. The target plugin can be determined by the job type of the target job, thereby enabling operations such as executing the target job, managing the target job, and synchronizing the status of the target job through the target plugin, which improves the convenience when executing, managing, or synchronizing the target job.

[0086] In this embodiment, the job plugins corresponding to different job types can support in-tree mode (i.e., run together with the source code corresponding to the method provided in this embodiment) or out-of-tree mode (i.e., run separately from the source code corresponding to the method provided in this embodiment and run independently).

[0087] In this embodiment, after the acquisition unit 701 acquires the target job, the first determination unit 702 determines the target machine cluster in the candidate machine clusters in the private cloud environment based on the target job. In this embodiment, there are multiple candidate machine clusters in the private cloud environment. Different candidate machine clusters can have the same cluster type or different cluster types, such as Kubernetes clusters, Yarn clusters, etc.

[0088] When the first determining unit 702 determines the target machine cluster among the candidate machine clusters in the private cloud environment based on the target job, the optional implementation method may be: obtaining the job information of the target job; and taking the machine cluster in the candidate machine cluster that matches the obtained job information as the target machine cluster.

[0089] In other words, the first determining unit 702 can determine the target machine cluster based on the job information of the target job, so that the determined target machine cluster is more matched with the target job and the accuracy of the determined target machine cluster is improved.

[0090] When the first determining unit 702 selects the machine cluster in the candidate machine cluster that matches the acquired job information as the target machine cluster, it may also adopt the following method: obtain the resource request amount of the target job according to the job information; determine the target machine cluster from the candidate machine cluster according to the resource request amount and the resource amount of the candidate machine cluster.

[0091] In other words, the first determining unit 702 determines the target machine cluster from the candidate machine clusters by the amount of resource requests in the job information, so that the determined target machine cluster has sufficient machine resources to execute the target job, thereby improving the success rate of the target job execution.

[0092] When determining the target machine cluster among the candidate machine clusters based on the resource request amount and the resource amount of the candidate machine clusters, the first determining unit 702 may use the candidate machine cluster whose resource amount can meet the resource request amount as the target machine cluster; if there are multiple candidate machine clusters that can meet the resource request amount, the candidate machine cluster with the lowest resource utilization rate may be used as the target machine cluster, or a randomly selected candidate machine cluster may be used as the target machine cluster.

[0093] When determining the target machine cluster among the candidate machine clusters in the private cloud environment based on the target job, the first determining unit 702 may also include the following: performing permission verification on the private cloud environment; and, if the permission verification result determines that the private cloud environment can execute the target job, performing the operation of determining the target machine cluster among the candidate machine clusters in the private cloud environment based on the target job.

[0094] In other words, the first determining unit 702 ensures that the private cloud environment can execute the target job by verifying the private cloud environment, thus avoiding the problem that the private cloud environment cannot execute the job due to lack of relevant permissions after the target node is determined, thereby improving the success rate of job execution.

[0095] In this embodiment, after the first determining unit 702 determines the target machine cluster, the second determining unit 703 obtains the node information of the nodes in the target machine cluster through the node query interface, and determines the target node in the target machine cluster based on the node information and the target job; in this embodiment, the node is a machine located in the machine cluster.

[0096] Specifically, when the second determining unit 703 obtains the node information of the nodes in the target machine cluster through the node query interface and determines the target node in the target machine cluster based on the node information and the target job, the optional implementation method can be: obtaining the job information of the target job; taking the nodes corresponding to the node information that matches the job information as candidate nodes; and determining the target node from the candidate nodes.

[0097] In other words, the second determining unit 703 obtains the node information of the nodes contained in the target machine cluster through the node query interface connected to the private cloud environment, and then determines the target node based on the node information and job information, thereby achieving the purpose of relying only on the node query permissions in the cluster, without adding a large number of additional components to the private cloud environment to cause intrusion into the private cloud environment, thereby enhancing the security of the private cloud environment.

[0098] When the second determining unit 703 selects nodes corresponding to node information that matches the job information as candidate nodes, the optional implementation methods may be as follows: determine the resource request amount of the target job based on the job information; determine the available resource amount of the node based on the node information; and select nodes whose available resource amount is greater than the resource request amount as candidate nodes.

[0099] In other words, the second determining unit 703 determines candidate nodes by the resource request amount of the target job and the available resource amount of the node, ensuring that the determined candidate nodes have sufficient resources to process the target data corresponding to the target job, thereby improving the success rate of job execution.

[0100] When the second determining unit 703 selects nodes with available resources greater than resource requests as candidate nodes, the optional implementation method is as follows: select nodes with available resources greater than resource requests as initial nodes; determine the identification information and / or tags of the initial nodes based on the node information of the initial nodes; and select the initial nodes corresponding to the identification information and / or tags that match the job information as candidate nodes.

[0101] When the second determining unit 703 selects the initial node corresponding to the identification information and / or tag that matches the job information as a candidate node, it can select the initial node corresponding to the identification information of the job specified node as a candidate node, and it can also select the initial node corresponding to the tag of the job specified node as a candidate node.

[0102] When determining the target node from the candidate nodes, the second determining unit 703 can sort the candidate nodes according to their resource utilization rate or available resource quantity, and then take the top N candidate nodes as the target nodes, where N is a positive integer greater than or equal to 1; or it can randomly select multiple nodes from the candidate nodes as the target nodes.

[0103] In this embodiment, after the second determining unit 703 determines the target node, the execution unit 704 sends the target data corresponding to the target job to the target node through the job creation interface, so that the target node can create the target job and process the target data.

[0104] The execution unit 704 can first obtain the target data corresponding to the target job through the job query interface, and then send the obtained target data to the target node through the job creation interface.

[0105] When the execution unit 704 sends the target data corresponding to the target job to the target node through the job creation interface, the optional implementation method can be: split the target job into multiple sub-target jobs; send the sub-target data corresponding to the multiple sub-target jobs to multiple target nodes through the job creation interface, so that the multiple target nodes can create sub-target jobs and process the sub-target data respectively.

[0106] In other words, the execution unit 704 can also split the target job, and then multiple target nodes can execute the split sub-target jobs respectively, so as to achieve the purpose of parallel processing of the target job and thus improve the execution efficiency of the target job.

[0107] When the execution unit 704 sends the target data corresponding to the target job to the target node through the job creation interface, if the target plugin corresponding to the target job is determined, the execution unit 704 sends the target data corresponding to the target job to the target node through the job creation interface and the target plugin. Specifically, the target data is first sent to the target plugin through the job creation interface, and then the target plugin sends the received target data to the target node.

[0108] The execution device 700 for a job in the private cloud environment of this embodiment may further include an operation unit 705, which is used to perform the following: obtain an operation request corresponding to the target job; and send the operation request to the target node through the job management interface so that the target node can process the target job according to the operation request.

[0109] In other words, the operation unit 705 interacts with the target node through the job management interface, enabling it to operate on the target job being executed on the target node, thereby improving the flexibility of job execution in a private cloud environment.

[0110] The operation request acquired by the operation unit 705 is an operation request made by the user for the target job. The operation corresponding to the operation request can be such as deleting the target job, stopping the target job, or updating the target job.

[0111] When the operation unit 705 sends an operation request to the target node through the job management interface, if there is a job plugin corresponding to the target job, the operation unit 705 can first send the operation request to the target plugin through the job management interface, and then the target plugin can send the received operation request to the target node.

[0112] It is understood that the job query interface, node query interface, job creation interface, job management interface, etc. in this embodiment can be a unified interface, that is, the unified interface can realize the functions of job query, node query, job creation, job management, etc.

[0113] The execution device 700 for jobs in the private cloud environment of this embodiment may further include a synchronization unit 706, which is used to perform the following: obtain the job status sent by the target node through the job management interface; and send the job status to the storage device through the job query interface so that the storage device can display the job status.

[0114] In other words, the synchronization unit 706 interacts with the target node through the job query interface and the job management interface, and can synchronize the job status of the target node when executing the target job. This allows users to have a clearer understanding of the execution status of the target job, and then to perform corresponding operations on the target job based on the execution status, thereby improving the convenience of executing the target job in a private cloud environment.

[0115] The job status received by the synchronization unit 706 can be whether the target node has completed the execution of the target job, or whether the target node has encountered a fault while executing the target job.

[0116] After the synchronization unit 706 sends the job status to the storage device through the job query interface, the storage device can display the job status to the user. The user can then perform operations such as updating, deleting, or stopping the target job based on the displayed job status.

[0117] When the synchronization unit 706 obtains the job status sent by the target node through the job management interface, if there is a job plugin corresponding to the target job, the synchronization unit 706 can obtain the job status sent by the target node in the job plugin through the job management interface.

[0118] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0119] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0120] Figure 8 is a block diagram of an electronic device for executing a method of operation in a private cloud environment according to an embodiment of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0121] As shown in Figure 8, device 800 includes a computing unit 801, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 802 or a computer program loaded from storage unit 808 into random access memory (RAM) 803. RAM 803 can also store various programs and data required for the operation of device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via bus 804. Input / output (I / O) interface 805 is also connected to bus 804.

[0122] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of display, speaker, etc.

[0123] Storage unit 808, such as a disk, optical disk, etc.; and communication unit 809, such as a network interface card, modem, wireless transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0124] The computing unit 801 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), and a graphics processing unit (GPU).

[0125] Graphics processing units (GPUs), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processors (DSPs), and any appropriate

[0126] Processors, controllers, microcontrollers, etc. The computing unit 801 executes the various methods and processes described above, such as the method for executing a job in a private cloud environment. For example, in some embodiments, the method for executing a job in a private cloud environment may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 808.

[0127] 5. In some embodiments, part or all of the computer program may be transmitted via ROM 802 and / or

[0128] The communication unit 809 is loaded and / or installed onto the device 800. When the computer program is loaded...

[0129] When RAM 803 is executed by computing unit 801, it can perform one or more steps of the job execution method in the private cloud environment described above. Alternatively, in other embodiments, computing unit 801

[0130] It can be configured to perform operations in a private cloud environment through any other appropriate means (e.g., by means of firmware).

[0131] Various implementations of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include implementations in one or more computer programs, wherein the one or more computer programs...

[0132] It can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, and can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0133] 0. Program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to the processor or controller of an execution device operating in a general-purpose computer, special-purpose computer, or other programmable private cloud environment, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0134] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0135] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for showing information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0136] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0137] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service system that addresses the management difficulties and weak business scalability inherent in traditional physical hosts and VPS (Virtual Private Server) services. Servers can also be servers for distributed systems or servers incorporating blockchain technology.

[0138] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0139] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for executing a job in a private cloud environment, comprising: The target job is obtained through the job query interface, and the target plugin corresponding to the target job is determined; based on the target job, the target machine cluster in the candidate machine cluster in the private cloud environment is determined. The node information of the nodes in the target machine cluster is obtained through the node query interface, and the target node in the target machine cluster is determined based on the node information and the target job. Through the job creation interface and the target plugin, the target data corresponding to the target job is sent to the target node so that the target node can create the target job and process the target data.

2. The method according to claim 1, wherein, The step of determining the target machine cluster in the candidate machine clusters in the private cloud environment based on the target job includes: obtaining the job information of the target job; and selecting the machine cluster in the candidate machine clusters that matches the job information as the target machine cluster.

3. The method according to claim 1, wherein, The step of determining the target machine cluster among the candidate machine clusters in the private cloud environment based on the target job includes: performing permission verification on the private cloud environment; and, if the permission verification result indicates that the private cloud environment can execute the target job, performing the operation of determining the target machine cluster among the candidate machine clusters in the private cloud environment based on the target job.

4. The method according to claim 1, wherein, The step of obtaining node information of nodes in the target machine cluster through a node query interface and determining the target node in the target machine cluster based on the node information and the target job includes: obtaining job information of the target job; selecting nodes corresponding to node information that matches the job information as candidate nodes; and determining the target node from the candidate nodes.

5. The method according to claim 4, wherein, The step of selecting nodes corresponding to node information matching the job information as candidate nodes includes: determining the resource request amount of the target job based on the job information; determining the available resource amount of the node based on the node information; and selecting nodes whose available resource amount is greater than the resource request amount as candidate nodes.

6. The method according to claim 5, wherein, The step of selecting nodes whose available resources are greater than the resource request amount as candidate nodes includes: selecting nodes whose available resources are greater than the resource request amount as initial nodes; determining the identification information and / or tags of the initial nodes based on the node information of the initial nodes; and selecting the initial nodes corresponding to the identification information and / or tags that match the job information as candidate nodes.

7. The method according to claim 1, wherein, Sending target data corresponding to the target job to the target node through the job creation interface includes: splitting the target job into multiple sub-target jobs; sending sub-target data corresponding to the multiple sub-target jobs to multiple target nodes through the job creation interface, so that the multiple target nodes can create sub-target jobs and process the sub-target data respectively.

8. The method according to claim 1 further includes: obtaining an operation request corresponding to the target job; sending the operation request to the target node through the job management interface and the target plugin, so that the target node can process the target job according to the operation request.

9. The method according to claim 1 further includes: obtaining the job status sent by the target node through the job management interface; and sending the job status to the storage device through the job query interface for the storage device to display the job status.

10. The method according to claim 9, wherein, The step of obtaining the job status sent by the target node through the job management interface includes: obtaining the job status sent by the target node through the job management interface and the target plugin.

11. An execution device for operating in a private cloud environment, comprising: The acquisition unit is used to acquire the target job through the job query interface and determine the target plugin corresponding to the target job; The first determining unit is used to determine the target machine cluster among the candidate machine clusters in the private cloud environment based on the target job. The second determining unit is used to obtain node information of nodes in the target machine cluster through a node query interface, and determine the target node in the target machine cluster based on the node information and the target job. An execution unit is used to send target data corresponding to the target job to the target node through the job creation interface and the target plugin, so that the target node can create the target job and process the target data.

12. The apparatus according to claim 11, wherein, When the first determining unit determines the target machine cluster among the candidate machine clusters in the private cloud environment based on the target job, it specifically performs the following: obtaining the job information of the target job; and taking the machine cluster in the candidate machine cluster that matches the job information as the target machine cluster.

13. The apparatus according to claim 11, wherein, When the first determining unit determines the target machine cluster among the candidate machine clusters in the private cloud environment based on the target job, it specifically performs the following: performs permission verification on the private cloud environment. If the permission verification result indicates that the private cloud environment can execute the target job, then the operation of determining the target machine cluster among the candidate machine clusters in the private cloud environment based on the target job is performed.

14. The apparatus according to claim 11, wherein, When the second determining unit obtains node information of nodes in the target machine cluster through the node query interface and determines the target node in the target machine cluster based on the node information and the target job, it specifically performs the following: obtaining job information of the target job; taking the nodes corresponding to the node information that matches the job information as candidate nodes; and determining the target node from the candidate nodes.

15. The apparatus according to claim 14, wherein, The second determining unit, in selecting nodes corresponding to node information matching the job information as candidate nodes, includes: determining the resource request amount of the target job based on the job information; determining the available resource amount of the node based on the node information; and selecting nodes whose available resource amount is greater than the resource request amount as candidate nodes.

16. The apparatus according to claim 15, wherein, When the second determining unit selects a node whose available resources are greater than the resource request amount as a candidate node, it specifically performs the following steps: selecting a node whose available resources are greater than the resource request amount as an initial node; determining the identification information and / or tag of the initial node based on the node information of the initial node; and selecting the initial node corresponding to the identification information and / or tag that matches the job information as the candidate node.

17. The apparatus according to claim 11, wherein, When the execution unit sends the target data corresponding to the target job to the target node through the job creation interface, it specifically performs the following: splitting the target job into multiple sub-target jobs; The sub-target data corresponding to the multiple sub-target jobs is sent to multiple target nodes through the job creation interface, so that the multiple target nodes can create sub-target jobs and process the sub-target data respectively.

18. The apparatus of claim 11 further includes an operation unit configured to: acquire an operation request corresponding to the target job; and send the operation request to the target node via a job management interface and the target plug-in, so that the target node can process the target job according to the operation request.

19. The apparatus according to claim 11 further includes a synchronization unit, configured to: obtain the job status sent by the target node through the job management interface; and send the job status to the storage device through the job query interface for the storage device to display the job status.

20. An electronic device, comprising: At least one processor; And a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-10.

21. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-10.

22. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-10.

23. An execution system for operating in a private cloud environment, comprising the electronic device of claim 20, a machine cluster in a private cloud environment, and a storage device.

Images