Job scheduling systems, methods, and media in hybrid clouds

Abstract

The present application relates to the field of computer technology, and in particular to a job scheduling system, method and medium in a hybrid cloud. The system includes a business logic layer, a message middle layer and multiple work domains, the business logic layer includes a job receiving component and a job issuing component, the message middle layer includes a work domain job issuing queue, and each work domain includes a port forwarding component and at least one work node; the job receiving component is used to receive a job execution request sent by a client, and the job execution request includes an identifier of the work domain and job content; the job issuing component is used to send the identifier of the work domain and the job content to the work domain job issuing queue; the work domain job issuing queue is used to issue the job content to the port forwarding component in the target work domain; the port forwarding component is used to send the job content to the target work node corresponding to the job content; the work node is used to execute the job content. The system provided by the present application can efficiently manage a multi-cloud environment and is easy to expand.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a job scheduling system, method and medium in a hybrid cloud. Background Art

[0002] With the continuous development of the information industry, more and more companies have begun to build hybrid clouds. Hybrid clouds have the higher security of private clouds and the rich computing resources of public clouds, making important contributions to enterprise data security and work efficiency. However, in general, the networks in multiple cloud environments in hybrid clouds are isolated from each other, which causes unnecessary trouble for enterprises in processing data.

[0003] In the current related technology, an asynchronous job scheduling engine in a private cloud environment is proposed. The business logic layer of the engine receives job tasks and sends the content to the message queue in a specific organizational form. The working node listens to the message queue, obtains the job content from the message queue, and then parses the content and calls the application interface to execute the task. That is, multiple scheduling engines are deployed in multiple cloud environments to realize cloud computing.

[0004] However, deploying multiple scheduling engines in multiple cloud environments cannot achieve efficient and unified management of multi-cloud environments. Summary of the invention

[0005] The present application provides a job scheduling system, method and medium in a hybrid cloud to achieve efficient and unified management of a multi-cloud environment.

[0006] In the first aspect, the present application provides a job scheduling system in a hybrid cloud, the system comprising a business logic layer, a message middle layer and multiple work domains, the business logic layer comprising a job receiving component and a job issuing component, the message middle layer comprising a work domain job issuing queue, each of the multiple work domains comprising a port forwarding component and at least one work node, the business logic layer and the message middle layer being deployed in the same network; the job receiving component being used to receive a job execution request sent by a client, the job execution request comprising an identifier of the work domain and job content; the job issuing component being used to send the identifier of the work domain and the job content to the work domain job issuing queue; the work domain job issuing queue being used to issue the job content to the port forwarding component in the target work domain corresponding to the identifier of the work domain; the port forwarding component being used to send the job content to the target work node corresponding to the job content; the work node being used to execute the job content.

[0007] In this system, the purpose of forwarding client requests to the corresponding work domain is achieved through the business logic layer and message middle layer in the same network, as well as the port forwarding components in different networks. The connectivity of the work domain job dispatching queue and the port forwarding components in the corresponding work domain provides a basis for efficient and unified management of multi-cloud environments.

[0008] In combination with the first aspect, in a possible implementation, the message intermediate layer also includes a job result queue, and the business logic layer also includes a job result processing component and a business database; the target working node is used to send the job result corresponding to the job content to the port forwarding component; the port forwarding component is used to send the job result to the job result queue; the job result queue is used to send the job result to the job result processing component; the job result processing component is used to store the job result in the business database.

[0009] In this implementation, starting from the job result queue, the job result processing component and the business database, the job result processing component determines the return result after the client sends the request for processing.

[0010] In combination with the first aspect, in a possible implementation method, the business logic layer also includes a synchronization task sending component, the message middle layer also includes a work domain synchronization task component queue, and each of the multiple work domains also includes a script warehouse, a synchronization component and an inventory database; the synchronization task sending component is used to send inventory information synchronization tasks, node load information synchronization tasks and script information synchronization tasks to the work domain synchronization task component queue; the synchronization component is used to listen to the synchronization tasks in the work domain synchronization task component queue, and according to the type of the synchronization task, obtain the inventory information, node load information and script information in the target work domain.

[0011] In this implementation, the inventory information, node load information and script information in the work domain are obtained through the script warehouse, synchronization components and inventory database, which provides a basis for responding to subsequent requests sent by the client.

[0012] In combination with the first aspect, in a possible implementation method, the business logic layer also includes a synchronization task result processing component, and the message intermediate layer also includes a synchronization component result queue; the synchronization component result queue is used to obtain the synchronization result corresponding to the synchronization task through the port forwarding component; the synchronization task result processing component is used to monitor the synchronization component result queue, and store the synchronization results in the synchronization component result queue to the business database.

[0013] In this implementation, the synchronization results are stored in the business database through the synchronization task result processing component and the synchronization component result queue for subsequent query use.

[0014] In a second aspect, the present application provides a job scheduling method in a hybrid cloud, the method being applied to a job scheduling system in a hybrid cloud, the system comprising a business logic layer, a message middle layer and multiple work domains, the business logic layer comprising a job receiving component and a job issuing component, the message middle layer comprising a work domain job issuing queue, each of the multiple work domains comprising a port forwarding component and at least one work node, the business logic layer and the message middle layer being deployed in the same network, the method comprising: the job receiving component receiving a job execution request sent by a client, the job execution request comprising an identifier of the work domain and job content; the job issuing component sending the identifier of the work domain and the job content to the work domain job issuing queue; the work domain job issuing queue sending the job content to the port forwarding component in the target work domain corresponding to the identifier of the work domain; the port forwarding component sending the job content to the target work node corresponding to the job content; the work node executing the job content.

[0015] In this method, the purpose of forwarding client requests to the corresponding work domain is achieved through the business logic layer and message middle layer in the same network, as well as the port forwarding components in different networks. The connectivity of the work domain job dispatching queue and the port forwarding components in the corresponding work domain provides a basis for efficient and unified management of multi-cloud environments.

[0016] In combination with the second aspect, in a possible implementation, the message intermediate layer also includes a job result queue, the business logic layer also includes a job result processing component and a business database, and the method also includes: the target working node sends the job result corresponding to the job content to the port forwarding component, and the port forwarding component sends the job result to the job result queue; the job result queue sends the job result to the job result processing component; and the job result processing component stores the job result in the business database.

[0017] In this implementation, starting from the job result queue, the job result processing component and the business database, the job result processing component determines the return result after the client sends the request for processing.

[0018] In combination with the second aspect, in a possible implementation, the business logic layer also includes a synchronization task issuing component, the message middle layer also includes a work domain synchronization task component queue, each of the multiple work domains also includes a script warehouse, a synchronization component and an inventory database, and the method also includes: the synchronization task issuing component sends inventory information synchronization tasks, node load information synchronization tasks and script information synchronization tasks to the work domain synchronization task component queue; the synchronization component listens to the synchronization tasks in the work domain synchronization task component queue, and obtains inventory information, node load information and script information in the target work domain according to the type of the synchronization task.

[0019] In this implementation, the inventory information, node load information and script information in the work domain are obtained through the script warehouse, synchronization components and inventory database, which provides a basis for responding to subsequent requests sent by the client.

[0020] In combination with the second aspect, in a possible implementation, the business logic layer also includes a synchronization task result processing component, the message intermediate layer also includes a synchronization component result queue, and the method also includes: the synchronization component result queue obtains the synchronization result corresponding to the synchronization task through the port forwarding component; the synchronization task result processing component listens to the synchronization component result queue, and stores the synchronization results in the synchronization component result queue to the business database.

[0021] In this implementation, the synchronization results are stored in the business database through the synchronization task result processing component and the synchronization component result queue for subsequent query use.

[0022] In the third aspect, the present application provides a job scheduling device in a hybrid cloud, the device comprising: a receiving module, used to receive a job execution request sent by a client, the job execution request comprising an identifier of a work domain and job content; a sending module, used to send the identifier of the work domain and the job content to the work domain job issuance queue; the sending module is also used to send the job content to a port forwarding component in a target work domain corresponding to the identifier of the work domain; the sending module is also used to send the job content to a target work node corresponding to the job content; an execution module, used to execute the job content.

[0023] In combination with the third aspect, in a possible implementation method, the sending module is also used to send the job results corresponding to the job content to the port forwarding component, and the sending module is also used to send the job results to the job result queue; the sending module is also used to send the job results to the job result processing component; the storage module is used to store the job results in the business database.

[0024] In combination with the third aspect, in a possible implementation method, the sending module is also used to send inventory information synchronization tasks, node load information synchronization tasks and script information synchronization tasks to the working domain synchronization task component queue; the acquisition module is used to listen to the synchronization tasks in the working domain synchronization task component queue, and according to the type of the synchronization task, obtain the inventory information, node load information and script information in the target working domain.

[0025] In combination with the third aspect, in a possible implementation method, the acquisition module is also used to obtain the synchronization results corresponding to the synchronization task through the port forwarding component, and to monitor the synchronization component result queue, and the storage module is also used to store the synchronization results in the synchronization component result queue to the business database.

[0026] In a fourth aspect, the present application provides a job scheduling system in a hybrid cloud, comprising: a memory and a processor; the memory is used to store program instructions; the processor is used to call the program instructions in the memory to execute the method described in the second aspect or any possible implementation method thereof.

[0027] In a fifth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the method described in the second aspect or any possible implementation method thereof.

[0028] In a sixth aspect, the present application provides a computer program product, wherein the computer program product includes a computer program, and when the computer program is executed by a processor, the method described in the second aspect or any possible implementation method thereof is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0030] Figure 1 A schematic diagram of an application scenario architecture provided for an embodiment of the present application;

[0031] Figure 2 A schematic diagram of a job scheduling system in a hybrid cloud provided by an embodiment of the present application;

[0032] Figure 3 A schematic diagram of a job scheduling system in a hybrid cloud provided by another embodiment of the present application;

[0033] Figure 4 A process of a method for scheduling jobs in a hybrid cloud provided in one embodiment of the present application Figure 1 ;

[0034] Figure 5 A process of a method for scheduling jobs in a hybrid cloud provided in one embodiment of the present application Figure 2 ;

[0035] Figure 6 A process of a method for scheduling jobs in a hybrid cloud provided in one embodiment of the present application Figure 3 ;

[0036] Figure 7 A process of a method for scheduling jobs in a hybrid cloud provided in one embodiment of the present application Figure 4 ;

[0037] Figure 8 A process of a method for scheduling jobs in a hybrid cloud provided in one embodiment of the present application Figure 5 ;

[0038] Fig. 9 A process of a method for scheduling jobs in a hybrid cloud provided in one embodiment of the present application Figure 6 ;

[0039] Fig.10 A schematic block diagram of a job scheduling device in a hybrid cloud provided by an embodiment of the present application;

[0040] Fig.11 A system schematic diagram provided for another embodiment of the present application.

[0041] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0042] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0043] With the rapid development of the information technology industry, more and more companies have begun to build hybrid clouds. Hybrid clouds have the higher security of private clouds and the rich computing resources of public clouds, making important contributions to enterprise data security and work efficiency. However, in general, the networks between multiple cloud environments in a hybrid cloud are isolated from each other, that is, the cloud environments are not interconnected, which causes unnecessary trouble for enterprises in processing data.

[0044] The specific application scenario diagram of this application is as follows Figure 1 As shown, the scenario includes: cloud server 101, cloud server 102, cloud server 103 and cloud server 104.

[0045] Among them, cloud server 101, cloud server 102, cloud server 103 and cloud server 104 are only used as examples, and there is no limit on the number of cloud servers. It is assumed that cloud server 101 is a private cloud, which can include servers on a private network, and cloud server 102 is a public cloud, which can include servers on a public network. Cloud server 103 and cloud server 104 can be any one of a public cloud, a private cloud or other clouds respectively.

[0046] Optionally, cloud server 101, cloud server 102, cloud server 103 and cloud server 104 are cloud resource pools that are managed independently.

[0047] In one possible implementation, cloud server 101 provides services for the user's client. When the user initiates a job execution request through the client, the job execution request requires data support from cloud server 102. At this time, cloud server 101 and cloud server 102 establish a connection to achieve data interaction.

[0048] In the current related technology, in order to realize data interaction between different cloud servers, a job scheduling engine is designed, which adopts a single cloud architecture. When facing a multi-cloud operation and maintenance scenario, since multiple cloud environments often exist in different networks (for example, cloud server 101 and cloud server 102 are in different network environments), they are not connected. In order to achieve the operation and maintenance goals, it is necessary to deploy a complete set of job scheduling engines in multiple cloud environments at the same time, or to open up the network access relationship between the working node and all managed servers (that is, the servers in cloud server 101 and cloud server 102) to maintain a set of job scheduling engines.

[0049] However, both of the above solutions have obvious disadvantages: repeated deployment consumes unnecessary computing resources and increases operation and maintenance costs, is inconvenient for unified management of multi-cloud environments, and opening a large number of network access relationships is not safe and may not comply with network control policies.

[0050] Therefore, how to achieve efficient and unified management of multi-cloud environments has become a technical problem that needs to be solved urgently.

[0051] In view of this, the present application provides a job scheduling system and method in a hybrid cloud, aiming to solve the above technical problems of the prior art.

[0052] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0053] Figure 2 A schematic diagram of a job scheduling system in a hybrid cloud provided by an embodiment of the present application. Figure 2 As shown, the job scheduling system includes a business logic layer 201, a message middle layer 202 and a plurality of work domains 203 ( Figure 2 Taking two working domains as an example), the business logic layer 201 includes a job receiving component 2011, a job issuing component 2012, a job result processing component 2013 and a business database 2014; the message middle layer 202 includes a working domain job issuing queue 2021 and a job result queue 2022; the working domain 203 includes a port forwarding component 2031 and at least one working node 2032.

[0054] Optionally, the working area 203 may be the above Figure 1 Any cloud server in .

[0055] Among them, the business logic layer 201 and the message middle layer 202 are deployed in the same network. The business logic layer 201, the message middle layer 202 and the network access of each work domain 203 are generally not connected. During the deployment of the new work domain 203, the network access relationship from the internal port forwarding component 2031 of the work domain 203 to the service port of the message middle layer 202 is opened.

[0056] It should be noted that the business logic layer 201 and the message middle layer 202 are deployed in the same network, that is, the business logic layer 201 and the message middle layer 202 are deployed in a private cloud environment or in a public cloud environment at the same time, and the connection between the business logic layer 201 and the message middle layer 202 does not need to be forwarded through a gateway device.

[0057] The business logic layer 201 provides services to the outside in the form of a representational state transfer application programming interface (REST API). Its main responsibilities include: job content analysis, legality judgment, job data storage, job scheduling, synchronous task distribution, message queue result processing, etc.

[0058] Specifically, the job receiving component 2011 is used to receive a job execution request sent by a client, and the job execution request includes an identifier of a work domain and job content.

[0059] Understandably, after receiving the job execution request from the client, the job receiving component 2011 will first determine the source of the job execution request and the legitimacy of the job content. The specific determination method is as follows: the job execution request content must contain the identification (parameter) of the work domain, and the identification of the work domain is consistent with the work domain information stored in the platform (that is, the work domain information stored in the platform stores the identification of the work domain corresponding to the job execution request content). If it is legal, the job content is parsed, and the job content and the identification of the work domain are saved in the job pool of the database, and the job pool is maintained in the business database 2014.

[0060] The job sending component 2012 is used to send the identification of the work domain and the job content to the work domain job sending queue.

[0061] In other words, the job dispatching component 2012 will periodically read the work domain information in the business database 2014, retrieve the undispatched jobs from the job pool according to the read work domain information, and dispatch the jobs to the corresponding job dispatching queues in the message middle layer 202 according to the work domain identifiers of the jobs. For example, jobs with the work domain 1 identifier will be dispatched to the work domain 1 job dispatching queues of the message middle layer 202.

[0062] It should be noted that after being parsed, the job content will be marked as having been issued or not issued.

[0063] The operation result processing component 2013 is used to store the operation result in the business database 2014 .

[0064] Understandably, the job result processing component 2013 will monitor the job result queue 2022, and after receiving the information, parse the work domain information inside the information and store the result in the business database 2014.

[0065] The business database 2014 uses a relational database management system (mySql) database, and the work domain information will be stored in the business database. The work domain information will include the function description of the work domain, the unique identifier in text form, whether it is abandoned, the connection information of the inventory database, the external inventory interface address, etc. The number of stored work domain information corresponds to the number of work domains deployed on the platform. When a new work domain 203 is deployed, the corresponding work domain information needs to be added.

[0066] The message middle layer 202 provides services using a message-oriented middleware (Rabbitmq) tool. Multiple domain job dispatch queues and multiple domain synchronization task dispatch queues as well as job result queues and synchronization task result queues are created in Rabbitmq.

[0067] Specifically, the work domain job delivery queue 2021 is used to deliver the job content to the port forwarding component in the target work domain corresponding to the work domain identifier, and the job result queue 2022 is used to send the job result to the job result processing component.

[0068] Among them, the multiple working domains 203 are cloud resource pools that are managed independently of each other.

[0069] Specifically, the port forwarding component 2031 is used to send the job content to the target working node corresponding to the job content, and accordingly, the target working node is used to send the job result corresponding to the job content to the port forwarding component 2031.

[0070] Understandably, since the network between the message middle layer 202 and the working domain 203 is generally not connected, and multiple components in the working domain 203 need to access the message middle layer 202 at the same time, in order to avoid opening too many unnecessary access relationships, a port forwarding component 2031 is set up as a proxy for the message middle layer 202. This component is implemented using an open source software (Nginx) tool.

[0071] The working node 2032 is used to execute the job content.

[0072] Figure 3 A schematic diagram of a job scheduling system in a hybrid cloud provided by another embodiment of the present application. Figure 3 As shown, the business logic layer 201 also includes a synchronization task issuing component 2015 and a synchronization task result processing component 2016; the message middle layer 202 includes a work domain synchronization task component queue 2023 and a synchronization component result queue 2024; the work domain 203 includes a script warehouse 2033, a synchronization component 2034 and a list database 2035.

[0073] Among them, the synchronization task sending component 2015 is used to send inventory information synchronization tasks, node load information synchronization tasks and script information synchronization tasks to the working domain synchronization task component queue; the synchronization task result processing component 2016 is used to monitor the synchronization tasks in the working domain synchronization task component queue, and according to the type of synchronization task, obtain the inventory information, node load information and script information in the target working domain.

[0074] The synchronization component 2034 monitors the corresponding work domain synchronization task component queue 2023. When a message is received, the task type is first determined. When the task type is 'invertories_info', the synchronization component 2034 will synchronize the inventory information in the message to the inventory database 2035, and return the result to the synchronization component result queue 2024. When the task type is 'script_info', the synchronization component 2034 will synchronize the script information carried in the message to the script warehouse 2033, and the synchronization component 2034 calls the open source version control system (subversion, SVN) API to achieve it, and then return the result to the synchronization component result queue 2024. When the task type is 'node_info', the synchronization component 2034 will log in to the node in a secure shell protocol (SSH), and then obtain information such as memory and central processing unit (CPU), and then return the result to the synchronization component result queue 2024.

[0075] In the script warehouse 2033, the script is a template for the working node to operate on the target, and the script warehouse 2033 is maintained using the SVN version management tool and mounted on the synchronization component 2034 and the working node 2032 in the working domain in a shared storage manner.

[0076] In the list database 2035, when a target of a certain working node 2032 is in the list, the target state is considered to be available, otherwise the target is ignored when executing the task. The list uses a non-relational database (remote dictionary server, redis) with a kv structure, and the list data is shared among the working nodes 2032.

[0077] Among them, the synchronization task issuing component 2015 is responsible for the tasks of each domain inventory information, script information, and node load information. This component will start three threads: inventory information synchronization task issuing thread, node load information synchronization task issuing thread, and script information synchronization task issuing thread.

[0078] Specifically, the execution steps of the three threads are as follows:

[0079] 1. Inventory information synchronization task dispatching thread: The inventory refers to a collection of target servers. After the thread is started, it reads the work domain information, requests the external inventory interface in the work domain information and obtains the inventory information. After obtaining the inventory information, add the inventory information identifier 'invertories_info' and the work domain identifier to it. Then send the entire message to the corresponding work domain synchronization task component queue 2023 in the message middle layer 202, so that the synchronization component 2034 of the work domain can read the message and update the inventory to the work domain. For example, the inventory information of work domain 1 is sent to the synchronization task component queue of work domain 1.

[0080] 2. Node load information synchronization task dispatching thread: After the thread is started, it will periodically read the work domain information, and then send information with 'node_info' and the work domain identifier to the corresponding work domain synchronization task component queue 2023 in the message middle layer 202 to notify the work domain synchronization component 2034 to obtain the work node load information and return it to the business logic layer 201.

[0081] 3. Script information synchronization task dispatch thread: A script is a file written in a certain format using a specific descriptive language and executable on a specific target. After the thread is started, it will periodically read the work domain information, and then send information with 'script_info' and the work domain identifier to the corresponding work domain synchronization task component queue 2023 in the message middle layer 202 to notify the work domain synchronization component 2034 to obtain the script information in the script warehouse 2033 in the work domain and then return to the business logic layer 201.

[0082] The above introduction is a detailed description of the job scheduling system in the hybrid cloud provided in the embodiment of the present application. The following describes in detail the job scheduling method in the present application with a specific implementation example.

[0083] Based on the above system architecture, Figure 4 A process of a method for scheduling jobs in a hybrid cloud provided in one embodiment of the present application Figure 1 .like Figure 4 As shown, the method provided in the embodiment of the present application includes S401, S402, S403, S404 and S405. Figure 4 The various steps in the method shown.

[0084] S401, the job receiving component receives a job execution request sent by a client, where the job execution request includes an identifier of a work domain and job content.

[0085] Optionally, the user sends a job execution request to the cloud server through the client, and the job receiving component in the cloud server performs corresponding preprocessing after receiving the job execution request sent by the client. Specifically, Figure 5A process of a method for scheduling jobs in a hybrid cloud provided in one embodiment of the present application Figure 2 ,like Figure 5 As shown, this step may include the following implementations:

[0086] S4011, the job receiving component obtains a job execution request.

[0087] S4012, the business logic layer determines whether the source and format of the job execution request are legal based on the job execution request.

[0088] S4013: After the request is legal, determine whether the work domain parameter in the job execution request is consistent with the work domain information stored in the business database.

[0089] S4014, after the consistency, parse the work content in the job execution request, and save the job content together with the identification of the work domain into the job pool of the business database.

[0090] Among them, the job pool is maintained in the business database.

[0091] S402, the job issuing component sends the identification of the work domain and the job content to the work domain job issuing queue.

[0092] Optionally, the job dispatch component periodically reads the work domain information in the business database, retrieves undispatched jobs from the job pool based on the read work domain information, and distributes the jobs to the corresponding job dispatch queue in the message middle layer based on the work domain identifier of the jobs.

[0093] For example, a job with a work domain 1 identifier will be sent to the corresponding position of work domain 1 through the work domain 1 job sending queue of the message middle layer.

[0094] Specifically, Figure 6 A process of a method for scheduling jobs in a hybrid cloud provided in one embodiment of the present application Figure 3 ,like Figure 6 As shown, this step may also include the following implementation:

[0095] S4021, the job distribution component starts.

[0096] S4022, initialize the working domain information.

[0097] S4023, searching for unissued jobs in the job pool.

[0098] S4024, distribute the job to its corresponding job delivery queue in the message middle layer according to the work domain identifier of the job.

[0099] S403, the working domain job sending queue sends the job content to the port forwarding component in the target working domain corresponding to the working domain identifier.

[0100] In this step, a job with a work domain identifier already exists in the corresponding task dispatch queue in the message middle layer. At this time, according to the work domain identifier of the job, the job content is sent to the port forwarding component in the target work domain corresponding to the work domain identifier, so that the target work domain can perform the job according to the specific job content.

[0101] S404, the port forwarding component sends the job content to the target working node corresponding to the job content.

[0102] In this step, after receiving the job content, the port forwarding component sends the job content to the corresponding target working node according to the instruction of the job content, so that the job execution request sent by the client is executed, thereby obtaining the request result.

[0103] Optionally, since the network between the message middle layer and the working domain is generally not connected (for example, the private cloud and the public cloud are not in the same network), and multiple components in the working domain need to access the message middle layer at the same time. In order to avoid opening too many unnecessary access relationships, a port forwarding component is set up as a proxy for the message middle layer. This component can be implemented using the Nginx tool.

[0104] S405: The working node executes the job content.

[0105] In this step, when the target corresponding to the working node is in the inventory database, the target status is considered available, otherwise the target is ignored when executing the task. The inventory database uses a redis memory database with a kv structure, and the inventory data is shared among the working nodes.

[0106] Furthermore, the work node based on Ansible API listens to the message queue corresponding to its work domain. When the work node receives the job content, it parses the data of the job content and creates a job execution process. After the job content is executed, the execution result is returned to the job result queue.

[0107] At this point, the response to a job execution request is completed.

[0108] The embodiment of the present application provides a job scheduling method in a hybrid cloud, wherein a job receiving component receives a job execution request sent by a client, a job issuing component sends an identifier of a work domain and job content to a job issuing queue of the work domain, and then the job issuing queue of the work domain sends the job content to a port forwarding component in a target work domain corresponding to the identifier of the work domain, and then the port forwarding component sends the job content to a target work node corresponding to the job content, and finally the work node executes the job content to return an execution result. In this implementation method, the purpose of forwarding requests sent by the client to the corresponding work domain is achieved through the business logic layer and the message middle layer in the same network, and the port forwarding components in different networks, and the connectivity of the work domain job issuing queue and the port forwarding components in the corresponding work domain provides a basis for efficient and unified management of a multi-cloud environment.

[0109] Based on the above embodiments, Figure 7 A process of a method for scheduling jobs in a hybrid cloud provided in one embodiment of the present application Figure 4 ,like Figure 7 As shown, this step may also include the following implementation:

[0110] S701, the synchronization task delivery component sends the inventory information synchronization task, the node load information synchronization task and the script information synchronization task to the work domain synchronization task component queue.

[0111] In this step, the synchronization task delivery component is responsible for tasks of each work domain list information, script information, and node load information.

[0112] Optionally, this component will start three threads: an inventory information synchronization task delivery thread, a node load information synchronization task delivery thread, and a script information synchronization task delivery thread.

[0113] S702, the synchronization component monitors the synchronization tasks in the work domain synchronization task component queue, and obtains inventory information, node load information and script information in the target work domain according to the type of the synchronization task.

[0114] Optionally, Figure 8 A process of a method for scheduling jobs in a hybrid cloud provided in one embodiment of the present application Figure 5 , combined with Figure 8 , the steps of the three threads are described as follows:

[0115] Among them, the steps of the three threads are executed simultaneously and are:

[0116] S7021, the synchronization task distribution component is started.

[0117] Optionally, for the script information synchronization thread, there are the following steps:

[0118] S70221, start the script information synchronization thread.

[0119] S70231, obtain work domain information.

[0120] S70241, generate script information.

[0121] Specifically, a message with 'script_info' and a work domain identifier is sent to the corresponding work domain synchronization task component queue in the message middle layer to notify the work domain synchronization component to obtain the script information in the script repository in the work domain.

[0122] Optionally, for the inventory information synchronization thread, there are the following steps:

[0123] S70222, start the inventory information synchronization thread.

[0124] S70232, obtain work domain information.

[0125] S70242, request external list interface.

[0126] S70252, obtain the list information after the request is successful.

[0127] Specifically, after the thread is started, the work domain information is read, and the external list interface in the work domain information is requested to obtain the list information.

[0128] Optionally, for the node information synchronization thread, there are the following steps:

[0129] S70223, start the node information synchronization thread.

[0130] S70233, obtain work domain information.

[0131] S70243, generate node load information.

[0132] Specifically, a message with 'node_info' and a working domain identifier is sent to the corresponding domain synchronization task component queue in the message middle layer to notify the working domain synchronization component to obtain the working node load information.

[0133] S703, the synchronization component result queue obtains the synchronization result corresponding to the synchronization task through the port forwarding component.

[0134] In this step, the synchronization component result queue obtains the synchronization result corresponding to the synchronization task through the port forwarding component, that is, obtains the list information, node load information and script information.

[0135] Optionally, a message with 'invertories_info' and a work domain identifier is sent to a synchronization component result queue of a corresponding work domain in the message middle layer.

[0136] Optionally, the information with 'node_info' and the work domain identifier is sent to the synchronization component result queue of the corresponding work domain in the message middle layer.

[0137] Optionally, a message with 'script_info' and a work domain identifier is sent to the synchronization task component queue of the corresponding work domain in the message middle layer to notify the work domain synchronization component to obtain the script information in the script repository in the work domain.

[0138] S704, the synchronization task result processing component monitors the synchronization component result queue, and stores the synchronization results in the synchronization component result queue into the business database.

[0139] In this step, the synchronization results (list information, node load information, and script information) in the synchronization component result queue are returned to the business logic layer, and are stored in the corresponding location in the business database.

[0140] Optionally, the synchronization component listens to the corresponding work domain synchronization task component queue. When a message is received, the task type is first determined. When the task type is 'invertories_info', the synchronization component synchronizes the inventory information in the message to the inventory database, and returns the result to the synchronization component result queue. When the task type is 'script_info', the synchronization component synchronizes the script information carried in the message to the script repository. The synchronization is achieved by calling the SVN API by the synchronization component, and then returns the result to the synchronization component result queue. When the task type is 'node_info', the synchronization component logs in to the node via SSH, obtains information such as memory and CPU, and then returns the result to the synchronization component result queue.

[0141] Specifically, Fig. 9 A process of a method for scheduling jobs in a hybrid cloud provided in one embodiment of the present application Figure 6 , describe this step:

[0142] S7041, monitor the synchronization task component queue.

[0143] S7042, obtain synchronization task message.

[0144] S7043, determine the task type.

[0145] S7044, synchronization list task, synchronization script task and synchronization node information task.

[0146] S7045, parse inventory data, parse script data and SSH login to the node.

[0147] S7046, synchronize to the inventory database, call the API to update script data and obtain memory and CPU information.

[0148] Specifically, in steps S7044, S7045 and S7046, different tasks are executed according to different task types.

[0149] The embodiment of the present application provides a job scheduling method in a hybrid cloud, wherein a synchronization task issuing component sends inventory information synchronization tasks, node load information synchronization tasks and script information synchronization tasks to a work domain synchronization task component queue, the synchronization component listens to the synchronization tasks in the work domain synchronization task component queue, and obtains inventory information, node load information and script information in a target work domain according to the type of the synchronization task, then the synchronization component result queue obtains the synchronization result corresponding to the synchronization task through a port forwarding component, and finally the synchronization task result processing component listens to the synchronization component result queue, and stores the synchronization results in the synchronization component result queue to a business database. In this method, the synchronization results are stored in a business database through the synchronization task result processing component and the synchronization component result queue for subsequent query use.

[0150] Fig.10 A job scheduling device 1000 in a hybrid cloud provided in an embodiment of the present application is shown. The device 1000 includes: a receiving module 1001, a sending module 1002, an execution module 1003, a storage module 1004 and an acquisition module 1005.

[0151] Among them, the receiving module 1001 is used to receive a job execution request sent by the client, and the job execution request includes the identification of the work domain and the job content; the sending module 1002 is used to send the identification of the work domain and the job content to the work domain job sending queue, and the sending module 1002 is also used to send the job content to the port forwarding component in the target work domain corresponding to the identification of the work domain; the sending module 1002 is also used to send the job content to the target work node corresponding to the job content; the execution module 1003 is used to execute the job content.

[0152] As an example, the apparatus 1000 may be used to perform Figure 4 and Figure 7 In the method shown, for example, the receiving module 1001 is used to execute S401, the sending module 1002 is used to execute S402, S403, S404 and S701, the executing module 1003 is used to execute S405, the acquiring module 1005 is used to execute S702 and S703, and the storing module 1004 is used to execute S704.

[0153] In one possible implementation, the sending module 1002 is also used to send the job results corresponding to the job content to the port forwarding component, and the sending module 1002 is also used to send the job results to the job result queue; the sending module 1002 is also used to send the job results to the job result processing component; the storage module 1004 is used to store the job results to the business database.

[0154] In one possible implementation, the sending module 1002 is also used to send inventory information synchronization tasks, node load information synchronization tasks and script information synchronization tasks to the working domain synchronization task component queue; the obtaining module 1005 is used to listen to the synchronization tasks in the working domain synchronization task component queue, and according to the type of synchronization task, obtain the inventory information, node load information and script information in the target working domain.

[0155] In a possible implementation, the acquisition module 1005 is also used to obtain the synchronization result corresponding to the synchronization task through the port forwarding component, and to monitor the synchronization component result queue. The storage module 1004 is also used to store the synchronization results in the synchronization component result queue to the business database.

[0156] Fig.11 A system schematic diagram provided for another embodiment of the present application. Fig.11 The system shown can be used to execute the method described in any of the aforementioned embodiments.

[0157] like Fig.11 As shown, the system 1100 of this embodiment includes: a memory 1101, a processor 1102, a communication interface 1103 and a bus 1104. The memory 1101, the processor 1102 and the communication interface 1103 are connected to each other through the bus 1104.

[0158] The memory 1101 may be a read only memory (ROM), a static storage device, a dynamic storage device or a random access memory (RAM). The memory 1101 may store a program. When the program stored in the memory 1101 is executed by the processor 1102, the processor 1102 is used to execute each step of the method in the above embodiment.

[0159] The processor 1102 can adopt a general-purpose CPU, a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits to execute relevant programs to implement the various methods in the embodiments of the present application.

[0160] The processor 1102 may also be an integrated circuit chip with signal processing capability. In the implementation process, each step of the method of the embodiment of the present application may be completed by an integrated logic circuit of hardware in the processor 1102 or by instructions in the form of software.

[0161] The processor 1102 may also be a general purpose processor, a digital signal processor (DSP), an ASIC, a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. The methods, steps, and logic block diagrams disclosed in the embodiments of the present application may be implemented or executed. The general purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0162] The steps of the method disclosed in the embodiment of the present application can be directly embodied as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 1101, and the processor 1102 reads the information in the memory 1101, and completes the functions required to be performed by the units included in the device of the present application in combination with its hardware.

[0163] The communication interface 1103 may use, but is not limited to, a transceiver or other transceiver device to implement communication between the system 1100 and other devices or communication networks.

[0164] The bus 1104 may include a path for transmitting information between various components of the system 1100 (eg, the memory 1101 , the processor 1102 , and the communication interface 1103 ).

[0165] It should be understood that the system 1100 shown in the embodiment of the present application may be an electronic device, or may also be a chip configured in the electronic device.

[0166] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0167] In this application, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, ab, ac, bc or abc, where a, b, c can be single or multiple.

[0168] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0169] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0170] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0171] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0172] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0173] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0174] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk or an optical disk.

[0175] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A job scheduling system in a hybrid cloud, characterized in that: The system includes a business logic layer, a message middle layer and multiple work domains, the business logic layer includes a job receiving component and a job issuing component, the message middle layer includes a work domain job issuing queue, each of the multiple work domains includes a port forwarding component and at least one work node, the business logic layer and the message middle layer are deployed in the same network; the port forwarding component is implemented using the Nginx tool; The job receiving component is used to receive a job execution request sent by a client, wherein the job execution request includes an identifier of a work domain and job content; The job issuing component is used to send the identifier of the work domain and the job content to the work domain job issuing queue; The work domain job delivery queue is used to deliver the job content to the port forwarding component in the target work domain corresponding to the identifier of the work domain; The port forwarding component is used to send the job content to the target working node corresponding to the job content; The working node is used to execute the job content; The business logic layer also includes a synchronization task issuing component and a synchronization task result processing component, the message intermediate layer also includes a work domain synchronization task component queue and a synchronization component result queue, and each of the multiple work domains also includes a script warehouse, a synchronization component and a list database; The synchronization task issuing component is used to send the list information synchronization task, the node load information synchronization task and the script information synchronization task to the work domain synchronization task component queue; The synchronization component is used to monitor the synchronization tasks in the work domain synchronization task component queue, and obtain inventory information, node load information and script information in the target work domain according to the type of the synchronization task; The synchronization component result queue is used to obtain the synchronization result corresponding to the synchronization task through the port forwarding component; The synchronization task result processing component is used to monitor the synchronization component result queue and store the synchronization results in the synchronization component result queue to the business database.

2. The system according to claim 1, characterized in that The message middle layer also includes a job result queue, and the business logic layer also includes a job result processing component and a business database; The target working node is used to send the job result corresponding to the job content to the port forwarding component; The port forwarding component is used to send the job result to the job result queue; The job result queue is used to send the job result to the job result processing component; The operation result processing component is used to store the operation result in the business database.

3. A method for scheduling jobs in a hybrid cloud, characterized in that: The method is applied to a job scheduling system in a hybrid cloud, the system comprising a business logic layer, a message middle layer and multiple work domains, the business logic layer comprising a job receiving component and a job issuing component, the message middle layer comprising a work domain job issuing queue, each of the multiple work domains comprising a port forwarding component and at least one work node, the port forwarding component being implemented using an Nginx tool; The business logic layer and the message intermediate layer are deployed in the same network, and the method includes: The job receiving component receives a job execution request sent by a client, wherein the job execution request includes an identifier of a work domain and job content; The job issuing component sends the identifier of the work domain and the job content to the work domain job issuing queue; The work domain job delivery queue delivers the job content to the port forwarding component in the target work domain corresponding to the identifier of the work domain; The port forwarding component sends the job content to a target working node corresponding to the job content; The working node executes the job content; The business logic layer further includes a synchronization task issuing component and a synchronization task result processing component, the message intermediate layer further includes a work domain synchronization task component queue and a synchronization component result queue, each of the multiple work domains further includes a script warehouse, a synchronization component and a list database, and the method further includes: The synchronization task issuing component sends the inventory information synchronization task, the node load information synchronization task and the script information synchronization task to the work domain synchronization task component queue; The synchronization component monitors the synchronization tasks in the work domain synchronization task component queue, and acquires inventory information, node load information and script information in the target work domain according to the type of the synchronization task; The synchronization component result queue obtains the synchronization result corresponding to the synchronization task through the port forwarding component; The synchronization task result processing component monitors the synchronization component result queue, and stores the synchronization results in the synchronization component result queue to the business database.

4. The method according to claim 3, characterized in that The message intermediate layer also includes a job result queue, the business logic layer also includes a job result processing component and a business database, and the method also includes: The target working node sends the job result corresponding to the job content to the port forwarding component; The port forwarding component sends the job result to the job result queue; The job result queue sends the job result to the job result processing component; The operation result processing component stores the operation result in the business database.

5. A job scheduling system in a hybrid cloud, characterized in that: include: Memory and processor; The memory is used to store program instructions; The processor is used to call the program instructions in the memory to execute the method according to claim 3 or 4.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to claim 3 or 4 when executed by a processor.

Images