Distributed service scheduling system based on HTTP (Hyper Text Transport Protocol)

Abstract

The invention provides a distributed service scheduling system based on HTTP, and the system comprises an operation and maintenance deployment service which is responsible for deploying a distributed service; the data sharing center is used for providing various data interaction methods for data interaction between distributed business services; the service registration center is used for providing load balancing and interface proxy functions for distributed business services; the workflow editing visual interface is used for providing visual interface editing workflow basic information; the scheduling data management service is responsible for storing workflow basic information edited and configured on the workflow editing visual interface; and the scheduling operation service is responsible for controlling the scheduling process of the whole distributed business service, editing workflow basic information based on a workflow editing visual interface, rapidly editing the scheduling business process based on different distributed business services, and being compatible with different communication middleware through the data sharing center. And data interaction requirements of different businesses can be met.

Description

Technical Field

[0001] This invention relates to the field of distributed services, and more particularly to an HTTP-based distributed service scheduling system. Background Technology

[0002] With the continuous development of distributed technologies and the increasing scale and complexity of enterprise applications, higher demands are being placed on distributed service and business developers.

[0003] Early services, due to their limited computer resources, poor scalability and reliability, relied on a single service to provide all the functions of a large application. If a single function malfunctioned, the entire application would be restricted. With the development of Internet technology, distributed architecture has become the mainstream backend technology, including distributed databases and distributed middleware. Application business applications also exhibit characteristics of being distributed, multi-node, and highly concurrent. Distributed frameworks support application business to fully utilize the computing resources of polymorphic servers.

[0004] As distributed technologies continue to develop, various message middleware and distributed communication frameworks are constantly iterating and upgrading. While improving reliability and scalability, the requirements for developers are also increasing. Commonly used distributed scheduling frameworks include Elastic-Job and Quartz, but their application scenarios and functions are relatively limited. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to overcome the defects of the existing technology. The present invention proposes a distributed service scheduling system based on HTTP.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: In a first aspect, the present invention provides a distributed service scheduling system based on HTTP, comprising: Operations and deployment services, responsible for deploying distributed business services; The data sharing center provides various data interaction methods for data exchange between distributed business services; The service registry provides load balancing and interface proxy functions for distributed business services; The workflow editing visual interface provides a visual interface for editing basic workflow information; The scheduling data management service is responsible for storing the basic workflow information that is edited and configured on the workflow editing visualization interface; The scheduling and operation service is responsible for controlling the scheduling process of the entire distributed business service, and provides functions such as workflow parsing, parameter configuration, scheduling and operation, scheduling control and operation status data monitoring. The workflow editing interface allows users to edit basic workflow information, quickly edit and schedule business processes based on different distributed business services, and is compatible with different communication middleware through a data sharing center to adapt to different business data interaction needs.

[0007] Preferably, the operation and maintenance deployment service sends the deployment information of the distributed business services to the service registry center, which then manages the deployed distributed business services.

[0008] Preferably, the deployment information includes service information, the service's IP address, and port information.

[0009] Preferably, the distributed business service and the scheduling operation service only transmit the communication middleware configuration information of the data sharing center.

[0010] Preferably, the data sharing center is responsible for managing each communication middleware and providing corresponding interaction interfaces to distributed business services.

[0011] Preferably, the service registry includes three modules: Nginx, Consul, and Consul proxy service.

[0012] Preferably, the service registry receives a request to schedule and run a service, searches for a distributed business service with the same URL based on the request, and sends the service to the corresponding distributed business service according to the configured load balancing strategy, and the distributed business service processes the HTTP request.

[0013] Preferably, the distributed business service provides interface functions to the outside world based on the HTTP protocol, and at the same time, a unified communication protocol is formulated based on the HTTP protocol.

[0014] In a second aspect, the present invention provides an electronic device, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of an HTTP-based distributed service scheduling system.

[0015] Thirdly, the present invention provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of an HTTP-based distributed service scheduling system.

[0016] Compared with the prior art, the beneficial effects of the present invention include: editing basic workflow information based on a workflow editing visualization interface, quickly editing and scheduling business processes based on different distributed business services, and being compatible with different communication middleware through a data sharing center to adapt to the data interaction needs of different businesses.

[0017] Furthermore, by using the HTTP protocol as the communication framework between the scheduling service and different distributed business services, it is possible to adapt to different languages. Whether it is C++, Java or Python, by adapting the HTTP protocol of the scheduling framework, it can be used and integrated into the scheduling framework, which can meet the needs of multi-scenario and multi-business integration.

[0018] Secondly, the management of distributed business services is achieved through operation and maintenance deployment services and service registration centers. The operation and maintenance deployment services manage the basic information of distributed business services, and the service registration centers realize load balancing and service proxying of distributed business services. This can meet the functional call requirements of single-machine or distributed scenarios. The scheduling and running services communicate with distributed business services through the proxy of the service registration centers. It can dynamically expand based on different server clusters, make full use of the hardware resources of the server clusters, and meet the needs of distributed scheduling. Attached Figure Description

[0019] The disclosure of this invention is illustrated with reference to the accompanying drawings. It should be understood that the drawings are for illustrative purposes only and are not intended to limit the scope of protection of this invention. In the drawings, the same reference numerals are used to refer to the same parts. Wherein: Figure 1 The diagram illustrates the business process of a distributed service scheduling framework proposed according to an embodiment of the present invention.

[0020] Figure 2 The diagram illustrates a data sharing center module according to one embodiment of the present invention.

[0021] Figure 3 The diagram illustrates a service registry module according to one embodiment of the present invention.

[0022] Figure 4 The diagram illustrates a visualization of basic workflow information according to an embodiment of the present invention.

[0023] Figure 5 The diagram illustrates a scheduling and operation service workflow according to an embodiment of the present invention. Detailed Implementation

[0024] It is readily understood that, based on the technical solution of this invention, those skilled in the art can propose various interchangeable structural methods and implementations without altering the essential spirit of the invention. Therefore, the following detailed embodiments and accompanying drawings are merely illustrative examples of the technical solution of this invention and should not be considered as the entirety of the invention or as limitations or restrictions on the technical solution of this invention.

[0025] Example 1, referring to Figure 1-5 As an embodiment of the present invention, an HTTP-based distributed service scheduling system is provided, comprising: S100: The operation and maintenance deployment service is mainly responsible for deploying distributed business services and sending the deployment information of distributed business services, including service information, service IP and port information, to the service registry center, which then manages the deployed distributed business services.

[0026] S200: The data sharing center mainly serves the data interaction between distributed business services, providing multiple data interaction methods. Distributed business services and scheduling and operation services only transmit the communication middleware configuration information of the data sharing center. Actual parameter data is interacted through communication middleware such as Redis, MongoDB, Ceph and RabbitMQ provided by the data sharing center. The data sharing center is responsible for managing each communication middleware and providing corresponding interaction interfaces to distributed business services.

[0027] S300: The service registry includes three modules: Nginx, Consul, and Consul proxy service.

[0028] S301: The service registry is mainly responsible for providing load balancing and interface proxy functions for distributed business services. After the distributed business services that provide the same service are deployed by the operation and maintenance deployment service, they will transmit the service information to the Consul proxy service, and the Consul proxy service will register with the HTTP service registry of Consul and Nginx. The Consul proxy service will periodically communicate with the distributed business services to keep them alive and update the status of each distributed business service in a timely manner.

[0029] S302: The service registry receives requests to schedule and run services, searches for distributed business services that provide the same URL based on the requests, and sends the services to the corresponding distributed business services according to the configured load balancing strategy. The distributed business services then process the HTTP requests.

[0030] S400: Distributed business services provide interface functions to the outside world based on the HTTP protocol. At the same time, based on the HTTP protocol, a unified communication protocol is formulated to support the integrated scheduling of distributed business services developed in multiple programming languages, including C++, Python, and Java.

[0031] S401: The HTTP service interface protocol provided by the distributed business service to the outside world has been redesigned and the protocol interaction content has been re-encapsulated. The HTTP interface protocol for communication between the distributed business service and the scheduling and operation service mainly contains data sharing configuration information. The distributed business service accesses input parameter data and output parameter data based on the data sharing configuration information.

[0032] S402: The data sharing configuration information for distributed business services includes the communication middleware configuration information for storing input parameter data and output parameter data, including Redis key-value pair information, Mongo primary key information, RabbitMQ queue information, and Ceph file path information. Depending on the different data sharing configuration information for each input parameter and output parameter, the corresponding parameter data is stored from different middleware based on the data sharing center, and different data sharing configurations can be adopted according to different business requirements.

[0033] S403: The data sharing configuration information of distributed business services is configured by the scheduling and operation service. The scheduling and operation service parses the entire distributed scheduling process based on the workflow basic information and configures the data sharing configuration of each distributed business service. The main content of the communication between the scheduling and operation service and the distributed business services based on the HTTP protocol is the data sharing configuration information. The input and output parameter data are exchanged by the data sharing center.

[0034] S500: The scheduling data management service is responsible for storing the basic workflow information that is edited and configured on the workflow editing visualization interface, and transmitting the basic workflow information when the scheduling operation service receives a scheduling request.

[0035] S501: Workflow editing visual interface provides a visual interface for editing basic workflow information.

[0036] S502: Workflow basic information is divided into four basic elements on the workflow editing visualization interface: connection, interface, input label and output label.

[0037] S503: Input and output labels on the workflow editing visualization interface are used to indicate the input and output parameter data required by the workflow basic information. At the same time, based on the connection information, they identify the data interaction relationship between the current input and output parameter data and the interface provided by the distributed business service.

[0038] S504: The interface for basic workflow information on the workflow editing visualization interface is provided by the distributed business service. The distributed business service provides an HTTP interface based on the HTTP protocol and can be edited as a basic element on the workflow editing visualization interface.

[0039] S505: Workflow basic information. The black circle in the upper left corner of each interface box on the workflow editing visualization interface represents the input parameters of the current interface. One black circle represents one input parameter, and multiple circles constitute the complete interface input parameters.

[0040] S506: Workflow basic information. The black circle in the lower right corner of each interface box on the workflow editing visualization interface represents the output parameter of the current interface. One black circle represents one input parameter, and multiple circles constitute the complete interface output parameter.

[0041] S507: The connections on the workflow editing visualization interface are used to identify the data interaction relationship between different interfaces. Each connection identifies a certain output parameter from one interface to a certain input parameter of another interface. Each connection needs to be configured with connection attributes, including data sharing configuration information of interface parameters, which is consistent with the data sharing configuration information described in S401.

[0042] S600: The scheduling and operation service is responsible for the scheduling process control of the entire distributed business service, and provides functions such as workflow parsing, parameter configuration, scheduling and operation, scheduling control and operation status data monitoring.

[0043] S601: After receiving an external scheduling request, the scheduling operation service obtains basic workflow information from the scheduling data management service, then performs workflow parsing to obtain the entire scheduling process information.

[0044] S602: After the scheduling and operation service completes the parsing of the basic workflow information, it initializes the parameters of each interface based on the basic information, including data sharing configuration information, HTTP request information of the interface, and scheduling logic sequence parameters of each interface.

[0045] S603: After the scheduling and operation service completes the parameter configuration, it requests the interface functions of the distributed business services in sequence according to the scheduling logic order parameters, and calls the interface functions through the service registry center, directly transmitting the data sharing configuration information to the interface through HTTP requests.

[0046] S604: After the scheduling and operation service calls the interface function through the service registration center, the called interface obtains the input parameter data from the data sharing center according to the data sharing configuration information transmitted by the scheduling and operation service, performs business processing according to the interface function, and stores the output parameter data in the data sharing center according to the data sharing configuration information after the business processing is completed.

[0047] S605: The scheduling and running service obtains the parameters returned by the interface of the distributed business service, determines whether the current call was successful based on the returned parameters, and determines whether the current scheduling process has ended based on the scheduling logic sequence parameters. If it has not ended, the scheduling process continues to be executed.

[0048] S606: After the scheduling process of the scheduling service is completed, the intermediate data and result data of the scheduling process can be obtained based on the data sharing configuration information, and workflow running instance data can be generated and returned to the caller.

[0049] S700: Configures the data interaction relationship between various distributed business service interfaces based on the workflow editing visual interface, realizes interface communication between scheduling and running services and distributed business services based on HTTP protocol and service registry center, and provides a variety of communication middleware through data sharing center to adapt to different business communication needs.

[0050] This embodiment also provides an electronic device, including: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, they implement the steps of an HTTP-based distributed service scheduling system.

[0051] This embodiment also provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of an HTTP-based distributed service scheduling system.

[0052] The storage medium proposed in this embodiment belongs to the same inventive concept as the implementation of an HTTP-based distributed service scheduling system proposed in the above embodiments. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0053] Based on the above description of the implementation methods, those skilled in the art can clearly understand that the present invention can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of the various embodiments of the present invention.

[0054] The technical scope of this invention is not limited to the content described above. Those skilled in the art can make various modifications and variations to the above embodiments without departing from the technical concept of this invention, and all such modifications and variations should fall within the protection scope of this invention.

Claims

1. An HTTP-based distributed traffic steering system, characterized in that, The application relates to an HTTP-based distributed business scheduling system. The operation and deployment service is responsible for deploying distributed business services. The data sharing center provides various data interaction methods for data interaction between distributed business services. The service registration center provides load balancing and interface proxy functions for distributed business services. The workflow editing visual interface provides a visual interface for editing workflow basic information. The scheduling data management service is responsible for storing workflow basic information configured on the workflow editing visual interface. The scheduling running service is responsible for scheduling process control of the entire distributed business service, and provides workflow analysis, parameter configuration, scheduling running, scheduling control and running state data monitoring functions. The workflow basic information is edited based on the workflow editing visual interface, the scheduling business process is quickly edited based on different distributed business services, and different communication middleware is compatible through the data sharing center to adapt to different business data interaction requirements.

2. The HTTP-based distributed traffic steering system of claim 1, wherein, The operation and deployment service sends deployment information of the distributed business service to the service registration center, and the service registration center manages the deployed distributed business service.

3. The HTTP-based distributed traffic steering system of claim 2, wherein, The deployment information includes service information, IP and port information of the service.

4. The HTTP-based distributed traffic steering system of claim 1, wherein, Only the communication middleware configuration information of the data sharing center is transmitted between the distributed business service and the scheduling running service.

5. The HTTP-based distributed traffic steering system of claim 1, wherein, The data sharing center is responsible for managing various communication middleware and providing corresponding interaction interfaces to the distributed business service.

6. The HTTP-based distributed traffic steering system of claim 1, wherein, The service registration center includes three modules of Nginx, Consul and Consul agent service.

7. The HTTP-based distributed traffic steering system of claim 6, wherein, The service registration center receives a request of the scheduling running service, finds distributed business services providing the same URL according to the request, sends the services to corresponding distributed business services according to a configured load balancing strategy, and processes HTTP requests by the distributed business services.

8. The HTTP-based distributed traffic steering system of claim 7, wherein, The distributed business service provides interface functions to the outside according to the HTTP protocol, and formulates a unified communication protocol based on the HTTP protocol. 9.An electronic device comprising: a memory and a processor; the memory is configured to store computer executable instructions, and the processor is configured to execute the computer executable instructions, and the computer executable instructions, when executed by the processor, implement the steps of the HTTP-based distributed business scheduling system according to any one of claims 1 to 8. 10.A computer readable storage medium storing computer executable instructions, and the computer executable instructions, when executed by a processor, implement the steps of the HTTP-based distributed business scheduling system according to any one of claims 1 to 8.

Images