Distributed system decentralized real-time business scheduling management method and system

Abstract

The present invention relates to a distributed system decentralized real-time business scheduling management method and system. The method includes: using multiple subsystems to directly perform business calls and communications without going through a business management subsystem; deploying service meshes to the multiple subsystems accordingly, and using multiple service meshes to proxy the multiple subsystems for external and external requests; using the business management subsystem to interact with multiple service meshes to collaboratively implement filtering, forwarding and mangle processing. The present invention can solve the versatility of business tracking in a distributed system architecture, and provide functions and services such as decentralized business real-time tracking and error handling mechanisms in a distributed system.

Description

Technical Field

[0001] The present invention relates to the technical field of business scheduling management methods, and in particular to a distributed system decentralized real-time business scheduling management method and system. Background Art

[0002] There are three common types of existing distributed system architecture designs: one is a loose architecture, that is, each subsystem can be designed heterogeneously and even run on different CPUs and different operating systems. The distributed system only defines the way each subsystem provides services to the outside world, such as through REST, RPC, etc. All aspects involved in service governance require self-developed support; one is a unified framework design, such as using Dubbo or SpringCloud as the only framework, and each subsystem runs under the same framework. The framework itself provides many functions for distributed service governance, such as service registration and discovery, and a unified log management mechanism; there is also a container-based distributed system design, which does not require a unified framework for each system, but also provides many service governance mechanisms for distributed systems.

[0003] The purpose of business management in distributed systems is to track processes, perform business statistics, trace and handle business processing errors, and conduct user behavior analysis. Currently, no matter which distributed system architecture is used, business tracking, business process control, and process tracing are all difficult problems. There are generally two existing methods for business scheduling management:

[0004] (1) Centralized business scheduling, with the business management subsystem as the central node. The subsystems in the distributed system do not interact directly with each other, and all businesses are uniformly scheduled through the business management subsystem. Its advantages are that it is easy to implement business tracking, business statistics, error handling, user behavior analysis, etc. The disadvantage is that the central node of the distributed system is the business management subsystem. Once the central node goes down or the service quality decreases, it will affect the operation of all businesses. In addition, all requests are forwarded through the business management subsystem, and the system performance may be slightly reduced.

[0005] (2) Each business in the distributed system runs independently and can call each other freely. According to the requirements of business tracking, a unified log is generated, and then a special processing system analyzes the log, traces the process, and performs user behavior analysis. The advantage is that the distributed system has no central node, which reduces the probability of system errors. The disadvantage is that the real-time performance of business tracking is poor and an additional error handling mechanism is required. Summary of the invention

[0006] In order to solve the universality of business tracking in a distributed system architecture and implement a real-time business management method such as decentralized business real-time tracking and error handling mechanism in a distributed system, the present invention provides a distributed system decentralized real-time business scheduling management method and system.

[0007] In order to achieve the above-mentioned object of the invention, the technical solution of the present invention is:

[0008] The present invention provides a distributed system decentralized real-time service scheduling management method, comprising:

[0009] Use multiple subsystems to directly call and communicate business without going through the business management subsystem;

[0010] Deploy the service mesh to the multiple subsystems accordingly, and use the multiple service meshes to proxy the multiple subsystems' requests to the outside and from the outside;

[0011] The business management subsystem is used to interact with multiple service meshes to collaboratively implement filtering, forwarding, and mangle processing.

[0012] According to one aspect of the present invention, the filtering process includes:

[0013] Utilize the plurality of service meshes to analyze the pressure of the corresponding subsystems, generate an intrusion detection alarm message or a traffic peak alarm message, and notify the business management subsystem;

[0014] When the service management subsystem receives the intrusion detection alarm message of the subsystem, the service management subsystem automatically limits the flow and blocks the IP via the service mesh;

[0015] When the business management subsystem receives the traffic peak alarm message of the subsystem, it dynamically adds processing nodes to smooth out the traffic peak.

[0016] According to one aspect of the present invention, the forwarding process includes:

[0017] When the service mesh detects that the corresponding subsystem is down, it notifies the business management subsystem;

[0018] The business management subsystem broadcasts the received downtime message to all other service meshes.

[0019] According to one aspect of the present invention, the forwarding process further includes:

[0020] After the business management subsystem receives the downtime message and broadcasts it to all other service meshes, the business management subsystem issues an alarm, dynamically allocates new processing nodes, and recycles the original processing nodes to achieve automatic operation and maintenance of the distributed system.

[0021] According to one aspect of the present invention, the process of mangle processing includes:

[0022] Using a plurality of the service meshes to automatically record the start time and scheduling parameters of the business management subsystem scheduling according to the tracking rules, and sending the recorded information to the business management subsystem;

[0023] The business management subsystem is used to perform real-time process restoration based on the record information reported by multiple service meshes, generate time statistics, and perform user behavior analysis.

[0024] The present invention also provides a distributed system decentralized real-time business scheduling management system implemented by the above method, including: a business management subsystem, multiple subsystems and multiple service meshes,

[0025] Each subsystem is deployed with a corresponding service mesh, so that the subsystem can interact with the business management subsystem through the corresponding service mesh;

[0026] The subsystem is used to directly request services provided by other subsystems in the distributed system to achieve interaction between the subsystems;

[0027] The service mesh is used to proxy the requests of the multiple subsystems to the outside and from the outside;

[0028] The business management subsystem is used to interact with the multiple service meshes and collaboratively implement filtering, forwarding and mangle processing.

[0029] According to another aspect of the present invention, when the business management subsystem and the multiple service meshes cooperate to implement filtering processing, the multiple service meshes analyze the pressure of the corresponding subsystems, generate an intrusion detection alarm message or a traffic peak alarm message, and notify the business management subsystem;

[0030] When the service management subsystem receives the intrusion detection alarm message of the subsystem, the service management subsystem automatically limits the flow and shields the IP via the multiple service meshes;

[0031] When the business management subsystem receives the traffic peak alarm message of the subsystem, it dynamically adds processing nodes to smooth out the traffic peak.

[0032] According to another aspect of the present invention, when the service management subsystem and the multiple service meshes cooperate to implement forwarding processing, the multiple service meshes detect that the corresponding subsystem is down and notify the service management subsystem;

[0033] The business management subsystem broadcasts the received downtime message to all other service meshes.

[0034] According to another aspect of the present invention, after the business management subsystem receives the downtime message and broadcasts it to all other service meshes, the business management subsystem issues an alarm, and the subsystem dynamically allocates new processing nodes and recycles the original processing nodes to achieve automatic operation and maintenance of the distributed system.

[0035] According to another aspect of the present invention, when the business management subsystem and the multiple service meshes collaborate to implement mangle processing, the multiple service meshes automatically record the start time and scheduling parameters of the business management subsystem scheduling according to the tracking rules, and send the recorded information to the business management subsystem;

[0036] The business management subsystem is used to perform real-time process restoration according to the record information reported by the multiple service meshes, generate time statistics, and perform user behavior analysis.

[0037] Beneficial effects:

[0038] According to the solution of the present invention, through the distributed system decentralized real-time business scheduling management method and the constructed distributed system decentralized real-time business scheduling management system architecture, the problem of the universality of business tracking in the distributed system architecture can be solved, and each service mesh and the business management subsystem work together to realize the functions and services such as real-time tracking, service monitoring, flow control and fuse mechanism in business management. The service mesh is deployed together with the subsystems of the distributed system, and each subsystem does not interact directly with the business management subsystem, and even each subsystem does not perceive the existence of the servicemesh, which can avoid the problem that once the central node of the centralized business management subsystem goes down or the service quality decreases, it will affect the operation of all businesses, and can also realize decentralized real-time tracking and error handling mechanisms. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A flowchart schematically showing a method for decentralized real-time service scheduling management in a distributed system according to an embodiment of the present invention;

[0040] Figure 2A schematic diagram of the architecture of a distributed system decentralized real-time business scheduling and management system according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0041] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0042] The present invention will be described in detail below in conjunction with the accompanying drawings and specific embodiments. The embodiments cannot be described one by one here, but the embodiments of the present invention are not therefore limited to the following embodiments.

[0043] like Figure 1 As shown, the distributed system decentralized real-time business scheduling management method of this embodiment can solve the universal problem of business tracking, that is, the method realizes the cross-distributed processing framework property and can be applied to three distributed processing frameworks. The three frameworks are loose architecture, unified framework design and container-based distributed system design. The mechanism and method for implementing decentralized business real-time tracking and error handling in a distributed system. The method includes the following steps:

[0044] Use multiple subsystems to directly call and communicate business without going through the business management subsystem;

[0045] Deploy the service mesh to multiple subsystems accordingly, and use multiple service meshes to proxy requests from multiple subsystems to the outside world and from the outside world;

[0046] The business management subsystem interacts with multiple service meshes to collaboratively implement filtering, forwarding, and mangle processing, thereby achieving business management, real-time process tracking, and error handling.

[0047] Among them, the filtering process includes: using multiple service meshes to analyze the pressure of the corresponding subsystem, generating intrusion detection alarm messages or traffic peak alarm messages, and notifying the business management subsystem; when the business management subsystem receives the intrusion detection alarm message of the subsystem, the business management subsystem can remind the user, or automatically limit the flow and block the IP through the service mesh; when the business management subsystem receives the traffic peak alarm message of the subsystem, the subsystem can remind the user, if the distributed system supports dynamic hardware resource management, it can dynamically increase processing nodes to smooth the traffic peak.

[0048] The forwarding process includes: when the service mesh detects that a subsystem is down, it notifies the business management subsystem; the business management subsystem broadcasts the received downtime message to all other service meshes. When another subsystem A wants to call the above-mentioned downtime subsystem, the local servicemesh deployed in subsystem A directly returns a failure message and blows the fuse in time to avoid an avalanche effect on the server. After the business management subsystem obtains the downtime message and broadcasts it to all other service meshes, the business management subsystem can alarm and remind users. If the distributed system supports dynamic hardware resource management, the subsystem can dynamically allocate a new processing node and recycle the original processing node to realize intelligent and automatic operation and maintenance of the distributed system.

[0049] The mangle processing process includes: using multiple service meshes to automatically record the start time and scheduling parameters of the business management subsystem according to tracking rules, and sending the recorded information to the business management subsystem; using the business management subsystem to perform real-time process restoration, production time statistics, and user behavior analysis based on the recorded information reported by multiple service meshes.

[0050] like Figure 2 As shown, the distributed system decentralized real-time business scheduling and management system of this embodiment is composed of a business management subsystem and several service meshes and several subsystems. The service mesh is deployed together with the subsystems of the distributed system. The subsystems do not interact directly with the business management subsystem, and even the subsystems are not aware of the existence of the service mesh. The service mesh is a universal proxy service that can be deployed on any platform and is independent of the framework adopted by the distributed processing system. Through several service meshes and business management subsystems, services and functions such as real-time tracking, service monitoring, flow control and fuse mechanisms in business management are provided together. The several subsystems here refer to such as Figure 2 Subsystem 1 and subsystem 2 shown may represent subsystems such as data access, processing, display, and distribution.

[0051] The distributed system decentralized real-time service scheduling and management system of this embodiment is applied and implemented through the above-mentioned distributed system decentralized real-time service scheduling and management method. Figure 2 As shown in the figure, the system mainly includes: a business management subsystem, multiple subsystems and multiple service meshes. Each subsystem is deployed with a corresponding service mesh, so that the subsystem can interact with the business management subsystem through the corresponding service mesh;

[0052] The subsystem is used to directly request services provided by other subsystems in the distributed system to achieve interaction between subsystems;

[0053] The service mesh is used to proxy requests from multiple subsystems to and from the outside world;

[0054] The business management subsystem is used to interact with multiple service meshes and collaborate to implement filtering, forwarding, and mangle processing. Service mesh is deployed on the servers where all subsystems are located. The service mesh proxies external requests. The service mesh performs filter, forward, mangle and other processing during the proxy process. Then, servicemesh collaborates with the business management subsystem to implement business management, real-time process tracking, error handling, etc.

[0055] When the business management subsystem and multiple service meshes work together to implement filtering processing, multiple service meshes analyze the pressure of the corresponding subsystems, generate intrusion detection alarm messages or traffic peak alarm messages, and notify the business management subsystem;

[0056] When the business management subsystem receives the intrusion detection alarm message from the subsystem, the business management subsystem automatically limits the flow and blocks the IP through the service mesh;

[0057] When the business management subsystem receives the traffic peak alarm message from the subsystem, it dynamically adds processing nodes to smooth out the traffic peak.

[0058] When the business management subsystem collaborates with multiple service meshes to implement forwarding processing, multiple service meshes detect that the corresponding subsystems are down and notify the business management subsystem;

[0059] The business management subsystem broadcasts the received downtime message to all other service meshes. After that, the business management subsystem issues an alarm, dynamically allocates new processing nodes, and recycles the original processing nodes to achieve automatic operation and maintenance of the distributed system.

[0060] When the business management subsystem collaborates with multiple service meshes to implement mangle processing, multiple service meshes automatically record the start time and scheduling parameters of the business management subsystem according to the tracking rules, and send the recorded information to the business management subsystem;

[0061] The business management subsystem is used to perform real-time process restoration, production time statistics, and user behavior analysis based on the record information reported by the service mesh.

[0062] According to the above scheme of the present invention, the above decentralized service scheduling method and system ensure that the scheduling between subsystems does not need to go through the service management system. Each subsystem in the distributed processing system will deploy a service mesh. The subsystem's external requests and requests from the outside will all go through this service mesh. This service mesh performs service tracking, recording, filtering, and forwarding during the service forwarding process. In addition, the service mesh has the function of monitoring the subsystem. Since the service mesh is a lightweight service deployed with the subsystem, it does not affect business efficiency and stability. The business management subsystem interacts with each service mesh, can summarize the information reported by each service mesh for real-time process tracking, and can send filtering, forwarding, and mangle rules to each service mesh to achieve the purpose of service control.

[0063] The above is only one embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A decentralized real-time business scheduling management method for a distributed system. include: Use multiple subsystems to directly call and communicate business without going through the business management subsystem; Deploy the service mesh to the multiple subsystems accordingly, and use the multiple service meshes to proxy the multiple subsystems' requests to the outside and from the outside; Utilize the business management subsystem to interact with multiple service meshes to collaboratively implement filtering, forwarding, and mangle processing; The forwarding process includes: When the service mesh detects that the corresponding subsystem is down, it notifies the business management subsystem; The business management subsystem broadcasts the received downtime message to all other service meshes; when another subsystem A needs to call the downtime subsystem, the service mesh deployed in the subsystem A directly returns a failure message and is disconnected in time; After the business management subsystem receives the downtime message and broadcasts it to all other service meshes, the business management subsystem issues an alarm, and the subsystem dynamically allocates new processing nodes and recycles the original processing nodes to achieve automatic operation and maintenance of the distributed system; The mangle processing process includes: Using a plurality of the service meshes to automatically record the start time and scheduling parameters of the business management subsystem scheduling according to the tracking rules, and sending the recorded information to the business management subsystem; The business management subsystem is used to perform real-time process restoration based on the record information reported by multiple service meshes, generate time statistics, and perform user behavior analysis.

2. The method according to claim 1, It is characterized in that The filtering process includes: Utilize the plurality of service meshes to analyze the pressure of the corresponding subsystems, generate an intrusion detection alarm message or a traffic peak alarm message, and notify the business management subsystem; When the service management subsystem receives the intrusion detection alarm message of the subsystem, the service management subsystem automatically limits the flow and blocks the IP via the service mesh; When the business management subsystem receives the traffic peak alarm message of the subsystem, it dynamically adds processing nodes to smooth out the traffic peak.

3. A distributed system decentralized real-time business scheduling management system implemented by the method according to claim 1 or 2, It is characterized in that include: Business management subsystem, multiple subsystems and multiple service meshes, Each subsystem is deployed with a corresponding service mesh, so that the subsystem can realize the interaction between the subsystem and the business management subsystem through the corresponding service mesh; The subsystem is used to directly request services provided by other subsystems in the distributed system to achieve interaction between the subsystems; The service mesh is used to proxy the requests of the multiple subsystems to the outside and from the outside; The business management subsystem is used to interact with the multiple service meshes and collaboratively implement filtering, forwarding and mangle processing.

4. The system according to claim 3, It is characterized in that When the business management subsystem and the multiple service meshes cooperate to implement filtering processing, the multiple service meshes analyze the pressure of the corresponding subsystems, generate an intrusion detection alarm message or a traffic peak alarm message, and notify the business management subsystem; When the service management subsystem receives the intrusion detection alarm message of the subsystem, the service management subsystem automatically limits the flow and shields the IP via the multiple service meshes; When the business management subsystem receives the traffic peak alarm message of the subsystem, it dynamically adds processing nodes to smooth out the traffic peak.

5. The system according to claim 3, It is characterized in that When the business management subsystem and the multiple service meshes collaborate to implement forwarding processing, the multiple service meshes detect that the corresponding subsystem is down and notify the business management subsystem; The business management subsystem broadcasts the received downtime message to all other service meshes.

6. The system according to claim 5, It is characterized in that After the business management subsystem receives the downtime message and broadcasts it to all other service meshes, the business management subsystem issues an alarm, dynamically allocates new processing nodes, and recycles the original processing nodes to achieve automatic operation and maintenance of the distributed system.

7. The system according to claim 3, It is characterized in that When the business management subsystem and the multiple service meshes collaborate to implement mangle processing, the multiple service meshes automatically record the start time and scheduling parameters of the business management subsystem scheduling according to the tracking rules, and send the recorded information to the business management subsystem; The business management subsystem is used to perform real-time process restoration according to the record information reported by the multiple service meshes, generate time statistics, and perform user behavior analysis.

Images