A gateway service system and method

Abstract

The application discloses a gateway service system and method, the system comprises a server module and a client module; the server module carries out routing operation according to a rule filter when receiving a downstream service request sent by the client module, the rule filter comprises a pre-filter, a middle filter, a post-filter and an exception filter; the middle filter is used for load balancing and protocol analysis, and single-asynchronous request processing or double-asynchronous request processing is selected according to the length of access delay time of a downstream processing request; the weight in load balancing adopts a preheating mechanism, and increases from a preset initial value to a preset weight value within a period of time; the application can flexibly configure filtering rules, optimize load balancing strategies, and improve the reliability and robustness of gateway service.

Description

TECHNICAL FIELD

[0001] The present application relates to a gateway, in particular to a gateway service system and method. BACKGROUND

[0002] In the era of the Internet, as the complexity of the business system increases, monolithic services are split according to different business logic and business modules. With the increase of function modules, the function entry of the program will be more and more, and the gateway system as an internal and external bridge provides services by exposing HTTP interfaces to the outside, manages all interfaces exposed by the business system to the outside, and distributes requests to internal business systems. The access request is unified and converged. However, the existing gateway system cannot flexibly configure filtering rules to let requests pass through different filtering rules for interception or statistics; frequent context switching reduces the performance of the gateway service. SUMMARY

[0003] The purpose of the present application is to provide a high-performance gateway service system that can flexibly configure filtering rules, adapt to multiple service protocols, and reduce context switching. The second purpose of the present application is to provide a gateway service method.

[0004] Technical solution: The gateway service system disclosed by the present application comprises a server module and a client module.

[0005] The server module performs routing operations according to a rule filter when receiving a downstream service request sent by the client module, and the rule filter comprises a pre-filter, a middle filter, a post-filter and an exception filter.

[0006] The pre-filter comprises authentication and authorization, flow control and black and white list authentication.

[0007] The middle filter is used for load balancing and protocol analysis, and selects single asynchronous request processing or double asynchronous request processing according to the length of the access delay time of the downstream processing request; the single asynchronous request processing is to take out a thread from a thread pool to process the request sent by the client, and synchronously wait for the response of the client module; the double asynchronous request processing is to take out a thread from a thread pool to process the request sent by the client, and process other requests when no response is received from the downstream service.

[0008] The post-filter is used for index analysis after the downstream service returns the response result to the client, and the index comprises response time and access times.

[0009] The exception filter is used for writing exception information back to the client module when exception information occurs, and executing the post-filter for index analysis.

[0010] The client module is used for parsing and registering the downstream service.

[0011] Further, the load balancing includes a weighted random strategy and a weighted round robin strategy, the weighted random strategy is to add weights of all servers to obtain a weight sum, select a random number between 1 and the weight sum, and sequentially subtract the weight of each server, and select a server whose result after subtraction is not greater than 0; the weighted round robin strategy is to preferentially select a server with the highest weight at each round robin; the weight of the server is increased from a preset initial value to a preset weight value within a period of time.

[0012] Further, the server module is internally provided with a cache area for storing configuration information of the gateway service, filter rule information and registration table information of the downstream service; the main thread puts the downstream service request into the cache area, and the background thread asynchronously executes the downstream service request;

[0013] Further, in the cache area, each variable is filled with a plurality of padding values before and after the variable, and each variable occupies a cache line.

[0014] Further, when the downstream service upgrades or a new function is put on line, a to-be-upgraded label is marked for the downstream service, the downstream service completes the received request and then performs offline and upgrade operations; the proportion of the downstream service marked with the to-be-upgraded label in all downstream services does not exceed a first threshold value.

[0015] Further, the client registers the downstream service to a registration center cluster, the client saves the IP address, port number and service path of the downstream service in a service configuration information table and sends the same to the registration center cluster for storage; the registration center cluster includes a main registration center server and a backup registration center server, and switches to the backup registration center server when the main registration center server loses heartbeat.

[0016] Further, the client includes a protocol adaptation unit and a registration center client unit, the protocol adaptation unit is used for protocol parsing of the downstream service, and the registration center client unit is used for registration to the registration center cluster according to the protocol of the downstream service.

[0017] Further, the gateway service system further includes a console module, the console module includes a definition service unit for determining gateway service number, version number, protocol and service path, a server configuration unit for configuring access address, routing label and server weight, and a rule configuration unit for configuring rules of the front filter, middle filter, rear filter and abnormal filter.

[0018] The gateway service method provided by the application, a client module parses and registers a downstream service, and a server module performs routing operation according to a rule filter when receiving a downstream service request sent by the client module, the rule filter comprising a pre-filter, a middle-filter, a post-filter and an exception filter;

[0019] The pre-filter comprises authentication authorization, flow control and black and white list authentication,

[0020] The middle-filter is used for load balancing and protocol resolution, and selects single-asynchronous request processing or double-asynchronous request processing according to the length of access delay time of a downstream processing request; the single-asynchronous request processing is to take out a thread from a thread pool to process a request sent by a client, and synchronously waits for a response of the client module; the double-asynchronous request processing is to take out a thread from the thread pool to process a request sent by a client, and processes other requests when a response of the downstream service is not received;

[0021] The post-filter is used for index analysis after a downstream service returns a response result to a client, the index comprising response time and access times;

[0022] The exception filter is used for writing back exception information to a client module when the exception information appears, and performing the post-filter to perform index analysis.

[0023] The computer readable storage medium provided by the application stores a computer program, and the computer program is executed by a processor to realize the gateway service method.

[0024] Advantages: compared with the prior art, the application has the following advantages:

[0025] (1) a complete gateway service system is constituted by a server module, a client module and a console module, the gateway system adopts a distributed architecture, and the single-point failure problem is solved;

[0026] (2) the weight in the load balancing strategy adopts a preheating mechanism, when a service is just online, the requests are not all allocated to the server, but the weight is increased with the lapse of time until the preheating time ends to reach the initial set weight; a plurality of load balancing strategies, i.e. random and polling strategies, are supported, and when the above strategies are executed, the weight parameter setting is also considered preferentially, and the higher the weight parameter is, the higher the probability of responding to the random processing request is;

[0027] (3) In the gateway service, the asynchronous design is adopted when forwarding the request to downstream for processing, and the single-asynchronous mode or the double-asynchronous mode is switched dynamically according to the processing time of the downstream service. If the downstream processing time is long, the double-asynchronous mode, i.e., the request is asynchronous and the response is asynchronous, is switched. If the downstream processing time is short, the single-asynchronous mode is switched, and the excessive context switching is avoided;

[0028] (4) The gateway service adopts the local cache strategy to store the pulled metadata, and the robustness of the service is improved as a whole. Since the gateway service is a CPU-intensive service, the operating system core and thread are bound to reduce the context switching of the thread:

[0029] (5) In the process of processing the client request, the pre-filter, the middle-filter, the post-filter and the exception filter are sequentially passed through, and the filtering information can be flexibly configured to filter the request through different rules;

[0030] (6) After the filters are classified according to the execution stage, they are also classified according to the function of the filter, such as the black and white list plug-in and the statistical analysis plug-in. In order to facilitate the management of the request path and the filtering plug-in, the rule is introduced, a path is bound to a rule, a rule is bound to multiple plug-ins, and the rule can be used for multiple service paths. BRIEF DESCRIPTION OF DRAWINGS

[0031] Figure 1 The gateway service system architecture of the application is shown.

[0032] Figure 2 The gateway service starting flowchart in the embodiment of the application is shown.

[0033] Figure 3 The request processing flowchart in the embodiment of the application is shown.

[0034] Figure 4 The registration center client unit processing flowchart in the embodiment of the application is shown.

[0035] Figure 5 The downstream service upgrade flowchart in the embodiment of the application is shown.

[0036] Figure 6 The console module running flowchart in the embodiment of the application is shown. DETAILED DESCRIPTION

[0037] The technical solutions of the application will be further described below with reference to the drawings.

[0038] As shown in Figure 1 The gateway service system comprises a server module, a client module and a console module.

[0039] The service end module is a core module of the gateway system. When the module is started, firstly, a running parameter or a configuration file is loaded to start the network service, and the corresponding service registration and rule information is pulled from the registration center service end to complete the start of the self container. During the running, the request is handed over to the working thread for processing by the forwarding thread, and the request is encapsulated in the context environment. Through the loaded filter rule, the filter is effectively organized and serialized, and the request is sequentially passed through the filter to finally complete the routing request and forwarding.

[0040] The client module includes a protocol adaptation unit and a registration center client unit. The protocol adaptation unit defines the specifications accessed by different types of client modules, analyzes the access path exposed by the downstream service end, and registers the service information to the registration center service end. The registration center client unit defines the listening service interface and the registration service interface, abstracts the application service access, and can select different technical components to realize the registration center service end function. The bottom layer of the registration center service end mainly stores the downstream service registration information, the basic information of the gateway, and the rule configuration information.

[0041] The console module completes the configuration of the rule through the front-end interface, and realizes the mutual binding of different rules and specific request paths. The console module can directly interact with the registration center service end.

[0042] The service end module specifically includes the following contents:

[0043] (1) Configuration file container

[0044] During the starting process of the gateway system, the gateway configuration file is automatically loaded. In the embodiment, the priority of the configuration file is designed. The priority of the default running parameter is the highest, followed by the Java virtual machine parameter, then the environment variable, and finally the configuration file parameter. When loading, the order is opposite to the above order. The configuration file is loaded first, then the environment variable, then the Java virtual machine parameter, and finally the running parameter. When a new parameter appears, the previous parameter is covered, so as to ensure the priority defined before.

[0045] (2) Request flow

[0046] After the gateway system starts, the internal network server is started and listens to the request for service outside. After the user sends a request, the network server receives the request, and the request thread puts it into the buffer area. Then the working thread takes the request from the buffer area and checks the legality of the request. If it is a legal request, it will start to build the request context object, locate the resources to be accessed by parsing the url in the access request, determine the possible service instances to be routed according to the service definition and other related information in the cache, construct the filter chain according to the filtering rules, execute the serialization filtering logic, and forward the request to the downstream. Finally, the service response of the downstream is forwarded to the request client.

[0047] When the gateway system starts, the corresponding service instance definition information is loaded from the registration center server, including service unique number, service version number, service protocol (HTTP protocol, RPC protocol, etc.), matching path name, and operation rules and current registration path information required for each method. These service configuration registration information is stored in the registration center server in the form of a permanent node.

[0048] When the service instance starts, it is stored in the registration center server in the form of a temporary node. When the downstream service is online or offline, the gateway system is notified in real time, and the gateway system dynamically updates the registration table information in the memory. The client module performs dynamic routing operation according to the latest service configuration registration information and the rules in the filter when making a request.

[0049] (3) Rule filter

[0050] According to the category of the filter, it is mainly divided into four categories: pre-filter, middle filter, post-filter and exception filter.

[0051] Among them, the pre-filter and post-filter can have multiple, and the execution order is prioritized according to the serial number, and is executed in series or in parallel according to the relevance. The main function of the middle filter is routing, but it can adapt to the protocol and forward according to the specific protocol of the lower layer. It can adapt to HTTP protocol, RPC protocol, etc. According to different protocols, the information to be filled in is unpacked and parsed and encapsulated, such as RPC protocol needs to fill in the name of the calling class and the parameters of the calling method. The exception filter is generally 1. If an error occurs in the execution of the filter chain, the exception filter will be triggered. The exception filter generally classifies the error information, and the regularized error information is written back to the client module.

[0052] According to different requests of the client module, the underlying performs different types of plug-in logic. First, the plug-ins are classified according to functions, such as authentication and authorization plug-ins for login verification requests; traffic control plug-ins for dynamically controlling request traffic; black and white list plug-ins for intercepting malicious requests; protocol analysis plug-ins for analyzing different protocol requests and refilling or organizing content; timeout plug-ins for internally defining timeout time, such as 3 seconds, if the actual request processing exceeds 3 seconds, the request processing can be directly interrupted, and a timeout exception is returned to the client; statistical analysis plug-ins, such as statistical request response time, daily successful access times, failed access times, and longest response request time. After the plug-ins are classified according to functions, the plug-ins of different functions are classified according to pre-filter, middle-filter, and post-filter. For example, authentication and authorization plug-ins, traffic control plug-ins, and black and white list plug-ins belong to pre-filter; load balancing belongs to middle-filter; and analysis and statistical plug-ins belong to post-filter.

[0053] Since the client module requests the backend service according to the url control access, the overall filtering rules are organized and split in path units, and the design principle is that one path binds one rule, one rule can be configured to multiple different paths, and one rule can contain multiple plug-ins; the execution order of the plug-ins is sorted by setting the priority value of the plug-in, and the larger the value, the later the execution.

[0054] The load balancing filter obtains the load balancing strategy configured for the path according to the configuration object in the request context, and performs load balancing operation from the instance information cached in the memory table. According to different requests, a weighted random strategy or a weighted round-robin strategy can be configured, and each server is configured with default weight information. The default weight of the embodiment is 100, and the weight has a preheating mechanism. In the embodiment, the server is preheated for 10 minutes, for example, the weight is increased by 10% every minute, and the weight becomes 100 in 10 minutes.

[0055] The weighted random strategy is as follows: assuming that there are three machines installed with downstream services, and the service weights of each machine are 100, 60, and 40. First, sum all the weights, 100+60+40=200, then a random number between 1 and 200 is generated, assuming that the random number is 150, then according to the random number, the weights of the three machines are subtracted in turn, if the result is less than or equal to 0, the machine is selected. The result is 50 after the random number 150 is subtracted from the weight 100 of the first machine, which is greater than 0, so the machine is not selected. The result is -10 after 50 is subtracted from 60, so the second machine is selected. According to the random number probability, only the random number between 1 and 100 will select the first machine, the random number between 101 and 160 will select the second machine, and the random number between 161 and 200 will randomly select the third machine. Therefore, although it is a random algorithm, the setting of the machine weight will still affect the selection of the machine service instance.

[0056] The weighted round robin strategy also makes full use of the weight. Each time the round robin is polled, the service instance with the highest weight is selected to provide services, and the services are provided in order from high to low according to the service weight. Due to the preheating mechanism, the service instance will automatically grow to the set weight value over time. Therefore, after each round of polling to provide services, the latest weight is used to reorder and provide external instance services.

[0057] When the post-filter is used for time-consuming statistics, the time of the request is first instrumented, and the time is divided into four parts: the first is the request time sent by the client module, the gateway service receives the request time of the client module, which is the request start time; the second is the time when the gateway service sends the request, which is the routing start time; the third is the time when the gateway service receives the request, which is the routing request receiving time; the fourth is the time when the gateway service writes back the request, which is the request end and write-back request time. The fourth time minus the first time is the entire request time-consuming, and the third time minus the second time is the entire routing call time. By comparing with the request timeout time (the request timeout time is set to 1s-2s in this embodiment, if the time threshold is exceeded, the first type of request timeout is determined to be a slow request call, and the second type is classified as a slow routing forwarding). After obtaining the time-consuming of each request, the number of requests that can be processed per second needs to be counted, and the processing idea is to count the total number of requests within a certain time and then divide by the total time. This will cause a lock competition problem in the calculation process, which will cause a loss of performance. The calculation process of this embodiment is to put it into a circular array for calculation, and use the array to divide slots, divide 1 second into 20 slots, each slot is 50 milliseconds, the first 50 milliseconds, write into the first slot for counting, the second 50 milliseconds, write into the second slot for counting, we can count the total value of the 20 slots within the current 1 second to process.

[0058] In order to prevent multi-threaded high concurrent read and write of data of a slot, a slot is segmented, each thread is bound to a fixed segment space for read and write processing operation. If the number of threads is large, the threads in each segment space use a lock-free design, and use comparison and re-setting to count, which can perform data statistics with higher performance. Since it is a circular array, after a certain time, the previous data will be covered and used in a loop. Since the array is pre-allocated, it will reduce a lot of memory fragmentation, and the overall performance will be better. The completed statistical data is placed in a buffer queue, and then an asynchronous thread takes out the data from the memory buffer queue, uses a non-falling disk read-write method, and directly forwards it to the message middleware. By avoiding disk IO, the statistical data can be processed more real-time and efficiently.

[0059] The execution flow of the whole filter is as follows, first, the client module sends a request, and through multiple pre-filters, the execution is performed in a serial manner according to the priority value. If the set priority is the same, and it is judged that the current filter has no correlation with other filters (the correlation is configured by the console module), it will be automatically converted into parallel execution. In this way, in the case of a time-consuming filter, the execution efficiency can be greatly improved. Then through the middle filter, which corresponds to the load balancing plug-in, the request is forwarded to the downstream in a double-asynchronous manner, and needs to be adapted according to the protocol of the downstream. After the downstream service returns the response, the response result is directly written back to the client module. After returning to the client module, the post-filter plug-in is executed, and the plug-in logic is executed in order according to the priority. In any link in the execution of the plug-in logic, if an exception occurs, it will be transferred to the exception filter for execution, and the error information will be written back to the client module. After writing back to the client module, the post-filter will continue to be executed and analyzed and counted.

[0060] (4) Cache

[0061] When the gateway service starts, it will set a large heap memory for the virtual machine, and put its basic information into the old generation, so that it is fixed in the old generation, reducing the time of young generation recycling.

[0062] When the gateway service starts, the configuration information of the gateway service is put into the cache area, and the filter rule information loaded by the gateway service is put into the cache area, and the registration table service information of the downstream providing service is put into the cache area. When the entire gateway processes the request, a large amount of cache is also used. Since the main work of the gateway is to forward the request or perform the filter rule filtering, most of the operations are not time-consuming operations, which can be determined as CPU-intensive operations, and the overall use of the disk is less. At this time, the size of the thread pool is set to N+1, N represents the number of cores of the current computer, in order to prevent context switching at the operating system level, the operating system core and a working thread are forcibly bound, so that the thread does not appear context switching, and the performance of the gateway service is improved.

[0063] In the process of processing the request of the gateway, the main thread is responsible for forwarding the request and putting the request into the cache area, and then the background thread uniformly processes it. By using the cache queue, large flow is first carried, and then the request is uniformly processed. The bottom layer uses the built-in buffer queue, and the internal data is identified and managed by the sequentially increasing serial number. When performing traditional data acquisition and calculation, the modulo operation is modified to bitwise AND operation, which can better improve the efficiency. The buffer line of a general operating system is 64 bytes, and a long integer variable is 8 bytes. If there are two threads A and thread B that need to access variables a and b, there will be a competition problem. Thread B needs to wait for thread A to complete the access operation before continuing to access. In this embodiment, 7 values are filled to the left of variable a, and 7 values are filled to the right of variable a. No matter where variable a is located at the beginning, in the middle, or at the end of the buffer line, it will be filled with values. In this way, variable a can occupy a cache line. In this way, different variables will occupy different cache lines, so that there is no competition problem when multiple threads access concurrently, and the concurrent performance can be better improved. Generally, the above buffer line will be pre-filled with data to save the process of creating a new object and reduce garbage collection. In the later stage, the delete operation can be directly updated without the delete operation. In this way, the cache queue can have better performance.

[0064] (5) Asynchronous

[0065] The asynchronous design is mainly divided into single asynchronous design and double asynchronous design. The single asynchronous design is mainly to optimize the synchronous serialization execution. Compared with the synchronous serialization execution, such as synchronous processing request, 5 requests need to be sent downstream. If the processing delay of each request is 1 second, the processing time of 5 requests is 5 seconds. And the single asynchronous evolution will make the 5 requests asynchronous, so the delay of the request to be processed is 1 second. The single asynchronous design is to take a thread from the thread pool to process the request when the request is received, and the main thread continues to go downstream, but the thread taken from the thread pool synchronously waits for the client response operation. The double asynchronous design is to further optimize the single asynchronous processing on the basis of the single asynchronous processing. The thread in the single asynchronous thread pool does not wait for the return of the downstream, but can process other tasks allocated by the main thread. When the downstream service returns the response, another idle thread is taken from the thread pool to process the request operation. The sending of the request and the processing of the response are asynchronous operations.

[0066] The single asynchronous request processing and the double asynchronous request processing are dynamically switched according to the length of the downstream processing request task. When the downstream processing task has a long time consumption (the access delay is greater than or equal to 100 milliseconds), or the performance of the downstream is not very good, the double asynchronous processing mode is enabled. However, if the performance of the downstream service is very good and the time consumption of the request processing is short, the single asynchronous mode is automatically switched. Because the double asynchronous processing will cause a large number of context switches, which indirectly reduces the processing performance. Therefore, the return time of the downstream processing request is collected in the gateway service at regular intervals. According to the processing request time, the single asynchronous mode or the double asynchronous mode is dynamically switched to better play the performance of the gateway.

[0067] (6) Downstream service upgrade

[0068] The downstream instance service of the gateway service needs to be upgraded in version as the customer puts forward new requirements, and the original program package needs to be replaced at this time, but since the entire downstream service is uninterrupted, it needs to continuously provide services to the client. The embodiment adopts a label method to perform rolling upgrade on the downstream service. First, the service to be upgraded in the downstream is marked with a label, and in the embodiment, no more than 50% of the servers are marked with a label to be upgraded, and in actual use, other number of servers can be selected for upgrade. After the client module sends a request to the gateway, the gateway will filter according to the label, select an instance from the service without a marked label according to the load balancing strategy, and send the request to the specified service instance for request processing. The service marked with the label to be upgraded needs to process the requests sent to it for processing in priority, and after processing these requests, the current service instance will be directly offline, and once the offline is successful, the operation and maintenance personnel can deploy the new version of the package and perform version upgrade operation. After the new version of the package is successfully deployed, the traffic to the newly deployed online package is gradually increased according to the weight value and preheating mechanism, and finally reaches the set weight value. After the service marked with the label is upgraded, another batch of servers can repeat the above operation until the entire cluster service is upgraded completely, achieving the effect of rolling upgrade.

[0069] By using the label in the gateway, part of the application service can be online with new functions, and part of the users can experience the new functions. For example, the module needing to upgrade the service is marked with a label, and after the upgrade service processes the previous requests, it can be offline, and then replaced with a deployment package with new functions and provided to the outside. The user policy and filtering policy are synchronized to the gateway service, and the gateway sends the requests of a small number of users to the deployment package with new functions according to the user filtering policy and the service label, so that part of the users can experience the new service function, and the function is optimized according to the feedback of the customers in the later period, or the number of label servers is dynamically adjusted to let more people experience the new function module and put forward optimization suggestions.

[0070] The client module specifically includes the following contents:

[0071] When the downstream service deployment is developed, a client module needs to be introduced. The client module abstracts the registration manager and dynamically determines whether the current downstream service is an HTTP protocol service or an RPC protocol service through code annotation when the downstream service is started. According to different protocol definitions, different registration methods are used to register the registration center server. In the registration process, the IP address and port number of the service instance are encapsulated into service instance definition information in the client module, and the method and path information of the external service are dynamically scanned and encapsulated into instance method attributes, which are saved in the instance registration information table. Finally, the service configuration information table is sent to the registration center server, and the registration center server saves the service configuration registration information. After that, the gateway service can obtain the corresponding service configuration registration information from the registration center server, and different load balancing strategies are used for different requests.

[0072] The registration center cluster specifically includes the following contents:

[0073] The registration center server mainly saves the configuration information of the gateway, the rule information of the path access, etc. The registration center client module built in the gateway is used to abstract the way of accessing the registration center server, so that different types of registration center servers can be connected and accessed. The self-developed registration center server can also be used, or the open source Zookeeper or Nacos registration center server can also be used. The registration center cluster mainly provides two core services to the outside, the listening service and the registration service. The listening service mainly synchronizes the gateway service dynamically when new information is stored in the registration center server or the configuration rule information needs to be deleted from the registration center server. The gateway service can modify the registration information in real time according to the notification of the registration center server.

[0074] The registration center server stores the core data of the gateway. In order to prevent single point failure of the registration center server, which causes the gateway service to be unable to normally pull the registration information and perform initialization operation or modify the routing strategy, the registration center cluster of the embodiment adopts a double center plus local cache scheme. When the gateway service starts, the registration center client module initiates a registration request to two registration center servers at the same time, and the pulled information is preferentially saved in the memory buffer, and then written to the local file by an asynchronous thread. Usually only one registration center is in active state, and the other registration center server is in non-active state. When one of the registration center servers loses heartbeat, it switches to the other registration center server to provide service. When both registration center servers are down, the gateway service preferentially loads the registration table from its own local memory. When the gateway service restarts, if the registration center server has not recovered, it will preferentially load the service rule configuration information from the local disk and provide services to the outside at the same time.

[0075] The console module specifically includes the following:

[0076] The core purpose of the console is to provide a visual operating interface for the external configuration of gateway services. Its internal services are mainly divided into definition services, instance configuration services, and rule configuration services.

[0077] The definition service mainly defines the unique number and version number of the service, and defines the basic protocol of the service and the path required to match the current service instance, and provides services to the outside with this path. The path is uploaded to the registry service side by the client module with the full path of the service call, and then the specific method and specific rule are bound together by the console module, and the information is synchronized to the registry service side.

[0078] The instance configuration service mainly configures the instance access address information, mainly including specific IP address, port information, routing label information, or weight configuration information of the current service instance. The load balancing strategy will distribute requests according to the weight configuration of the service instance.

[0079] The rule configuration service mainly configures specific rule information, which mainly configures filter rules by putting multiple filter plug-ins into a collection. One rule can correspond to multiple plug-ins, and the priority and relevance are configured in the rule information.

[0080] All configuration information will be saved to the registry service side. The gateway service will automatically pull the corresponding configuration information from the registry service side when starting, and put the configuration information into the memory of the gateway service, and use the configuration information to initialize its rules and filter interception plug-ins. When the request arrives later, it can directly access the interception filtering operation of the request, and forward according to the configured rules.

[0081] As shown in Figure 2 , the gateway service startup process includes the following steps:

[0082] 1.1) When the gateway service starts, load the configuration file required by the service. When loading the configuration file, load the parameters in the configuration file first, then load the parameters in the environment variable, then load the parameters in the virtual machine, and finally load the parameters in the running command line. The loaded parameters will automatically overwrite the previously loaded parameters, so that the running parameters have the highest priority, followed by virtual machine parameters, then environment variables, and finally configuration file parameters.

[0083] 1.2) Initialize the registry center client unit, first load the registry center address from the configuration file, try to connect the registry center server, then pull the service instance information from the registry center server, filter according to the configuration rule information, the registry center server increases the listening, when the registry center server listens to the new instance online, or the configuration file changes, the gateway service will be notified. The gateway service also registers itself in the registry center server, which is convenient for the client module to obtain the corresponding gateway address from the registry center server.

[0084] 1.3) Initialize the plug-in service, load the specified external collection log information plug-in through the service call interface, start the sending thread to connect the message middleware, and when the gateway external service generates statistical logs, directly forward the statistical information to the message middleware. Because the plug-in is used for integration, different types of message middleware can be connected to collect statistical information.

[0085] 1.4) Initialize the internal container, first initialize the network listening service, start the client module side listening, when the client module request is sent, the client module request can be processed. Initialize the internal work thread pool, which mainly consists of the following parts:

[0086] Forwarding thread, responsible for receiving the request sent by the client, and putting the request into the buffer queue,

[0087] Work thread, the background work thread takes out the request from the buffer queue and processes it evenly.

[0088] In the process of initializing the thread pool, the thread and the number of CPU cores are bound to reduce the context switching of the thread.

[0089] 1.5) Add exception listening to the gateway service, when the gateway service appears abnormal downtime, before the gateway service stops and destroys, it will automatically close the connection of the previous registry center server and the resources occupied in the thread pool, better releasing the resources.

[0090] As shown in Figure 3 , the request processing flow includes the following steps:

[0091] 2.1) After the gateway service receives the request sent by the client module, the request is converted into an internal request of the gateway service.

[0092] 2.2) The forwarding thread puts the request into the buffer queue, and the work thread takes out the gateway internal request from the buffer queue and processes it.

[0093] 2.3) The worker thread first hands the request to the pre-filter for processing. Before processing, it first determines whether the priority of the request filter is consistent and whether there is a correlation. If the priorities are inconsistent, it can only be executed serially. If the priorities are consistent, it determines whether there is a correlation. If there is no correlation, the pre-filter is executed in parallel. Otherwise, the pre-filter is executed serially. During the execution, it needs to be sorted according to priority.

[0094] 2.4) After the pre-filter completes its execution, the request will be handed over to the middle filter for processing. The middle filter parses the access address in the request and selects the specified load balancing strategy according to the configuration rules. When selecting the load balancing strategy, the weight factor and the preheating mechanism will be taken into account.

[0095] 2.5) After internal policy selection, the middle filter selects one server from multiple servers, performs protocol adaptation, and forwards the request to that server.

[0096] 2.6) During request sending, the default dual-asynchronous mode is used. A separate thread sends the request, and after sending, the thread does not wait synchronously but continues processing other requests. For responses returned from downstream requests, a new thread is selected for processing. During processing, the response time is monitored; if the response time is short, it will automatically switch to single-asynchronous mode to reduce context switching.

[0097] 2.7) After the intermediate filter completes the processing of the request, it directly forwards the request to the downstream server.

[0098] 2.8) After the downstream server finishes processing the request, it performs post-filter processing. The post-filter processes the request in order of priority, such as performing statistical operations like request time analysis.

[0099] 2.9) If an error occurs during the processing of any filter, the request will be forwarded to the exception filter for processing. The exception filter will classify the error information and return the error classification information to the client module.

[0100] 2.10) After the exception filter writes the request back to the client module, it forwards the request to the post-filter for processing.

[0101] like Figure 4 As shown, the client unit processing flow of the registration center includes the following steps:

[0102] 3.1) After each gateway service starts, it will first load the locally persisted registry file, initialize the registry file, and put it into memory.

[0103] 3.2) Gateway service registers to two registry service endpoints one by one using the registry client unit, then pulls the registry information and rule information from one of the registry service endpoints, and updates and covers the registry in the memory.

[0104] 3.3) The information between the two registries is synchronized in real time. When a new downstream service is online, the gateway service is notified in real time to update the registry.

[0105] 3.4) In the registry cluster, there is always an active registry service endpoint and an inactive registry service endpoint, and the two registry service endpoints are connected through heartbeat. When one registry service endpoint is down and loses heartbeat, the other registry service endpoint will take over and become an active registry service endpoint to continue to provide services.

[0106] As shown in Figure 5 , the downstream service upgrade process includes the following steps:

[0107] 4.1) The console module marks the service instance to be upgraded with a label to be upgraded. In order not to affect the entire downstream service to provide services to the outside, no more than 50% of the machines are marked with the label to be upgraded. The label information is saved to the registry service endpoint.

[0108] 4.2) The label information is synchronized by the registry service endpoint to the downstream service and the gateway service.

[0109] 4.3) When the gateway service receives a new client module request, it will not forward the user request to the downstream service marked with the label to be upgraded, but to the downstream service without the label to be upgraded.

[0110] 4.4) The downstream service marked with the label to be upgraded will be offline after processing the current request. At this time, a new software package can be enabled for deployment.

[0111] 4.5) After deployment, the gateway service reacquires the registration information from the registry service endpoint, and according to the preheating degree of the newly online service and the load balancing strategy of the current service method, the request is selectively forwarded to the newly deployed server for processing.

[0112] As shown in Figure 6 , the console module running process includes the following steps:

[0113] 5.1) The user defines the service through the console module, such as the protocol of the service and the method of providing services to the outside, and sets the default parameters of each filter rule. It can also mark specific labels for the service instance that has been online.

[0114] 5.2) Synchronize the configuration parameters set by the console module to the registry center cluster.

[0115] 5.3) When a new service is online and registered to the registry center service end, the console module realizes the new online service instance in real time through the registry center service end.

[0116] 5.4) The gateway service indirectly synchronizes the service information and the specific path service method which should use which filter for interception processing set by the console module through synchronizing the registration table information from the registry center service end.

[0117] 5.5) When an external request is sent to the gateway service, the gateway uses the initialized filter rule for processing.

[0118] Based on the same inventive concept, the gateway service method client module parses and registers the downstream service, and the service end module performs routing operation according to the rule filter when receiving the downstream service request sent by the client module, wherein the rule filter includes a pre-filter, a middle-filter, a post-filter and an exception filter;

[0119] The pre-filter includes authentication authorization, flow control and black and white list authentication,

[0120] The middle-filter is used for load balancing and protocol analysis, and selects single asynchronous request processing or double asynchronous request processing according to the length of the access delay time of the downstream processing request; the single asynchronous request processing is to take out a thread from a thread pool to process the request sent by the client, and synchronously waits for the response of the client module; the double asynchronous request processing is to take out a thread from a thread pool to process the request sent by the client, and process other requests when the response of the downstream service is not received;

[0121] The post-filter is used for index analysis after the downstream service returns the response result to the client, and the index includes response time and access times;

[0122] The exception filter is used for writing exception information back to the client module when an access exception occurs, and executing the post-filter for index analysis.

[0123] Based on the same inventive concept, the computer readable storage medium of the application stores a computer program, and the computer program is executed by a processor to realize the gateway service method.

[0124] The computer readable storage medium can include a RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer.

[0125] The processor is configured to execute a computer program stored in the memory to implement the various steps in the methods involved in the above embodiments.

Claims

1. A gateway service system, characterized in that, Includes server-side and client-side modules; When the server module receives a downstream service request sent by the client module, it performs routing operations according to the rule filter, which includes a pre-filter, a middle filter, a post-filter, and an exception filter. The pre-filter includes authentication and authorization, traffic control, and blacklist / whitelist authentication; The central filter is used for load balancing and protocol parsing, and selects between single asynchronous request processing and dual asynchronous request processing based on the access latency of downstream processing requests. Single asynchronous request processing involves selecting a thread from the thread pool to handle the client's request while synchronously waiting for the client module's response. Dual asynchronous request processing involves selecting a thread from the thread pool to handle the client's request and processing other requests if no response is received from the downstream service. The load balancing includes a weighted random strategy and a weighted round-robin strategy. The weighted random strategy sums the weights of all servers to obtain a weighted sum, then selects a random number between 1 and this weighted sum and subtracts it from the weight of each server sequentially, selecting the server whose subtraction result is not greater than 0. The weighted round-robin strategy prioritizes the server with the highest weight in each round-robin iteration. A preheating mechanism is used, where the server's weight increases from a preset initial value to a preset weight value over a period of time. The post-filter is used to perform metric analysis after the downstream service returns the response result to the client. The metrics include response time and number of accesses. When the post-filter is used for time consumption statistics, after obtaining the time consumption of each request, it is put into a circular array for calculation. The array is divided into slots, with 1 second divided into 20 segments, each segment being 50 milliseconds. Within the first 50 milliseconds, the data is written to the first slot for counting, within the second 50 milliseconds, the data is written to the second slot for counting, and so on. The processing is performed by calculating the sum of the counts of these 20 slots within the current 1 second. For each slot segment, each thread is bound to a fixed segment space for read and write operations. If there are many threads, a lock-free design is used for each thread in each segment space, and the counting is performed by comparison and resetting. The anomaly filter is used to write the anomaly information back to the client module when an anomaly occurs, and then execute the post-filter to perform indicator analysis. The client module is used to parse and register downstream services.

2. The gateway service system according to claim 1, characterized in that, The server module has a cache area for storing gateway service configuration information, filter rule information, and downstream service registry information; the main thread puts downstream service requests into the cache area, and the background thread executes the downstream service requests asynchronously.

3. The gateway service system according to claim 2, characterized in that, In the cache area, each variable is padded with several padding values ​​before and after it, and each variable occupies one cache line.

4. The gateway service system according to claim 1, characterized in that, When a downstream service is upgraded or a new feature is launched, the downstream service is marked with an upgrade tag. After the downstream service completes the received request, it will be taken offline and upgraded. The proportion of downstream services marked with the upgrade tag to all downstream services shall not exceed the first threshold.

5. The gateway service system according to claim 1, characterized in that, The client registers the downstream service with the registration center cluster. The client saves the IP address, port number, and service path of the downstream service in the service configuration information table and sends it to the registration center cluster for storage. The registration center cluster includes a primary registration center server and a backup registration center server. When the primary registration center server loses its heartbeat, it switches to the backup registration center server.

6. The gateway service system according to claim 5, characterized in that, The client includes a protocol adaptation unit and a registration center client unit. The protocol adaptation unit is used to parse the protocol of the downstream service, and the registration center client unit is used to register with the registration center cluster according to the protocol of the downstream service.

7. The gateway service system according to claim 1, characterized in that, It also includes a console module, which includes a definition service unit for determining the gateway service number, version number, protocol and service path, a server configuration unit for configuring access address, route label and server weight, and a rule configuration unit for configuring the rules of the pre-filter, middle filter, post-filter and exception filter.

8. A gateway service method, characterized in that, The client module parses and registers downstream services. When the server module receives a downstream service request sent by the client module, it performs routing operations according to the rule filters, which include pre-filters, mid-filters, post-filters, and exception filters. The pre-filter includes authentication and authorization, traffic control, and blacklist / whitelist authentication; The central filter is used for load balancing and protocol parsing, and selects between single asynchronous request processing and dual asynchronous request processing based on the access latency of downstream processing requests. Single asynchronous request processing involves selecting a thread from the thread pool to handle the client's request while synchronously waiting for the client module's response. Dual asynchronous request processing involves selecting a thread from the thread pool to handle the client's request and processing other requests if no response is received from the downstream service. The load balancing includes a weighted random strategy and a weighted round-robin strategy. The weighted random strategy sums the weights of all servers to obtain a weighted sum, then selects a random number between 1 and this weighted sum and subtracts it from the weight of each server sequentially, selecting the server whose subtraction result is not greater than 0. The weighted round-robin strategy prioritizes the server with the highest weight in each round-robin iteration. A preheating mechanism is used, where the server's weight increases from a preset initial value to a preset weight value over a period of time. The post-filter is used to perform metric analysis after the downstream service returns the response result to the client. The metrics include response time and number of accesses. When the post-filter is used for time consumption statistics, after obtaining the time consumption of each request, it is put into a circular array for calculation. The array is divided into slots, with 1 second divided into 20 segments, each segment being 50 milliseconds. Within the first 50 milliseconds, the data is written to the first slot for counting, within the second 50 milliseconds, the data is written to the second slot for counting, and so on. The processing is performed by calculating the sum of the counts of these 20 slots within the current 1 second. For each slot segment, each thread is bound to a fixed segment space for read and write operations. If there are many threads, a lock-free design is used for each thread in each segment space, and the counting is performed by comparison and resetting. The anomaly filter is used to write the anomaly information back to the client module when an anomaly occurs, and then execute the post-filter to perform indicator analysis.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the gateway service method according to claim 8.

Images