Dynamically adjusted timeout value for proxy server

Abstract

A method for dynamically adjusting proxy server timeout includes identifying each micro-service within a proxy server and mapping a topology of the micro-services, assigning each micro-service to one of an application layer, a middleware layer, and an infrastructure layer. Each representational state transition (REST) application programming interface (API) call relationship between each microservice and each other microservice is defined. A corresponding regression model defining a response time for each microservice is determined based at least in part on the set of available response time predictors. A sequence model is constructed for at least one microservice in the application layer. An incoming REST API call is predicted and a possible sequence model corresponding to the predicted incoming REST API call is identified. A timeout value of the predicted REST API call within the proxy server is updated based on the sequentially predicted response times.

Description

Background Technology

[0001] This invention generally relates to timeout values ​​for services within a proxy server, and more specifically, to dynamically adjusted timeout values ​​for a proxy server.

[0002] Interconnected computer systems and programs often use applications and software that can use proxy servers to invoke other applications or software within the interconnected system. In such an example, a call for relevant information is made to the interconnected system. To prevent indefinite waiting in the event of an error, a timeout value is applied to the request. When a request has been pending for longer than the timeout value, the request is canceled and an error can be reported. When a given call is particularly complex, a timeout value set to too short a duration may prevent the request from being completed before the timeout expires. Conversely, a timeout value set too long can cause a considerable delay in reporting an error when one does occur. Summary of the Invention

[0003] Embodiments of the present invention relate to a computer-implemented method for dynamically adjusting timeout values ​​in a proxy server. Non-limiting examples of the computer-implemented method include dynamically adjusting proxy server timeouts, comprising identifying each microservice within the proxy server and mapping the topology of the microservices within the proxy server, assigning each microservice to one of the application layer, middleware layer, and infrastructure layer of the proxy server. Defining each Representational State Transition (REST) ​​application programming interface (API) call relationship between each microservice and each other microservice within the proxy server. For each microservice in the proxy server, determining a corresponding regression model defining the response time of each microservice based at least in part on a set of available response time predictors. Constructing a sequence model for at least one microservice in the application layer. Predicting incoming REST API calls and identifying possible sequence models corresponding to the predicted incoming REST API calls. Sequentially predicting the response time corresponding to each microservice in the sequence using the corresponding regression model. Updating the timeout value of the predicted REST API calls within the proxy server based on the sequentially predicted response times. This method can be performed using a computerized system stored on a memory storage device or using a distributed computing system.

[0004] Additional technical features and benefits are achieved through the technology of this invention. Embodiments and aspects of the invention are described in detail herein and are considered part of the claimed subject matter. For a better understanding, refer to the specific embodiments and accompanying drawings. Attached Figure Description

[0005] The details of the proprietary rights described herein are specifically pointed out and explicitly claimed in the claims at the end of the specification. The foregoing and other features and advantages of the embodiments of the invention will be apparent from the following detailed description taken in conjunction with the accompanying drawings, wherein:

[0006] Figure 1 A cloud computing environment according to an embodiment of the present invention is described;

[0007] Figure 2 The microservice topology within a proxy server is described based on an example.

[0008] Figure 3 An example dependency table for a service is described;

[0009] Figure 4 A sample historical data table is depicted;

[0010] Figure 5 An example sequence model of application-layer microservices that can have multiple Representational State Transition (REST) ​​application programming interface (API) calls is depicted;

[0011] Figure 6 A dynamic timeout system implemented in a production environment, based on an example, is described; and

[0012] Figure 7 Example time predictions for application service sequences are depicted.

[0013] The figures depicted herein are illustrative. Many variations may be made to the figures or operations described herein without departing from the scope of the invention. For example, actions may be performed in a different order, or actions may be added, deleted, or modified. Furthermore, the term "coupling" and its variations describe a communication path between two elements and do not imply a direct connection between elements without an intermediate element / connection between them. All such variations are considered part of the specification.

[0014] In the accompanying drawings and the following detailed description of the disclosed embodiments, the various elements shown in the figures are labeled with two or three numbers. With a few exceptions, the leftmost number of each reference numeral corresponds to the figure in which its element is first shown. Detailed Implementation

[0015] Various embodiments of the invention are described herein with reference to the accompanying drawings. Alternative embodiments of the invention can be devised without departing from its scope. Various connections and positional relationships (e.g., above, below, adjacent, etc.) between elements are illustrated in the following description and drawings. Unless otherwise stated, these connections and / or positional relationships can be direct or indirect, and the invention is not intended to be limiting in this respect. Therefore, coupling of entities can refer to direct or indirect coupling, and positional relationships between entities can be direct or indirect positional relationships. Furthermore, the various tasks and process steps described herein can be incorporated into a more comprehensive program or process with additional steps or functions not described in detail herein.

[0016] The following definitions and abbreviations are used to interpret the claims and specification. As used herein, the terms “comprises,” “comprising,” “includes,” “including,” “has,” “having,” “contains,” or “containing,” or any other variation thereof, are intended to cover non-exclusive inclusion. For example, a composition, mixture, process, method, article, or apparatus that comprises a list of elements is not necessarily limited to those elements, but may include other elements not expressly listed or inherent to such compositions, mixtures, processes, methods, articles, or apparatus.

[0017] Additionally, the term "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any embodiment or design described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments or designs. The terms "at least one" and "one or more" can be understood to include any integer greater than or equal to one, i.e., one, two, three, four, etc. The term "multiple" can be understood to include any integer greater than or equal to two, i.e., two, three, four, five, etc. The term "connection" can include both indirect "connection" and direct "connection."

[0018] The terms “about,” “substantially,” “approximately,” and their variations are intended to include the degree of error associated with a measurement based on a specific quantity of equipment available at the time of application submission. For example, “about” could include a range of ±8%, 5%, or 2% of a given value.

[0019] For the sake of brevity, conventional techniques related to the manufacture and use of aspects of the present invention may or may not be described in detail herein. In particular, various aspects of the computing systems and specific computer programs used to implement the various technical features described herein are well known. Therefore, for the sake of brevity, many conventional implementation details are only briefly mentioned or omitted entirely herein, without providing well-known system and / or process details.

[0020] It should be understood that although this disclosure includes a detailed description of cloud computing, the implementation of the teachings described herein is not limited to a cloud computing environment. Rather, embodiments of the invention can be implemented in conjunction with any other type of computing environment now known or developed hereafter.

[0021] Computing environment 100 includes examples of environments for executing at least some computer code related to performing the methods of the present invention, such as dynamically updating the timeout of a proxy server at block 150. In addition to block 150, computing environment 100 includes, for example, a computer 101, a wide area network (WAN) 102, an end-user equipment (EUD) 103, a remote server 104, a public cloud 105, and a private cloud 106. In this embodiment, computer 101 includes a processor set 110 (including processing circuitry 120 and a cache 121), communication infrastructure 111, volatile memory 112, persistent storage device 113 (including an operating system 122 and block 150, as described above), a peripheral device set 114 (including a user interface (UI), a device set 123, a storage device 124, and an Internet of Things (IoT) sensor set 125), and a network module 115. Remote server 104 includes a remote database 132. The public cloud 105 includes a gateway 130, a cloud orchestration module 131, a host physical machine set 132, a virtual machine set 143, and a container set 144.

[0022] Computer 101 may take the form of a desktop computer, laptop computer, tablet computer, smartphone, smartwatch or other wearable computer, mainframe computer, quantum computer, or any other form of computer or mobile device now known or to be developed in the future capable of running programs, accessing networks, or querying databases (such as remote database 132). As is well understood in the field of computer technology, and depending on the technology, the execution of the computer-implemented method may be distributed among multiple computers and / or multiple locations. On the other hand, in this presentation of computing environment 100, the detailed discussion focuses on a single computer, specifically computer 101, to keep the presentation as simple as possible. Computer 101 may be located in the cloud, even if it is... Figure 1 The computer 101 is not shown in the cloud. On the other hand, except to any extent that can be definitively indicated, the computer 101 is not required to be in the cloud.

[0023] Processor set 110 includes one or more computer processors of any type now known or to be developed in the future. Processing circuitry 120 may be distributed across multiple packages, such as multiple coordinated integrated circuit chips. Processing circuitry 120 may implement multiple processor threads and / or multiple processor cores. Cache 121 is memory located within the processor chip package(s) and is typically used for data or code that should be readily accessible by the threads or cores running on processor set 110. Cache memory is typically organized into multiple levels based on its relative proximity to the processing circuitry. Alternatively, some or all of the cache for the processor set may be located “off-chip.” In some computing environments, processor set 110 may be designed to work with qubits and perform quantum computing.

[0024] Computer-readable program instructions are typically loaded onto computer 101 to cause the processor set 110 of computer 101 to perform a series of operational steps to implement a computer-implemented method, such that the instructions, when executed, instantiate the method specified in the flowcharts and / or descriptive descriptions of the computer-implemented method included in this document (collectively, the “method of the invention”). These computer-readable program instructions are stored in various types of computer-readable storage media, such as cache 121 and other storage media discussed below. The program instructions and associated data are accessed by the processor set 110 to control and direct the execution of the method of the invention. In computing environment 100, at least some of the instructions for performing the method of the invention may be stored in persistent storage device 113 in block 150.

[0025] Communication structure 111 is a signal transmission path that allows various components of computer 101 to communicate with each other. Typically, this structure consists of switches and conductive paths, such as switches and conductive paths forming buses, bridges, physical input / output ports, etc. Other types of signal communication paths can be used, such as fiber optic communication paths and / or wireless communication paths.

[0026] Volatile memory 112 is any type of volatile memory now known or to be developed in the future. Examples include dynamic random access memory (RAM) or static RAM. Typically, volatile memory is characterized by random access, but this is not necessary unless explicitly indicated. In computer 101, volatile memory 112 is located in a single package and is internal to computer 101; however, alternatively or additionally, volatile memory may be distributed across multiple packages and / or located externally relative to computer 101.

[0027] The persistent storage device 113 is any form of non-volatile storage device for a computer, now known or to be developed in the future. The non-volatility of this storage device means that the stored data is maintained regardless of whether power is supplied to the computer 101 and / or directly to the persistent storage device 113. The persistent storage device 113 may be a read-only memory (ROM), but typically, at least a portion of the persistent storage device allows data to be written, deleted, and rewritten. Some common forms of persistent storage devices include hard disks and solid-state storage devices. The operating system 122 may take several forms, such as various known proprietary operating systems or open-source portable operating system interface type operating systems employing a kernel. The code included in box 150 generally includes at least some of the computer code involved in performing the methods of the present invention.

[0028] Peripheral device set 114 includes a collection of peripheral devices for computer 101. Data communication connections between peripheral devices and other components of computer 101 can be implemented in various ways, such as Bluetooth connectivity, near field communication (NFC) connectivity, connections via cables (such as Universal Serial Bus (USB) type cables), plug-in connections (e.g., Secure Digital (SD) cards), connections via local area communication networks, and even connections via wide area networks such as the Internet. In various embodiments, UI device set 123 may include components such as displays, speakers, microphones, wearable devices (e.g., goggles and smartwatches), keyboards, mice, printers, touchpads, game controllers, and haptic devices. Storage device 124 is an external storage device (such as an external hard drive) or a pluggable storage device (such as an SD card). Storage device 124 can be persistent and / or volatile. In some embodiments, storage device 124 may take the form of a quantum computing storage device for storing data in the form of qubits. In embodiments requiring computer 101 to have substantial storage (e.g., where computer 101 locally stores and manages a large database), this storage can be provided by peripheral storage devices designed to store very large amounts of data, such as a storage area network (SAN) shared by multiple geographically distributed computers. The IoT sensor set 125 consists of sensors that can be used in IoT applications. For example, one sensor could be a thermometer, while another could be a motion detector.

[0029] Network module 115 is a collection of computer software, hardware, and firmware that allows computer 101 to communicate with other computers via WAN 102. Network module 115 may include hardware such as a modem or Wi-Fi transceiver, software for packetizing and / or unpacking data for transmission over a communication network, and / or web browser software for transmitting data over the Internet. In some embodiments, the network control and network forwarding functions of network module 115 are performed on the same physical hardware device. In other embodiments (e.g., embodiments utilizing software-defined networking (SDN), the control and forwarding functions of network module 115 are performed on physically separate devices, such that the control function manages several different network hardware devices. Computer-readable program instructions for performing the methods of the present invention can typically be downloaded to computer 101 from an external computer or external storage device via a network adapter card or network interface included in network module 115.

[0030] WAN 102 is any wide area network (e.g., the Internet) capable of transmitting computer data over non-local distances using any technology now known or to be developed in the future for transmitting computer data. In some embodiments, a WAN may be replaced by and / or supplemented by a local area network (LAN) designed to transmit data between devices located in a local area, such as a Wi-Fi network. WANs and / or LANs typically include computer hardware such as copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and edge servers.

[0031] End User Equipment (EUD) 103 is any computer system used and controlled by an end user (e.g., a customer of the enterprise operating computer 101) and can take any of the forms discussed above in connection with computer 101. EUD 103 typically receives helpful and useful data from the operation of computer 101. For example, assuming computer 101 is designed to provide recommendations to the end user, these recommendations are typically transmitted from network module 115 of computer 101 to EUD 103 via WAN 102. In this way, EUD 103 can display or otherwise present the recommendations to the end user. In some embodiments, EUD 103 can be a client device, such as a thin client, a heavy client, a mainframe computer, a desktop computer, etc.

[0032] Remote server 104 is any computer system that provides at least some data and / or functionality to computer 101. Remote server 104 can be controlled and used by the same entity operating computer 101. Remote server 104 represents (multiple) machines that collect and store helpful and useful data for use by other computers, such as computer 101. For example, if computer 101 is designed and programmed to provide recommendations based on historical data, that historical data can be provided to computer 101 from a remote database 132 of remote server 104.

[0033] Public cloud 105 is any computer system available to multiple entities, providing on-demand availability of computer system resources and / or other computing capabilities, particularly data storage (cloud storage) and computing power, without the need for direct, active management by users. Cloud computing typically leverages resource sharing to achieve consistency and economies of scale. Direct and active management of the computing resources of public cloud 105 is performed by the computer hardware and / or software of cloud orchestration module 131. The computing resources provided by public cloud 105 are typically implemented by virtual computing environments running on various computers constituting host physical set 132, which is a universe of physical computers in and / or available to public cloud 105. Virtual computing environments typically take the form of virtual machines from virtual machine set 143 and / or containers from container set 144. It should be understood that these VCEs can be stored as images and can be transferred between various physical machine hosts as images or after the VCEs are instantiated. Cloud orchestration module 131 manages the transfer and storage of images, deploys new instantiations of VCEs, and manages the active instantiation of VCE deployments. Gateway 130 is a collection of computer software, hardware, and firmware that allow public cloud 105 to communicate via WAN 102.

[0034] Now, we will provide some further explanation of Virtualized Computing Environments (VCEs). A VCE can be stored as an "image." A new active instance of a VCE can be instantiated from an image. Two familiar types of VCEs are virtual machines and containers. A container is a VCE that uses operating system-level virtualization. This refers to an operating system feature where the kernel allows multiple isolated user-space instances, called containers, to exist. From the perspective of the programs running within them, these isolated user-space instances typically appear as real computers. Computer programs running on a regular operating system can utilize all the resources of that computer, such as connected devices, files and folders, network shares, CPU power, and quantifiable hardware capabilities. However, programs running within a container can only use the contents of the container and the devices allocated to the container; this is a feature known as containerization.

[0035] Private cloud 106 is similar to public cloud 105, except that computing resources are only available for use by a single enterprise. While private cloud 106 is depicted communicating with WAN 102, in other embodiments, private cloud may be completely disconnected from the internet and accessible only via a local / private network. A hybrid cloud is a combination of multiple clouds of different types (e.g., private, community, or public cloud types), typically implemented by different vendors. Each of the multiple clouds remains a separate and discrete entity, but the larger hybrid cloud architecture is bound together by standardization or proprietary technology that enables orchestration, management, and / or data / application portability across the multiple component clouds. In this embodiment, public cloud 105 and private cloud 106 are both part of a larger hybrid cloud.

[0036] For example, one or more embodiments described herein can utilize machine learning techniques to perform prediction and / or classification tasks. In one or more embodiments, machine learning functions can be implemented using artificial neural networks (ANNs) that have the ability to be trained to perform functions. In machine learning and cognitive science, ANNs are a family of statistical learning models inspired by the biological neural networks of animals, particularly the brain. ANNs can be used to estimate or approximate systems and functions that depend on a large number of inputs. Convolutional neural networks (CNNs) are a class of deep feedforward ANNs that are particularly useful in tasks such as, but not limited to, analyzing visual images and natural language processing (NLP). Recurrent neural networks (RNNs) are another class of deep feedforward ANNs and are particularly useful in tasks such as, but not limited to, handwriting recognition and speech recognition without segmented connections. Other types of neural networks are also known and can be used according to one or more embodiments described herein.

[0037] An ANN can be embodied as a "neuromorphic" system of interconnected processor elements that act as analog "neurons" and exchange "messages" with each other in the form of electronic signals. Similar to the so-called "plasticity" of synaptic neurotransmitter connections that carry messages between biological neurons, the connections in an ANN that carry electronic messages between analog neurons are provided with numerical weights corresponding to the strength of a given connection. These weights can be adjusted and tuned empirically, allowing the ANN to adapt to the input and learn. For example, an ANN for handwriting recognition is defined by a set of input neurons that can be activated by pixels of an input image. After being weighted and transformed by a function determined by the network designer, the activations of these input neurons are then passed to other downstream neurons, often referred to as "hidden" neurons. This process is repeated until an output neuron is activated. The activated output neuron determines which character was input.

[0038] Containers are Virtual CEs (VCEs) that utilize operating system-level virtualization. This refers to an operating system feature where the kernel allows multiple isolated user-space instances, called containers, to exist. From the perspective of a program running within one of these instances, they typically behave like a real computer. A computer program running on a regular operating system can utilize all of that computer's resources, such as connected devices, files and folders, network shares, CPU power, and quantifiable hardware capabilities. However, a program running within a container can only use the contents of the container and the devices allocated to it; this is a characteristic known as containerization.

[0039] Turning now to an overview of techniques more specifically related to aspects of the invention, a proxy server is an intermediary server or middleman server that forwards content requests from multiple clients to different servers over the Internet. A reverse proxy server is a type of proxy server that typically resides behind a firewall in a private network and directs client requests to the appropriate backend server. Reverse proxies provide an additional level of abstraction and control to ensure a smooth flow of network traffic between clients and servers. Common uses of reverse proxy servers include load balancing, web acceleration, and security and anonymity.

[0040] Typically, users need to set request timeouts for each microservice in the network to avoid long wait times in case of any issues with the server. For example, NGINX (a popular proxy server) defaults to a request timeout of 60 seconds. The request timeout defines the duration for which a microservice will wait for a response from the proxy server. After the timeout period, the microservice provides a "504: Gateway Timeout" error and responds as defined within that service. Specific default responses depend on the service, and the type of default response is general. NGINX is used as an example proxy server in this document, and it should be understood that the concepts described herein are general to all proxy servers and are not limited to NGINX-based proxy servers.

[0041] In some instances, a request or request type may inherently exceed the default or set request timeout, regardless of how quickly the proxy server can respond. These requests are called "long-running requests." In such examples, users may need to increase the request timeout in NGINX to service long-running requests. If the user does not increase the request timeout value, NGINX will give a "504: Gateway Timeout" error.

[0042] Typically, timeout values ​​are fixed values ​​in the proxy server's configuration file. Once a timeout value is set for a service, it remains constant for all API requests to that service. Setting a timeout value is inflexible because real-world network implementations involve numerous and complex user scenarios, and each implementation is affected by dependencies on infrastructure layer services, hardware resources, request parameters, and so on.

[0043] If users set a small value for the timeout, they may frequently encounter 504 errors. If users set a very large value for the timeout, they may wait for a long time for a REST API response when an error occurs in the service. Setting an appropriate timeout value in the configuration always depends on the engineer's experience.

[0044] To overcome current shortcomings, the system disclosed in this paper uses an intelligent approach to dynamically adjust the timeout value of the reverse proxy, which can be used in complex products constructed from large-scale microservices.

[0045] Turning now to an overview of various aspects of the invention, one or more embodiments of the invention address the aforementioned shortcomings of the prior art by providing an intelligent method for dynamically adjusting the timeout value of a reverse proxy, which can be used in complex products constructed from large-scale microservices. Typically, the system builds a dependency topology for all microservices based on the REST API call relationships between them. Simultaneously, the system identifies application layer services, middleware layer services, and infrastructure layer services, and builds a regression model for the response time corresponding to each REST API in the microservices.

[0046] Turning now to a more detailed description of various aspects of the invention, in some examples, a sequence model is built in each application-layer service based on historical data regarding REST API call sequences. This process predicts the probability of upcoming REST API calls in each application microservice of the production environment based on the latest REST API calls and searches for downstream services in the topology based on the predicted REST API calls. The process further uses a regression model to predict response times to update the timeout value for each microservice in the topology layer by layer, from the bottom-level services to the top-level services. This process is controlled by an on-demand timer. Therefore, this process provides automatic and dynamic determination of the timeout value for each service and updates the proxy server with appropriate values ​​based on the determined timeout values.

[0047] Advantageously, this process allows for the construction of dependency topologies for all microservices based on REST API call relationships between microservices. It also identifies application-layer services, middleware-layer services, and infrastructure-layer services.

[0048] Continue to refer to Figure 1 The computing system Figure 2Figure 8 illustrates the process generally described above according to a particular implementation. Initially, the dependency topology 200 (in...) Figure 2 (As shown in the diagram). Within the dependency topology, each microservice (which may alternatively be referred to as a microservice) 210, 220, 230 is categorized into one of the following: an application service layer 202 that interfaces with a graphical user interface (GUI) 240; a middleware layer 204 that interfaces with one or more services in the application service layer 202; and an infrastructure layer 206 that interfaces with one or more services in the middleware layer 204. Figure 2 As shown in the example, each microservice 210, 220, 230 connects to one or more other services 210, 220, 230 or GUI 240 interfaces via REST API call 250.

[0049] Once the dependency topology is determined, the dependencies between services 210, 220, and 230 on a given REST API are established. Each microservice 210, 220, and 230 provides multiple REST APIs 250. When a REST API 250 is executed within a microservice 210, 220, or 230, that microservice 210, 220, or 230 calls one or more REST APIs 250 from other services 250. Dependencies are used to create a dependency table 300 for each microservice 210, 220, and 230. Figure 2 An exemplary dependency table for application services 210, 220, and 230 is shown. It should be understood that an actual implementation will include substantially more services 210, 220, and 230 in each layer 202, 204, and 206. As an example, an actual implementation may include at least services 210, 220, and 230 in each layer.

[0050] Dependency table 300 identifies the REST API call relationships with other (downstream) services 210, 220, and 230. For infrastructure layer service 230, the value of API calls from downstream services should be inapplicable (N / A) 310, as it should not call any REST API from other services 210, 220, and 230.

[0051] For each REST API in microservices 210, 220, and 230, information related to real-time resources 410 from historical deployments and executions (referred to as historical data in this document) is collected, such as available memory, disk usage, CPU usage, network throughput, etc. Similarly, when microservices 210, 220, and 230 are deployed on physical machines or virtual machines, parameter information 420 about the REST API is collected, such as query parameters, headers, bodies, etc. Historical time consumed by downstream REST API calls 430 is collected, and the response time of those REST API calls is collected 440. Historical data is collected in a historical data table 400, an example of which is shown below. Figure 4 As shown in the image.

[0052] exist Figure 4 The historical data collected in Table 400 is used to build a regression model for the REST API data, using response time as the target. Predictors for the regression model include real-time resources 410, parameter information about the REST API 420, and the time consumption of downstream REST API calls 430. REST API calls in the downstream column of Infrastructure Layer 206 Service 230 have no time consumption because it is assumed that Infrastructure Layer 206 Service 230 should not make any further REST API calls from other services.

[0053] After building the regression model, a sequence model was built for each application layer microservice 210, 220, and 230 based on historical data about REST API call sequences. Figure 5 An example sequence model 550 is shown for an application layer 202 microservice 510 that can have multiple REST API calls 250. The sequence model 550 includes different rows for each possible permutation of the REST API calls 250, and a confidence value for the permutation occurring.

[0054] When the system is implemented in a production environment, such as Figure 6 As shown, each time the application microservice 510 is used, the system predicts the probability of an upcoming REST API call based on a regression model 550. The predicted REST API call is then used to search for downstream services within the topology and determine possible REST API call stacks 560. As used herein, a REST API call stack refers to the sequence of REST API calls required to complete the utilization of the application microservice 510.

[0055] Once the possible REST API call stack 560 is identified, the identified regression model is used to determine the response time of each microservice 510, 522, 532 in stack 560. Figure 7An example operation is illustrated, where the response time is initially determined using microservice 532 in the bottom layer (infrastructure service 206), then microservice 522 in the middle layer (middleware service 204), and finally microservice 510 in the top layer (application service 202). At each layer 202, 204, 206, corresponding regression models 534, 524, 514 determine predicted times 536, 526, 516. Each layer 202, 204, 206 provides the corresponding predicted times 516, 526, 536 to the layer above, and uses the predicted times 536, 526 from the lower layers in regression models 524, 514. As an example, middleware service layer 204 receives the predicted time 536 from infrastructure service layer 206 and uses the predicted time 536 in its regression model 524 to determine the middleware service layer's predicted time 526. Sequential time prediction results in a total time prediction output (predicted time 516) from the application service layer 202, which takes into account the possible times of each microservice 510, 522, 532 in the prediction sequence.

[0056] In some examples, the sequence of REST API calls is repeated for each possible application. Figure 7 The process outlined in [the document] is as follows. In other examples, the process is repeated only for sequences of REST API calls that have a probability of occurrence higher than a specified likelihood. In other examples, the process runs only for the most likely or N most likely sequences.

[0057] Finally, the timeout value for each microservice is updated using a value based on the predicted time, rather than on arbitrary settings of the proxy server or the default timeout setting. In some examples, this process is controlled on demand by a timer and executed automatically for all application-layer services. When used, each service has a defined list of predicted times to different REST APIs, and the maximum value across all REST APIs is set as the service's timeout value.

[0058] The system described above provides an intelligent approach that dynamically adjusts the timeout value of a reverse proxy in a complex product built from large-scale microservices by applying artificial intelligence (AI) methods based on learned previous timeout adjustments.

[0059] The above aspects of the invention overcome the shortcomings of the prior art by advantageously allowing dynamic adjustment of the timeout value for the reverse proxy server, wherein this adjustment takes historical time into account to create a predicted timeout value. The predicted timeout value provides an intelligent way to update the timeout values ​​of all services with different values.

[0060] This invention can be a system, method, and / or computer program product at any possible level of technical detail integration. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of the invention.

[0061] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0062] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, optical fiber transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0063] The computer program instructions used to perform the operations of this invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, integrated circuit configuration data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer, partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet through an Internet microservice provider). In some embodiments, electronic circuits, such as programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), are personalized by utilizing the status information of the computer-readable program instructions. These electronic circuits can execute the computer-readable program instructions to implement various aspects of the invention.

[0064] Various aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0065] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create components that implement the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions includes an article of manufacture comprising instructions that implement aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0066] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0067] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions mentioned in the blocks may not occur in the order shown in the figures. For example, two blocks shown consecutively may actually be executed substantially simultaneously, or these blocks may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0068] Various embodiments of the invention have been described for illustrative purposes, but are not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope of the invention described. The terminology used herein has been chosen to best explain the principles of the embodiments, their practical application, or technical improvements to techniques found in the market, or to enable those skilled in the art to understand the embodiments described herein.

[0069] In a preferred embodiment of the invention described herein, a method is provided comprising: identifying each microservice within a proxy server and mapping the topology of the microservices within the proxy server; assigning each microservice to one of the application layer, middleware layer, and infrastructure layer of the proxy server; defining each Representational State Transition (REST) ​​application programming interface (API) call relationship between each microservice within the proxy server and each other microservice; for each microservice in the proxy server, determining a corresponding regression model defining the response time of each microservice based at least in part on a set of available response time predictors; constructing a sequence model for at least one microservice in the application layer; predicting incoming REST API calls and identifying possible sequence models corresponding to the predicted incoming REST API calls; sequentially predicting the response time corresponding to each microservice in the sequence using the corresponding regression model; and updating the timeout value of the predicted REST API calls within the proxy server based on the sequentially predicted response times. Preferably, the set of available response time predictors includes real-time resources, REST API parameter information, and the downstream duration of the REST API calls. Preferably, constructing a sequence model for at least one application layer microservice includes constructing a different sequence model for each application layer microservice. Preferably, each sequence model is constructed based on historical data. Preferably, the historical data includes the proxy server's available memory, disk usage, CPU usage, and network throughput. Preferably, sequentially predicting the response time corresponding to each microservice in the sequence includes: first predicting the response time of the last microservice in the sequence, then providing the predicted response time of the last microservice to the regression model of the microservice immediately preceding it in the sequence, and iterating the response time prediction from the last microservice to the first microservice in the sequence. Preferably, the first microservice in the sequence corresponds to the application service layer.

Claims

1. A computer-implemented method, comprising: Each microservice within the proxy server is identified using a computer processor, and the topology of the microservices within the proxy server is mapped. The topology includes each microservice within the proxy server and each Representational State Transition (REST) ​​Application Programming Interface (API) call relationship between each microservice and each other microservice. The topology identifies one of the application layer, middleware layer, and infrastructure layer corresponding to each microservice. The computer processor is used to determine a regression model corresponding to each microservice, the regression model defining the response time of each microservice based at least in part on a set of available response time predictors; Build a sequence model for at least one microservice corresponding to the application layer; The computer processor is used to predict incoming REST API calls and identify possible sequence patterns corresponding to the predicted incoming REST API calls. The corresponding regression model is used to sequentially predict the response time for each microservice in the sequence; as well as Using the computer processor, the predicted timeout values ​​for REST API calls within the proxy server are updated based on sequentially predicted response times.

2. The computer-implemented method according to claim 1, wherein, The set of available response time predictors includes real-time resources, REST API parameter information, and the downstream duration of the REST API call.

3. The computer-implemented method according to claim 1, wherein, Building a sequence model for at least one microservice corresponding to the application layer includes building a sequence model for each microservice corresponding to the application layer.

4. The computer-implemented method according to claim 3, wherein, Each sequence model is constructed by the processor based on historical data.

5. The computer-implemented method according to claim 4, wherein, The historical data includes the proxy server's available storage, disk usage, CPU usage, and network throughput.

6. The computer-implemented method according to claim 1, wherein, Sequentially predicting the response time corresponding to each microservice in the sequence includes: initially predicting the response time of the last microservice in the sequence, providing the predicted response time of the last microservice in the sequence to the regression model of the microservice immediately preceding it in the sequence, and iterating the response time prediction from the last microservice in the sequence to the first microservice in the sequence.

7. The computer-implemented method according to claim 6, wherein, The first microservice in the sequence corresponds to the application service layer.

8. A system comprising: Multiple interconnected computing systems, the multiple interconnected computing systems including at least one proxy server containing multiple microservices; Each microservice within the proxy server is assigned to one of the application service layer, middleware service layer, and infrastructure layer. A computer processor configured to dynamically update the timeout value corresponding to each application-layer microservice of the proxy server by: Map the topology of the microservices within the proxy server; Define the Representational State Transfer (REST) ​​Application Programming Interface (API) call relationships between each microservice and each other microservice within the proxy server; For each microservice in the proxy server, the corresponding regression model defining the response time is determined at least in part based on the set of available response time predictors; Build a sequence model for at least one microservice in the application layer; Predict incoming REST API calls and identify possible sequence models corresponding to the execution of the predicted incoming REST API calls; The corresponding regression model is used to sequentially predict the response time for each microservice in the sequence; as well as The timeout values ​​for predicted REST API calls within the proxy server are updated based on the sequentially predicted response times.

9. The system according to claim 8, wherein, The set of available response time predictors includes real-time resources, REST API parameter information, and the downstream duration of the REST API call.

10. The system according to claim 9, wherein, Building a sequence model for at least one application-layer microservice involves building a different sequence model for each application-layer microservice.

11. The system according to claim 10, wherein, Each sequence model is built based on historical data.

12. The system according to claim 11, wherein, The historical data includes the proxy server's available storage, disk usage, CPU usage, and network throughput.

13. A computer program product, comprising: A non-transitory computer-readable medium storing instructions for causing a processor to implement a dynamic microservice timeout update method within a proxy server, the method comprising: Identify each microservice within the proxy server and map the topology of the microservices within the proxy server, assigning each microservice to one of the application layer, middleware layer, and infrastructure layer of the proxy server; Define the Representational State Transfer (REST) ​​Application Programming Interface (API) call relationships between each microservice and each other microservice within the proxy server; For each microservice in the proxy server, the corresponding regression model defining the response time of each microservice is determined at least in part based on the set of available response time predictors; Build a sequence model for at least one microservice in the application layer; Predict incoming REST API calls and identify possible sequence patterns corresponding to the predicted incoming REST API calls; The corresponding regression model is used to sequentially predict the response time for each microservice in the sequence; and The timeout values ​​for predicted REST API calls within the proxy server are updated based on the sequentially predicted response times.

Images