Techniques for configurable caching, deduplication, and rate limit processing

Abstract

There is provided a method for configurable caching, deduplication and rate limit processing of user interface data requests according to an embodiment, the method comprising receiving a data request initiated by a user interface element of a user interface; determining whether a deduplication cache includes a deduplication entry associated with the data request; in response to determining that the deduplication cache does not include the deduplication entry, determining whether the primary cache includes a cache entry associated with the data request; in response to determining that the primary cache does not include the cache entry or determining that the cache entry has expired, determining whether the data request is associated with a rate-limited set of application programming interface endpoints; and in response to determining that the data request is associated with the rate limited group, making the data request to a backend system after a predefined server retry cycle associated with the rate limited group has elapsed.

Description

[0001] Claiming priority

[0002] This application claims the benefit of U.S. patent application 18 / 345,243, filed on June 30, 2023, entitled “TECHNOLOGIES FOR CONFIGURABLE CACHING, DEDUPLICATION, AND RATE LIMIT HANDLING”. Background Art

[0003] Web development has traditionally followed the formula of building single-page applications (SPAs) or multi-page applications, where each page implements its own processing of application programming interface (API) requests, caches, settings, permissions, events, and records. Page-level data storage is a common solution, but it is linked to a specific framework and requires revisions when implementing solutions across multiple frameworks. This approach results in code duplication across multiple small repositories, or placing all dependent processing code in the same repository. Duplication of code leads to duplicate work during debugging, dependency upgrades, or regular maintenance, as well as inconsistencies across pages. On the other hand, making all code dependent on a service layer within one repository causes the repository to become large, complex, and difficult to manage, slowing down the maintenance of existing features and the development of new features. When moving between pages and user interface frameworks, components must also be rebuilt (e.g., due to differences in the specific implementation of page-level or framework-level logic). Summary of the invention

[0004] One embodiment is directed to unique systems, components, and methods for configurable caching, deduplication, and rate-limiting processing of user interface data requests. Other embodiments are directed to apparatus, systems, devices, hardware, methods, and combinations thereof, for configurable caching, deduplication, and rate-limiting processing of user interface data requests.

[0005] According to an embodiment, a method for configurable caching, deduplication, and rate limit processing for user interface data requests is provided, which may include receiving a data request initiated by a user interface element of a user interface; determining whether a deduplication cache includes a deduplication entry associated with the data request; in response to determining that the deduplication cache does not include the deduplication entry, determining whether a primary cache different from the deduplication cache includes a cache entry associated with the data request; in response to one of (i) determining that the primary cache does not include the cache entry associated with the data request or (ii) the cache entry associated with the data request has expired, determining whether the data request is associated with a rate-limited application programming interface endpoint group; and in response to determining that the data request is associated with the rate-limited group, making the data request to a back-end system after a predefined server retry period associated with the rate-limited group has passed.

[0006] In some embodiments, the deduplication entry may indicate that a previous data request identical to the data request initiated by the user interface element of the user interface was made within a threshold time period.

[0007] In some embodiments, the method may also include, in response to determining that the primary cache includes the cache entry associated with the data request, determining whether the data request is associated with a predefined maximum cache age.

[0008] In some embodiments, the method may also include, in response to determining that the data request is associated with the predefined maximum cache age, determining whether the cache entry associated with the data request has expired based on the creation time of the cache entry and the predefined maximum cache age.

[0009] In some embodiments, the method may also include, in response to determining that the data request is not associated with the predefined maximum cache age, determining whether the cache entry associated with the data request has expired based on an expiration time of the cache entry defined by the cache entry.

[0010] In some embodiments, the method may further include, in response to determining that the data request is not associated with any rate limited group, immediately making the data request to the backend system.

[0011] In some embodiments, the method may also include, in response to failing to satisfy the data request, receiving an error associated with the data request; determining whether the error associated with the data request is a retryable error; in response to determining that the error associated with the data request is the retryable error, determining whether a predefined retry limit has been reached; and in response to determining that the predefined retry limit has not been reached, retrying the data request.

[0012] In some embodiments, retrying the data request may include, in response to determining that the retryable error is a 429 error, retrying the data request after the predefined server retry period.

[0013] In some embodiments, the predefined retry limit may be the maximum number of retry attempts that may be made.

[0014] In some embodiments, determining whether the data request is associated with the rate-limited API endpoint group may include determining whether the API endpoint to which the data request is directed is included in the rate-limited API endpoint group.

[0015] In some embodiments, the deduplication cache may be configured to store one or more commitments, wherein each of the one or more commitments is associated with a previous corresponding data request.

[0016] In some embodiments, the deduplication cache may be configured to store data returned in response to a previous request for data with a configurable timeout.

[0017] In some embodiments, the primary cache may be configured to store data returned in response to a previous request for data.

[0018] According to another embodiment, a system for configurable caching, deduplication, and rate limit processing for user interface data requests is provided, the system may include a deduplication cache; a primary cache different from the deduplication cache; at least one cache; and at least one memory, the at least one memory including a plurality of instructions stored thereon, the plurality of instructions being responsive to being executed by the at least one processor such that the system receives a data request initiated by a user interface element of a user interface; determines whether the deduplication cache includes a deduplication entry associated with the data request; in response to determining that the deduplication cache does not include the deduplication entry, determines whether the primary cache includes a cache entry associated with the data request; in response to one of (i) determining that the primary cache does not include the cache entry associated with the data request or (ii) the cache entry associated with the data request has expired, determines whether the data request is associated with a rate-limited application programming interface endpoint group; and in response to determining that the data request is associated with the rate-limited group, makes the data request to a backend system after a predefined server retry period associated with the rate-limited group has passed.

[0019] In some embodiments, the deduplication entry may indicate that a previous data request identical to the data request initiated by the user interface element of the user interface was made within a threshold time period.

[0020] In some embodiments, the multiple instructions may also cause the system to: in response to determining that the main cache includes the cache entry associated with the data request, determine whether the data request is associated with a predefined maximum cache age; in response to determining that the data request is associated with the predefined maximum cache age, determine whether the cache entry associated with the data request has expired based on a creation time of the cache entry and the predefined maximum cache age; and in response to determining that the data request is not associated with the predefined maximum cache age, determine whether the cache entry associated with the data request has expired based on an expiration time of the cache entry defined by the cache entry.

[0021] In some embodiments, the multiple instructions may also cause the system to receive an error associated with the data request in response to failure to satisfy the data request; determine whether the error associated with the data request is a retryable error; in response to determining that the error associated with the data request is the retryable error, determine whether a predefined retry limit has been reached; and in response to determining that the predefined retry limit has not been reached, retry the data request.

[0022] In some embodiments, retrying the data request may include, in response to determining that the retryable error is a 429 error, retrying the data request after the predefined server retry period.

[0023] In some embodiments, the predefined retry limit may be the maximum number of retry attempts that may be made.

[0024] In some embodiments, the deduplication cache may be configured to store one or more commitments, wherein each of the one or more commitments is associated with a previous corresponding data request, and the primary cache may be configured to store data returned in response to the previous data request.

[0025] This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to be used as an aid in limiting the scope of the claimed subject matter.Other embodiments, forms, features, and aspects of the present application will become apparent from the description and drawings provided herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The concepts described herein are illustrated in the accompanying drawings by way of example and not by way of limitation. For simplicity and clarity of illustration, the elements shown in the drawings are not necessarily drawn to scale. Where deemed appropriate, reference numerals have been repeated in the drawings to indicate corresponding or similar elements.

[0027] Figure 1 depicts a simplified block diagram of at least one embodiment of a system for configurable caching, deduplication, and rate-limiting processing of user interface data requests;

[0028] Figure 2 is a simplified block diagram of at least one embodiment of a computing device; and

[0029] Figure 3 to Figure 4 is a simplified flow chart of at least one embodiment of a method for configurable caching, deduplication, and rate-limiting processing of user interface data requests. DETAILED DESCRIPTION

[0030] Although the concepts of the present disclosure are susceptible to various modifications and alternative forms, specific embodiments have been shown by way of example in the drawings and will be described in detail herein. However, it should be understood that it is not intended to limit the concepts of the present disclosure to the specific forms disclosed, but on the contrary, it is intended to cover all modifications, equivalent forms and alternatives consistent with the present disclosure and the appended claims.

[0031] References in the specification to "one embodiment," "an embodiment," "an exemplary embodiment," etc. indicate that the described embodiment may include a particular feature, structure, or characteristic, but each embodiment may or may not necessarily include the particular feature, structure, or characteristic. In addition, such phrases do not necessarily refer to the same embodiment. It should also be understood that although reference to a "preferred" component or feature may indicate the desirability of a particular component or feature with respect to an embodiment, the present disclosure is not thereby limited with respect to other embodiments in which such a component or feature may be omitted. In addition, when a particular feature, structure, or characteristic is described in conjunction with an embodiment, it is considered to be within the knowledge of those skilled in the art to implement such a feature, structure, or characteristic in conjunction with other embodiments, whether or not explicitly described. In addition, in various embodiments, the particular features, structures, or characteristics may be combined in any suitable combination and / or sub-combination.

[0032] Additionally, it should be understood that items included in a list in the format “at least one of A, B, and C” may mean (A); (B); (C); (A and B); (B and C); (A and C); or (A, B, and C). Similarly, items listed in the format “at least one of A, B, or C” may mean (A); (B); (C); (A and B); (B and C); (A and C); or (A, B, and C). Furthermore, with respect to the claims, the use of words and phrases such as “one,” “at least one,” and / or “at least a portion” should not be construed as limited to only one such element, unless specifically stated to the contrary, and the use of phrases such as “at least a portion” and / or “a portion” should be construed to cover both embodiments including only a portion of such elements and embodiments including the entire such elements, unless specifically stated to the contrary.

[0033] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or a combination thereof. The disclosed embodiments may also be implemented as instructions carried or stored on one or more transient or non-transient machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. A machine-readable storage medium may be embodied as any storage device, mechanism, or other physical structure for storing or transmitting information in a machine-readable form (e.g., a volatile or non-volatile memory, a media disk, or other media device).

[0034] In the accompanying drawings, some structural or method features may be shown in a specific arrangement and / or order. However, it should be understood that such specific arrangement and / or order may not be required. On the contrary, in some embodiments, unless otherwise indicated, such features may be arranged in a different manner and / or order than shown in the illustrative figures. In addition, the inclusion of structural or method features in a specific figure does not mean that such features are required in all embodiments, and in some embodiments, such features may not be included or may be combined with other features.

[0035] Traditional web development paradigms lead to duplication of work, inefficiency and inconsistency. Each of the existing libraries for implementing API processing, caching, logging and other features is typically narrowly focused on specific functions, and fails to provide the key features of minimizing requests to cloud services and handling decentralized microservices 429 errors. Cache solutions are very advanced, general, cannot be cached based on parameters, and are not dynamically configurable. Prior art leads to a large waste of development and computing resources (e.g., CPU cycles, RAM, storage devices, etc.). At both the front end (e.g., client device) and the back end (e.g., cloud device), resources are wasted due to inefficient retries and request processing. Although microservices allow economies of scale because they allow multiple developers to build and deploy service layers, this allows not using common libraries and re-implementing and duplicating code. Similar benefits and problems also exist in micro applications.

[0036] It should be understood that the technology described herein allows to drive independent component development via the general service orchestration of request, data and configuration services.For example, entry methods are provided for standard RESTful API calls, and the configurable cache, deduplication and logic for processing HTTP 429 too many request status codes or errors that can be parameterized and integrated with these entry points. The cache combined with request deduplication as described herein can provide data flow minimization for RESTful API calls, which improves user interface (UI) performance, and allows multiple user interface components to be deployed separately, without considering the API flow managed to the cloud service layer. It should also be understood that the technology described herein can be used as a public service layer, which allows user interface components to move freely between micro applications due to the self-contained and web framework-unknown nature of the solution. In addition, these technologies allow to reduce the number of times of re-requesting data from the service layer, reduce the processing across the cloud stack, and make the entire solution more efficient and sustainable.

[0037] In addition, as described herein, modular and self-contained API packages can be used to encapsulate logic built on top of the basic API to handle more complex API interactions and data manipulation. In particular, RESTful methods can be processed in a common way in basic objects (e.g., encoded once), wherein cache and retry as described herein can be utilized in these calls. This ensures that the implementer of a specific API does not need to care about or re-implement the basic processing of retry, deduplication or RESTful methods. For example, a specific API may involve a POST operation, and then asynchronously returns a file download URL. In some embodiments, the entire call flow can be encapsulated in an API package at one time, so that the developer does not have to re-implement the behavior. In another exemplary embodiment, a user identifier can be returned by a specific API, but the final developer expects the user's name. In such embodiments, appropriate API calls can be paired with cache to reveal a user name answer, rather than a user identifier, which ensures universal, consistent and streamlined coding from the perspective of a user interface developer. It should also be understood that the API package allows the API to be grouped together and written to a separate file, so that only the API required by the user interface is merged into the package, which ensures that the size is minimized. Additionally, developers familiar with a particular API can write a handler once to account for any special caching or rate-limiting behaviors that should be considered during development, reducing code duplication and errors.

[0038] As described below, for example in Figure 3 to Figure 4 In method 300, configurable caching, deduplication, and / or 429 logic may be used in conjunction with one another. For example, a parameterized configurable cache may be utilized because RESTful API data responses are highly dependent on the request context, query, and / or body. Therefore, the configurable cache utilizes various input parameters and control over the lifespan of cache entries (e.g., time to live (TTL)) to allow global, grouped, or individual settings for requests. In other words, in some embodiments, the cache may be configured for each application, for each endpoint, and / or for each individual type of request. In addition, the service may have a default configuration of TTL set at the page level, which may be overridden individually.

[0039] In some embodiments, cache grouping can be used to group data sets together, which allows set operations to be performed on the cache, such as manual invalidation of linked data. It should be understood that such features may be useful when managing sets of requests that are interrelated, such as a group of requests to retrieve information from or about a single entity. For example, cache grouping can allow subsequent steps of requests that are known to affect the cache entries of the request to change those cache entries as a single operation rather than a separate cache entry operation. Therefore, cache grouping can allow faster writes and / or otherwise more efficient cache operations. In addition, rate limit grouping and maximum cache aging parameters can also be used, as described in more detail below.

[0040] Retry logic with advanced 429 logic can be implemented to handle the changing and sometimes interconnected rate limits of the microservice architecture. For example, rate limit groups can be used in conjunction with advanced 429 logic to "forward" 429 errors to all affected requests, which prevents requests from being made within a group that has received a 429 status code before the associated wait time has passed. It should be understood that each API endpoint can define a separate rate limit, and that rate limits and rate limit groups can depend on backend services. For example, when developing an API package, a specific API endpoint associated with or assigned to a specific rate limit group can be predefined by a domain expert. The speed and efficiency of the user interface can therefore be improved by preventing requests from failing to bind. In addition, as described in more detail below, the 429 logic can cause requests blocked by the 429 status code to be merged, reducing instances of repeated 429 errors waiting for requests.

[0041] It should be understood that different API parts may have different retry logic. For example, some APIs may consider different error codes to be retryable due to specific implementations. Retry groups can be used to group API requests that share common retry behaviors. Doing so allows for the provision of scalable retry logic that can be customized to suit the needs of a group of endpoints. For example, a group of requests with shared errors can be repaired by additional requests as part of the retry process. In various embodiments, for each retry group in the retry group, it is possible to define what constitutes a retryable error, as well as specific actions and / or other features of the retries and errors to be performed.

[0042] As described in more detail below, the system can utilize deduplication technology to merge identical requests into a single request within a configurable time period so that these requests are only processed once. In other words, the duplication technology effectively acts as a short-term deduplication cache. Although not necessarily a "cache" in nature, for the sake of simplicity and brevity of description, the deduplication cache is described as such herein. In an exemplary embodiment, the deduplication cache stores commitments associated with data requests, rather than the actual data objects requested (which may be stored in a "primary cache" or other memory). Deduplication ensures that if the user interface element is not debounced, the request will not spam the back-end server. This allows different user interface components to be developed independently without knowing each other or considering making multiple identical calls to the back-end server.

[0043] In some embodiments, page configuration settings can be established at the page level, but if a specific application requires customized settings, they can also be overwritten. Depending on the specific application, micro applications can utilize different behavior patterns. For example, it may be expected that data is only cached for 10 seconds instead of 120 seconds. Therefore, the page can configure the behavior mode at the highest level, and it affects all components individually. Therefore, even if more than a dozen components are developed, it is not necessary to know the cache window because the page knows the cache window and attributes it to each component (for example, if it is not overwritten at a lower level, the advanced configuration is pushed down). In this way, it should be understood that the page configuration scheme can provide a default configuration for rapid development, while also allowing personalized configuration of feature switching, licensing services, data services, notification services, retries, deduplication cleaning, application-level caches and / or other features or parameters.

[0044] Reference now Figure 1 , a system 100 for configurable caching, deduplication, and rate limiting processing of user interface data requests may include a front-end system 102, a network 104, and a back-end system 106. Additionally, in the exemplary embodiment, the front-end system 102 includes a primary cache 108 and a deduplication cache 110, and the back-end system 106 includes an application programming interface (API) 112.

[0045] Despite Figure 1In the exemplary embodiment of the system 100, only one front-end system 102, one network 104, and one back-end system 106 are shown, but in other embodiments, the system 100 may include multiple front-end systems 102, networks 104, and / or back-end systems 106. For example, in some embodiments, multiple back-end systems 106 can be used to perform the various functions described herein. In addition, in some embodiments, one or more systems described herein may be excluded from the system 100, one or more systems described as independent may form part of another system, and / or one or more systems described as forming part of another system may be independent. In addition, it should be understood that the system 100 described herein may be independent of the system 100. Figure 1 The system architecture represented by system 100 in FIG. 1 is for illustrative purposes only, and system 100 may include additional and / or alternative devices / systems in other system architectures. In addition, it should be understood that each of front-end system 102, network 104, and / or back-end system 106 may include multiple devices and / or subsystems. For example, in some system architectures, back-end system 106 may include at least an application server and a database server. In a cloud-based embodiment, front-end system 102 and / or back-end system 106 or portions thereof may be embodied as a server-agnostic computing solution similar to that described below.

[0046] The front-end system 102 may be embodied as any one or more types of devices / systems capable of performing the functions described herein. For example, in an exemplary embodiment, the front-end system 102 includes a client device configured to launch a user interface with which a user can interact. When a user interacts with the user interface, an element of the user interface may initiate a data request directed to a specific API 112 of the back-end system 106 (e.g., in response to a user clicking a user interface element such as a button). In an exemplary embodiment, the front-end system 102 includes a primary cache 108 and a deduplication cache 110.

[0047] The primary cache 108 is configured to store data results (e.g., data objects) for data requests made via a user interface, while the deduplication cache 110 stores promises, which essentially only indicate that it will have results at a certain point in time, but do not reference the primary cache 108. In some embodiments, as described herein, the deduplication cache 110 stores data returned in response to a previous data request with a configurable timeout. It should be understood that a promise is an object that represents the eventual completion or failure of an asynchronous operation and its resulting value rather than the value itself. As described below, the deduplication cache 110 can be utilized to ensure that if multiple identical data requests are made within a short predefined time period, these data requests are merged into a single request, thereby avoiding spamming the backend system 106. It should be understood that each of the primary cache 108 and the deduplication cache 110 can be embodied as or include any type of memory, data storage device, and / or data structure for storing data and performing the functions described herein. In addition, although the primary cache 108 and the deduplication cache 110 are described as forming part of the front-end system 102, it should be understood that in other embodiments, the primary cache 108 and / or the deduplication cache 110 may form part of one or more other devices / systems of the system 100.

[0048] The network 104 may be embodied as any one or more types of communication networks capable of facilitating communication between various devices communicatively connected via the network 104. Thus, the network 104 may include one or more networks, routers, switches, access points, hubs, computers, and / or other intermediate network devices. For example, the network 104 may be embodied as or otherwise include one or more cellular networks, telephone networks, local or wide area networks, publicly available global networks (e.g., the Internet), ad hoc networks, short-range communication links, or combinations thereof. In some embodiments, the network 104 may include a circuit-switched voice or data network, a packet-switched voice or data network, and / or any other network capable of carrying voice and / or data. Specifically, in some embodiments, the network 104 may include an Internet Protocol (IP)-based and / or an Asynchronous Transfer Mode (ATM)-based network. In some embodiments, the network 104 may handle voice traffic (e.g., via a Voice over IP (VOIP) network), web traffic (e.g., such as Hypertext Transfer Protocol (HTTP) traffic and Hypertext Markup Language (HTML) traffic), and / or other network traffic depending on the specific implementation of the system 100 and / or devices communicating with each other. In various embodiments, the network 104 may include analog or digital wired and wireless networks (e.g., IEEE 802.11 networks, public switched telephone networks (PSTN), integrated services digital networks (ISDN), and digital subscriber lines (xDSL)), third generation (3G) mobile telecommunication networks, fourth generation (4G) mobile telecommunication networks, fifth generation (5G) mobile telecommunication networks, wired Ethernet networks, private networks (e.g., such as intranets), radio, television, cable, satellite, and / or any other delivery or tunneling mechanism for carrying data, or any suitable combination of such networks. The network 104 may enable connections between the various devices / systems 102, 106 of the system 100. It should be understood that the various devices / systems 102, 106 may communicate with each other via different networks 104 depending on the source and / or destination devices / systems 102, 106.

[0049] The backend system 106 may be embodied as any one or more types of devices / systems capable of performing the functions described herein. As shown, the backend system 106 includes various APIs 112 and corresponding API endpoints from which the frontend system 102 makes data requests. As described above, it should be understood that the number and type of devices included in the backend system 106 may vary depending on the specific architecture of the system 100. It should be understood that each of the frontend system 102, the network 104, and the backend system 106 may be embodied as any type of device / system, collection of devices / systems, and / or portions thereof suitable for performing the functions described herein (e.g., Figure 2 computing device 200), or executed by, or forming a part of, or associated with.

[0050] Reference now Figure 2 , a simplified block diagram of at least one embodiment of a computing device 200 is shown. The exemplary computing device 200 depicts at least one embodiment of each of the computing devices, systems, servers, controllers, switches, gateways, engines, modules and / or computing components described herein (e.g., for the sake of brevity of description, they may be interchangeably referred to as computing devices, servers or modules). For example, various computing devices may be processes or threads running on one or more processors of one or more computing devices 200, which may execute computer program instructions and interact with other system modules to perform various functions described herein. Unless otherwise expressly limited, the functions described with respect to multiple computing devices may be integrated into a single computing device, or the various functions described with respect to a single computing device may be distributed across several computing devices. In addition, with respect to the computing system described herein, depending on the specific embodiment, various servers and their computer devices may be located at a local computing device 200, a remote computing device 200 (e.g., off-site or in a cloud-based or cloud computing environment, e.g., in a remote data center connected via a network), or some combination thereof. In some embodiments, functionality provided by a server located on an off-site computing device may be accessed and provided via a virtual private network (VPN) as if such server were on-site, or the functionality may be provided using Software as a Service (SaaS) (accessed over the Internet using various protocols), such as by exchanging data via Extensible Markup Language (XML) and JSON, and / or the functionality may be accessed / utilized in other ways.

[0051] In some embodiments, computing device 200 may be embodied as a server, a desktop computer, a laptop computer, a tablet computer, a notebook, a netbook, an Ultrabook, or a laptop computer. TM , cellular telephones, mobile computing devices, smartphones, wearable computing devices, personal digital assistants, Internet of Things (IoT) devices, processing systems, wireless access points, routers, gateways, and / or any other computing devices, processing devices, and / or communication devices capable of performing the functions described herein.

[0052] The computing device 200 includes a processing device 202 that executes algorithms and / or processes data according to operating logic 208, an input / output device 204 that enables communication between the computing device 200 and one or more external devices 210, and a memory 206 that stores data received from the external device 210, for example, via the input / output device 204.

[0053] The input / output devices 204 allow the computing device 200 to communicate with external devices 210. For example, the input / output devices 204 may include a transceiver, a network adapter, a network card, an interface, one or more communication ports (e.g., a USB port, a serial port, a parallel port, an analog port, a digital port, VGA, DVI, HDMI, FireWire, CAT 5, or any other type of communication port or interface), and / or other communication circuitry. Depending on the particular computing device 200, the communication circuitry of the computing device 200 may be configured to use any one or more communication technologies (e.g., wireless or wired communications) and associated protocols (e.g., Ethernet, WiMAX, etc.) to achieve such communications. Input / output device 204 may include hardware, software and / or firmware suitable for performing the techniques described herein.

[0054] The external device 210 may be any type of device that allows data to be input or output from the computing device 200. For example, in various embodiments, the external device 210 may be embodied as one or more of the devices / systems described herein and / or a portion thereof. In addition, in some embodiments, the external device 210 may be embodied as another computing device, a switch, a diagnostic tool, a controller, a printer, a display, an alarm, a peripheral device (e.g., a keyboard, a mouse, a touch screen display, etc.), and / or any other computing device, processing device, and / or communication device capable of performing the functions described herein. In addition, in some embodiments, it should be understood that the external device 210 may be integrated into the computing device 200.

[0055] The processing device 202 may be embodied as any type of processor capable of performing the functions described herein. In particular, the processing device 202 may be embodied as one or more single-core or multi-core processors, microcontrollers or other processors or processing / control circuits. For example, in some embodiments, the processing device 202 may include or be embodied as an arithmetic logic unit (ALU), a central processing unit (CPU), a digital signal processor (DSP), a graphics processing unit (GPU), a field programmable gate array (FPGA), an application-specific integrated circuit (ASIC) and / or another suitable processor. The processing device 202 may be a programmable type, a dedicated hard-wired state machine or a combination thereof. In various embodiments, a processing device 202 with multiple processing units may utilize distributed, pipelined and / or parallel processing. In addition, the processing device 202 may be dedicated to the execution of only the operations described herein, or may be utilized in one or more additional applications. In an exemplary embodiment, the processing device 202 is programmable and executes algorithms and / or processes data according to the operation logic 208, as defined by programming instructions (such as software or firmware) stored in the memory 206. Additionally or alternatively, the operating logic 208 for the processing device 202 may be defined at least in part by hardwired logic or other hardware. Furthermore, the processing device 202 may include one or more components of any type suitable for processing signals received from the input / output device 204 or from other components or devices and providing desired output signals. Such components may include digital circuitry, analog circuitry, or a combination thereof.

[0056] The memory 206 may be one or more types of non-transitory computer-readable media, such as solid-state memory, electromagnetic memory, optical memory, or a combination thereof. In addition, the memory 206 may be volatile and / or non-volatile, and in some embodiments, some or all of the memory 206 may be of a portable type, such as a disk, tape, memory stick, cassette, and / or other suitable portable memory. In operation, the memory 206 may store various data and software used during operation of the computing device 200, such as an operating system, applications, programs, libraries, and drivers. It should be understood that in addition to or in lieu of storing programming instructions defining the operating logic 208, the memory 206 may store data manipulated by the operating logic 208 of the processing device 202, such as, for example, data representing signals received from and / or sent to the input / output device 204. As Figure 2 As shown, depending on the particular implementation, memory 206 can be included with and / or coupled to processing device 202. For example, in some implementations, processing device 202, memory 206, and / or other components of computing device 200 can form part of a system on a chip (SoC) and be incorporated on a single integrated circuit chip.

[0057] In some embodiments, various components of computing device 200 (e.g., processing device 202 and memory 206) may be communicatively coupled via an input / output subsystem, which may be embodied as circuitry and / or components to facilitate input / output operations with processing device 202, memory 206, and other components of computing device 200. For example, the input / output subsystem may be embodied as or otherwise include a memory controller hub, an input / output control hub, a firmware device, communication links (i.e., point-to-point links, bus links, wires, cables, optical guides, printed circuit board traces, etc.), and / or other components and subsystems to facilitate input / output operations.

[0058] In other embodiments, computing device 200 may include other or additional components, such as those commonly found in typical computing devices (e.g., various input / output devices and / or other components). It should also be understood that one or more of the components of computing device 200 described herein may be distributed across multiple computing devices. In other words, the techniques described herein may be employed by a computing system that includes one or more computing devices. Additionally, although in Figure 2 2, only a single processing device 202, I / O device 204, and memory 206 are illustratively shown, but it should be understood that in other embodiments, a particular computing device 200 may include multiple processing devices 202, I / O devices 204, and / or memories 206. In addition, in some embodiments, more than one external device 210 may communicate with the computing device 200.

[0059] The computing device 200 can be one of a plurality of devices connected to other systems / sources through a network connection or via a network. The network can be embodied as any one or more types of communication networks that can facilitate communication between various devices connected via network communication. Therefore, the network can include one or more networks, routers, switches, access points, hubs, computers, client devices, terminals, nodes and / or other intermediate network devices. For example, the network can be embodied as or otherwise include one or more cellular networks, telephone networks, local area networks or wide area networks, publicly available global networks (e.g., the Internet), self-organizing networks, short-range communication links or combinations thereof. In some embodiments, the network can include a circuit-switched voice or data network, a packet-switched voice or data network and / or any other network capable of carrying voice and / or data. In particular, in some embodiments, the network can include a network based on Internet Protocol (IP) and / or based on asynchronous transfer mode (ATM). In some embodiments, the network can handle voice traffic (e.g., via a voice over IP (VOIP) network), web traffic and / or other network traffic depending on the specific implementation of the system communicating with each other and / or the device. In various embodiments, the network may include analog or digital wired and wireless networks (e.g., IEEE 802.11 networks, public switched telephone networks (PSTN), integrated services digital networks (ISDN), and digital subscriber lines (xDSL)), third generation (3G) mobile telecommunication networks, fourth generation (4G) mobile telecommunication networks, fifth generation (5G) mobile telecommunication networks, wired Ethernet networks, private networks (e.g., such as intranets), radio, television, cable, satellite, and / or any other delivery or tunneling mechanism for carrying data, or any suitable combination of such networks. It should be understood that the various devices / systems may communicate with each other via different networks depending on the source and / or destination devices.

[0060] It should be understood that the computing device 200 can communicate with other computing devices 200 via any type of gateway or tunneling protocol (such as secure socket layer or transport layer security). The network interface may include a built-in network adapter (such as a network interface card) that is suitable for joining the computing device to any type of network capable of performing the operations described herein. In addition, the network environment can be a virtual network environment in which various network components are virtualized. For example, various machines can be virtual machines implemented as software-based computers running on physical machines. The virtual machines can share the same operating system, or in other embodiments, different operating systems can be run on each virtual machine instance. For example, a "hypervisor" type of virtualization is used, in which multiple virtual machines run on the same host physical machine, and each virtual machine acts as if it has its own dedicated box. In other embodiments, other types of virtualization can be adopted, such as, for example, networks (e.g., via software-defined networking) or functions (e.g., via network function virtualization).

[0061] Thus, one or more of the computing devices 200 described herein may be embodied as or form part of one or more cloud-based systems. In a cloud-based embodiment, the cloud-based system may be embodied as a server-obfuscated computing solution that, for example, executes multiple instructions on demand, contains logic to execute instructions only when prompted by a specific activity / triggering event, and does not consume computing resources when not in use. That is, the system may be embodied as a virtual computing environment (e.g., a distributed network of devices) residing "on" a computing system, wherein various virtual functions (e.g., lambda functions, Azure functions, Google cloud functions, and / or other suitable virtual functions) may be executed corresponding to the functions of the system described herein. For example, when an event occurs (e.g., data is transferred to the system for processing), the virtual computing environment may be communicated (e.g., via a request to an API of the virtual computing environment), whereby the API may route the request to the correct virtual function (e.g., a specific server-obfuscated computing resource) based on a set of rules. Thus, when a user makes a request for a data transfer (e.g., via an appropriate user interface to the system), an appropriate virtual function may be executed to perform an action before the instance of the virtual function is eliminated.

[0062] Reference now Figure 3 to Figure 4 In use, the system 100 may perform the method 300 for configurable caching, deduplication, and rate limiting processing of user interface data requests. It should be understood that unless otherwise stated, the specific blocks of the method 300 are shown by way of example, and such blocks may be combined or divided, added or removed, and / or reordered in whole or in part depending on the specific implementation. Because the system 100 may have a variety of different system architectures, for the sake of simplicity and brevity of description, Figure 3 to Figure 4 The blocks of method 300 are primarily described as being simply performed by system 100 .

[0063] The exemplary method 300 is Figure 3 , where a user interface element initiates a data request. It should be understood that the data request may reference a related API endpoint / URL and include any related parameters / data. In box 304, the system 100 determines whether the deduplication cache 110 includes a deduplication entry associated with the data request (e.g., indicating that the previous same data request was made within a configurable threshold time period). For example, as described above, the deduplication cache 110 may be configured to store commitments associated with corresponding data requests. Therefore, the various deduplication entries in the deduplication cache 110 indicate previous data requests made within a threshold time period (i.e., the expiration time period of the deduplication entry, after which it can be removed from the deduplication cache 110). For example, in one embodiment, the threshold time period may be 10ms, while in another embodiment, the threshold may be 100ms. It should be understood that certain HTTP methods are considered non-idempotent and are not typically cached. However, in an exemplary embodiment, the deduplication cache 110 still caches it, in a sense, it is the request or commitment itself rather than the actual result data that is cached. It should be appreciated that the deduplication cache 110 is utilized to ensure that duplicate data requests are not made within a predefined period of time (eg, due to a user inadvertently rapidly and repeatedly clicking on elements of a user interface).

[0064] If the system 100 determines in block 304 that an associated deduplication entry exists in the deduplication cache 110, the method 300 proceeds to block 316, where a data request is made and the data is returned to the user interface element. In other words, if an associated deduplication entry exists in the deduplication cache 110, the data request has been made and the system 100 may wait for the requested data to be returned. However, if the system 100 determines in block 304 that the deduplication cache 110 does not include an associated deduplication entry, the method 300 proceeds to block 306, where the system 100 determines whether the primary cache 108 includes a cache entry associated with the data request.

[0065] If the system 100 determines in block 306 that the primary cache 108 includes a cache entry associated with the data request, the method 300 proceeds to block 308, where the system 100 determines whether the data request is associated with a predefined maximum cache age. In other words, the system 100 determines whether the data request includes a maximum cache age parameter that overrides any expiration time provided by the cache entry itself. It should be understood that different components may have different understandings of what is acceptable in terms of the TTL of a data response. For example, one component may only accept data that is at most two minutes old, while another component may accept data that is ten minutes old.

[0066] If the system 100 determines in block 308 that the data request is associated with a predefined maximum cache age, the method 300 proceeds to block 310, in which the system 100 uses the maximum cache age to calculate the expiration of the cache entry. More specifically, the system 100 may determine whether the cache entry associated with the data request has expired based on the creation time of the cache entry and the predefined maximum cache age (e.g., the expiration is the maximum cache age after the cache entry creation time). However, if the system 100 determines in block 308 that the data request is not associated with a predefined maximum cache age, the method 300 proceeds to block 312, in which the system 100 uses the expiration of the cache entry associated with the data request, so that the system 100 may determine whether the cache entry associated with the data request has expired based on the expiration time defined by the cache entry itself. From blocks 310, 312, the method 300 proceeds to block 314, in which the system 100 determines whether the cache entry has expired. If not, the method 300 proceeds to block 316 where a request for data is made and the data is returned to the user interface element.

[0067] Returning to block 306, if the system 100 determines that the primary cache 108 does not include a cache entry associated with the data request, or if the system 100 determines in block 314 that the cache entry exists but has expired, the method 300 proceeds to Figure 4 The process proceeds to block 318, where the system 100 determines whether the data request is associated with a rate-limited API endpoint group. As described above, rate-limited API endpoint groups may be predefined by the system 100 to help avoid transmitting certain unnecessary data requests. For example, in an embodiment, the GET_USER and GET_USERS endpoints may belong to the same rate-limited group, such that if the system 100 already knows that the GET_USER request has been rate-limited, the GET_USERS request should be postponed in order to avoid the inevitable 429 error.

[0068] If the system 100 determines in block 318 that the data request is associated with a rate limited group, the method 300 proceeds to block 320 where the system 100 delays a corresponding server retry period before making the data request to the backend system 106 in block 322. If the system 100 determines in block 318 that the data request is not associated with a rate limited group, the method 300 proceeds to block 322 where the data request may be made without delay.

[0069] In block 324, the system 100 determines whether the data request was successful. If so, the method 300 returns to Figure 3 316, in which a data request is made and the data is returned to the user interface element. However, if the data request fails, an error or status code associated with the failure to satisfy the data request may be provided, and the method 300 proceeds to a box 326, in which the system 100 determines whether the error associated with the data request is a retryable error. If not, the method 300 proceeds to a box 328, in which the data request failure is returned to the user interface element. However, if the system 100 determines in box 326 that the error is retryable, the method 300 proceeds to a box 330, in which the system 100 determines whether a predefined retry limit has been reached. It should be understood that the predefined retry limit may be configurable, and the maximum number of retry attempts that may be made may be defined. In some embodiments, the system 100 may utilize different retry attempts for different situations (e.g., different API endpoints, different error types, etc.). If the predefined retry limit has been reached, the method 300 proceeds to a box 328, in which the data request failure is returned to the user interface element.

[0070] However, if the predefined retry limit has not been reached, the system 100 may retry the data request. More specifically, in an exemplary embodiment, the method 300 proceeds to block 332, in which the system 100 determines whether the retryable error is a 429 error (e.g., associated with an HTTP 429 Too Many Requests status code or error). If so, the method 300 proceeds to block 320, in which the system 100 delays a corresponding server retry period (e.g., provided by a 429 error) before retrying the data request. However, if the retryable error is not a 429 error, the method 300 proceeds to block 322, in which the data request may be made without delay.

[0071] Although blocks 302 through 332 are described in a relatively serial manner, it should be understood that in some embodiments, the various blocks of method 300 may be performed in parallel.

Claims

1. A method for configurable caching, deduplication, and rate limiting processing of user interface data requests, the method comprising: receiving a data request initiated by a user interface element of the user interface; determining whether a deduplication cache includes a deduplication entry associated with the data request; In response to determining that the deduplication cache does not include the deduplication entry, determining whether a primary cache different from the deduplication cache includes a cache entry associated with the data request; determining whether the data request is associated with a rate-limited application programming interface endpoint group in response to one of (i) determining that the primary cache does not include the cache entry associated with the data request or (ii) determining that the cache entry associated with the data request has expired; as well as In response to determining that the data request is associated with the rate limited group, the data request is made to a backend system after a predefined server retry period associated with the rate limited group has elapsed. 2 . The method of claim 1 , wherein the deduplicated entry indicates that a previous data request identical to the data request initiated by the user interface element of the user interface was made within a threshold time period.

3. The method of claim 1, further comprising, in response to determining that the primary cache includes the cache entry associated with the data request, determining whether the data request is associated with a predefined maximum cache age.

4. The method according to claim 3 further includes, in response to determining that the data request is associated with the predefined maximum cache age, determining whether the cache entry associated with the data request has expired based on the creation time of the cache entry and the predefined maximum cache age.

5. The method according to claim 3 further includes, in response to determining that the data request is not associated with the predefined maximum cache age, determining whether the cache entry associated with the data request has expired based on the expiration time of the cache entry defined by the cache entry. 6 . The method of claim 1 , further comprising, in response to determining that the data request is not associated with any rate limited group, immediately making the data request to the backend system.

7. The method according to claim 1, further comprising: receiving an error associated with the data request in response to failure to satisfy the data request; determining whether the error associated with the data request is a retryable error; responsive to determining that the error associated with the data request is the retryable error, determining whether a predefined retry limit has been reached; as well as In response to determining that the predefined retry limit has not been reached, the data request is retried.

8. The method of claim 7, wherein retrying the data request comprises, in response to determining that the retryable error is a 429 error, retrying the data request after the predefined server retry period.

9. The method of claim 7, wherein the predefined retry limit is a maximum number of retry attempts that can be made.

10. The method of claim 1, wherein determining whether the data request is associated with the rate-limited API endpoint group comprises determining whether an API endpoint to which the data request is directed is included in the rate-limited API endpoint group.

11. The method of claim 1, wherein the deduplication cache is configured to store one or more commitments, wherein each of the one or more commitments is associated with a previous corresponding data request.

12. The method of claim 1, wherein the deduplication cache is configured to store data returned to a previous data request with a configurable timeout.

13. The method of claim 1, wherein the primary cache is configured to store data returned in response to a previous request for data.

14. A system for configurable caching, deduplication, and rate limiting processing of user interface data requests, the system comprising: Deduplication cache; a primary cache different from the deduplication cache; at least one processor; as well as at least one memory including a plurality of instructions stored thereon, the plurality of instructions, in response to being executed by the at least one processor, causing the system to: receiving a data request initiated by a user interface element of the user interface; determining whether the deduplication cache includes a deduplication entry associated with the data request; In response to determining that the deduplication cache does not include the deduplication entry, determining whether the primary cache includes a cache entry associated with the data request; In response to one of (i) determining that the primary cache does not include the cache entry associated with the data request or (ii) the cache entry associated with the data request has expired, determining whether the data request is associated with a rate-limited application programming interface endpoint group; as well as In response to determining that the data request is associated with the rate limited group, the data request is made to a backend system after a predefined server retry period associated with the rate limited group has elapsed. 15 . The system of claim 14 , wherein the deduplicated entry indicates that a previous data request identical to the data request initiated by the user interface element of the user interface was made within a threshold time period.

16. The system of claim 14, wherein the plurality of instructions further cause the system to: In response to determining that the primary cache includes the cache entry associated with the data request, determining whether the data request is associated with a predefined maximum cache age; In response to determining that the data request is associated with the predefined maximum cache age, determining whether the cache entry associated with the data request has expired based on a creation time of the cache entry and the predefined maximum cache age; as well as In response to determining that the data request is not associated with the predefined maximum cache age, a determination is made as to whether the cache entry associated with the data request has expired based on an expiration time of the cache entry defined by the cache entry.

17. The system of claim 14, wherein the plurality of instructions further cause the system to: receiving an error associated with the data request in response to failure to satisfy the data request; determining whether the error associated with the data request is a retryable error; responsive to determining that the error associated with the data request is the retryable error, determining whether a predefined retry limit has been reached; as well as In response to determining that the predefined retry limit has not been reached, the data request is retried.

18. The system of claim 17, wherein retrying the data request comprises, in response to determining that the retryable error is a 429 error, retrying the data request after the predefined server retry period.

19. The system of claim 17, wherein the predefined retry limit is a maximum number of retry attempts that can be made.

20. The system of claim 14, wherein the deduplication cache is configured to store one or more commitments, wherein each of the one or more commitments is associated with a previous corresponding data request; and The primary cache is configured to store data returned in response to a previous data request.

Images