Technology for configurable caching, deduplication, and rate-limiting processing

Abstract

A method for configurable caching, deduplication, and rate-limiting processing for user interface data requests according to an embodiment is provided, the method comprising receiving a data request initiated by a user interface element of a user interface; determining whether a deduplication cache includes a deduplication entry associated with the data request; in response to determining that the deduplication cache does not include the deduplication entry, determining whether the primary cache includes a cache entry associated with the data request; in response to determining that the primary cache does not include the cache entry or determining that the cache entry has expired, determining whether the data request is associated with a rate-limited application programming interface endpoint group; and in response to determining that the data request is associated with the rate-limited group, making the data request to a back-end system after a predefined server retry period associated with the rate-limited group has passed.

Description

[0001] Claim priority

[0002] This application claims the benefit of U.S. patent application 18 / 345,243, filed on June 30, 2023, entitled “TECHNOLOGIES FOR CONFIGURABLE CACHING, DEDUPLICATION, AND RATE LIMIT HANDLING.” Background Art

[0003] Web development has traditionally followed the formula of building single-page applications (SPAs) or multi-page applications, where each page implements its own handling of application programming interface (API) requests, caching, settings, permissions, events, and logging. Page-level data storage is a common solution, but it is linked to a specific framework and requires revisions when implementing a solution across multiple frameworks. This approach results in code duplication across multiple small repositories, or placing all dependent processing code in the same repository. The duplication of code leads to duplicated work during debugging, dependency upgrades, or regular maintenance, as well as inconsistencies across pages. On the other hand, having all code rely on a service layer within one repository causes the repository to become large, complex, and difficult to manage, slowing down the maintenance of existing features and the development of new features. Components must also be rebuilt when moving between pages and user interface frameworks (e.g., due to differences in the specific implementation of page-level or framework-level logic). Summary of the Invention

[0004] One embodiment relates to unique systems, components, and methods for configurable caching, deduplication, and rate-limiting processing of user interface data requests. Other embodiments relate to apparatus, systems, devices, hardware, methods, and combinations thereof, for configurable caching, deduplication, and rate-limiting processing of user interface data requests.

[0005] According to an embodiment, a method for configurable caching, deduplication, and rate limiting processing of user interface data requests is provided, which may include receiving a data request initiated by a user interface element of a user interface; determining whether a deduplication cache includes a deduplication entry associated with the data request; in response to determining that the deduplication cache does not include the deduplication entry, determining whether a primary cache different from the deduplication cache includes a cache entry associated with the data request; in response to one of (i) determining that the primary cache does not include the cache entry associated with the data request or (ii) the cache entry associated with the data request has expired, determining whether the data request is associated with a rate-limited application programming interface endpoint group; and in response to determining that the data request is associated with the rate-limited group, making the data request to a back-end system after a predefined server retry period associated with the rate-limited group has passed.

[0006] In some embodiments, the deduplication entry may indicate that a previous data request identical to the data request initiated by the user interface element of the user interface was made within a threshold time period.

[0007] In some embodiments, the method may further include, in response to determining that the primary cache includes the cache entry associated with the data request, determining whether the data request is associated with a predefined maximum cache age.

[0008] In some embodiments, the method may also include, in response to determining that the data request is associated with the predefined maximum cache age, determining whether the cache entry associated with the data request has expired based on the creation time of the cache entry and the predefined maximum cache age.

[0009] In some embodiments, the method may also include, in response to determining that the data request is not associated with the predefined maximum cache age, determining whether the cache entry associated with the data request has expired based on the expiration time of the cache entry defined by the cache entry.

[0010] In some embodiments, the method may further include, in response to determining that the data request is not associated with any rate limited group, immediately making the data request to the backend system.

[0011] In some embodiments, the method may further include, in response to the data request being failed to satisfy, receiving an error associated with the data request; determining whether the error associated with the data request is a retryable error; in response to determining that the error associated with the data request is the retryable error, determining whether a predefined retry limit has been reached; and in response to determining that the predefined retry limit has not been reached, retrying the data request.

[0012] In some embodiments, retrying the data request may include retrying the data request after the predefined server retry period in response to determining that the retryable error is a 429 error.

[0013] In some embodiments, the predefined retry limit may be the maximum number of retry attempts that may be made.

[0014] In some embodiments, determining whether the data request is associated with the rate-limited API endpoint group may include determining whether the API endpoint to which the data request is directed is included in the rate-limited API endpoint group.

[0015] In some embodiments, the deduplication cache may be configured to store one or more commitments, where each of the one or more commitments is associated with a previous corresponding data request.

[0016] In some embodiments, the deduplication cache may be configured to store data returned in response to a previous request for data with a configurable timeout.

[0017] In some embodiments, the primary cache may be configured to store data returned in response to a previous request for data.

[0018] According to another embodiment, a system for configurable caching, deduplication, and rate-limiting processing of user interface data requests is provided, which system may include a deduplication cache; a primary cache different from the deduplication cache; at least one cache; and at least one memory, the at least one memory including a plurality of instructions stored thereon, the plurality of instructions being responsive to execution by the at least one processor causing the system to receive a data request initiated by a user interface element of a user interface; determine whether the deduplication cache includes a deduplication entry associated with the data request; responsive to determining that the deduplication cache does not include the deduplication entry, determine whether the primary cache includes a cache entry associated with the data request; responsive to one of (i) determining that the primary cache does not include the cache entry associated with the data request or (ii) the cache entry associated with the data request has expired, determine whether the data request is associated with a rate-limited application programming interface endpoint group; and responsive to determining that the data request is associated with the rate-limited group, make the data request to a back-end system after a predefined server retry period associated with the rate-limited group has elapsed.

[0019] In some embodiments, the deduplication entry may indicate that a previous data request identical to the data request initiated by the user interface element of the user interface was made within a threshold time period.

[0020] In some embodiments, the plurality of instructions may further cause the system to: in response to determining that the primary cache includes the cache entry associated with the data request, determine whether the data request is associated with a predefined maximum cache age; in response to determining that the data request is associated with the predefined maximum cache age, determine whether the cache entry associated with the data request has expired based on a creation time of the cache entry and the predefined maximum cache age; and in response to determining that the data request is not associated with the predefined maximum cache age, determine whether the cache entry associated with the data request has expired based on an expiration time of the cache entry defined by the cache entry.

[0021] In some embodiments, the plurality of instructions may further cause the system to, in response to a failure to satisfy the data request, receive an error associated with the data request; determine whether the error associated with the data request is a retryable error; in response to determining that the error associated with the data request is the retryable error, determine whether a predefined retry limit has been reached; and in response to determining that the predefined retry limit has not been reached, retry the data request.

[0022] In some embodiments, retrying the data request may include retrying the data request after the predefined server retry period in response to determining that the retryable error is a 429 error.

[0023] In some embodiments, the predefined retry limit may be the maximum number of retry attempts that may be made.

[0024] In some embodiments, the deduplication cache may be configured to store one or more commitments, where each of the one or more commitments is associated with a previous corresponding data request, and the primary cache may be configured to store data returned in response to the previous data request.

[0025] This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to be used as an aid in limiting the scope of the claimed subject matter. Other embodiments, forms, features, and aspects of the present application will become apparent from the description and drawings provided herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The concepts described herein are illustrated in the accompanying drawings by way of example and not by way of limitation. For simplicity and clarity of illustration, the elements shown in the drawings are not necessarily drawn to scale. Where deemed appropriate, reference numerals have been repeated in the drawings to indicate corresponding or similar elements.

[0027] Figure 1 depicts a simplified block diagram of at least one embodiment of a system for configurable caching, deduplication, and rate-limiting processing of user interface data requests;

[0028] Figure 2 is a simplified block diagram of at least one embodiment of a computing device; and

[0029] Figures 3 and 4 is a simplified flow chart of at least one embodiment of a method for configurable caching, deduplication, and rate-limiting processing of user interface data requests. DETAILED DESCRIPTION

[0030] While the concepts of the present disclosure are susceptible to various modifications and alternative forms, specific embodiments have been shown by way of example in the drawings and will be described in detail herein. However, it should be understood that there is no intention to limit the concepts of the present disclosure to the specific forms disclosed, but on the contrary, the intention is to cover all modifications, equivalents, and alternatives consistent with this disclosure and the appended claims.

[0031] References in the specification to "one embodiment," "an embodiment," "an exemplary embodiment," etc., indicate that the embodiment being described may include a particular feature, structure, or characteristic, but every embodiment may or may not necessarily include the particular feature, structure, or characteristic. Furthermore, such phrases do not necessarily refer to the same embodiment. It should also be understood that although reference to a "preferred" component or feature may indicate the desirability of a particular component or feature with respect to an embodiment, the present disclosure is not thereby limited with respect to other embodiments in which such component or feature may be omitted. Furthermore, when a particular feature, structure, or characteristic is described in conjunction with an embodiment, it is deemed to be within the knowledge of those skilled in the art to implement such feature, structure, or characteristic in conjunction with other embodiments, whether or not explicitly described. Furthermore, in various embodiments, the particular features, structures, or characteristics may be combined in any suitable combinations and / or subcombinations.

[0032] Additionally, it should be understood that items included in a list in the format “at least one of A, B, and C” can mean (A); (B); (C); (A and B); (B and C); (A and C); or (A, B, and C). Similarly, items listed in the format “at least one of A, B, or C” can mean (A); (B); (C); (A and B); (B and C); (A and C); or (A, B, and C). Furthermore, with respect to the claims, the use of words and phrases such as “a,” “an,” “at least one,” and / or “at least a portion” should not be construed as limiting to only one such element, unless specifically stated to the contrary, and the use of phrases such as “at least a portion” and / or “a portion” should be construed to encompass both embodiments including only a portion of such elements and embodiments including the entire such elements, unless specifically stated to the contrary.

[0033] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or a combination thereof. The disclosed embodiments may also be implemented as instructions carried or stored on one or more transient or non-transient machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. A machine-readable storage medium may be embodied as any storage device, mechanism, or other physical structure for storing or transmitting information in a machine-readable form (e.g., a volatile or non-volatile memory, a media disk, or other media device).

[0034] In the accompanying drawings, some structural or method features may be shown in a particular arrangement and / or ordering. However, it should be understood that such a particular arrangement and / or ordering may not be required. Instead, in some embodiments, unless otherwise indicated, such features may be arranged in a manner and / or order different from that shown in the illustrative figures. In addition, the inclusion of a structural or method feature in a particular figure does not imply that such feature is required in all embodiments, and in some embodiments, such feature may not be included or may be combined with other features.

[0035] The traditional web development paradigm leads to duplication of work, inefficiency and inconsistency. Each of the existing libraries for implementing API processing, caching, logging and other features is typically narrowly focused on a specific function and fails to provide the key features of minimizing requests to cloud services and handling decentralized microservice 429 errors. Cache solutions are very advanced and general, cannot be cached based on parameters, and are not dynamically configurable. Existing technologies lead to a large waste of development and computing resources (e.g., CPU cycles, RAM, storage devices, etc.). At both the front end (e.g., client device) and the back end (e.g., cloud device), resources are wasted due to inefficient retries and request processing. Although microservices allow economies of scale because they allow multiple developers to build and deploy service layers, this allows for not using common libraries and reimplementing and duplicating code. Similar benefits and problems exist in micro-applications.

[0036] It should be understood that the technology described herein allows for independent component development to be driven via a common service orchestration of request, data, and configuration services. For example, entry methods are provided for standard RESTful API calls, and integrated with these entry points can be parameterized configurable caches, deduplication, and logic for handling HTTP 429 too many request status codes or errors. The cache combined with request deduplication as described herein can provide data flow minimization for RESTful API calls, which improves user interface (UI) performance and allows multiple user interface components to be deployed separately without having to consider the API flow to the cloud service layer. It should also be understood that the technology described herein can be used as a public service layer that allows user interface components to move freely between micro applications due to the self-contained and web framework-agnostic nature of the solution. In addition, these technologies allow reducing the number of times data is re-requested from the service layer, reducing processing across the cloud stack, and making the entire solution more efficient and sustainable.

[0037] Furthermore, as described herein, modular and self-contained API wrappers can be used to encapsulate logic built on top of a base API to handle more complex API interactions and data manipulation. Specifically, RESTful methods can be handled in a common manner within the base object (e.g., coded once), with caching and retries as described herein being leveraged within these calls. This ensures that implementers of a particular API do not need to worry about or reimplement retries, deduplication, or the basic handling of RESTful methods. For example, a particular API may involve a POST operation that then asynchronously returns a file download URL. In some embodiments, the entire call flow can be encapsulated in an API wrapper at once, eliminating the need for developers to reimplement this behavior. In another exemplary embodiment, a user identifier may be returned by a particular API, but ultimately the developer expects the user's name. In such embodiments, appropriate API calls can be paired with a cache to expose a username response, rather than a user identifier, ensuring universal, consistent, and streamlined coding from the user interface developer's perspective. It should also be understood that API wrappers allow APIs to be grouped together and written into separate files, so that only the APIs required by the user interface are incorporated into the wrapper, minimizing size. Additionally, developers familiar with a specific API can write a handler once to account for any special caching or rate-limiting behaviors that should be considered during development, reducing code duplication and errors.

[0038] As described below, for example in Figures 3 and 4 In method 300, configurable caching, deduplication, and / or 429 logic may be used in conjunction with one another. For example, a parameterized configurable cache may be utilized because RESTful API data responses are highly dependent on the request context, query, and / or body. Thus, the configurable cache utilizes various input parameters and controls over the lifespan of cache entries (e.g., time to live (TTL)) to allow global, grouped, or individual settings for requests. In other words, in some embodiments, the cache may be configured per application, per endpoint, and / or per individual type of request. Additionally, the service may have a default configuration of TTL set at the page level, which may be overridden individually.

[0039] In some embodiments, cache grouping can be used to group data sets together, which allows set operations to be performed on the cache, such as manual invalidation of linked data. It should be understood that such features may be useful in managing sets of interrelated requests, such as a set of requests to retrieve information from or about a single entity. For example, cache grouping can allow subsequent steps of a request that is known to affect the cache entries of the request to change those cache entries as a single operation rather than separate cache entry operations. Thus, cache grouping can allow for faster writes and / or otherwise more efficient cache operations. Additionally, rate limit grouping and maximum cache aging parameters can also be used, as described in more detail below.

[0040] Retry logic with advanced 429 logic can be implemented to handle the varying and sometimes interconnected rate limits of a microservice architecture. For example, rate limit grouping can be used in conjunction with advanced 429 logic to "forward" 429 errors to all affected requests, which prevents requests from being made within a group that has received a 429 status code before the associated wait time has elapsed. It should be understood that each API endpoint can define a separate rate limit, and that the rate limits and rate limit groupings can depend on the backend service. For example, when developing an API wrapper, the specific API endpoints associated with or assigned to a particular rate limit grouping can be predefined by domain experts. The speed and efficiency of the user interface can therefore be improved by preventing requests from failing to bind. In addition, as described in more detail below, the 429 logic can cause requests blocked by a 429 status code to be merged, reducing instances of repeated 429 errors for waiting requests.

[0041] It should be understood that different parts of an API may have different retry logic. For example, some APIs may consider different error codes to be retryable due to specific implementations. Retry groups can be used to group API requests that share a common retry behavior. Doing so allows for the provision of scalable retry logic that can be customized to suit the needs of a group of endpoints. For example, a group of requests with a shared error can be repaired by an additional request as part of the retry process. In various embodiments, for each retry group in the retry group, it can be defined what constitutes a retryable error, as well as the specific actions and / or other characteristics of the retry and error to be performed.

[0042] As described in more detail below, the system can utilize deduplication techniques to merge identical requests into a single request over a configurable period of time so that these requests are only processed once. In other words, the deduplication techniques effectively act as a short-term deduplication cache. Although not necessarily a "cache" in nature, for the sake of simplicity and brevity of description, a deduplication cache is described herein as such. In an exemplary embodiment, the deduplication cache stores commitments associated with data requests, rather than the actual data objects requested (which may be stored in a "primary cache" or other memory). Deduplication ensures that requests do not spam the back-end server if the user interface elements are not debounced. This allows different user interface components to be developed independently without having to know about each other and without having to consider making multiple identical calls to the back-end server.

[0043] In some embodiments, page configuration settings can be established at the page level, but can also be overridden if a specific application requires customized settings. Depending on the specific application, micro-applications can utilize different behavioral patterns. For example, it may be desired that data be cached for only 10 seconds instead of 120 seconds. Therefore, the page can configure the behavioral pattern at the highest level, and it affects all components individually. Therefore, even if more than a dozen components are developed, there is no need to know the cache window because the page knows the cache window and attributes it to each component (for example, if it is not overridden at a lower level, the high-level configuration is pushed down). In this way, it should be understood that the page configuration scheme can provide a default configuration for rapid development while also allowing personalized configuration of feature toggles, licensing services, data services, notification services, retries, deduplication cleaning, application-level caches, and / or other features or parameters.

[0044] Now refer to Figure 1 , a system 100 for configurable caching, deduplication, and rate-limiting processing of user interface data requests may include a front-end system 102, a network 104, and a back-end system 106. Additionally, in the exemplary embodiment, the front-end system 102 includes a primary cache 108 and a deduplication cache 110, and the back-end system 106 includes an application programming interface (API) 112.

[0045] Despite Figure 1In the exemplary embodiment of the system 100, only one front-end system 102, one network 104, and one back-end system 106 are shown, but in other embodiments, the system 100 may include multiple front-end systems 102, networks 104, and / or back-end systems 106. For example, in some embodiments, multiple back-end systems 106 may be used to perform the various functions described herein. In addition, in some embodiments, one or more systems described herein may be excluded from the system 100, one or more systems described as independent may form part of another system, and / or one or more systems described as forming part of another system may be independent. In addition, it should be understood that the system 100 described herein may be independent of the network 104. Figure 1 The system architecture represented by system 100 in FIG. 1 is for illustrative purposes only, and system 100 may include additional and / or alternative devices / systems in other system architectures. Furthermore, it should be understood that each of front-end system 102, network 104, and / or back-end system 106 may include multiple devices and / or subsystems. For example, in some system architectures, back-end system 106 may include at least an application server and a database server. In cloud-based embodiments, front-end system 102 and / or back-end system 106, or portions thereof, may be embodied as server-agnostic computing solutions similar to those described below.

[0046] The front-end system 102 may be embodied as any one or more types of devices / systems capable of performing the functions described herein. For example, in an exemplary embodiment, the front-end system 102 includes a client device configured to launch a user interface with which a user can interact. When a user interacts with the user interface, elements of the user interface may initiate data requests directed to specific APIs 112 of the back-end system 106 (e.g., in response to a user clicking a user interface element such as a button). In an exemplary embodiment, the front-end system 102 includes a primary cache 108 and a deduplication cache 110.

[0047] The primary cache 108 is configured to store data results (e.g., data objects) for data requests made via a user interface, while the deduplication cache 110 stores promises, which essentially only indicate that it will have results at some point in time, but do not reference the primary cache 108. In some embodiments, as described herein, the deduplication cache 110 stores data returned in response to a previous data request with a configurable timeout. It should be understood that a promise is an object that represents the eventual completion or failure of an asynchronous operation and its resulting value, rather than the value itself. As described below, the deduplication cache 110 can be utilized to ensure that if multiple identical data requests are made within a short predefined time period, the data requests are merged into a single request, thereby avoiding spamming the backend system 106. It should be understood that each of the primary cache 108 and the deduplication cache 110 can be embodied as or include any type of memory, data storage device, and / or data structure for storing data and performing the functions described herein. Furthermore, although the primary cache 108 and the deduplication cache 110 are described as forming part of the front-end system 102, it should be understood that in other embodiments, the primary cache 108 and / or the deduplication cache 110 may form part of one or more other devices / systems of the system 100.

[0048] The network 104 may be embodied as any one or more types of communication networks capable of facilitating communication between various devices communicatively connected via the network 104. Thus, the network 104 may include one or more networks, routers, switches, access points, hubs, computers, and / or other intermediate network devices. For example, the network 104 may be embodied as or otherwise include one or more cellular networks, telephone networks, local or wide area networks, publicly available global networks (e.g., the Internet), ad hoc networks, short-range communication links, or combinations thereof. In some embodiments, the network 104 may include a circuit-switched voice or data network, a packet-switched voice or data network, and / or any other network capable of carrying voice and / or data. Specifically, in some embodiments, the network 104 may include an Internet Protocol (IP)-based and / or an Asynchronous Transfer Mode (ATM)-based network. In some embodiments, the network 104 may handle voice traffic (e.g., via a Voice over IP (VOIP) network), web traffic (e.g., such as Hypertext Transfer Protocol (HTTP) traffic and Hypertext Markup Language (HTML) traffic), and / or other network traffic depending on the specific embodiment of the system 100 and / or the devices communicating with each other. In various embodiments, the network 104 may include analog or digital wired and wireless networks (e.g., IEEE 802.11 networks, public switched telephone networks (PSTN), integrated services digital networks (ISDN), and digital subscriber lines (xDSL)), third generation (3G) mobile telecommunication networks, fourth generation (4G) mobile telecommunication networks, fifth generation (5G) mobile telecommunication networks, wired Ethernet networks, private networks (e.g., such as intranets), radio, television, cable, satellite, and / or any other delivery or tunneling mechanism for carrying data, or any suitable combination of such networks. The network 104 may enable connectivity between the various devices / systems 102, 106 of the system 100. It should be understood that, depending on the source and / or destination device / system 102, 106, the various devices / systems 102, 106 may communicate with each other via different networks 104.

[0049] The backend system 106 may be embodied as any one or more types of devices / systems capable of performing the functions described herein. As shown, the backend system 106 includes various APIs 112 and corresponding API endpoints from which the frontend system 102 makes data requests. As described above, it should be understood that the number and type of devices included in the backend system 106 may vary depending on the specific architecture of the system 100. It should be understood that each of the frontend system 102, the network 104, and the backend system 106 may be embodied as any type of device / system, collection of devices / systems, and / or portions thereof suitable for performing the functions described herein (e.g., Figure 2 computing device 200), or executed by, or forming a part of, or associated with.

[0050] Now refer to Figure 2 , a simplified block diagram of at least one embodiment of a computing device 200 is shown. The exemplary computing device 200 depicts at least one embodiment of each of the computing devices, systems, servers, controllers, switches, gateways, engines, modules, and / or computing components described herein (e.g., for the sake of brevity of description, they may be interchangeably referred to as computing devices, servers, or modules). For example, various computing devices may be processes or threads running on one or more processors of one or more computing devices 200 that execute computer program instructions and interact with other system modules to perform the various functions described herein. Unless otherwise expressly limited, the functions described with respect to multiple computing devices may be integrated into a single computing device, or the various functions described with respect to a single computing device may be distributed across several computing devices. Furthermore, with respect to the computing systems described herein, depending on the particular embodiment, the various servers and their computing devices may be located on a local computing device 200, a remote computing device 200 (e.g., off-site or in a cloud-based or cloud computing environment, e.g., in a remote data center connected via a network), or some combination thereof. In some embodiments, the functionality provided by a server on a computing device located off-site may be accessed and provided via a virtual private network (VPN) as if such server were on-site, or the functionality may be provided using Software as a Service (SaaS) (accessed over the Internet using various protocols), such as by exchanging data via Extensible Markup Language (XML) and JSON, and / or the functionality may be accessed / utilized in other manners.

[0051] In some embodiments, the computing device 200 may be embodied as a server, a desktop computer, a laptop computer, a tablet computer, a notebook computer, a netbook computer, an Ultrabook computer, or a laptop computer. TM , cellular telephones, mobile computing devices, smartphones, wearable computing devices, personal digital assistants, Internet of Things (IoT) devices, processing systems, wireless access points, routers, gateways and / or any other computing devices, processing devices and / or communication devices capable of performing the functions described herein.

[0052] The computing device 200 includes a processing device 202 that executes algorithms and / or processes data according to operating logic 208, an input / output device 204 that enables communication between the computing device 200 and one or more external devices 210, and a memory 206 that stores data received from the external devices 210, for example, via the input / output device 204.

[0053] The input / output devices 204 allow the computing device 200 to communicate with external devices 210. For example, the input / output devices 204 may include a transceiver, a network adapter, a network card, an interface, one or more communication ports (e.g., a USB port, a serial port, a parallel port, an analog port, a digital port, VGA, DVI, HDMI, FireWire, CAT 5, or any other type of communication port or interface), and / or other communication circuitry. Depending on the particular computing device 200, the communication circuitry of the computing device 200 may be configured to use any one or more communication technologies (e.g., wireless or wired communication) and associated protocols (e.g., Ethernet, WiMAX, etc.) to enable such communications. Input / output devices 204 may include hardware, software, and / or firmware suitable for performing the techniques described herein.

[0054] The external device 210 can be any type of device that allows data to be input or output from the computing device 200. For example, in various embodiments, the external device 210 can be embodied as one or more of the devices / systems described herein and / or a portion thereof. Furthermore, in some embodiments, the external device 210 can be embodied as another computing device, a switch, a diagnostic tool, a controller, a printer, a display, an alarm, a peripheral device (e.g., a keyboard, a mouse, a touch screen display, etc.), and / or any other computing device, processing device, and / or communication device capable of performing the functions described herein. Furthermore, in some embodiments, it should be understood that the external device 210 can be integrated into the computing device 200.

[0055] The processing device 202 may be embodied as any type of processor capable of performing the functions described herein. In particular, the processing device 202 may be embodied as one or more single-core or multi-core processors, microcontrollers, or other processors or processing / control circuits. For example, in some embodiments, the processing device 202 may include or be embodied as an arithmetic logic unit (ALU), a central processing unit (CPU), a digital signal processor (DSP), a graphics processing unit (GPU), a field programmable gate array (FPGA), an application-specific integrated circuit (ASIC), and / or another suitable processor. The processing device 202 may be a programmable type, a dedicated hard-wired state machine, or a combination thereof. In various embodiments, a processing device 202 with multiple processing units may utilize distributed, pipelined, and / or parallel processing. In addition, the processing device 202 may be dedicated to the execution of only the operations described herein, or may be utilized in one or more additional applications. In an exemplary embodiment, the processing device 202 is programmable and executes algorithms and / or processes data according to the operating logic 208, as defined by programming instructions (such as software or firmware) stored in the memory 206. Additionally or alternatively, the operating logic 208 for the processing device 202 may be at least partially defined by hardwired logic or other hardware. Furthermore, the processing device 202 may include one or more components of any type suitable for processing signals received from the input / output device 204 or from other components or devices and providing desired output signals. Such components may include digital circuitry, analog circuitry, or a combination thereof.

[0056] The memory 206 may be one or more types of non-transitory computer-readable media, such as solid-state memory, electromagnetic memory, optical memory, or a combination thereof. Furthermore, the memory 206 may be volatile and / or non-volatile, and in some embodiments, some or all of the memory 206 may be of a portable type, such as a disk, tape, memory stick, cassette, and / or other suitable portable memory. In operation, the memory 206 may store various data and software used during operation of the computing device 200, such as an operating system, applications, programs, libraries, and drivers. It should be understood that in addition to or in lieu of storing programming instructions defining the operating logic 208, the memory 206 may store data manipulated by the operating logic 208 of the processing device 202, such as, for example, data representing signals received from and / or sent to the input / output device 204. As Figure 2 As shown, depending on the particular embodiment, memory 206 can be included with and / or coupled to processing device 202. For example, in some embodiments, processing device 202, memory 206, and / or other components of computing device 200 can form part of a system on a chip (SoC) and be incorporated onto a single integrated circuit chip.

[0057] In some embodiments, various components of computing device 200 (e.g., processing device 202 and memory 206) may be communicatively coupled via an input / output subsystem, which may be embodied as circuitry and / or components to facilitate input / output operations with processing device 202, memory 206, and other components of computing device 200. For example, the input / output subsystem may be embodied as or otherwise include a memory controller hub, an input / output control hub, a firmware device, communication links (i.e., point-to-point links, bus links, wires, cables, optical guides, printed circuit board traces, etc.), and / or other components and subsystems to facilitate input / output operations.

[0058] In other embodiments, the computing device 200 may include other or additional components, such as those commonly found in typical computing devices (e.g., various input / output devices and / or other components). It should also be understood that one or more of the components of the computing device 200 described herein may be distributed across multiple computing devices. In other words, the techniques described herein may be employed by a computing system that includes one or more computing devices. Additionally, although Figure 2 2 , only a single processing device 202, I / O device 204, and memory 206 are illustratively shown, but it should be understood that in other embodiments, a particular computing device 200 may include multiple processing devices 202, I / O devices 204, and / or memories 206. Furthermore, in some embodiments, more than one external device 210 may be in communication with the computing device 200.

[0059] The computing device 200 can be one of a plurality of devices connected to other systems / sources via a network or via a network. The network can be embodied as any one or more types of communication networks capable of facilitating communication between various devices connected via network communication. Thus, the network can include one or more networks, routers, switches, access points, hubs, computers, client devices, terminals, nodes, and / or other intermediate network devices. For example, the network can be embodied as or otherwise include one or more cellular networks, telephone networks, local area networks or wide area networks, publicly available global networks (e.g., the Internet), ad hoc networks, short-range communication links, or combinations thereof. In some embodiments, the network can include a circuit-switched voice or data network, a packet-switched voice or data network, and / or any other network capable of carrying voice and / or data. In particular, in some embodiments, the network can include a network based on Internet Protocol (IP) and / or based on Asynchronous Transfer Mode (ATM). In some embodiments, the network can handle voice traffic (e.g., via a Voice over IP (VOIP) network), web traffic, and / or other network traffic depending on the specific implementation of the systems and / or devices communicating with each other. In various embodiments, the network may include analog or digital wired and wireless networks (e.g., IEEE 802.11 networks, public switched telephone networks (PSTN), integrated services digital networks (ISDN), and digital subscriber lines (xDSL)), third generation (3G) mobile telecommunication networks, fourth generation (4G) mobile telecommunication networks, fifth generation (5G) mobile telecommunication networks, wired Ethernet networks, private networks (e.g., such as intranets), radio, television, cable, satellite, and / or any other delivery or tunneling mechanism for carrying data, or any suitable combination of such networks. It should be understood that the various devices / systems may communicate with each other via different networks depending on the source and / or destination device.

[0060] It should be understood that the computing device 200 can communicate with other computing devices 200 via any type of gateway or tunneling protocol (such as secure socket layer or transport layer security). The network interface may include a built-in network adapter (such as a network interface card) that is suitable for connecting the computing device to any type of network capable of performing the operations described herein. In addition, the network environment can be a virtual network environment in which various network components are virtualized. For example, the various machines can be virtual machines implemented as software-based computers running on physical machines. The virtual machines can share the same operating system, or in other embodiments, different operating systems can be run on each virtual machine instance. For example, a "hypervisor" type of virtualization is used, in which multiple virtual machines run on the same host physical machine, and each virtual machine acts as if it has its own dedicated box. In other embodiments, other types of virtualization can be employed, such as, for example, networks (e.g., via software-defined networking) or functions (e.g., via network function virtualization).

[0061] Thus, one or more of the computing devices 200 described herein may be embodied as or form part of one or more cloud-based systems. In a cloud-based embodiment, the cloud-based system may be embodied as a server-obfuscated computing solution that, for example, executes multiple instructions on demand, contains logic to execute instructions only when prompted by specific activities / triggering events, and consumes no computing resources when not in use. That is, the system may be embodied as a virtual computing environment (e.g., a distributed network of devices) residing "on" the computing system, wherein various virtual functions (e.g., Lambda Functions, Azure Functions, Google Cloud Functions, and / or other suitable virtual functions) may be executed corresponding to the functions of the system described herein. For example, when an event occurs (e.g., data is transferred to the system for processing), the virtual computing environment may be communicated with (e.g., via a request to an API of the virtual computing environment), whereby the API may route the request to the correct virtual function (e.g., a specific server-obfuscated computing resource) based on a set of rules. Thus, when a user makes a request for a data transfer (e.g., via an appropriate user interface to the system), the appropriate virtual function may be executed to perform an action before the instance of the virtual function is eliminated.

[0062] Now refer to Figures 3 and 4 In use, the system 100 may perform a method 300 for configurable caching, deduplication, and rate limiting of user interface data requests. It should be understood that, unless otherwise noted, specific blocks of the method 300 are shown by way of example, and such blocks may be combined or divided, added or removed, and / or reordered in whole or in part depending on the specific implementation. Because the system 100 may have a variety of different system architectures, for the sake of simplicity and brevity of description, Figures 3 and 4 The blocks of method 300 are primarily described as being simply performed by system 100 .

[0063] The exemplary method 300 is to Figure 3 , where a user interface element initiates a data request. It should be understood that the data request may reference a relevant API endpoint / URL and include any relevant parameters / data. In box 304, the system 100 determines whether the deduplication cache 110 includes a deduplication entry associated with the data request (e.g., indicating that a previous identical data request was made within a configurable threshold time period). For example, as described above, the deduplication cache 110 may be configured to store commitments associated with corresponding data requests. Thus, the various deduplication entries in the deduplication cache 110 indicate previous data requests made within a threshold time period (i.e., an expiration time period for the deduplication entry, after which it may be removed from the deduplication cache 110). For example, in one embodiment, the threshold time period may be 10ms, while in another embodiment, the threshold may be 100ms. It should be understood that certain HTTP methods are considered non-idempotent and are not typically cached. However, in an exemplary embodiment, the deduplication cache 110 still caches them, in a sense that the request or commitment itself is cached rather than the actual result data. It will be appreciated that the deduplication cache 110 is utilized to ensure that duplicate data requests are not made within a predefined period of time (eg, due to a user inadvertently clicking rapidly and repeatedly on elements of a user interface).

[0064] If the system 100 determines in block 304 that an associated deduplication entry exists in the deduplication cache 110, the method 300 proceeds to block 316, where a data request is made and the data is returned to the user interface element. In other words, if an associated deduplication entry exists in the deduplication cache 110, the data request has been made, and the system 100 can wait for the requested data to be returned. However, if the system 100 determines in block 304 that the deduplication cache 110 does not include an associated deduplication entry, the method 300 proceeds to block 306, where the system 100 determines whether the primary cache 108 includes a cache entry associated with the data request.

[0065] If the system 100 determines in block 306 that the primary cache 108 includes a cache entry associated with the data request, the method 300 proceeds to block 308, where the system 100 determines whether the data request is associated with a predefined maximum cache age. In other words, the system 100 determines whether the data request includes a maximum cache age parameter, which overrides any expiration time provided by the cache entry itself. It should be understood that different components may have different interpretations of what is acceptable in terms of the TTL for a data response. For example, one component may only accept data that is at most two minutes old, while another component may accept data that is at most ten minutes old.

[0066] If the system 100 determines in block 308 that the data request is associated with a predefined maximum cache age, the method 300 proceeds to block 310, where the system 100 uses the maximum cache age to calculate the expiration of the cache entry. More specifically, the system 100 may determine whether the cache entry associated with the data request has expired based on the creation time of the cache entry and a predefined maximum cache age (e.g., the expiration is the maximum cache age after the creation time of the cache entry). However, if the system 100 determines in block 308 that the data request is not associated with a predefined maximum cache age, the method 300 proceeds to block 312, where the system 100 uses the expiration of the cache entry associated with the data request, so that the system 100 can determine whether the cache entry associated with the data request has expired based on the expiration time defined by the cache entry itself. From blocks 310 and 312, the method 300 proceeds to block 314, where the system 100 determines whether the cache entry has expired. If not, the method 300 proceeds to block 316 where a request for data is made and the data is returned to the user interface element.

[0067] Returning to block 306, if the system 100 determines that the primary cache 108 does not include a cache entry associated with the data request, or if the system 100 determines in block 314 that a cache entry exists but has expired, the method 300 proceeds to Figure 4 The system 100 then proceeds to block 318, where it determines whether the data request is associated with a rate-limited API endpoint group. As described above, rate-limited API endpoint groups can be predefined by the system 100 to help avoid transmitting certain unnecessary data requests. For example, in an embodiment, the GET_USER and GET_USERS endpoints can belong to the same rate-limited group, such that if the system 100 already knows that the GET_USER request has been rate-limited, the GET_USERS request should be postponed to avoid the inevitable 429 error.

[0068] If the system 100 determines in block 318 that the data request is associated with a rate limited group, the method 300 proceeds to block 320 where the system 100 delays for a corresponding server retry period before making the data request to the backend system 106 in block 322. If the system 100 determines in block 318 that the data request is not associated with a rate limited group, the method 300 proceeds to block 322 where the data request may be made without delay.

[0069] In block 324, the system 100 determines whether the data request was successful. If so, the method 300 returns to Figure 3 316, in which a data request is made and the data is returned to the user interface element. However, if the data request fails, an error or status code associated with the failure to satisfy the data request may be provided, and method 300 proceeds to box 326, in which the system 100 determines whether the error associated with the data request is a retryable error. If not, method 300 proceeds to box 328, in which the data request failure is returned to the user interface element. However, if the system 100 determines in box 326 that the error is retryable, method 300 proceeds to box 330, in which system 100 determines whether a predefined retry limit has been reached. It should be understood that the predefined retry limit may be configurable and may define the maximum number of retry attempts that can be made. In some embodiments, system 100 may utilize different retry attempts for different situations (e.g., different API endpoints, different error types, etc.). If the predefined retry limit has been reached, method 300 proceeds to box 328, in which the data request failure is returned to the user interface element.

[0070] However, if the predefined retry limit has not been reached, the system 100 may retry the data request. More specifically, in the exemplary embodiment, the method 300 proceeds to block 332, where the system 100 determines whether the retryable error is a 429 error (e.g., associated with an HTTP 429 Too Many Requests status code or error). If so, the method 300 proceeds to block 320, where the system 100 delays the corresponding server retry period (e.g., provided by the 429 error) before retrying the data request. However, if the retryable error is not a 429 error, the method 300 proceeds to block 322, where the data request may be made without delay.

[0071] Although blocks 302 through 332 are described in a relatively serial manner, it should be understood that in some embodiments, the various blocks of method 300 may be performed in parallel.

Claims

1. A method for configurable caching, deduplication, and rate-limiting processing of user interface data requests, the method comprising: receiving data requests initiated by user interface elements of the user interface; determining whether a deduplication cache includes a deduplication entry associated with the data request; In response to determining that the deduplication cache does not include the deduplication entry, determining whether a primary cache different from the deduplication cache includes a cache entry associated with the data request; determining whether the data request is associated with a rate-limited application programming interface endpoint group in response to one of (i) determining that the primary cache does not include the cache entry associated with the data request or (ii) determining that the cache entry associated with the data request has expired; as well as In response to determining that the data request is associated with the rate limited group, the data request is made to a backend system after a predefined server retry period associated with the rate limited group has elapsed. 2 . The method of claim 1 , wherein the deduplicated entry indicates that a previous data request identical to the data request initiated by the user interface element of the user interface was made within a threshold time period.

3. The method of claim 1 , further comprising, in response to determining that the primary cache includes the cache entry associated with the data request, determining whether the data request is associated with a predefined maximum cache age.

4. The method according to claim 3 further includes, in response to determining that the data request is associated with the predefined maximum cache age, determining whether the cache entry associated with the data request has expired based on the creation time of the cache entry and the predefined maximum cache age.

5. The method according to claim 3 further includes, in response to determining that the data request is not associated with the predefined maximum cache age, determining whether the cache entry associated with the data request has expired based on the expiration time of the cache entry defined by the cache entry. 6 . The method of claim 1 , further comprising, in response to determining that the data request is not associated with any rate-limited group, immediately making the data request to the backend system.

7. The method according to claim 1, further comprising: receiving an error associated with the data request in response to a failure to satisfy the data request; determining whether the error associated with the data request is a retryable error; In response to determining that the error associated with the data request is the retryable error, determining whether a predefined retry limit has been reached; as well as In response to determining that the predefined retry limit has not been reached, the data request is retried.

8. The method of claim 7, wherein retrying the data request comprises retrying the data request after the predefined server retry period in response to determining that the retryable error is an HTTP 429 error.

9. The method of claim 7, wherein the predefined retry limit is a maximum number of retry attempts that can be made.

10. The method of claim 1, wherein determining whether the data request is associated with the rate-limited API endpoint group comprises determining whether an API endpoint to which the data request is directed is included in the rate-limited API endpoint group.

11. The method of claim 1 , wherein the deduplication cache is configured to store one or more commitments, wherein each of the one or more commitments is associated with a previous corresponding data request.

12. The method of claim 1, wherein the deduplication cache is configured to store data returned to a previous data request with a configurable timeout.

13. The method of claim 1, wherein the primary cache is configured to store data returned in response to a previous data request.

14. A system for configurable caching, deduplication, and rate-limiting processing of user interface data requests, the system comprising: Deduplication cache; a primary cache distinct from the deduplication cache; at least one processor; as well as at least one memory including a plurality of instructions stored thereon, the plurality of instructions, in response to being executed by the at least one processor, causing the system to: receiving data requests initiated by user interface elements of the user interface; determining whether the deduplication cache includes a deduplication entry associated with the data request; responsive to determining that the deduplication cache does not include the deduplication entry, determining whether the primary cache includes a cache entry associated with the data request; determining whether the data request is associated with a rate-limited application programming interface endpoint group in response to one of (i) determining that the primary cache does not include the cache entry associated with the data request or (ii) the cache entry associated with the data request has expired; as well as In response to determining that the data request is associated with the rate limited group, the data request is made to a backend system after a predefined server retry period associated with the rate limited group has elapsed. 15 . The system of claim 14 , wherein the deduplicated entry indicates that a previous data request identical to the data request initiated by the user interface element of the user interface was made within a threshold time period.

16. The system of claim 14, wherein the plurality of instructions further cause the system to: In response to determining that the primary cache includes the cache entry associated with the data request, determining whether the data request is associated with a predefined maximum cache age; In response to determining that the data request is associated with the predefined maximum cache age, determining whether the cache entry associated with the data request has expired based on a creation time of the cache entry and the predefined maximum cache age; as well as In response to determining that the data request is not associated with the predefined maximum cache age, a determination is made as to whether the cache entry associated with the data request has expired based on an expiration time of the cache entry defined by the cache entry.

17. The system of claim 14, wherein the plurality of instructions further cause the system to: receiving an error associated with the data request in response to a failure to satisfy the data request; determining whether the error associated with the data request is a retryable error; In response to determining that the error associated with the data request is the retryable error, determining whether a predefined retry limit has been reached; as well as In response to determining that the predefined retry limit has not been reached, the data request is retried.

18. The system of claim 17, wherein retrying the data request comprises retrying the data request after the predefined server retry period in response to determining that the retryable error is an HTTP 429 error.

19. The system of claim 17, wherein the predefined retry limit is a maximum number of retry attempts that can be made.

20. The system of claim 14, wherein the deduplication cache is configured to store one or more commitments, wherein each of the one or more commitments is associated with a previous corresponding data request; and The primary cache is configured to store data returned in response to a previous data request.

Images