Configurable caching, duplication prevention, and rate limiting techniques

Abstract

In accordance with an embodiment, a method for configurable caching, duplication prevention, and rate limiting processing of user interface data requests includes receiving a data request initiated by a user interface element of a user interface; determining whether an anti-duplication cache includes an anti-duplication entry associated with the data request; in response to determining that the anti-duplication cache does not include the anti-duplication entry, determining whether a main cache includes a cache entry associated with the data request; in response to determining that the main cache does not include the cache entry or that the cache entry has expired, determining whether the data request is associated with a rate limiting group of an application programming interface endpoint; and in response to determining that the data request is associated with the rate limiting group, making the data request to a backend system after a predetermined server retry period associated with the rate limiting group.

Description

[Technical Field]

[0001] (Priority Claim) This application claims the benefit of U.S. Patent Application No. 18 / 345,243, filed June 30, 2023, entitled "TECHNOLOGIES FOR Configurable Caching, Deduplication, and Rate Limit Handling." [Background technology]

[0002] Web development traditionally follows a pattern of building single-page applications (SPAs) or multi-page applications, where each page implements its own handling of application programming interface (API) requests, caching, configuration, permissions, events, and logs. While page-level data stores are a common solution, they are tied to specific frameworks and require modifications when implementing solutions across multiple frameworks. This approach leads to code duplication across multiple small repositories or to all code relying on processes located in the same repository. Code duplication leads to duplication of effort during debugging, dependency upgrades, or routine maintenance, as well as inconsistencies across pages. On the other hand, having all code rely on a service layer in a single repository leads to a large, complex, and difficult-to-manage repository, slowing down the process of maintaining existing features and the development of new ones. Components also must be rebuilt when moved between pages and user interface frameworks (e.g., due to differences in page- or framework-level logic implementation). Summary of the Invention

[0003] One embodiment is directed to unique systems, components, and methods for configurable caching, duplication prevention, and rate limiting of user interface data requests. Other embodiments are directed to apparatus, systems, devices, hardware, methods, and combinations thereof for configurable caching, duplication prevention, and rate limiting of user interface data requests.

[0004] According to an embodiment, a method for configurable caching, duplication prevention, and rate limiting processing of user interface data requests may include receiving a data request initiated by a user interface element of a user interface; determining whether an anti-duplication cache includes an anti-duplication entry associated with the data request; in response to determining that the anti-duplication cache does not include the anti-duplication entry, determining whether a main cache, different from the anti-duplication cache, includes a cache entry associated with the data request; in response to one of (i) determining that the main cache does not include a cache entry associated with the data request or (ii) determining that the cache entry associated with the data request has expired, determining whether the data request is associated with a rate limiting group of an application programming interface endpoint; and in response to determining that the data request is associated with the rate limiting group, making the data request to a backend system after a predetermined server retry period associated with the rate limiting group.

[0005] In some embodiments, the anti-duplicate entry may indicate that a previous data request identical to the data request initiated by a user interface element of the user interface was made within a threshold period of time.

[0006] In some embodiments, the method may further include, in response to determining that the main cache contains a cache entry associated with the data request, determining whether the data request is associated with a predetermined maximum cache retention period.

[0007] In some embodiments, the method may further include, upon determining that the data request is associated with a predetermined maximum cache retention period, determining whether a cache entry associated with the data request has expired based on the creation date and time of the cache entry and the predetermined maximum cache retention period.

[0008] In some embodiments, the method may further include, in response to determining that the data request is not associated with a predetermined maximum cache retention period, determining whether a cache entry associated with the data request has expired based on an expiration date and time of the cache entry defined by the cache entry.

[0009] In some embodiments, the method may include immediately making the data request to the backend system in response to determining that the data request is not associated with any rate limiting group.

[0010] In some embodiments, the method may further include, in response to a failure to fulfill the data request, receiving an error associated with the data request; determining whether the error associated with the data request is a retryable error; in response to determining that the error associated with the data request is a retryable error, determining whether a predetermined retry limit has been reached; and in response to determining that the predetermined retry limit has not been reached, retrying the data request.

[0011] In some embodiments, retrying the data request may include retrying the data request after a predetermined server retry period in response to determining that the retryable error is a 429 error.

[0012] In some embodiments, the predetermined retry limit may be the maximum number of retry attempts that can be made.

[0013] In some embodiments, determining whether the data request is associated with a rate limiting group of application programming interface endpoints may include determining whether the application programming interface endpoint to which the data request is directed is included in the rate limiting group of application programming interface endpoints.

[0014] In some embodiments, the anti-duplication cache may be configured to store one or more probabilities, each of the one or more probabilities being associated with a respective previous data request.

[0015] In some embodiments, the anti-duplication cache may be configured to store data returned in response to a previous data request with a configurable timeout.

[0016] In some embodiments, the main cache may be configured to store data returned in response to a previous data request.

[0017] According to another embodiment, a system for configurable caching, duplication prevention, and rate limiting processing of user interface data requests may include an anti-duplication cache, a main cache distinct from the anti-duplication cache, at least one processor, and at least one memory storing a plurality of instructions that, when executed by the at least one processor, cause the system to receive a data request initiated by a user interface element of a user interface, determine whether the anti-duplication cache includes an anti-duplication entry associated with the data request, determine whether the main cache includes a cache entry associated with the data request upon a determination that the anti-duplication cache does not include an anti-duplication entry, determine whether the data request is associated with a rate limiting group of an application programming interface endpoint upon a determination that the data request is associated with the rate limiting group, and make the data request to a backend system after a predetermined server retry period associated with the rate limiting group upon a determination that the data request is associated with the rate limiting group.

[0018] In some embodiments, the anti-duplicate entry may indicate that a previous data request identical to the data request initiated by a user interface element of the user interface was made within a threshold period of time.

[0019] In some embodiments, the instructions may further cause the system, upon a determination that the main cache includes a cache entry associated with the data request, to determine whether the data request is associated with a predetermined maximum cache retention period; upon a determination that the data request is associated with the predetermined maximum cache retention period, to determine whether the cache entry associated with the data request has expired based on a creation date and time of the cache entry and the predetermined maximum cache retention period; and, upon a determination that the data request is not associated with the predetermined maximum cache retention period, to determine whether the cache entry associated with the data request has expired based on an expiration date and time of the cache entry defined by the cache entry.

[0020] In some embodiments, the instructions may further cause the system, in response to a failure to fulfill the data request, to receive an error associated with the data request; determine whether the error associated with the data request is a retryable error; in response to a determination that the error associated with the data request is a retryable error, determine whether a predetermined retry limit has been reached; and in response to a determination that the predetermined retry limit has not been reached, to retry the data request.

[0021] In some embodiments, retrying the data request may include retrying the data request after a predetermined server retry period upon determining that the retryable error is a 429 error.

[0022] In some embodiments, the predetermined retry limit may be the maximum number of retry attempts that can be made.

[0023] In some embodiments, the anti-duplication cache may be configured to store one or more probabilities, each of which is associated with a respective previous data request, and the main cache may be configured to store data returned in response to the previous data request.

[0024] This Summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to be used as an aid in limiting the scope of the claimed subject matter. Further embodiments, forms, features, and aspects of the present application will become apparent from the description and figures provided herewith. [Brief explanation of the drawings]

[0025] The concepts described herein are by way of example and not limitation in the accompanying drawings. For simplicity and clarity of illustration, elements illustrated in the figures have not necessarily been drawn to scale. Where considered appropriate, reference labels have been repeated among the figures to indicate corresponding or analogous elements. [Figure 1] 1 depicts a simplified block diagram of at least one embodiment of a system for configurable caching, duplicate prevention, and rate limiting of user interface data requests. [Figure 2] FIG. 1 is a simplified block diagram of at least one embodiment of a computing device. [Figure 3] FIG. 1 is a simplified flow diagram of at least one embodiment of a method for configurable caching, duplication prevention, and rate limiting of user interface data requests. [Figure 4] FIG. 1 is a simplified flow diagram of at least one embodiment of a method for configurable caching, duplication prevention, and rate limiting of user interface data requests. DETAILED DESCRIPTION OF THE INVENTION

[0026] While the concepts of the present disclosure are susceptible to various modifications and alternative forms, specific embodiments have been shown by way of example in the drawings and are herein described in detail. It should be understood, however, that there is no intention to limit the concepts of the present disclosure to the particular forms disclosed, but on the contrary, the intention is to cover all modifications, equivalents, and alternatives consistent with the scope of this disclosure and the appended claims.

[0027] References herein to "one embodiment," "an embodiment," "an illustrative embodiment," and the like indicate that the described embodiment may include a particular feature, structure, or characteristic, but that all embodiments may or may not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases do not necessarily refer to the same embodiment. It should be further understood that references to "preferred" components or features may indicate the desirability of a particular component or feature with respect to an embodiment, but that the present disclosure is not so limited with respect to other embodiments that may omit such component or feature. Furthermore, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that it is within the knowledge of one skilled in the art to implement such feature, structure, or characteristic in connection with other embodiments, whether or not explicitly described. Furthermore, particular features, structures, or characteristics may be combined in any suitable combinations and / or subcombinations in various embodiments.

[0028] Additionally, it should be understood that items in a listing of the form "at least one of A, B, and C" can mean (A), (B), (C), (A and B), (B and C), (A and C), or (A, B, and C). Similarly, items listed in the form "at least one of A, B, or C" can mean (A), (B), (C), (A and B), (B and C), (A and C), or (A, B, and C). Further, with respect to the claims, use of words and phrases such as "a," "an," "at least one," and / or "at least a portion" should be construed as limiting to only one of such elements unless specifically stated to the contrary, and use of phrases such as "at least a portion" and / or "a portion" should be construed to encompass both embodiments including only a portion of such elements and embodiments including the entirety of such elements unless specifically stated to the contrary.

[0029] The disclosed embodiments may, in some cases, be implemented in hardware, firmware, software, or a combination thereof. The disclosed embodiments may also be implemented as instructions stored on or executed by one or more transitory or non-transitory machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. A machine-readable storage medium may be embodied as any storage device, mechanism, or other physical structure for storing or transmitting information in a form readable by a machine (e.g., volatile or non-volatile memory, media disk, or other media device).

[0030] In the drawings, some structural or method features may be shown in a specific arrangement and / or ordering. However, it should be understood that such specific arrangement and / or ordering may not be required. Rather, in some embodiments, such features may be arranged in a different manner and / or order than that shown in the illustrative drawings, unless indicated to the contrary. Additionally, the inclusion of a structural or method feature in a particular figure does not imply that such feature is required in all embodiments, and in some embodiments, it may not be included or may be combined with other features.

[0031] Traditional web development paradigms lead to duplication of effort, inefficiencies, and inconsistencies. Existing libraries for implementing API handling, caching, logging, and other features are generally narrowly focused on specific functionality and fail to minimize requests to cloud services and provide key features for handling 429 errors in distributed microservices. Caching solutions are very high-level and general, cannot cache based on parameters, and cannot be dynamically configured. Existing technologies result in substantial waste of development and computing resources (e.g., CPU cycles, RAM, storage, etc.). Resources are wasted through retries and inefficient processing of requests on both the front end (e.g., client devices) and back end (e.g., cloud devices). Microservices enable economies of scale, but also allow multiple developers to build and deploy the service layer, allowing for the neglect of common libraries and the leeway to reimplement and copy code. Similar benefits and challenges exist for microapps.

[0032] It should be appreciated that the techniques described herein enable common services to orchestrate the whole through request, data, and configuration services, facilitating the development of standalone components. For example, input methods can be provided for standard RESTful API calls, and integrated with these entry points can provide parameterized, configurable caching, deduplication, and logic for handling HTTP 429 Too Many Requests status codes or errors. Caching combined with request deduplication as described herein can provide data flow minimization for RESTful API calls, which improves user interface (UI) performance and allows multiple user interface components to be deployed independently without concerns about managing API flows to a cloud services layer. It should be further appreciated that the techniques described herein can be used as a common services layer, thereby allowing user interface components to move freely between microapps because of the self-contained, web-framework-agnostic solution. Additionally, the techniques enable reduced re-requesting of data from the services layer, reducing processing across the cloud stack and making the overall solution more efficient and sustainable.

[0033] Furthermore, as described herein, modular, self-contained API wrappers may be used to encapsulate logic built on top of a base API to handle more complex API interactions and data manipulation. In particular, RESTful methods may be handled in a generic manner (e.g., coded once) in a base object, where caching and retries, as described herein, may be leveraged in these calls. This ensures that implementers of a particular API do not need to concern themselves with or reimplement the retries, duplication prevention, or basic processing of RESTful methods. For example, a particular API may include a POST operation that asynchronously returns a file by URL. In some embodiments, the entire call flow may be encapsulated once within the API wrapper, so that developers do not need to reimplement that behavior. In another exemplary embodiment, a user identifier may be returned by a particular API, but the end developer desires the user's name. In such an embodiment, appropriate API calls may be paired with a cache to expose the response with the username rather than the user identifier, thereby ensuring common, consistent, and streamlined coding from a user interface developer's perspective. It should be further appreciated that API wrappers allow APIs to be grouped together and written to individual files, ensuring that size is minimized, so that only the APIs needed by the user interface are included in the package. Additionally, developers familiar with a particular API can write handlers once to handle any special caching or rate limiting behavior that should be considered during development, reducing code duplication and bugs.

[0034] As described below, for example, in method 300 of FIGS. 3 and 4 , configurable caching, duplicate prevention, and / or 429 logic may be used in conjunction with one another. For example, because RESTful API data responses are highly dependent on the context, query, and / or body of a request, a parameterized configurable cache may be utilized. Thus, the configurable cache utilizes various input parameters, along with control around the lifespan (e.g., time-to-live (TTL)) of cache entries, to allow global, grouped, or individual configuration for requests. In other words, in some embodiments, caching may be configured for each application, for each endpoint, and / or for each individual type of request. Additionally, a service may have a default setting for TTL set at the page level, which may be individually overridden.

[0035] In some embodiments, cache grouping may be used to group sets of data together, which allows for configuration operations on the cache, such as manual invalidation of associated data. It should be appreciated that such a feature may be useful in managing a set of interrelated requests, such as a group of requests retrieving information from or about a single entity. For example, cache grouping may allow post-request steps known to affect the request's cache entries to modify those cache entries as a single operation, instead of individual cache entry operations. Thus, cache grouping may allow for cache operations that are quicker to write and / or otherwise more efficient. Additionally, as described in more detail below, rate limit grouping and cache maximum age parameters may also be used.

[0036] Retry logic, along with advanced 429 logic, can be implemented to handle the varying and sometimes interconnected rate limits of a microservice architecture. For example, rate limit grouping can be used with advanced 429 logic to “forward” a 429 error to all affected requests, preventing requests in a group that received a 429 status code from being made before the associated wait time has elapsed. It should be understood that each API endpoint may define a separate rate limit, and rate limits and rate limit groupings can depend on the backend service. The specific API endpoints associated with or assigned to specific rate limit groupings can be predefined by domain experts, for example, when the API wrapper is developed. This can improve the speed and efficiency of the user interface by preventing requests that would otherwise fail from being made. Additionally, as described in more detail below, 429 logic can consolidate requests blocked by a 429 status code, reducing instances of duplicate 429 errors from waiting requests.

[0037] It should be understood that different API sections may have different retry logic. For example, some APIs may consider different error codes to be retryable simply due to the specific implementation. Retry groupings can be leveraged to group API requests that share a common retry behavior. Doing so allows for providing extensible retry logic that can be customized to fit the needs of a group of endpoints. For example, a group of requests with a shared error can be modified by additional requests made as part of the retry process. In various embodiments, for each retry grouping, what constitutes a retryable error can be defined, along with retries and specific actions to be taken for the error and / or other characteristics.

[0038] As described in more detail below, the system may utilize anti-duplication techniques to consolidate multiple identical requests into a single request within a configurable period of time to ensure that a request is processed only once. In other words, the replication techniques effectively function as a short-lived anti-duplication cache. While not necessarily a "cache" per se, the anti-duplication cache is described as such herein for ease of explanation and brevity. In an illustrative embodiment, the anti-duplication cache stores the likelihoods associated with data requests, rather than the actual data objects requested (which may be stored in the "main cache" or other memory). Anti-duplication ensures that requests are not excessively sent to the backend server if user interface elements are not debounced. This allows different user interface components to be developed independently, without knowledge of each other and without concern about making the same calls to the backend server multiple times.

[0039] In some embodiments, page configuration settings may be established at the page level, but may be overridden if a particular application requires custom settings. Depending on the particular application, microapps may utilize different behavior patterns. For example, it may be desirable for data to be cached for only 10 seconds rather than 120 seconds. Thus, a page may be able to set a behavior pattern at the top level, which affects all of the components individually. Thus, even if many components are developed, they do not need to know the cache window because the page knows it and attributes it to each individual component (e.g., if not overridden at a lower level, it trickles down). Thus, while the page configuration approach can provide default settings for rapid development, it should be understood that individualized feature switching, authorization services, data services, notification services, retries, anti-duplication cleanup, application-level caching, and / or other feature or parameter settings are also possible.

[0040] 1, a system 100 for configurable caching, duplication prevention, and rate limiting processing of user interface data requests may include a front-end system 102, a network 104, and a back-end system 106. Additionally, in the illustrative embodiment, the front-end system 102 includes a main cache 108 and a duplication prevention cache 110, and the back-end system 106 includes an application programming interface (API) 112.

[0041] Although the illustrative embodiment of FIG. 1 shows only one front-end system 102, one network 104, and one back-end system 106, in other embodiments, system 100 may include multiple front-end systems 102, networks 104, and / or back-end systems 106. For example, in some embodiments, multiple back-end systems 106 may be used to perform various functions described herein. Furthermore, in some embodiments, one or more of the systems described herein may be excluded from system 100, one or more of the systems described as being independent may form part of another system, and / or one or more of the systems described as forming part of another system may be independent. Additionally, it should be understood that the system architecture represented by system 100 of FIG. 1 is for illustrative purposes only, and that system 100 may include additional and / or alternative devices / systems in other system architectures. Further, it should be understood that each of the front-end system 102, the network 104, and / or the back-end system 106 may include multiple devices and / or subsystems. For example, in some system architectures, the back-end system 106 may include at least an application server and a database server. In cloud-based embodiments, the front-end system 102 and / or the back-end system 106, or portions thereof, may be embodied as a server-less computing solution similar to that described below.

[0042] The front-end system 102 may be embodied as any one or more types of devices / systems capable of performing the functions described herein. For example, in an illustrative embodiment, the front-end system 102 includes a client device configured to launch a user interface with which a user can interact. As the user interacts with the user interface, elements of the user interface may initiate data requests directed to particular APIs 112 of the back-end system 106 (e.g., in response to the user clicking a user interface element such as a button). In an illustrative embodiment, the front-end system 102 includes a main cache 108 and an anti-duplication cache 110.

[0043] The main cache 108 is configured to store data results of data requests made via a user interface (e.g., data objects), while the anti-duplication cache 110 stores probabilities, which essentially indicate that a result will be available at some point in the future, but without a reference to the main cache 108. In some embodiments, as described herein, the anti-duplication cache 110 stores data returned in response to a previous data request with a configurable timeout. It should be understood that a probability is an object that represents the eventual completion or failure of an asynchronous operation and the resulting value, rather than the value itself. As described below, the anti-duplication cache 110 can be utilized to ensure that if multiple identical data requests are made within a short, predetermined period of time, the data requests are consolidated into a single request to avoid excessive transmissions to the backend system 106. It should be understood that each of the main cache 108 and the anti-duplication cache 110 may be embodied as or include any type of memory, data storage, and / or data structure for storing data and performing the functions described herein. Furthermore, although the main cache 108 and the anti-duplication cache 110 are depicted as forming part of the front-end system 102, it should be understood that in other embodiments, the main cache 108 and / or the anti-duplication cache 110 may form part of one or more other devices / systems of the system 100.

[0044] Network 104 may be embodied as any one or more types of communications network capable of facilitating communication between various devices communicatively connected via network 104. Accordingly, network 104 may include one or more networks, routers, switches, access points, hubs, computers, and / or other intervening network devices. For example, network 104 may be embodied as or otherwise include one or more mobile communications networks, telephone networks, local or wide area networks, publicly available global networks (e.g., the Internet), ad hoc networks, short-range communications links, or combinations thereof. In some embodiments, network 104 may include circuit-switched voice or data networks, packet-switched voice or data networks, and / or any other networks capable of carrying voice and / or data. In particular, in some embodiments, network 104 may include Internet Protocol (IP)-based and / or asynchronous transfer mode (ATM)-based networks. In some embodiments, network 104 may handle voice traffic (e.g., via a Voice over IP (VOIP) network), web traffic (e.g., hypertext transfer protocol (HTTP) traffic and hypertext markup language (HTML) traffic), and / or other network traffic, depending on the particular embodiment and / or the devices of system 100 that communicate with each other.In various embodiments, the network 104 may include analog or digital wired and wireless networks (e.g., IEEE 802.11 networks, Public Switched Telephone Networks (PSTN), Integrated Services Digital Networks (ISDN), and Digital Subscriber Lines (xDSL)), Third Generation (3G) mobile networks, Fourth Generation (4G) mobile networks, Fifth Generation (5G) mobile networks, wired Ethernet networks, private networks (e.g., intranets, etc.), radio, television, cable, satellite, and / or any other delivery or tunneling mechanism for carrying data, or any suitable combination of such networks. The network 104 may enable connectivity between the various devices / systems 102, 106 of the system 100. It should be understood that the various devices / systems 102, 106 may communicate with each other over different networks 104 depending on the source and / or destination device / system 102, 106.

[0045] The backend system 106 may be embodied as any one or more types of devices / systems capable of performing the functions described herein. As shown, the backend system 106 includes various APIs 112 and corresponding API endpoints from which the frontend system 102 makes data requests. As mentioned above, it should be understood that the number and types of devices included in the backend system 106 may vary depending on the particular architecture of the system 100. It should be understood that the frontend system 102, the network 104, and the backend system 106 may each be embodied as, performed by, form a part of, or be associated with any type of device / system, collection of devices / systems, and / or portions thereof (e.g., computing device 200 of FIG. 2 ) suitable for performing the functions described herein.

[0046] 2, a simplified block diagram of at least one embodiment of a computing device 200 is shown. The illustrative computing device 200 depicts at least one embodiment of each of the computing devices, systems, services, controllers, switches, gateways, engines, modules, and / or computing components (e.g., which may be collectively referred to interchangeably as computing devices, servers, or modules for ease of description) described herein. For example, various computing devices may be processes or threads executing on one or more processors of one or more computing devices 200, which processors may execute computer program instructions and interact with other system modules to perform various functions described herein. Unless specifically limited otherwise, functionality described in the context of multiple computing devices may be integrated into a single computing device, or various functionality described in the context of a single computing device may be distributed across several computing devices. Furthermore, in connection with the computing systems described herein, the various servers and their computer devices may be located on a local computing device 200, a remote computing device 200 (e.g., off-site or in a cloud-based or cloud computing environment, e.g., a remote data center connected via a network), or some combination thereof, depending on the particular embodiment.In some embodiments, functionality provided by a server located on an off-site computing device may be accessed and provided via a virtual private network (VPN) as if such server were on-site, or functionality may be provided using software as a service (SaaS) accessed over the internet using various protocols, e.g., by exchanging data via extensible markup language (XML), JSON, and / or functionality may be otherwise accessed / utilized.

[0047] In some embodiments, computing device 200 may be embodied as a server, a desktop computer, a laptop computer, a tablet computer, a notebook, a netbook, an Ultrabook™, a mobile phone, a mobile computing device, a smartphone, a wearable computing device, a personal digital assistant, an Internet of Things (IoT) device, a processing system, a wireless access point, a router, a gateway, and / or any other computing, processing, and / or communication device capable of performing the functions described herein.

[0048] The computing device 200 includes a processing device 202 that executes algorithms and / or processes data according to operational logic 208, an input / output device 204 that enables communication between the computing device 200 and one or more external devices 210, and a memory 206 that stores data received from the external device 210 via the input / output device 204, for example.

[0049] The input / output devices 204 enable the computing device 200 to communicate with external devices 210. For example, the input / output devices 204 may include a transceiver, a network adapter, a network card, an interface, one or more communication ports (e.g., a USB port, a serial port, a parallel port, an analog port, a digital port, VGA, DVI, HDMI, FireWire, CAT5, or any other type of communication port or interface), and / or other communication circuitry. The communication circuitry of the computing device 200 may be configured to conduct such communication using any one or more communication technologies (e.g., wireless or wired communication) and associated protocols (e.g., Ethernet, Bluetooth, Wi-Fi, WiMAX, etc.) depending on the particular computing device 200. The input / output devices 204 may include hardware, software, and / or firmware suitable for implementing the techniques described herein.

[0050] External device 210 may be any type of device that allows data to be input or output from computing device 200. For example, in various embodiments, external device 210 may be embodied as one or more of the devices / systems described herein and / or portions thereof. Furthermore, in some embodiments, external device 210 may be embodied as another computing device, a switch, a diagnostic tool, a controller, a printer, a display, an alarm, a peripheral device (e.g., a keyboard, a mouse, a touchscreen display, etc.), and / or any other computing, processing, and / or communication device capable of performing the functions described herein. Furthermore, it should be understood that in some embodiments, external device 210 may be integrated into computing device 200.

[0051] Processing device 202 may be embodied as any type of processor capable of performing the functions described herein. In particular, processing device 202 may be embodied as one or more single-core or multi-core processors, microcontrollers, or other processors or processing / control circuitry. For example, in some embodiments, processing device 202 may include or be embodied as an arithmetic logic unit (ALU), a central processing unit (CPU), a digital signal processor (DSP), a graphics processing unit (GPU), a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), and / or another suitable processor. Processing device 202 may be of a programmable type, a state machine implemented by dedicated hardware, or a combination thereof. A processing device 202 having multiple processing units may utilize distributed processing, pipelined processing, and / or parallel processing in various embodiments. Furthermore, processing device 202 may be dedicated to performing only the operations described herein, or may be utilized in one or more additional applications. In an illustrative embodiment, processing device 202 is programmable, executing algorithms and / or processing data according to operating logic 208 defined by programming instructions (e.g., software or firmware) stored in memory 206. Additionally or alternatively, operating logic 208 of processing device 202 may be defined at least in part by hardwired logic or other hardware. Furthermore, processing device 202 may include one or more components of any type suitable for processing signals received from input / output device 204 or from other components or devices and providing a desired output signal.Such components may include digital circuits, analog circuits, or a combination thereof.

[0052] Memory 206 may be of one or more types of non-transitory computer-readable media, such as solid-state memory, electromagnetic memory, optical memory, or combinations thereof. Furthermore, memory 206 may be volatile and / or non-volatile, and in some embodiments, some or all of memory 206 may be of a portable type, such as a disk, tape, memory stick, cartridge, and / or other suitable portable memory. In operation, memory 206 may store various data and software used during operation of computing device 200, such as an operating system, applications, programs, libraries, and drivers. It should be understood that memory 206 may store data manipulated by operational logic 208 of processing device 202, such as, for example, data representing signals received from and / or transmitted to input / output devices 204, in addition to or instead of storing programming instructions defining operational logic 208. As shown in FIG. 2, memory 206 may be included with and / or coupled to processing device 202, depending on the particular embodiment. For example, in some embodiments, processing device 202, memory 206, and / or other components of computing device 200 may form part of a system-on-a-chip (SoC) and be integrated into a single integrated circuit chip.

[0053] In some embodiments, various components of computing device 200 (e.g., processing device 202 and memory 206) may be communicatively coupled via an input / output subsystem, which may be embodied as circuits and / or components for facilitating input / output operations with processing device 202, memory 206, and other components of computing device 200. For example, the input / output subsystem may be embodied as or otherwise include a memory controller hub, an input / output control hub, firmware devices, communication links (i.e., point-to-point links, bus links, wires, cables, light guides, printed circuit board traces, etc.), and / or other components and subsystems for facilitating input / output operations.

[0054] Computing device 200 may, in other embodiments, include other or additional components, such as those commonly found in a typical computing device (e.g., various input / output devices and / or other components). It should further be understood that one or more of the components of computing device 200 described herein may be distributed across multiple computing devices. In other words, the techniques described herein may be employed by a computing system including one or more computing devices. Additionally, while only a single processing device 202, I / O device 204, and memory 206 are illustratively shown in FIG. 2 , it should be understood that a particular computing device 200 may, in other embodiments, include multiple processing devices 202, I / O devices 204, and / or memory 206. Furthermore, in some embodiments, two or more external devices 210 may communicate with computing device 200.

[0055] Computing device 200 may be one of multiple devices connected by a network or to other systems / resources via a network. The network may be embodied as any one or more types of communications network capable of facilitating communication between various devices communicatively connected via the network. Thus, the network may include one or more networks, routers, switches, access points, hubs, computers, client devices, endpoints, nodes, and / or other intervening network devices. For example, the network may be embodied as or otherwise include one or more cellular networks, telephone networks, local or wide area networks, publicly available global networks (e.g., the Internet), ad hoc networks, short-range communications links, or combinations thereof. In some embodiments, the network may include circuit-switched voice or data networks, packet-switched voice or data networks, and / or any other network capable of carrying voice and / or data. In particular, in some embodiments, the network may include an Internet Protocol (IP)-based network and / or an Asynchronous Transfer Mode (ATM)-based network. In some embodiments, the network may handle voice traffic (e.g., via a Voice over IP (VOIP) network), web traffic, and / or other network traffic, depending on the particular embodiment and / or the devices in the system that communicate with each other.In various embodiments, the networks may include analog or digital wired and wireless networks (e.g., IEEE 802.11 networks, Public Switched Telephone Networks (PSTN), Integrated Services Digital Networks (ISDN), and Digital Subscriber Lines (xDSL)), third generation (3G) mobile networks, fourth generation (4G) mobile networks, fifth generation (5G) mobile networks, wired Ethernet networks, private networks (e.g., intranets, etc.), radio, television, cable, satellite, and / or any other distribution or tunneling mechanism for carrying data, or any suitable combination of such networks. It should be understood that various devices / systems may communicate with each other over different networks depending on the source and / or destination device / system.

[0056] It should be understood that computing device 200 may communicate with other computing devices 200 via any type of gateway or tunneling protocol, such as Secure Sockets Layer or Transport Layer Security. Network interfaces may include built-in network adapters, such as network interface cards, suitable for interfacing a computing device to any type of network capable of performing the operations described herein. Furthermore, the network environment may be a virtual network environment in which various network components are virtualized. For example, the various machines may be virtual machines implemented as software-based computers running on a physical machine. The virtual machines may share the same operating system, or in other embodiments, different operating systems may run on each virtual machine instance. For example, a "hypervisor" type of virtualization is used in which multiple virtual machines run on the same host physical machine, each functioning as if it had its own dedicated box. Other types of virtualization may be employed in other embodiments, such as networks (e.g., via software-defined networking) or functions (e.g., via network function virtualization).

[0057] Accordingly, one or more of the computing devices 200 described herein may be embodied as or form a part of one or more cloud-based systems. In a cloud-based embodiment, the cloud-based system may be embodied, for example, as a server-agnostic computing solution that executes instructions on demand, executes instructions only when prompted by specific activities / triggers, and does not consume computing resources when not in use. That is, the system may be embodied as a virtual computing environment residing “on” a computing system (e.g., a distributed network of devices) in which various virtual functions (e.g., Lambda functions, Azure Functions, Google Cloud Functions, and / or other suitable virtual functions) may be executed corresponding to the functionality of the system described herein. For example, when an event occurs (e.g., data is transferred to the system for processing), the virtual computing environment may be communicated (e.g., via a request to the virtual computing environment's API), which may then route the request to the correct virtual function (e.g., a specific server-agnostic computing resource) based on a set of rules. Thus, when a request for transmission of data is made by a user (e.g., via an appropriate user interface to the system), an appropriate virtual function may be executed to perform the action before deleting the instance of the virtual function.

[0058] 3 and 4, in use, system 100 may perform a method 300 for configurable caching, duplication prevention, and rate limiting of user interface data requests. It should be understood that certain blocks of method 300 are illustrated by way of example, and that such blocks may be combined or divided, added or removed, and / or reordered in whole or in part, depending on the particular embodiment, unless stated to the contrary. Because system 100 may have a variety of different system architectures, the blocks of method 300 in FIGS. 3 and 4 are primarily described as simply being performed by system 100 for ease and brevity of explanation.

[0059] The illustrative method 300 begins at block 302 of FIG. 3 , where a user interface element initiates a data request. It should be understood that the data request may reference an associated API endpoint / URL and include any associated parameters / data. At block 304, the system 100 determines whether the anti-duplication cache 110 includes an anti-duplication entry associated with the data request (e.g., indicating that an identical previous data request occurred within a configurable threshold period). For example, as described above, the anti-duplication cache 110 may be configured to store a probability associated with each data request. Thus, various anti-duplication entries in the anti-duplication cache 110 indicate that the previous data request occurred within the threshold period (i.e., the expiration time of the anti-duplication entry, after which the anti-duplication entry may be removed from the anti-duplication cache 110). For example, in one embodiment, the threshold period may be 10 ms, and in another embodiment, the threshold period may be 100 ms. It should be understood that certain HTTP methods are considered non-idempotent and are generally not cached. However, in the illustrative embodiment, the anti-duplication cache 110 nevertheless caches them, in the sense that it is the requests or prospects themselves that are cached, rather than the actual resulting data. It should be appreciated that the anti-duplication cache 110 is utilized to ensure that duplicate data requests are not made within a given period of time (e.g., by a user inadvertently clicking a user interface element quickly and repeatedly).

[0060] If the system 100 determines in block 304 that there is an associated anti-duplication entry in the anti-duplication cache 110, the method 300 proceeds to block 316, where the data request is made and the data is returned to the user interface element. In other words, if there is an associated anti-duplication entry in the anti-duplication cache 110, the data request has already been made and the system 100 can wait for the requested data to be returned. However, if the system 100 determines in block 304 that the anti-duplication cache 110 does not contain an associated anti-duplication entry, the method 300 proceeds to block 306, where the system 100 determines whether the main cache 108 contains a cache entry associated with the data request.

[0061] If the system 100 determines in block 306 that the main cache 108 contains a cache entry associated with the data request, the method 300 proceeds to block 308, where the system 100 determines whether the data request is associated with a predetermined maximum cache age. In other words, the system 100 determines whether the data request includes a maximum cache age parameter that overrides any expiration date and time provided by the cache entry itself. It should be understood that different components may have different ideas about what is acceptable in terms of the TTL for a data response. For example, one component may only accept data that is up to two minutes old, while another component may accept data that is ten minutes old.

[0062] If the system 100 determines in block 308 that the data request is associated with a predetermined maximum cache retention period, the method 300 proceeds to block 310, where the system 100 calculates the expiration of the cache entry using the maximum cache retention period. More specifically, the system 100 may determine whether the cache entry associated with the data request has expired based on the creation date and time of the cache entry and the predetermined maximum cache retention period (e.g., the expiration is the maximum cache retention period after the creation date and time of the cache entry). However, if the system 100 determines in block 308 that the data request is not associated with a predetermined maximum cache retention period, the method 300 proceeds to block 312, where the system 100 uses the expiration date of the cache entry associated with the data request so that the system 100 can determine whether the cache entry associated with the data request has expired based on the expiration date and time defined by the cache entry itself. The method 300 proceeds from blocks 310 and 312 to block 314, where the system 100 determines whether the cache entry has expired. If not, the method 300 proceeds to block 316 where a data request is made and the data is returned to the user interface element.

[0063] Returning to block 306, if the system 100 determines that the main cache 108 does not contain a cache entry associated with the data request, or if the system 100 determines in block 314 that a cache entry exists but has expired, the method 300 proceeds to block 318 of FIG. 4 , where the system 100 determines whether the data request is associated with a rate limiting group of API endpoints. As described above, rate limiting groups of API endpoints may be predefined by the system 100 to help avoid sending certain unnecessary data requests. For example, in one embodiment, if the system 100 already knows that GET_USER requests are rate limited, the GET_USER and GET_USERS endpoints may belong to the same rate limiting grouping, such that the GET_USERS request should be postponed to avoid an inevitable 429 error.

[0064] If the system 100 determines in block 318 that the data request is associated with a rate limiting group, the method 300 proceeds to block 320, where the system 100 delays for a corresponding server retry period before making the data request to the backend system 106 in block 322. If the system 100 determines in block 318 that the data request is not associated with a rate limiting group, the method 300 proceeds to block 322, where the data request may be made without delay.

[0065] At block 324, the system 100 determines whether the data request was successful. If successful, the method 300 returns to block 316 of FIG. 3 , where the data request is made and the data is returned to the user interface element. However, if the data request is unsuccessful, an error or status code associated with the failure to fulfill the data request may be provided, and the method 300 proceeds to block 326, where the system 100 determines whether the error associated with the data request is a retryable error. If not, the method 300 proceeds to block 328, where a data request failure is returned to the user interface element. However, if the system 100 determines at block 326 that the error is retryable, the method 300 proceeds to block 330, where the system 100 determines whether a predetermined retry limit has been reached. It should be understood that the predetermined retry limit may be configurable and may define a maximum number of retry attempts that may be made. In some embodiments, the system 100 may utilize different retry attempts for different situations (e.g., different API endpoints, different error types, etc.). If the predetermined retry limit has been reached, the method 300 proceeds to block 328 where a data request failure is returned to the user interface element.

[0066] However, if a predetermined retry limit has not been reached, system 100 may retry the data request. More specifically, in the illustrative embodiment, method 300 proceeds to block 332, where system 100 determines whether the retryable error is a 429 error (e.g., associated with an HTTP 429 Too Many Requests status code or error). If it is a 429 error, method 300 proceeds to block 320, where system 100 delays for a corresponding server retry period (e.g., as provided by the 429 error) before retrying the data request. However, if the retryable error is not a 429 error, method 300 proceeds to block 322, where the data request may be made without delay.

[0067] Although blocks 302-332 are described in a relatively serial manner, it should be understood that various blocks of method 300 may be performed in parallel in some embodiments.

Claims

1. 1. A method for configurable caching, duplication prevention, and rate limiting of user interface data requests, the method comprising: receiving a data request initiated by a user interface element of a user interface; determining whether an anti-duplication cache includes an anti-duplication entry associated with the data request; In response to determining that the anti-duplication cache does not contain the anti-duplication entry, determining whether a main cache different from the anti-duplication cache contains a cache entry associated with the data request; determining whether the data request is associated with a rate limiting group of application programming interface endpoints in response to one of (i) determining that the main cache does not contain the cache entry associated with the data request, or (ii) determining that the cache entry associated with the data request has expired; In response to determining that the data request is associated with the rate limiting group, making the data request to a backend system after a predetermined server retry period associated with the rate limiting group.

2. The method of claim 1 , wherein the anti-duplicate entry indicates that a previous data request identical to the data request initiated by the user interface element of the user interface was made within a threshold period of time.

3. 2. The method of claim 1, further comprising, in response to determining that the main cache contains the cache entry associated with the data request, determining whether the data request is associated with a predetermined cache maximum retention period.

4. 4. The method of claim 3, further comprising, upon determining that the data request is associated with the predetermined maximum cache retention period, determining whether the cache entry associated with the data request has expired based on a creation date and time of the cache entry and the predetermined maximum cache retention period.

5. 4. The method of claim 3, further comprising, in response to determining that the data request is not associated with the predetermined maximum cache retention period, determining whether the cache entry associated with the data request has expired based on an expiration date and time of the cache entry defined by the cache entry.

6. The method of claim 1 , further comprising: upon determining that the data request is not associated with any rate limiting group, immediately making the data request to the backend system.

7. receiving an error associated with the data request in response to a failure to fulfill the data request; determining whether the error associated with the data request is a retryable error; In response to determining that the error associated with the data request is the retryable error, determining whether a predetermined retry limit has been reached; 2. The method of claim 1, further comprising: retrying the data request in response to determining that the predetermined retry limit has not been reached.

8. 8. The method of claim 7, wherein retrying the data request comprises, in response to determining that the retryable error is a 429 error, retrying the data request after the predetermined server retry period.

9. The method of claim 7 , wherein the predetermined retry limit is a maximum number of retry attempts that can be made.

10. 2. The method of claim 1, wherein determining whether the data request is associated with a rate limiting group of the application programming interface endpoints comprises determining whether an application programming interface endpoint to which the data request is directed is included in a rate limiting group of the application programming interface endpoints.

11. The method of claim 1 , wherein the anti-duplication cache is configured to store one or more probabilities, each of the one or more probabilities being associated with a respective previous data request.

12. The method of claim 1 , wherein the anti-duplication cache is configured to store data returned for a previous data request with a configurable timeout.

13. 2. The method of claim 1, wherein the main cache is configured to store data returned in response to a previous data request.

14. 1. A system for configurable caching, duplication prevention, and rate limiting of user interface data requests, the system comprising: an anti-duplication cache; a main cache distinct from the duplication prevention cache; at least one processor; and at least one memory containing a plurality of instructions stored therein, the instructions, when executed by the at least one processor, causing the system to: receiving a data request initiated by a user interface element of a user interface; determining whether the anti-duplication cache includes an anti-duplication entry associated with the data request; determining whether the main cache contains a cache entry associated with the data request in response to a determination that the anti-duplication cache does not contain the anti-duplication entry; determining whether the data request is associated with a rate limiting group of an application programming interface endpoint upon a determination of one of (i) the main cache not containing the cache entry associated with the data request, or (ii) the cache entry associated with the data request has expired; and Upon determining that the data request is associated with the rate limiting group, the system causes the data request to be made to a backend system after a predetermined server retry period associated with the rate limiting group.

15. The system of claim 14 , wherein the anti-duplicate entry indicates that a previous data request identical to the data request initiated by the user interface element of the user interface was made within a threshold period of time.

16. The instructions further include: determining whether the data request is associated with a predetermined cache maximum retention time upon determining that the main cache contains the cache entry associated with the data request; upon determining that the data request is associated with the predetermined maximum cache retention period, determining whether the cache entry associated with the data request has expired based on a creation date and time of the cache entry and the predetermined maximum cache retention period; 15. The system of claim 14, wherein upon determining that the data request is not associated with the predetermined maximum cache retention period, a determination is made as to whether the cache entry associated with the data request has expired based on an expiration date and time of the cache entry defined by the cache entry.

17. The instructions further include: receiving an error associated with the data request in response to a failure to fulfill the data request; determining whether the error associated with the data request is a retryable error; determining whether a predetermined retry limit has been reached in response to a determination that the error associated with the data request is a retryable error; 15. The system of claim 14, wherein a determination that the predetermined retry limit has not been reached causes the data request to be retried.

18. 20. The system of claim 17, wherein retrying the data request comprises retrying the data request after the predetermined server retry period upon determining that the retryable error is a 429 error.

19. 20. The system of claim 17, wherein the predetermined retry limit is a maximum number of retry attempts that can be made.

20. the anti-duplication cache is configured to store one or more probabilities, each of the one or more probabilities being associated with a respective previous data request; 15. The system of claim 14, wherein the main cache is configured to store data returned in response to a previous data request.

Images