High latency query optimization system

Abstract

The described technology provides a high latency query optimization method, comprising: receiving a data request from a client, the data request pointing to data stored in a plurality of data fragments; determining a set of operational parameters for the data fragment for retrieving data from the plurality of fragments; determining a partitioning factor based on the set of operating parameters; dividing the data request into a plurality of API requests based on the partitioning factor, wherein each API request in the API requests points to a part of the plurality of data fragments; and transmitting the plurality of API requests in parallel to a source API, the source API configured to perform a data query on the plurality of data fragments.

Description

Background Art

[0001] Systems and scenarios that need to obtain real-time data from multiple sources, for example, for the purpose of displaying information to users, are often constrained by how much data they can collect from slow data sources at query time. Here, data latency is the time it takes to store or retrieve a data packet. In computer networks and Internet communications, data latency is the time it takes for data to travel from source to destination, where the lower the network latency, the higher the speed and performance of the network. Summary of the invention

[0002] The described technology provides a high-latency query optimization method, including: receiving a data request from a client, the data request pointing to data stored in multiple data shards; determining a set of operating parameters of the data shards for retrieving data from the multiple shards; determining a chunking factor based on the set of operating parameters; dividing the data request into multiple API requests based on the chunking factor, each of the API requests pointing to a portion of the multiple data shards; and transmitting the multiple API requests in parallel to a source API, the source API being configured to perform data queries on the multiple data shards.

[0003] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

[0004] Other implementations are also described and documented herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] Figure 1 An example implementation of the high-latency query optimization system disclosed herein is illustrated.

[0006] Figure 2 Illustrated is an example operation of the high-latency query optimization system disclosed herein.

[0007] Figure 3 Alternative example operations of the high-latency query optimization system disclosed herein are illustrated.

[0008] Figure 4 An example graph illustrating the relationship between the number of samples that may be used by the Kalman filter of the high-latency query optimization system disclosed herein and various parameters of the query.

[0009] Figure 5 An example system is illustrated that may be used to implement the high-latency query optimization system disclosed herein. DETAILED DESCRIPTION

[0010] Systems and scenarios that need to obtain real-time data from multiple sources, for example, for the purpose of displaying information to users, are often constrained by how much data they can collect from slow data sources at query time. For example, a middle tier backend service that works with multiple clients is often required to collect data from many different data sources. Often, one or more of these multiple data sources may be slower at query time than other data sources in such multiple data sources. Therefore, the middle tier service may not be able to meet strict latency requirements.

[0011] In addition, for data-driven applications or websites, it is important to scale the data in a way that ensures the security and integrity of the data. The database that is the target of queries from such a middle-tier service can be organized into multiple data shards. Here, the data shards can be horizontal partitions of data in a database or search engine. Data sharding can include breaking the data into two smaller blocks. Such data shards can be referred to as logical shards. Such logical shards can be distributed across multiple database nodes and can be referred to as physical shards. In some implementations, such physical shards can hold data from various logical shards.

[0012] To scale the data load, each data shard may be maintained on a separate data server, where each shard stores rows of a database. The query generated by the middle tier service may involve collecting data from a single shard or may require collecting data from a large number of data shards of the database. In such an implementation, the latency of responses from one or more of the database shards may be greater than the latency of other data shards, and therefore, the latency of servicing data requests from clients may be blocked by the data shard with the highest latency (slowest response).

[0013] The high-latency query optimization system disclosed herein allows for the retrieval of data organized in data shards in a faster, more efficient and more flexible manner. Specifically, the technology disclosed herein includes a high-latency query optimization system, wherein a middle-tier backend service receives a data request from a client, the data request pointing to data stored in multiple (N) data shards, and splits the original request for the N data shards into smaller chunks. For example, each chunk of a request may include requests for N / F data shards, wherein the chunking factor F is determined based on the number of parameters related to the query system, such as the size of the data shards, the delay in completing requests from the data shards, the availability of bandwidth for the data shards, and the like.

[0014] Figure 1An implementation of a high latency query optimization system 100 disclosed herein is illustrated. The system 100 includes a middle-tier backend service 106 communicatively connected to a client 102. The middle-tier backend service 106 may also be connected to a source API 114 that communicates with a plurality of data shards 116. For example, the data shards 116 may be configured to host horizontal data of a database on a plurality of different data servers.

[0015] At 104, a client-side experience / user at a client 102 requests data from a backend service 106. For example, the data may include results collected from N data shards of a plurality of data shards 116. The middle-tier backend service 106 accepts the client request and divides the client request for the N data shards 116 into N / F data shard requests, where F is a chunking factor determined based on a plurality of data shard parameters. Thus, each of the N / F requests is directed to a subset of the N data shards 116. At 112, the middle-tier backend service 106 transmits the N / F data shard requests to a source API 114. In one implementation, the N / F data shard requests are transmitted to the source API 114 in parallel.

[0016] The source API 114 accepts N / F data shard requests from the middle-tier backend service 106. Subsequently, at 118, the source API 114 fans out the N / F data shard requests to individual data shards in the N data shards 116. At 118, in return for the N / F data shard requests, the data shard N returns the data to the middle-tier backend service 106. Since some of the data shards in the data shards 116 may have higher latency than other data shards, some of the N / F data shard requests may be returned earlier than other data shard requests. At 120, the source API 114 transmits responses to the N / F data shard requests to the middle-tier backend service 106. In one implementation, the source API 114 transmits the responses to the N / F data shard requests to the middle-tier backend service 106 in parallel. However, in one implementation, when the responses to the N / F data shard requests are received from the data shard 116, the responses to the N / F data shard requests are sent at different times.

[0017] At 122, the middle-tier backend service 106 performs a best-effort attempt upon collecting the responses to the N / F data shard requests returned to it from the source API 114. The middle-tier backend service 106 may also monitor latency parameters associated with the responses to the data shard requests. For example, the middle-tier backend service 106 may measure and tabulate the time taken to receive the responses. Alternatively, the middle-tier backend service 106 may also preemptively perform a ping test on each of the data shards, and tabulate the results of such ping tests as latency parameters associated with the data shards. In another implementation, the middle-tier backend service 106 may request the data shards to provide a decomposition of the latency, such as network latency, disk latency, etc.

[0018] For example, some data shards may have higher network latency due to traffic on the network connected to the data shards, while some other data shards may have higher disk latency due to the health of the disks storing the data, the number of requests to the disks, etc. Here, the network latency describes the delay that occurs during communication over the network to the data shards, while the disk latency may refer to the delay between the time when data is requested from the storage device configured for the data shard and the time when the data begins to be returned. In another implementation, the data shards may also notify the middle tier backend service 106 of other latency parameters related to the data shards, such as RAM latency for the local storage controller, CPU latency of the local storage controller, etc.

[0019] Based on the delay parameters of the various data shards 116, some of the responses to the N / F data shard requests may have arrived, while other responses may take longer to return. The middle-tier backend service 106 may stop waiting for responses that exceed a predetermined cutoff value and return partial results to the client 102. In one implementation, such a predetermined cutoff value may be determined based on the type of client, the type of client request, the identity of the data shard that lost the response, the values ​​of various delay parameters related to the data shard, etc.

[0020] In one implementation, the middle tier backend service 106 is further enhanced by utilizing a stateful Kalman filter 110, which allows the middle tier backend service 106 to self-select the best parameters at any given time without human intervention. The Kalman filter measures a series of measurements observed over time and produces estimates of unknown variables that tend to be more accurate than those based on a single measurement alone. Thus, for example, the stateful Kalman filter 110 may observe the delay parameters for the data shards over time and generate output estimates of the states of the delays for the various data shards.

[0021] In addition, the Kalman filter uses persistent state to balance the ratio between the blocking factor F, the observed latency of each chunk returned from the source API 114, and the total load on the high-latency query optimization system 100. Specifically, the persistent state of the system 100 can be a state that continues to exist even when there is no data access process. Therefore, the Kalman filter can use persistent state to balance the ratio between the blocking factor F, the observed latency of each chunk returned from the source API 114, and the total load on the high-latency query optimization system 100 in the absence of a query. Subsequently, at 124, the middle-tier backend service 106 accepts the results from the source API 114 and prepares a response to be returned to the client-side experience 102.

[0022] The query optimization system 100 provides advantages over other query systems because the latency of the middle-tier backend service 106 is guaranteed to be at most the latency deadline timeout value or less (if all chunks are returned from the source API within the deadline). Compared to existing implementations of query systems, the latency of the middle-tier backend service is determined by how long the call to the source API takes to complete and is not controlled by the middle-tier backend service.

[0023] In addition, the query optimization system 100 provides advantages over other query systems because it can be configured to give best-effort results by spreading the call to the source API over N / F chunks, thereby increasing the chance of partial data being returned. Subsequently, data is collected from the successful chunks, thereby allowing partial data to be returned to the client. The best-effort implementation of the query optimization system 100 also allows the system to cancel outbound calls that have not been completed in time, thereby further reducing the pressure on the query optimization system 100. In addition, using the query optimization system 100, delay parameters such as delay deadline, chunking factor F, etc. can be exposed and customizable.

[0024] In an alternative implementation, the query optimization system 100 may also use system operating parameters such as chunk size, fan-out factor, and total timeout value to control data retrieval delay. In addition, the query optimization system 100 may use the fan-out algorithm for scenarios where the original request from the client 102 is broken down into smaller chunks and expose parameters such as chunk size and fan-out factor. In addition, the query optimization system 100 may also impose a total duration on each or some of the parallel fan-out requests, and then, expose such total duration request as a parameter. In such an implementation, each fan-out request may also be allowed to return partial results and then release its resources before the timeout value is reached.

[0025] Figure 2Operation 200 of the high-latency query optimization system disclosed herein is illustrated. The operation receives a client-side experience / user request for data from a backend service. The requested data may include results collected from N data shards. Operation 204 accepts the user's request at the middle-tier backend service and divides the original list of N data shards into smaller chunks. In one implementation, the original list of N data shards may be divided by a chunking factor F so that each data shard request includes N / F data shards. Operation 206 transmits F data shard requests, each data shard request requests data from N / F data shards in parallel with the source API. Thus, for example, if the initial request is to request data from N=30 data shards and the chunking factor F is 5, each of the 5 requests is for data from 6 data shards.

[0026] The source API accepts requests for N / F data chunks from the middle-tier backend service. Each chunk can be processed by the source API by fanning out the data request to individual data shards. The source API receives data from the data shards, and each chunk is returned to the middle-tier backend service separately from the other chunks. Operation 208 receives the result of the data request from the source API at the middle-tier backend service. Operation 210 performs a best-effort attempt when collecting data from the N / F chunks returned to it from the source API. Since each data chunk request is receiving data from a different data shard, data from some data chunk requests may be received earlier than data from other data chunk requests in the data chunk requests. In one implementation, the middle-tier backend service determines the point at which it will group the data to satisfy the request from the user. Subsequently, operation 212 prepares a response to the client-side experience / user based on the determination of when to group the data.

[0027] Subsequently, operation 214 determines whether there are any source API requests with failed responses or responses that are only partially filled. For example, operation 214 determines whether a request for one or more data shards did not return data in a timely manner or the response provided an incomplete result. If so, operation 218 transmits a retry request to the source API to collect data for the failed or partially filled data. If all data has been collected according to operation 216, no action is required.

[0028] Figure 3An alternative operation 300 of the high latency query optimization system disclosed herein is illustrated. Specifically, operation 300 illustrates an implementation in which a stateful Kalman filter is used to self-select parameters for determining chunking of requests to N data shards. Thus, operation 304 determines one or more operating parameters for the N target data shards. For example, operation 304 may determine the latency of one or more of the N data shards, the availability of bandwidth to one or more of the N data shards, the size of the N data shards, the selection of an API for communicating with the N data shards, and the like. The Kalman filter allows the middle-tier backend service to self-select optimal parameters at any given time without human intervention. In one implementation, a persistent state is used to balance the ratio between the chunking factor F and the observed latency of each chunk returned from the source API.

[0029] Operation 306 uses a Kalman filter to determine a blocking factor F. Specifically, one or more of the operating parameters of the N data slices are input to a stateful Kalman filter, which generates a value for the blocking factor as an output. Operation 308 divides the original list of N data slices into smaller N / F chunks of data slice requests. Subsequently, operation 310 transmits the smaller chunks of the data slice requests to the source API. Once the source API has processed the request, operation 312 receives the results of the query from the source API. Operation 314 performs a best effort attempt when collecting data from the N / F chunks. For example, such a best effort attempt can be based on a timeout parameter or a total duration parameter set by the system.

[0030] Operation 316 prepares a response for the client-side experience based on the response from the source API. In one implementation, operation 318 transmits the performance metrics to a logging system. Specifically, the performance metrics can be used for learning and investigation purposes. Operation 320 uses the performance metrics to fine-tune operating parameters and algorithms used internally within the Kalman filter.

[0031] Figure 4 A graph 400 illustrating the relationship between the number of samples and various parameters of a query that can be used by a Kalman filter of a high latency query optimization system disclosed herein. As shown in graph 402, initially the number of data shards per block is set to 10. As data is returned from the data shards, a delay in receiving the data is observed. As shown in graph 404, the delay may begin to increase. As the delay increases, the number of data shards per block may be reduced to 8. As a result, as shown in graph 402, the delay may begin to stabilize, and therefore, the number of data shards per block remains at 8.

[0032] Although the above example uses delay for determining the blocking factor, other parameters such as CPU usage, number of contentions, RAM usage, number of threads, throttling statistics, etc. can be used to determine the number of blocks and / or the size of the blocks. In some embodiments, the Kalman filter can increase or decrease the number of blocks based on the observed values ​​of the parameters. Alternatively, the Kalman filter can increase or decrease the size of the blocks based on the observed values ​​of the parameters.

[0033] The high-latency query optimization system disclosed in this article provides multiple advantages over other implementations that do not use the technology disclosed in this article. For example, using the high-latency query optimization system disclosed in this article, it can be guaranteed that the delay of the middle-tier backend service is a predetermined delay cutoff timeout value, or if all blocks are returned from the source API at the deadline, the delay will be smaller. In addition, using the best-effort implementation of the high-latency query optimization system disclosed in this article, the number of calls is expanded on N / F blocks, thereby increasing the chance of partial data return. Subsequently, data is collected from successful blocks, which allows partial data to be returned to the client. In addition, in the illustrated implementation, parameters such as the delay cutoff timeout value and the block factor F are exposed and customizable. This allows the parameters to be continuously fine-tuned and optimized in real time based on system behavior using, for example, a Kalman filter. In addition, the best-effort implementation also allows the high-latency query optimization system to cancel outbound calls to the source API that are not completed in time, thereby further reducing the pressure on the system.

[0034] In an alternative implementation, the high-latency query optimization system uses various key factors, such as query time latency, data freshness, and completeness / coverage, to determine the chunking factor. Here, the query time latency indicates how long it takes for the client to return data from the server at query time, the data freshness indicates how up-to-date the data returned by the server is relative to the query time, and the completeness / coverage indicates how much data the server will return.

[0035] In some implementations, a combination of these factors can be used to determine the blocking factor. For example, when a combination of query-time delay and data freshness is used, the system acquires data from the source itself in real time to optimize data freshness, or it acquires as little data as possible to optimize query time delay, thereby sacrificing completeness / coverage. Alternatively, if a combination of data freshness and completeness / coverage is used to determine the blocking factor, the system acquires data from the source itself in real time to optimize data freshness, or it extracts as much data as possible to optimize completeness / coverage, thereby sacrificing query time delay. On the other hand, if the system uses query-time delay and completeness / coverage to determine the blocking factor, the system prepares the data asynchronously and makes it easily available through fast storage, thereby sacrificing data freshness.

[0036] Figure 5 An example system 500 is shown that can be used to implement the high latency query optimization system disclosed herein. Figure 5 Example hardware and operating environments include computing devices, such as general purpose computing devices in the form of computers 20, mobile phones, personal digital assistants (PDAs), tablet computers, smart watches, game remote controls, or other types of computing devices. Figure 5 In an implementation, for example, computer 20 includes processing unit 21, system memory 22, and system bus 23, which operably couples various system components including system memory 22 to processing unit 21. There may be only one or more than one processing unit 21, so that the processor of computer 20 includes a single central processing unit (CPU) or multiple processing units, which is generally referred to as a parallel processing environment. Computer 20 may be a conventional computer, a distributed computer, or any other type of computer; the implementation is not limited thereto.

[0037] The system bus 23 may be any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a switching fabric, a point-to-point connection, and a local bus using any of a variety of bus architectures. The system memory 22 may also be referred to simply as memory, and includes a read-only memory (ROM) 24 and a random access memory (RAM) 25. A basic input / output system (BIOS) 26, containing basic routines that help transfer information between elements within the computer 20, such as during startup, is stored in the ROM 24. The computer 20 also includes a hard disk drive 27 for reading from and writing to a hard disk (not shown), a disk drive 28 for reading from or writing to a removable disk 29, and an optical disk drive 30 for reading from or writing to a removable optical disk 31 (such as a CD ROM, DVD, or other optical media).

[0038] The computer 20 may be used to implement the high latency query optimization system disclosed herein. In one embodiment, a frequency unwrapping module including instructions for unwrapping frequencies based at least in part on a sampled reflected modulated signal may be stored in a memory of the computer 20, such as a read-only memory (ROM) 24 and a random access memory (RAM) 25.

[0039] In addition, instructions stored in the memory of computer 20 may be used to use Figure 5 Similarly, the instructions stored in the memory of the computer 20 can also be used to implement Figure 4 The memory of computer 20 may also include one or more instructions for implementing the high latency query optimization system disclosed herein.

[0040] The hard disk drive 27, magnetic disk drive 28, and optical disk drive 30 are connected to the system bus 23 via a hard disk drive interface 32, a magnetic disk drive interface 33, and an optical drive interface 34, respectively. The drives and their associated tangible computer-readable media provide non-volatile storage of computer-readable instructions, data structures, program modules, and other data for the computer 20. Those skilled in the art will appreciate that any type of tangible computer-readable media may be used in the exemplary operating environment.

[0041] A number of program modules may be stored on the hard disk, disk 29, optical disk 31, ROM 24 or RAM 25, including an operating system 35, one or more application programs 36, other program modules 37 and program data 38. A user may generate reminders on the personal computer 20 through input devices such as a keyboard 40 and a pointing device 42. Other input devices (not shown) may include a microphone (e.g., for voice input), a camera (e.g., for a natural user interface (NUI)), a joystick, a game pad, a satellite dish, a scanner, etc. These and other input devices are typically connected to the processing unit 21 through a serial port interface 46 coupled to the system bus 23, but may be connected through other interfaces, such as a parallel port, a game port or a universal serial bus (USB). A monitor 47 or other type of display device is also connected to the system bus 23 via an interface such as a video adapter 48. In addition to the monitor, the computer typically includes other peripheral output devices (not shown), such as speakers and printers.

[0042] The computer 20 can operate in a networked environment using logical connections to one or more remote computers, such as the remote computer 49. These logical connections are implemented by a communication device coupled to the computer 20 or a portion of the computer 20; the implementation is not limited to a particular type of communication device. The remote computer 49 can be another computer, a server, a router, a network PC, a client, a peer device, or other public network node, and typically includes many or all of the elements described above with respect to the computer 20. Figure 5 The logical connections depicted in 5 include a local area network (LAN) 51 and a wide area network (WAN) 52. Such networking environments are commonplace in office networks, enterprise-wide computer networks, intranets and the Internet, which are all types of networks.

[0043] When used in a LAN networking environment, the computer 20 is connected to the local area network 51 through a network interface or adapter 53, which is a type of communication device. When used in a WAN networking environment, the computer 20 typically includes a modem 54, a network adapter, a type of communication device, or any other type of communication device for establishing communications on the wide area network 52. The modem 54, which can be internal or external, is connected to the system bus 23 via the serial port interface 46. In a networking environment, the program engine or portions thereof depicted relative to the personal computer 20 can be stored in a remote memory storage device. It should be appreciated that the network connections shown are examples, and other units of communication devices for establishing a communication link between computers can be used.

[0044] In an example implementation, software or firmware instructions for the high latency query optimization system 510 may be stored in the system memory 22 and / or storage device 29 or 31 and processed by the processing unit 21. High latency query optimization system operations and data may be stored in the system memory 22 and / or storage device 29 or 31 as persistent data storage.

[0045] In contrast to tangible computer-readable storage media, intangible computer-readable communication signals may embody computer-readable instructions, data structures, program modules, or other data residing in a modulated data signal such as a carrier wave or other signal transmission mechanism. The term "modulated data signal" refers to a signal that has one or more characteristics set or changed in such a manner as to encode the information in the signal. By way of example, and not limitation, intangible communication signals include wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared, and other wireless media.

[0046] Some embodiments of the high-latency query optimization system may include an article of manufacture. The article of manufacture may include a tangible storage medium for storing logic. Examples of storage media may include one or more types of computer-readable storage media capable of storing electronic data, including volatile memory or non-volatile memory, removable or non-removable memory, erasable or non-erasable memory, writable or rewritable memory, etc. Examples of logic may include various software elements, such as software components, programs, applications, computer programs, applications, system programs, machine programs, operating system software, middleware, firmware, software modules, routines, subroutines, functions, methods, processes, software interfaces, application program interfaces (APIs), instruction sets, computing codes, computer codes, code segments, computer code segments, words, values, symbols, or any combination thereof. In one embodiment, for example, an article of manufacture may store executable computer program instructions, which, when executed by a computer, cause the computer to perform methods and / or operations according to the described embodiments. Executable computer program instructions may include any suitable type of code, such as source code, compiled code, interpreted code, executable code, static code, dynamic code, etc. Executable computer program instructions can be implemented according to a predefined computer language, method or syntax to instruct a computer to perform a specific function. Instructions can be implemented using any suitable high-level, low-level, object-oriented, visual, compiled and / or interpreted programming language.

[0047] The high-latency query optimization system disclosed herein may include various tangible computer-readable storage media and intangible computer-readable communication signals. Tangible computer-readable storage can be embodied by any available medium that can be accessed by the high-latency query optimization system disclosed herein, and includes volatile and non-volatile storage media, removable and non-removable storage media. Tangible computer-readable storage media do not include intangible and temporary communication signals, and include volatile and non-volatile, removable and non-removable storage media implemented by any method or technology for storing information such as computer-readable instructions, data structures, program modules or other data. Tangible computer-readable storage media include, but are not limited to: RAM, ROM, EEPROM, flash memory or other memory technology, CDROM, digital versatile disk (DVD) or other optical disk storage, cassettes, tapes, disk storage or other magnetic storage devices, or any other tangible medium that can be used to store desired information and can be accessed by the high-latency query optimization system disclosed herein. Compared with tangible computer-readable storage media, intangible computer-readable communication signals can embody computer-readable instructions, data structures, program modules or other data residing in modulated data signals (such as carrier waves or other signal transmission mechanisms). The term "modulated data signal" refers to a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, intangible communication signals include signals that move through a wired medium (such as a wired network or direct-wired connection) and signals that move through wireless media such as acoustic, RF, infrared and other wireless media.

[0048] The implementation described in this article is implemented as logical steps in one or more computer systems. Logical operations can be implemented as (1) a sequence of steps implemented as a processor executed in one or more computer systems, and (2) as interconnected machines or circuit modules within one or more computer systems. The implementation is a matter of choice depending on the performance requirements of the computer system utilized. Therefore, the logical operations that constitute the implementation described in this article are variously referred to as operations, steps, objects, or modules. In addition, it should be understood that unless otherwise explicitly stated, the logical operations can be performed in any order, otherwise the claim language inherently requires a specific order. The above description, examples, and data together with additional appendices provide a complete description of the structure and use of the exemplary implementation.

[0049] The method described in this article includes: receiving a data request from a client, the data request pointing to data stored in multiple data shards; determining a set of operating parameters of the data shards for retrieving data from the multiple data shards; determining a blocking factor based on the set of operating parameters; dividing the data request into multiple API requests based on the blocking factor, each of the API requests pointing to a portion of the multiple data shards; and transmitting the multiple API requests in parallel to a source API, the source API being configured to perform data queries on the multiple data shards.

[0050] Another implementation discloses one or more physically manufactured computer-readable storage media that encode computer-executable instructions for executing a computer process on a computer system, the computer process comprising: receiving a data request from a client, the data request pointing to data stored in multiple data shards; determining a set of delay parameters for the data shards for retrieving data from the multiple data shards; determining a blocking factor based on the set of delay parameters; dividing the data request into multiple API requests based on the blocking factor, each of the API requests pointing to a portion of the multiple data shards; and transmitting the multiple API requests in parallel to a source API, the source API being configured to perform data queries on the multiple data shards.

[0051] The system disclosed in this article includes a memory, one or more processing units, and a query optimization system stored in the memory and executable by the one or more processor units, the query optimization system encoding computer executable instructions on the memory for executing a computer process on the one or more processor units, the computer process including receiving a data request from a client, the data request pointing to data stored in multiple data shards, determining a set of delay parameters for the data shards used to retrieve data from the multiple data shards, determining a blocking factor based on the set of delay parameters, dividing the data request into multiple API requests based on the blocking factor, each of the API requests pointing to a portion of the multiple data shards, and transmitting the multiple API requests in parallel to a source API, the source API being configured to perform data queries on the multiple data shards.

[0052] The above description, examples and data provide a complete description of the structure and use of exemplary embodiments of the present invention. Since many implementations of the present invention can be carried out without departing from the spirit and scope of the present invention, the present invention resides in the appended claims. In addition, without departing from the enumerated claims, the structural features of different embodiments can be combined in another implementation.

Claims

1. A method comprising: Receiving a data request from a client, wherein the data request points to data stored in a plurality of data shards; determining a set of operating parameters for the data shard for retrieving data from the plurality of shards; determining a blocking factor based on the set of operating parameters; Dividing the data request into a plurality of API requests based on the blocking factor, each of the API requests pointing to a portion of the plurality of data shards; as well as The multiple API requests are transmitted in parallel to a source API, which is configured to perform data queries on the multiple data shards.

2. The method according to claim 1, wherein: The set of operating parameters also includes a delay parameter for one or more of the data slices.

3. The method according to claim 2, wherein: Determining the blocking factor based on the set of operating parameters further includes determining the blocking factor based on the set of operating parameters using a stateful Kalman filter.

4. The method according to claim 1, wherein: The set of operational parameters includes a query time latency parameter for one or more of the data shards and data freshness of the data received from the one or more of the data shards.

5. The method according to claim 1, wherein: The set of operational parameters includes a query time delay parameter for one or more of the data shards and integrity of the data received from the one or more of the data shards.

6. The method according to claim 1, further comprising: receiving data in response to the plurality of API requests; grouping the received data received based on a predetermined delay cutoff timeout value; as well as The packetized data is transmitted to the client.

7. The method according to claim 1, further comprising: In response to determining that one or more of the API requests have failed or have been partially filled resulting in lost data, one or more additional API requests are transmitted to the source API, and the source API is configured to perform a data query for the lost data on the multiple data shards.

8. One or more physically manufactured computer-readable storage media encoding computer-executable instructions for executing a computer process on a computer system, the computer process comprising: Receiving a data request from a client, wherein the data request points to data stored in a plurality of data shards; determining a set of latency parameters for the data shard for retrieving data from the plurality of shards; determining a blocking factor based on the set of delay parameters; Dividing the data request into a plurality of API requests based on the blocking factor, each of the API requests pointing to a portion of the plurality of data shards; as well as The multiple API requests are transmitted in parallel to a source API, which is configured to perform data queries on the multiple data shards.

9. One or more physically manufactured computer-readable storage media according to claim 8, wherein: Determining the blocking factor based on the set of delay parameters further includes determining the blocking factor based on the set of operating parameters using a stateful Kalman filter.

10. One or more physically manufactured computer-readable storage media according to claim 8, wherein: The set of delay parameters includes a query time delay parameter for one or more of the data shards and a data freshness of the data received from the one or more of the data shards.

11. A system comprising: Memory; one or more processor units; as well as a query optimization system stored in the memory and executable by the one or more processor units, the query optimization system encoding computer executable instructions on the memory for executing a computer process on the one or more processor units, the computer process comprising: Receiving a data request from a client, wherein the data request points to data stored in a plurality of data shards; determining a set of latency parameters for the data shard for retrieving data from the plurality of shards; determining a blocking factor based on the set of delay parameters; Dividing the data request into a plurality of API requests based on the blocking factor, each of the API requests pointing to a portion of the plurality of data shards; and The multiple API requests are transmitted in parallel to a source API, which is configured to perform data queries on the multiple data shards.

12. The system according to claim 11, wherein: Determining the blocking factor based on the set of delay parameters further comprises determining the blocking factor based on the set of delay parameters using a stateful Kalman filter.

13. The system according to claim 11, wherein: The set of delay parameters includes a query time delay parameter for one or more of the data shards and a data freshness of the data received from the one or more of the data shards.

14. The system according to claim 11, wherein: The computer process also includes: receiving data in response to the plurality of API requests; grouping the received data based on a predetermined delay cutoff timeout value; and The packetized data is transmitted to the client.

15. The system according to claim 11, wherein: The computer process also includes: in response to determining that one or more of the API requests have failed or have been partially filled resulting in lost data, transmitting one or more additional API requests to the source API, wherein the source API is configured to perform a data query for the lost data on the multiple data shards.

Images