Index data structures and graphical user interface

Abstract

To provide a device for computers to query and display collections of big data in a more efficient manner.SOLUTION: In a system 100, a server computer 103 associates a multi-level data structure hierarchy of index 103 with a particular date to reduce linearity of search. A client computer 112, 116, or 118 receiving results of a query stores the query results in a linked list with a record format consistent with the resulting records. The client computer also periodically updates a graphical user interface with the query results until all the results are received.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] (Related Applications) This application claims the benefit of U.S. Patent Application Serial No. 16 / 028641, filed July 6, 2018, the contents of which are hereby incorporated by reference in their entirety. [Background technology]

[0002] (background) Current computing devices and applications generate overwhelming amounts of data ("big data") that can be used for critical decisions, predictive analytics, business intelligence, etc. Such devices and applications may include, but are not limited to, sensors, mobile phones, laptops, and social networking applications. Fortunately, the cost of data storage has decreased. [Brief explanation of the drawings]

[0003] [Figure 1] FIG. 1 is an example system according to aspects of this disclosure. [Figure 2] FIG. 2 is an example flow diagram according to aspects of this disclosure. [Figure 3] 3 through 3D depict examples according to aspects of this disclosure. [Figure 3A] 3 through 3D depict examples according to aspects of this disclosure. [Figure 3B] 3 through 3D depict examples according to aspects of this disclosure. [Figure 3C] 3 through 3D depict examples according to aspects of this disclosure. [Figure 3D] 3 through 3D depict examples according to aspects of this disclosure. [Figure 4] FIG. 4 is another example flow diagram according to aspects of the disclosure. [Figure 5]5 through 5B are further example flow diagrams according to aspects of the present disclosure. [Figure 5A] 5 through 5B are further example flow diagrams according to aspects of the present disclosure. [Figure 5B] 5 through 5B are further example flow diagrams according to aspects of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0004] Big data offers many benefits, but processing such data in an efficient manner is a never-ending challenge. Traditional data processing cannot cope with the complexity and volume of big data collections. As a result, many different solutions for handling big data have evolved, and numerous database indexes have been designed. In some cases, certain queries are very common. For example, users often search for transactions that occurred on a particular date. Despite some sophistication of such queries, data indexes are not organized accordingly. On the client side, receiving and displaying the results of large data queries can also be challenging, as the results themselves are large and complex. This can lead to users experiencing slow response from their graphical user interfaces ("GUIs").

[0005] In view of the foregoing, an apparatus, a non-transitory computer-readable medium, and a method for querying and displaying collections of big data are disclosed herein. The techniques disclosed herein improve computer performance by enabling computers to query and display collections of big data in a more efficient manner. In turn, users experience less delay in the GUI, and the load placed on the database can be reduced.

[0006] In one example, an apparatus may include a memory and at least one processor that performs the following operations: generate an index in memory that includes a hierarchy of interconnected data structures and a hash table, where an entry in the hash table includes an association between a day and a first data structure in the hierarchy of interconnected data structures; receive a search request from a remote device having a plurality of search parameters including the day, search the hash table using the day for a memory address of a first data structure in the hierarchy of interconnected data structures, determine a plan for traversing the interconnected data structures based on the search parameters, initiate a search for the data sought by the search request in the first data structure according to the traversal plan, and return the results of the search to the remote device.

[0007] In a further example, a method is disclosed that may include: generating, by at least one processor, an index in memory that includes a hierarchy of interconnected data structures and a hash table, where an entry in the hash table includes an association between a day and a first data structure in the hierarchy of interconnected data structures; receiving, by the at least one processor, a search request from a remote device having a plurality of search parameters including the day; searching, by the at least one processor, the hash table for a memory address of a first data structure in the hierarchy of interconnected data structures using the day; determining, by the at least one processor, a plan for traversing the interconnected data structures based on the search parameters; initiating, by the at least one processor, a search for data sought by the search request in the first data structure according to the traversal plan; and returning, by the at least one processor, results of the search to the remote device.

[0008] In yet another example, an apparatus may include a memory, a display device, and at least one processor that performs the following operations: render a graphical user interface on the display device, transmit a search request to a remote server, receive, in response to the search request, a plurality of records where a format of each received record matches a format of a respective memory space allocated to each record, copy each received record to a separate allocated memory space, add each allocated memory space to a linked list of memory spaces in an order that reflects the order in which the plurality of records were received, and periodically render currently received records in the graphical user interface until all data records corresponding to the search request have been received.

[0009]

[0013] Several aspects, features, and advantages of the present disclosure will become apparent upon consideration of the following description of examples and accompanying drawings. The following description is not intended to limit the scope of the disclosure, and the scope of the disclosure is defined by the appended claims and their equivalents.

[0010] (Detailed explanation) FIG. 1 illustrates a schematic diagram of an exemplary system 100 for performing operations of the present disclosure. Computing devices 101, 112, 116, and 118 may each include devices capable of processing instructions and transmitting data to and receiving data from other computers. In the example of FIG. 1, computing device 101 may be a server computer including a large data repository organized as index 103. Computing devices 112, 116, and 118 may be client computers for querying data from computing device 101 and displaying query results. Each device illustrated in FIG. 1 may include all components typically associated with a computer. For example, each may have a keyboard and mouse and / or various other types of input devices, such as pen input, joystick, buttons, touchscreen, etc., as well as a display 114, which may include, for example, a CRT, LCD, plasma screen monitor, TV, projector, etc.

[0011] Each computing device shown in FIG. 1 may include at least one processor 102, such as a processor sold by Intel® Corporation, or any number of known processors. In another example, processor 102 may be an application-specific integrated circuit ("ASIC"). Each computing device may include memory 104 that holds instructions that can be retrieved and executed by processor 102. As mentioned above, memory 104 of computing device 101 may hold index 103. As described in more detail below, index 103 may be a database organized in a particular way to optimize queries against data occurring on a particular day. In one example, memory 104 may be a random access memory ("RAM") device. In a further example, memory 104 may be divided into multiple memory segments organized as dual in-line memory modules ("DIMMs"). Memory 104 may also include other types of devices, such as non-transitory computer-readable media. Non-transitory computer-readable media may include any of a number of physical media (e.g., electronic, magnetic, optical, electromagnetic, or semiconductor media). More specific examples of suitable non-transitory computer-readable media include, but are not limited to, removable magnetic computer disks such as floppy disks or hard drives, read-only memory ("ROM"), erasable programmable read-only memory, removable compact disks, or other storage devices that may be directly or indirectly coupled to a computing device. Memory 104 may also include any combination of one or more of the above and / or other devices. While all components of each computing device are functionally described as being within the same block, it will be understood that the components may or may not be maintained within the same physical housing.

[0012] 1 may be connected to one another via a network using network interface 108. The network may be a local area network ("LAN"), a wide area network ("WAN"), the Internet, etc. Network interface 108 may utilize a variety of protocols, including virtual private networks, local Ethernet networks, private networks utilizing one or more company-proprietary communication protocols, cellular and wireless networks, HTTP, and various combinations thereof. While only a few computers are depicted herein, it should be understood that the network may include additional interconnected computers.

[0013] As discussed above, instructions present in memory 104 may be retrieved and executed by processor 102. These instructions may include any collection of instructions to be executed directly (e.g., machine code) or indirectly (e.g., script) by processor 102. In this regard, the terms "instructions," "script," or "module" may be used interchangeably herein. Computer-executable instructions may be stored in any computer language or format, such as in modules of object code or source code. Furthermore, it should be understood that instructions may be implemented in the form of hardware, software, or a combination of hardware and software, and that examples herein are merely illustrative.

[0014] One example operation in accordance with the technology herein is illustrated in Figures 2-3D. Specifically, Figure 2 illustrates a flow diagram of an example method 200 for responding to a query. Figures 3-3D illustrate corresponding example operations. The operations illustrated in Figures 3-3D are described below with respect to the flow diagram of Figure 2.

[0015] 2, at least one processor may generate a hierarchy of interconnected data structures in memory, as shown in block 202. The hierarchy of interconnected data structures may be part of a database index and may include, but is not limited to, binary tree data structures, linked list data structures, vector data structures, etc. As will be described in further detail, each node of a data structure at a given level of the hierarchy may include a pointer to a corresponding node in a second data structure at the next level.

[0016] Referring now to block 204, at least one processor may generate a hash table in memory. A hash table may be another aspect of an index. In one example, each entry in the hash table may have an association between a day and a first data structure in a hierarchy of interconnected data structures. Referring now to FIG. 3, an illustrative hash table 302 is shown. Hash table 302 shows an association between day 304 (i.e., July 20, 2017) and data structure 308, and an association of day 306 (i.e., February 2, 2016) with data structure 310. In the example of FIG. 3, data structure 308 and data structure 310 are binary search trees ("BSTs"). Furthermore, data structure 308 and data structure 310 may each be the first data structure in a respective hierarchy of data structures. The illustrative hash table 302 allows queries involving days to be efficiently mapped to data associated with that day. Data structures 308 / 310 are each at the first level of a hierarchy that leads to the data sought by a query, where that data can be found. As explained in more detail below, an advantage of using a binary tree as the first layer is that the binary tree can reflect structural relationships in the data. Structural relationships can reduce the time it takes a computer to find the requested data.

[0017] Referring back to FIG. 2 , the at least one processor may receive a search request from a remote device, as shown in block 206. The search request may include a date and multiple other search parameters. In block 208, the at least one processor may use the date to search a hash table (e.g., hash table 302) for an associated memory address of a first data structure in a hierarchy of data structures. In block 210, the at least one processor of the computer may determine a plan for traversing the hierarchy of interconnected data structures based on the search parameters. In one example, the traversal plan may include determining a depth in the hierarchy where the sought data will be found based on the search parameters. The at least one processor may begin searching for the sought data in the first data structure in the hierarchy, as shown in block 212.

[0018] Referring now to FIG. 3A , a detailed description of data structure 308 is shown. As an example, the data may include transactions for a financial instrument, and the data sought by the query is all transactions for all trades executed on July 20, 2017, for the stock symbol “TTT.” As shown in FIG. 3 , the date July 20, 2017, is mapped to data structure 308. While the BST in FIG. 3A includes only seven nodes 312, 314, 316, 318, 320, 322, and 324, it is understood that a BST in a big data environment will have a much larger number of nodes, and FIG. 3A shows a small number of nodes for ease of illustration. Each memory node in this example holds a stock symbol. Root node 312 contains the stock symbol “MMM,” with nodes branching from the left of root node 312 alphabetically smaller than “MMM,” while nodes branching from the right are alphabetically larger than “MMM.” At least one processor may traverse the BST from root to leaf, making comparisons along the way to search for the "TTT" symbol. The BST is organized to allow each comparison to allow the processor to skip about half the tree, on average. In this way, the time it takes to look up a symbol may be proportional to the logarithm of the number of entries held in the tree, which is more efficient than a linear search.

[0019] The BST 308, in this example, may be the first data structure in a hierarchy of data structures. Each node in the BST shown in Figure 3A may include a pointer to a second data structure in the hierarchy. The first level of the hierarchy may include symbols, while the second level of the hierarchy may include transactions associated with that symbol for that particular day. In the example of FIG. 3A , node 312 includes a pointer to data structure 312A containing all transactions for the symbol “MMM” on July 20, 2017; node 320 includes a pointer to data structure 320A containing all transactions for the symbol “TTT” on July 20, 2017; node 314 includes a pointer to data structure 314A containing all transactions for the symbol “GGG” on July 20, 2017; node 316 includes a pointer to data structure 316A containing all transactions for the symbol “CCC” on July 20, 2017; node 318 includes a pointer to data structure 318A containing all transactions for the symbol “JJJ” on July 20, 2017; node 322 includes a pointer to data structure 322A containing all transactions for the symbol “PPP” on July 20, 2017; and node 324 includes a pointer to data structure 324A containing all transactions for the symbol “YYY” on July 20, 2017. Although the example of Figure 3A includes only two levels of data, a hierarchy may include as many levels as needed to address a particular query, i.e., the data may include more levels to make the search less linear.

[0020] Referring now to FIG. 3B, a detailed description of data structure 320A is shown. A processor may arrive at node 320 of the BST when searching for "TTT" transactions on July 20, 2017. However, in this working example, the first-level data structure 308 contains only symbols. Transactions may be found at the next level. As noted above, data structure 320A is at the second level of the hierarchy and contains all transactions for the symbol "TTT" on July 20, 2017. In the example of FIG. 3B, data structure 320A may be a vector data structure. The vector data structure of FIG. 3B shows three nodes 326, 328, and 330 representing sequences for the symbol "TTT" on July 20, 2017. Each node contains further linked lists branching off from it, representing the transactions for each sequence. For example, node 326 may represent an order with two transactions 326A and 326B branching off from it, node 328 may represent an order with three transactions 328A, 328B, and 328C branching off from it, and node 330 may represent an order with one transaction branching off from it. Referring back to the method of FIG. 2, the results of the query are returned as shown in block 214. Referring back to FIG. 3B, the data in vector data structure 320A may be sent back to the device requesting this information. While this example uses a vector data structure for the order, it will be understood that in other situations, other types of data structures may be used, such as an array, a linked list, etc.

[0021] As mentioned above, a data structure may contain more than two levels of data, depending on the most frequently encountered queries. In the example of FIG. 3C, a binary data structure 310 is shown with three levels branching off from it. As seen previously in FIG. 3, data structure 310 is associated with the date February 2, 2016 in hash table 302. The illustrative hierarchy of FIG. 3C is a three-tiered data hierarchy. Specifically, the example of FIG. 3C includes a level between symbols representing customers and transactions. This organization can be useful when queries for symbol transactions by a particular customer are common. In the example hierarchy of FIG. 3C, node 332 includes a symbol "MMM" and a pointer to a data structure 332A containing customers associated with the symbol "MMM," ​​which in turn includes a pointer to a transaction data structure 332B representing transactions for the symbol "MMM" by the customer. Node 344 includes a pointer to a data structure 344A containing the symbol "GGG" and customers involved with the symbol "GGG," which in turn contains a pointer to a transaction data structure 344B indicating transactions by customers for the symbol "GGG." Node 334 includes a pointer to a data structure 334A containing the symbol "TTT" and customers involved with the symbol "TTT," which in turn contains a pointer to a transaction data structure 334B indicating transactions by customers for the symbol "TTT." Node 342 includes a pointer to a data structure 342A containing the symbol "CCC" and customers involved with the symbol "CCC," which in turn contains a pointer to a transaction data structure 342B indicating transactions by customers for the symbol "CCC." Node 340 contains a pointer to a data structure 340A containing the symbol "JJJ" and customers involved with the symbol "JJJ", which in turn contains a pointer to a transaction data structure 340B indicating transactions for the symbol "JJJ" by customers.Node 338 contains a pointer to a data structure 338A containing the symbol "PPP" and customers who have engaged with the symbol "PPP", which in turn contains a pointer to a transaction data structure 338B indicating transactions by customers for the symbol "PPP". Node 336 contains a pointer to a data structure 336A containing the symbol "YYY" and customers who have engaged with the symbol "YYY", which in turn contains a pointer to a transaction data structure 336B indicating transactions by customers for the symbol "YYY".

[0022] Referring now to FIG. 3D, a detailed description of customer data structure 334A is shown. As seen in FIG. 3C, node 334 of the symbol "TTT" is associated with data structure 334A by way of a pointer. Data structure 334A can also be organized as a BST with a root node 352 containing customer "JPMorgan," a node 354 containing customer "Chase," and a node 356 containing customer "NFSC." FIG. 3C also depicts each customer node containing a respective pointer to the transaction data structure. In the event that a query is received for a transaction of "TTT" made by customer "Chase" on February 2, 2016, at least one processor may first traverse BST structure 310 to reach node 334 containing the symbol "TTT," which leads to BST structure 334A. The processor may then traverse BST structure 334A to find customer "Chase" at node 354, which leads to transaction data structure 334B. Data from transaction data structure 334B may be returned in response to a query. Adding an extra BST that includes the customer's name may improve the efficiency of searching for a customer's transactions that occurred on a particular day by making the search less linear.

[0023] The hierarchies described above are merely exemplary. As such, a hierarchy can include different combinations of data structures. For example, the first level can be a BST, the second level can be an array, and the third level can be another BST. The combination of data structures used in a particular hierarchy can address the most frequently encountered queries. A hierarchy for a particular system can be designed to reduce search linearity. However, different situations may call for different data structure arrangements.

[0024] Another area where performance of big data queries can be improved is in the client devices requesting the data. Example operations for displaying the results of a big data query are shown in Figures 4 through 5B. Specifically, Figure 4 shows a flow diagram of an example method 400 for displaying the results of a query, and Figures 5 through 5B show corresponding example operations. The operations shown in Figures 5 through 5B are described below in conjunction with the flow diagram of Figure 4.

[0025] Referring to FIG. 4 , at least one processor of the computing device renders a GUI on a screen, as shown in block 402, and transmits a search request to a remote server, as shown in block 404. In block 406, the at least one processor may receive multiple records in response to the search request, and in block 408, each received record may be copied to a separate allocated memory space. In one example, each received record matches the format of the respective memory space allocated to each record. Keeping the format between the memory space and the record consistent may allow the at least one processor to efficiently copy fields of each received record to corresponding fields in memory. As such, the time to process the resulting records may be reduced because there is no need to reformat fields, data types, or data objects.

[0026] 5, computing device 101 may be a server computer that provides records to computing device 112, which may be a requesting client computer. Computing device 101 is shown returning records 502 to computing device 112. The records may be displayed on display device 508.

[0027] Referring back to FIG. 4 , as shown in block 410, each allocated memory space may be added to a linked list of memory spaces. In one example, the order of the linked list may reflect the order in which the records were received. FIG. 5 shows a linked list 504 of records in memory 104. Referring back to FIG. 4 , as shown in block 412, currently received records may be periodically rendered to the GUI. In one example, records may be rendered after a predetermined period of time, such as every second. An advantage of periodically updating the GUI is that the user can still see the currently received results while the remaining results are still being transferred. Thus, if a network delay occurs, such delay may be hidden from the user because the user can already see the results on the GUI. In one example, the updates continue periodically until all results are received.

[0028] Referring to FIG. 5A, an illustrative GUI 510 with the results of a query of transaction data is shown. The GUI includes fields that would be typical for transaction data, such as, but not limited to, status, symbol, price, and size. In one example, at least one processor of a client device may detect user selection of a given row displayed on the GUI. In response to the selection, the processor may display aggregated data corresponding to the given row. As an example, a user may click on a symbol (“Sym”) row in GUI 510. As a result, at least one processor may render another GUI 514 that aggregates all sequences associated with the symbol row, as shown in FIG. 5B. In one example, in response to a user clicking on a symbol row, at least one processor may perform a linear traversal of a linked list (e.g., linked list 504) using the symbol row as a key. That is, a user may click on any row, and at least one processor may use the selected row as a key for the linear traversal.

[0029] Advantageously, the above-described apparatus, non-transitory computer-readable medium, and method generate an index comprising a hash table and a multi-level data structure that may be optimal for general queries about transactions that occurred on a particular date. Furthermore, the client device may allocate memory with a format that matches the format of the query results and periodically update the query results on a GUI. This time, even if there is a delay in retrieving the results, the user can still view the data on the GUI. Thus, the above-disclosed index and GUI techniques improve computer performance by accelerating big data queries at the server and accelerating the display of query results at the client.

[0030] While the disclosure herein has been described with reference to specific examples, it should be understood that these examples are merely illustrative of the principles of the disclosure. It should be understood, therefore, that numerous modifications may be made to the examples, and that other variations may be made without departing from the spirit and scope of this disclosure, as defined by the appended claims. Furthermore, although specific processes are shown in a particular order in the accompanying figures, such processes are not limited to any particular order unless such order is expressly set forth herein. Rather, various steps may be performed in different orders or simultaneously, and steps may be omitted or added.

[0031] (Addendum) (Appendix 1) Memory and generating an index in said memory including a hierarchy of interconnected data structures and a hash table, wherein an entry in said hash table includes an association between a date and a first data structure in said hierarchy of interconnected data structures; receiving a search request from a remote device, the search request having a plurality of search parameters including the date; searching the hash table for a memory address of the first data structure in the hierarchy of interconnected data structures using the date; determining a plan for traversing the interconnected data structure based on the search parameters; Initiating a search for the data sought by the search request in the first data structure according to the traversal plan; returning results of the search to the remote device. at least one processor; Equipped with Device.

[0032] (Appendix 2) the at least one processor is further configured to determine a plan for the traversal, such as to determine a depth to which the hierarchy should be traversed to obtain the data sought by the search request. 10. The apparatus described in Appendix 1.

[0033] (Appendix 3) the first data structure is at a first level in the hierarchy of interconnected data structures; 10. The apparatus described in Appendix 1.

[0034] (Appendix 4) the first data structure is a first binary tree data structure; 10. The apparatus described in Appendix 3.

[0035] (Appendix 5) the at least one processor is further configured to generate, at each node of the first binary tree data structure, a pointer to a second data structure at a second level in the hierarchy. 10. The apparatus described in Appendix 4.

[0036] (Appendix 6) the second data structure is a vector data structure; 10. The apparatus described in Appendix 5.

[0037] (Appendix 7) the second data structure is a second binary tree data structure. 10. The apparatus described in Appendix 5.

[0038] (Appendix 8) the at least one processor is further configured to generate, at each node of the second binary tree data structure, a pointer to a third data structure at a third level in the hierarchy. 10. The apparatus described in Appendix 7.

[0039] (Appendix 9) generating, by at least one processor, an index in memory including a hierarchy of interconnected data structures and a hash table, wherein an entry in the hash table includes an association between a date and a first data structure in the hierarchy of interconnected data structures; receiving, by the at least one processor, a search request from a remote device, the search request having a plurality of search parameters including the date; searching, by the at least one processor, the hash table for a memory address of the first data structure in the hierarchy of interconnected data structures using the date; determining, by the at least one processor, a plan for traversing the interconnected data structure based on the search parameters; initiating, by the at least one processor, a search for data sought by the search request in the first data structure according to the traversal plan; returning, by the at least one processor, results of the search to the remote device; Including, method.

[0040] (Appendix 10) determining the traversal plan includes determining, by the at least one processor, a depth to which the hierarchy should be traversed to obtain the data sought by the search request. The method described in Appendix 9.

[0041] (Appendix 11) the first data structure is at a first level in the hierarchy of interconnected data structures; The method described in Appendix 9.

[0042] (Appendix 12) the first data structure is a first binary tree data structure; The method described in Appendix 11.

[0043] (Appendix 13) generating, by the at least one processor, a pointer at each node of the first binary tree data structure to a second data structure at a second level in the hierarchy; 12. The method described in Appendix 12.

[0044] (Appendix 14) the second data structure is a vector data structure; 12. The method described in Appendix 12.

[0045] (Appendix 15) the second data structure is a second binary tree data structure. The method described in Appendix 14.

[0046] (Appendix 16) generating, by the at least one processor, a pointer at each node of the second binary tree data structure to a third data structure at a third level in the hierarchy; 16. The apparatus of claim 15.

[0047] (Appendix 17) Memory and A display device; Rendering a graphical user interface on the display device; Transmitting the search request to a remote server; receiving a plurality of records in response to the search request, the format of each received record matching the format of a respective memory space allocated to each respective record; Copy each received record into a separate allocated memory space; adding each allocated memory space to a linked list of memory spaces, the linked list indicating an order that reflects the order in which the plurality of records were received; periodically rendering currently received records in the graphical user interface until all data records corresponding to the search request have been received; at least one processor; Equipped with Device.

[0048] (Appendix 18) the at least one processor is configured to render the currently received recording upon expiration of a predetermined period of time; 18. The apparatus of claim 17.

[0049] (Appendix 19) The at least one processor: detecting a selection of a predetermined field displayed in the graphical user interface; responsive to said selection, rendering another graphical user interface including aggregated data corresponding to said predetermined field; and further configured as follows: 18. The apparatus of claim 17.

[0050] (Appendix 20) The predetermined period is about 1 second. 18. The apparatus of claim 17.

Claims

[Claim 1] Memory and generating an index in said memory comprising a hierarchy of interconnected data structures and a hash table, wherein an entry in said hash table contains an association between a date and a first data structure in said hierarchy of interconnected data structures; receiving a search request from a remote device, the search request having a plurality of search parameters including the date; searching the hash table for a memory address of the first data structure in the hierarchy of interconnected data structures using the date; determining a plan for traversing the interconnected data structure based on the search parameters; Initiating a search for the data sought by the search request in the first data structure according to the traversal plan; returning results of the search to the remote device. at least one processor; Equipped with Device.

Images