System and method for paginated query of real-time data

Abstract

The embodiment of the application relates to the technical field of big data, and particularly relates to a system and a method for paging query of real-time data. The system comprises a client node, a service node and at least two distributed data nodes. The service node is configured to send a paging query request matched with the data node to the data node in response to a paging query request sent by the client node. The data node is configured to take out data records meeting a query condition through a paging query adapter and return the data records to the service node in response to the paging query request sent by the service node. The service node is further configured to sort and integrate the data records meeting the paging query condition returned by different data nodes through a deployed paging query integrator, and return query result records of the paging query to the client node. The application supports paging query to meet the requirements of big data query and display, and meets the requirements of high efficiency and low cost, and achieves the effect of real-time data query.

Description

Technical Field

[0001] This invention relates to the field of big data technology, and in particular to a real-time data pagination query system and method. Background Technology

[0002] With the development of software technology and the popularization of software applications, software functions are becoming increasingly diverse, and the software used by users is often a distributed system in terms of technical implementation. For example, in some scenarios, users may request to query data, which may be distributed across different physical systems. Furthermore, because the amount of data to be queried is too large, it cannot be returned to the customer all at once, otherwise it would lead to excessively long transaction times; or the returned data may exceed the cache space or communication message length limits. Therefore, queries need to be paginated, meaning that only one page of data is retrieved and returned to the customer at a time, and the next query is performed only when the customer needs to continue querying the next page. Therefore, for scenarios where users use software in real time, ensuring that queries are returned in real time, minimizing query latency, and reducing system overhead and performance become particularly important. Summary of the Invention

[0003] This invention provides a real-time data pagination query system and method to support pagination queries to meet the requirements of large-scale data query and display, while also meeting the requirements of high efficiency and low cost.

[0004] In a first aspect, embodiments of the present invention provide a real-time data pagination query system, comprising: a client node, a business node, and at least two distributed data nodes, wherein a pagination query integrator is deployed on the business node, and a pagination query adapter is deployed on the data nodes; wherein:

[0005] The business node is configured to respond to a pagination query request sent by a client node and send a pagination query request that matches the data node to the data node.

[0006] The data node is configured to respond to the pagination query request sent by the business node, retrieve the data records that meet the query conditions through the pagination query adapter, and return them to the business node;

[0007] The business node is also configured to sort and integrate the data records that meet the pagination query conditions returned by different data nodes through the deployed pagination query integrator, and return the pagination query result records to the client node.

[0008] Secondly, this invention also provides a method for paginated querying of real-time data, the method comprising:

[0009] The business node responds to the pagination query request sent by the client node and sends a pagination query request that matches the data node to the data node; a pagination query integrator is deployed on the business node, and a pagination query adapter is deployed on the data node;

[0010] The data node responds to the pagination query request sent by the business node, retrieves the data records that meet the query conditions through the locally pre-deployed pagination query adapter, and returns them to the business node;

[0011] The business node also sorts and integrates the data records that meet the pagination query conditions returned by different data nodes through the deployed pagination query integrator, and returns the pagination query result records to the client node.

[0012] The real-time data pagination query system provided in this embodiment of the invention adds a pagination query integrator to the business nodes and a pagination query adapter to the data nodes. Each data node can retrieve data records that meet the conditions through the deployed pagination query adapter and return them to the business nodes. The business nodes can sort and integrate the data retrieved from each data node through the pagination query integrator and return it to the client node. There is no need to add an additional intermediate physical node to perform asynchronous data caching and integration of data from different data nodes. Only the corresponding software system needs to be added. It can directly access multiple data nodes and simultaneously query and sort data from the distributed heterogeneous system in real time. It supports pagination queries to meet the requirements of large-scale data query and display, and meets the requirements of high efficiency and low cost, achieving the effect of real-time data query.

[0013] The above description of the invention is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0014] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings. The drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0015] Figure 1 This is a structural block diagram of a real-time data pagination query system provided in an embodiment of the present invention;

[0016] Figure 2 This is a structural block diagram of a typical distributed query system provided in an embodiment of the present invention;

[0017] Figure 3 This is an architecture diagram of a real-time data pagination query system provided in an embodiment of the present invention;

[0018] Figure 4 This is a diagram illustrating the change process of the pagination query adder at different nodes during the first pagination query provided in this embodiment of the invention;

[0019] Figure 5 This is a diagram showing the changes in the pagination query adder at different nodes during a non-first pagination query, as provided in an embodiment of the present invention.

[0020] Figure 6 This is a schematic diagram of the data records returned by each data node during the first pagination query provided in this embodiment of the invention;

[0021] Figure 7 This is a schematic diagram of the data records returned by each data node during a non-first pagination query provided in an embodiment of the present invention;

[0022] Figure 8 This is a flowchart of a real-time data pagination query method provided in an embodiment of the present invention;

[0023] Figure 9 This is a flowchart of another real-time data pagination query method provided in this embodiment of the invention;

[0024] Figure 10 This is a flowchart of another real-time data pagination query method provided in this embodiment of the invention;

[0025] Figure 11 This is a diagram showing the changes in the pagination query add-on for different nodes during a pagination query, as provided in an embodiment of the present invention. Detailed Implementation

[0026] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0027] Before discussing the exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the operations (or steps) as sequential processes, many of these operations can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but it may also have additional steps not included in the figures. The process may correspond to a method, function, procedure, subroutine, subroutine, etc.

[0028] The pagination query system and method for real-time data provided in the embodiments of the present invention will be described in detail below through various embodiments and optional solutions of each embodiment.

[0029] Figure 1 This is a structural block diagram of a real-time data pagination query system provided in an embodiment of the present invention. This embodiment is applicable to situations involving pagination queries of real-time data. Figure 1 As shown, the real-time data pagination query system 100 in this embodiment may include the following: a client node 110, a business node 120, and at least two distributed data nodes 130. A pagination query integrator is deployed on the business node 120, and a pagination query adapter is deployed on the data nodes 130. Wherein:

[0030] Business node 120 is configured to respond to pagination query requests sent by client node 110 and send pagination query requests that match data node 130 to data node 130.

[0031] Data node 130 is configured to respond to pagination query requests sent by business node 120, retrieve data records that meet the query conditions through the pagination query adapter, and return them to business node 120.

[0032] Business node 120 is also configured to sort and integrate the data records that meet the pagination query conditions returned by different data nodes 130 through the deployed pagination query integrator, and return the pagination query result records to client node 110.

[0033] See Figure 2 A typical distributed query system usually consists of clients, business nodes, and at least two data nodes. Client nodes can be systems directly used by users, business nodes can be systems that process client requests, and data nodes can be systems that ultimately provide application services and store data. The services and data in a distributed system come from multiple independent systems. Users query data through clients, which send requests to business nodes. These business nodes then process the requests and request data from different data nodes. When the data distributed across different data nodes has logical relationships and needs to be sorted according to specific options, it is generally not possible to query each data node sequentially; that is, it is not possible to query only one data node at a time. Instead, data from multiple data nodes must be obtained simultaneously and sorted according to requirements.

[0034] See Figure 2Taking data nodes corresponding to an e-wallet system, a debit card system, and a credit card system as an example, the client app can manage a user's e-wallet, debit card, and credit card, and users can use these three systems to make online payments. When a user checks their bill, the app simultaneously displays the bills for all three payment methods: e-wallet, debit card, and credit card. Since users can choose any payment method—e-wallet, debit card, or credit card—the payment records for these three systems do not occur in a fixed order. Therefore, when displaying these records to the user, they need to be arranged in chronological order from most recent to oldest; otherwise, the display would appear disorganized.

[0035] Based on the above, see Figure 1 This application further develops upon a typical distributed query system by adding a pagination query integrator to business nodes and a pagination query adapter to data nodes. When constructing a real-time data pagination query system, the system defines the data nodes required for business node queries, the sorting rules and primary keys for the data in each data node, and the sorting rules for the aggregated data returned from each data node. The pagination query adapter performs data queries based on pagination query requests and retrieves the paginated data records; while the query integrator sorts and integrates the data returned from different data nodes to obtain the data results of this pagination query and returns them to the client.

[0036] The real-time data pagination query system provided in this embodiment of the invention does not require additional intermediate physical nodes to asynchronously cache and integrate data from different data nodes. Only the corresponding software system needs to be added. It can directly access multiple data nodes and simultaneously query and sort data from the distributed heterogeneous system in real time. It supports pagination queries to meet the requirements of querying and displaying large amounts of data, and meets the requirements of high efficiency and low cost. The queried data does not need to be transferred and can achieve the effect of real-time data query.

[0037] In one optional embodiment, this can be combined with various optional embodiments described above. See also... Figure 1 A pagination query attacher is deployed on client node 110, business node 120, and data node 130, respectively. The pagination query attacher is used to record and transmit the input component information of the pagination query start and end records and the output component information of the pagination output start and end records of the corresponding node to which the pagination query attacher belongs during each pagination query process. The input and output components of the pagination query attacher contain the information required for the pagination query. Optionally, the pagination query attacher, pagination query adapter, and pagination query integrator are implemented using software algorithms.

[0038] By employing the above method, pagination query adders are added to the client, business nodes, and data nodes. These adders record and transmit the information needed for pagination during the query process, achieving the effect of pagination queries in a distributed system and resolving the pagination problem when clients perform multiple queries. Furthermore, the added pagination query adders, adapters, and integrators only require software development on top of the existing system; no additional hardware such as servers and databases is needed, resulting in very low costs.

[0039] Optionally, the input component information of the pagination query adder includes the following parameters: a page jump direction indicator parameter, a start and end record key value parameter for the pagination query, and a start and end record quantity parameter for the pagination query. The start and end record key value parameter specifies the starting record of the current pagination query. For example, referring to Table 1, the specific input component information of the pagination query adder includes: a page jump indicator parameter PAGE_UP_DOWN, a start record key value PAGE_STA_KEY for the pagination query, a stop record key value PAGE_END_KEY for the pagination query, the number of records per page REC_IN_PAGE for the pagination query, and the page number to jump to in the pagination query.

[0040] Table 1. Parameter Definitions for the Pagination Query Add-in Component

[0041]

[0042] Optionally, the output component information of the pagination query adder includes the following parameters: the start and end record key values ​​for pagination output and the pagination query scroll-down end flag parameter. Referring to Table 2, the specific output component information of the pagination query adder includes: the start record key value PAGE_STA_KEY for pagination output, the end record key value PAGE_END_KEY for pagination output, the scroll-down end flag parameter CONV_NO_DATA for pagination query, and the total number of records in the pagination output TOTAL_REC.

[0043] Table 2. Parameter Definitions of the Pagination Query Add-on Output Component

[0044]

[0045] In one optional embodiment, this can be combined with various optional embodiments described above. See also... Figure 1 The pagination query method in this application adopts a page-turning method. The data structure of the pagination query add-in of both the client node and the business node is an array with a length of 3. The pagination query add-in of the business node corresponds to the pagination query add-in of at least two data nodes; for example, the business node corresponds to at least two data nodes, namely the e-wallet, debit card and credit card data nodes.

[0046] In one optional embodiment, this can be combined with various optional embodiments described above. See also... Figure 1 Client node 110 is configured to send user query requests to its main query service and the corresponding deployed pagination query adder. Furthermore, client node 110 is also configured to assemble the main query message using its local main query service and assemble the input component information using its locally deployed pagination query adder, and then send the corresponding pagination query request to business node 120.

[0047] In one optional embodiment, this can be combined with various optional embodiments described above. See also... Figure 1 Specifically, business node 120 is configured to send the main query message carried in the pagination query request to the main query service of business node 110; and to pass the input component information of the pagination query attacher on client node 110 carried in the pagination query request to the input component of the pagination query attacher on business node 120. Simultaneously, business node 120 is also specifically configured to assemble the main query message through its local main query service and assemble the input component information through its locally deployed pagination query attacher, and send corresponding matching pagination query requests to each data node 130. The pagination query requests then invoke each data node 130 to perform data pagination query services and send the input component information of the locally deployed pagination query attacher to the input component of the pagination query attacher on each data node.

[0048] In one optional embodiment, this can be combined with various optional embodiments described above. See also... Figure 1 Specifically, data node 130 is configured to send the main query message carried in the pagination query request from business node 120 to the main query service of data node 130, and to pass the input component information of the pagination query attacher on business node 120 carried in the pagination query request to the input component of the pagination query attacher on data node 130. Simultaneously, data node 130 is also configured to send the main query message carried in the pagination query request to the pagination query adapter of data node 130 through the local main query service, and to send the input component information of the locally deployed pagination query attacher to the locally deployed pagination query adapter; and to control the pagination query adapter to retrieve data records that meet the pagination query conditions based on the main query message and the input component information of the locally deployed pagination query attacher.

[0049] As an optional option, see Figure 1During each pagination query, the input component information of the pagination query add-on of the client node 110 is transmitted to the input component of the pagination query add-on of the business node 120 and recorded in the corresponding input component. Subsequently, the input component information of the pagination query add-on of the business node 120 is transmitted to the input component of the pagination query add-on of the data node 130 and recorded in the corresponding input component. This allows each data node 130 to activate the query adapter after receiving the pagination query request from the business node 120, and to query a new page of data records based on the main query message and the input component information of the locally deployed pagination query add-on.

[0050] Optionally, the main query message may include the user account to be queried, the start date of the record to be queried, and the end date of the record to be queried. See also Figure 3 Taking data nodes 1, 2, and 3 as an e-wallet system, a debit card system, and a credit card system, respectively, as an example, when the user's query record is a payment transaction record, the main query message can include the following: the user's payment account, the start date of the payment transaction, and the end date of the payment transaction. The main query message allows the data nodes to initially filter data records that meet the conditions. However, if there are many data records initially filtered, it is difficult to display them all on one page and they need to be displayed in pages. The input component information of the locally deployed pagination query add-in can instruct the data nodes to retrieve one page of data records for this pagination query.

[0051] Optionally, see Figure 4 When the pagination query request sent by client node 110 is the first query, the page jump direction indicator parameter and the start and end record key value parameter of the pagination query in the input component information of each node are empty, while the start and end record quantity parameter of the pagination query is a preset quantity value, which is convenient for filtering a specified number of data records. Moreover, the parameter values ​​in the input component information of the pagination query adder of different nodes are the same during a single pagination query. For example, the page jump direction indicator parameter PAGE_UP_DOWN, the start record key value PAGE_STA_KEY, and the end record key value PAGE_END_KEY are all empty, and the number of records per page REC_IN_PAGE is assigned a value of 20, that is, 20 data records are queried per page during pagination. In addition, because it is necessary to query data from three parts: e-wallet, debit card, and credit card, the values ​​of each parameter in the input component information of the pagination query adder are arrays containing the element values ​​of the three data nodes under the corresponding parameters.

[0052] Optionally, see Figure 4 and Figure 5When the pagination query request sent by client node 110 is not the first query, the input component information of the pagination query adder of client node 110 is determined based on the start and end record key value parameters of the pagination output in the output component information of the pagination query adder of client node 110 during the previous pagination query. For example, Figure 4 The output component information of the pagination query adder of the client node shown is used as the output component information of the pagination query adder during the previous pagination query. The key value of the first record of the pagination output and the key value of the last record of the pagination output in the output component information can be used as the key value of the first record of the pagination query and the key value of the last record of the pagination query in the input component information of the pagination query adder of the client node 110 during the current pagination query.

[0053] Optionally, when initiating another pagination query, the client node determines the value of the page jump direction indicator parameter PAGE_UP_DOWN based on the user's input, i.e., scrolling up or down; it determines the corresponding information of its query adder input component based on the pagination query adder output component information from the client node's previous pagination query, specifically the start record key and end record key value of the pagination query in the client query adder output component information; and it determines whether there are still records to be queried on the data node based on the pagination query end flag parameter in the pagination query adder output component information from the client node's previous pagination query.

[0054] In one optional embodiment, this can be combined with various optional embodiments described above. Retrieving data records that meet the pagination query conditions includes: searching for data records that meet the query conditions indicated by the main query message, and retrieving a preset number of data records from the searched data records that meet the query conditions indicated by the input component information of the pagination query add-in on the data node.

[0055] For the first pagination query, see [link / reference]. Figure 6 The data records returned by data node 1 that meet the query conditions indicated by the input component information of the pagination query adder on the data node include the data corresponding to DW1-DW20; the data records returned by data node 2 that meet the query conditions indicated by the input component information of the pagination query adder on the data node include the data corresponding to DC1-DC20; the data records returned by data node 3 that meet the query conditions indicated by the input component information of the pagination query adder on the data node include the data corresponding to CC1-CC20.

[0056] For non-first pagination queries, see [link / reference] Figure 7Data node 1 returns data records corresponding to DW8-DW27 that match the query conditions indicated by the input component information of the pagination query adder on the data node. Upon subsequent queries, due to insufficient data in data node 2, data node 2 returns data records corresponding to DC8-DC17 that match the query conditions indicated by the input component information of the pagination query adder on the data node. Similarly, upon subsequent queries, due to insufficient data in data node 3, data node 3 returns data records corresponding to CC7-CC16 that match the query conditions indicated by the input component information of the pagination query adder on the data node.

[0057] In one optional embodiment, this can be combined with various optional embodiments described above. Sorting and integrating data records returned by different data nodes that meet the pagination query conditions includes: integrating and sorting the data records returned by different data nodes that meet the pagination query conditions according to a preset sorting field agreed upon with the data nodes, and retrieving a preset number of data records that are ranked first from the integrated and sorted data records; for the retrieved preset number of data records, obtaining the start record key value and the end record key value of the sorted data records retrieved from each data node.

[0058] See Figure 6 and Figure 7 Whether it's the first pagination query or a subsequent query, the initial data selection often results in a large number of records that are difficult to display on a single page, necessitating pagination. Therefore, the data records returned by each data node are uniformly sorted and integrated. Then, based on the input information of the pagination query add-in, a preset number of data records are retrieved according to the query conditions. The number to the right of each column of data for each data node indicates the order of that data record in the set of data records returned by each data node (e.g., 60 records). Finally, the 20 records in the shaded area represent the data records that business node 120 needs to return to client node 110.

[0059] Using the above method, different data nodes can be sorted separately according to their respective systems, and then sorted uniformly by business nodes. This supports situations where different data nodes are heterogeneous data sources (i.e., the data structures of data records are different), making the application scenarios very broad. It solves the problem that data from different data nodes must have the same structure and cannot handle heterogeneous data, and solves the query problem of multiple heterogeneous data nodes.

[0060] In one optional embodiment, this can be combined with various optional embodiments described above. When the page jump direction indicator parameter is an upward query, the input component information of the pagination query adder is used to indicate the data records whose key value is greater than the key value of the starting record in the input component information; when the page jump direction indicator parameter is a downward query, the input component information of the pagination query adder is used to query the data records whose key value is less than the key value of the starting record in the input component information. Simultaneously, the pagination query processing is enhanced by sorting the data records by their key values ​​to obtain the preset number of data records indicated by the single-page record count of the pagination query in the input component of the pagination query adder.

[0061] As an optional solution, when designing a paginated query service, the key values ​​of the data records for each data node can be predetermined. Since each data node is a heterogeneous system with different key values ​​for its data records, each data node can query data records using these key values, ensuring that multiple queries do not contain duplicate data. For example, in an e-wallet system, the key value for payment records is wallet account + transaction date + sequence number within that date.

[0062] As an optional approach, when integrating and sorting data records that meet the pagination query criteria returned by different data nodes, the sorting field after the data from each data node is aggregated into the business node must be a field that exists in the data records of each data node. For example, in a payment system, the sorting field could be the transaction timestamp. For instance, the data records of the corresponding data nodes in e-wallet systems, debit card systems, and credit card systems all include the transaction timestamp field.

[0063] As an optional approach, for a single data node, the sorting result of the data records retrieved from that single data node by the business node is consistent with the sorting result of the key values ​​when querying data records on each data node separately. The key value sorting on each data node and the sorting by the business node are performed using a unified field. The fields used can differ, but the sorting result must be consistent for the records on each data node.

[0064] As an optional solution, business node 120 sends the input component information of the pagination query attacher of client node 110 to the input components of the pagination query attachers of different data nodes 130. Data node 130 activates its query adapter, queries a new page of data records based on the input component information in the pagination query attacher, and returns the returned data records to the business node through the output component of the query attacher. The query integrator of the business node obtains the page of data records returned by each data node, sorts them according to a unified sorting criterion, filters out the data from each node that appears on the result page, and forms an array of information from the output components of the business node's query attacher, which is then returned to the client. The client node receives the query result page from the business node and displays it to the user. Its pagination query attacher's output component receives the output component information from the business node and determines the input component information for the pagination query attacher for the next page turn based on this information.

[0065] As an optional solution, the pagination query integrator is implemented in software. It can sort the data returned by multiple data nodes according to a specified field and retrieve a specified number of data records. It can also retrieve the key values ​​of the first and last records of the sorted data retrieved from each data node, as well as the end flag of each data node's scroll-down.

[0066] Figure 8 This is a flowchart illustrating a paginated query method for real-time data provided in an embodiment of the present invention. This embodiment is applicable to situations involving paginated queries of real-time data. Figure 8 As shown, the real-time data pagination query method in this embodiment may include the following steps S810-S830:

[0067] S810, the business node responds to the pagination query request sent by the client node and sends a pagination query request matching the data node to the data node; wherein a pagination query integrator is deployed on the business node and a pagination query adapter is deployed on the data node.

[0068] S820: The data node responds to the pagination query request sent by the business node, retrieves the data records that meet the query conditions through the locally pre-deployed pagination query adapter, and returns them to the business node.

[0069] S830. The business node also sorts and integrates the data records that meet the pagination query conditions returned by different data nodes through the deployed pagination query integrator, and returns the pagination query result records to the client node.

[0070] Based on the above embodiments, optionally, a pagination query adder is deployed on the client node, the business node, and the data node respectively. The pagination query adder is used to record and transmit the input component information of the pagination query start and end record and the output component information of the pagination output start and end record in the node to which the corresponding pagination query adder belongs during each pagination query process.

[0071] Optionally, based on the above embodiments, before sending a pagination query request matching the data node to the data node in response to the pagination query request sent by the client node, the method further includes:

[0072] The client node sends the user query request to the client node's main query service and the corresponding deployed pagination query add-in; and, through the local main query service, it assembles the main query message and through the locally deployed pagination query add-in, it assembles the input component information and sends the corresponding pagination query request to the business node.

[0073] Based on the above embodiments, optionally, in response to a pagination query request sent by a client node, a pagination query request matching the data node is sent to the data node, including:

[0074] The business node sends the main query message carried in the pagination query request to the main query service of the business node and passes the input component information of the pagination query add-on on the client node carried in the pagination query request to the input component of the pagination query add-on on the business node; and, through the local main query service, it assembles the main query message and through the locally deployed pagination query add-on, it assembles the input component information and sends the corresponding matching pagination query request to each data node.

[0075] Based on the above embodiments, optionally, in response to the pagination query request sent by the business node, data records that meet the query conditions are retrieved through the pagination query adapter, including: the data node sends the main query message carried in the pagination query request to the main query service of the data node and transmits the input component information of the pagination query add-on on the business node carried in the pagination query request to the input component of the pagination query add-on on the data node;

[0076] The data node also sends the subject query message carried in the pagination query request to the pagination query adapter of the data node through the local subject query service and sends the input component information of the locally deployed pagination query add-on to the pagination query adapter; and controls the pagination query adapter to retrieve data records that meet the pagination query conditions based on the subject query message and the input component information of the locally deployed pagination query add-on.

[0077] Based on the above embodiments, optionally, the main query message includes the user account to be queried, the start date of the record to be queried, and the end date of the record to be queried.

[0078] The input component information of the pagination query adder includes the following parameters: page jump direction indicator parameter, start and end record key value parameter of pagination query, and start and end record quantity parameter of pagination query; the output component information of the pagination query adder includes the following parameters: start and end record key value parameter of pagination output, and down-scrolling end flag parameter of pagination query.

[0079] Based on the above embodiments, optionally, when the pagination query request sent by the client node is the first query, the page jump direction indicator parameter and the start and end record key value parameter of the pagination query in the input component information of the pagination query adder are empty, the start and end record quantity parameter of the pagination query is a preset quantity value, and the parameter values ​​in the input component information of the pagination query adder of different nodes are the same during a pagination query process.

[0080] Based on the above embodiments, optionally, when the pagination query request sent by the client node is not the first query, the input component information of the pagination query adder is determined based on the start and end record key value parameters of the pagination output in the output component information of the client node's pagination query adder during the previous pagination query.

[0081] Based on the above embodiments, optionally, when the page jump direction indicator parameter is an upward query, the input component information of the pagination query adder is used to indicate the query key value of data records whose key value is greater than the key value of the starting record in the input component information; when the page jump direction indicator parameter is a downward query, the input component information of the pagination query adder is used to query data records whose key value is less than the key value of the starting record in the input component information.

[0082] Based on the above embodiments, optionally, retrieving data records that meet the pagination query conditions includes:

[0083] Find the data records that match the query conditions indicated by the main query message, and extract the preset number of data records that match the query conditions indicated by the input component information of the pagination query add-in on the data node from the found data records.

[0084] Based on the above embodiments, optionally, the data records returned by different data nodes that meet the pagination query conditions are sorted and integrated, including:

[0085] The data records that meet the pagination query conditions returned by different data nodes are integrated and sorted according to the preset sorting fields agreed upon with the data nodes, and the preset number of data records that are sorted first are extracted from the integrated and sorted data records.

[0086] For a preset number of data records that have been retrieved and sorted, obtain the start record key and end record key of the data records retrieved from each data node after sorting.

[0087] Based on the above embodiments, optionally, for a single data node, the sorting result of the data records retrieved from the single data node by the business node is consistent with the sorting result of the key values ​​when querying data records on each data node.

[0088] Based on the above embodiments, optionally, the pagination query adder, the pagination query adapter, and the pagination query integrator are implemented using software algorithms.

[0089] The real-time data pagination query method provided in the embodiments of the present invention can be applied to the real-time data pagination query system provided in any of the embodiments of the present invention, and has the corresponding functions and beneficial effects of the real-time data pagination query system. For technical details not described in detail in the above embodiments, please refer to the real-time data pagination query system provided in any of the embodiments of this application.

[0090] Figure 9 This is a flowchart of another real-time data pagination query method provided in this embodiment of the invention. This embodiment of the invention further optimizes the aforementioned embodiments, and can be combined with various optional solutions from one or more of the above embodiments. For example... Figure 9 As shown, the pagination query method for real-time data in this embodiment of the invention may include the following steps S910-S980:

[0091] S910. In the case of the first pagination query, the client receives the user's query request and sends it to the client main query service and the client node pagination query adder; and the client node's main query service assembles the main query message and assembles the input component information through the locally deployed pagination query adder, and sends the corresponding pagination query request to the business node.

[0092] S920, the service node receives the pagination query request sent by the client node, interprets the input pagination query request message, and sends it to the service node's main query service and the service node's query adder; and the service node's main query service receives the main query message, while the service node's pagination query adder input component receives the client node's pagination query adder input component information and interprets and assembles it.

[0093] S930: The main query service of the business node assembles the main query message and assembles the input component information through the locally deployed pagination query adder, and sends the corresponding matching pagination query request to each data node to call the query service of each data node and send the input component information of the locally deployed pagination query adder to each data node.

[0094] S940, the data node receives a pagination query request sent by the service node, interprets the received query request message, and sends it to the data node main query service and the data node pagination query adder; and the data node main query service assembles the input message, and after the message is assembled, the main query sends it to the query adapter of the data node.

[0095] The main query service receives the user account, payment transaction start date, payment transaction end date, and other information from the query message. At the same time, the query add-in component receives the input component information from the business node's query add-in component and interprets and assembles it.

[0096] S950, the pagination query adapter of the data node queries the record of the first pagination query request, the data node assembles the output message, and sends it to the business node after the message assembly is completed.

[0097] See Figure 2 Taking data nodes corresponding to the e-wallet system, debit card system, and credit card system as examples, each data node's main query service assembles 20 records, while the data node's pagination query add-on output component assembles additional output component information; for example, querying records based on user account, payment transaction start date, and payment transaction end date, and returning the 20 largest records sorted in descending order by occurrence date and occurrence number.

[0098] S960, the pagination query integrator of business nodes arranges the records returned by different data nodes in reverse order according to the payment occurrence timestamp to obtain the final preset number of records.

[0099] join Figure 6 Taking data nodes 1, 2, and 3 as the e-wallet system, debit card system, and credit card system respectively, DW1-DW20 represent the sorted data returned by the e-wallet, DC1-DC20 represent the sorted data returned by the debit card system, and CC1-CC20 represent the sorted data returned by the credit card system. The number to the right of each column indicates the order of the record among the 60 records. The 20 records in the shaded area represent the data records that the business nodes need to return to the client. The integrated results are then returned to the main query and pagination query add-ons of the business nodes.

[0100] S970, the business node obtains the main query service to receive the final preset records, and the pagination query adder outputs the component assembly information.

[0101] See Figure 4 and Figure 6 Taking data nodes 1, 2, and 3 as corresponding to the e-wallet system, debit card system, and credit card system respectively as examples, the specific content of the output component information of the pagination query add-on returned by the corresponding data nodes (e-wallet, debit card, and credit card) received by the business node is as follows: The parameter values ​​of [PAGE_STA_KEY,PAGE_END_KEY,CONV_NO_DATA,TOTAL_RE] are [DW1,DW20,N,60], [DC1,DC20,N,70], and [CC1,CC20,N,50] respectively (for ease of explanation, data records and data key values ​​are represented the same way). After being sorted and integrated by the pagination query integrator of the business node, the three elements need to be changed to [DW1,DW7,N,60], [DC1,DC7,N,70], and [CC1,CC6,N,50], and returned to the client through the output component of the pagination query add-on.

[0102] S980: The client node's main query service receives the preset records of the business node's main query message, assembles and outputs the message, and at the same time, the pagination query adder output component receives the updated information from the business node's pagination query adder output component, prepares the data for the next query, and displays the final query results to the user.

[0103] Figure 10 This is a flowchart of another real-time data pagination query method provided in this embodiment of the invention. This embodiment of the invention further optimizes the aforementioned embodiments, and can be combined with various optional solutions from one or more of the above embodiments. For example... Figure 10 As shown, the pagination query method for real-time data in this embodiment of the invention may include the following steps:

[0104] S1010. In the case of a non-first pagination query, the client receives the user's query request and sends it to the client's main query service and the client node's pagination query adder; and the client node's main query service assembles the main query message and assembles the input component information through the locally deployed pagination query adder, and sends the corresponding pagination query request to the business node.

[0105] A second query refers to a user who has already queried the payment transaction history and obtained a certain page of records, and subsequently initiates a query to retrieve the next / previous page of data. During a second query, the client-side query appender output already includes the pagination information obtained from the previous query; in this case, PAGE_UP_DOWN can be either "up" or "down".

[0106] See Figure 5 and Figure 7 During this pagination query, the input component information in the client node's pagination query adder is assembled based on the output component information obtained from the previous pagination query adder. Taking data nodes 1, 2, and 3 as corresponding to the e-wallet system, debit card system, and credit card system respectively as examples, when the output component information of the pagination query add-on obtained from the previous query is the first pagination query, the specific output component information of the pagination query add-on is as follows: e-wallet [PAGE_UP_DOWN,PAGE_STA_KEY,PAGE_END_KEY,PAGE_JUMP,REC_IN_PAGE]=[2,DW1,DW7,20]; debit card [PAGE_UP_DOWN,PAGE_STA_KEY,PAGE_END_KEY,PAGE_JUMP,REC_IN_PAGE]=[2,DC1,DC7,20]; and credit card [PAGE_UP_DOWN,PAGE_STA_KEY,PAGE_END_KEY,PAGE_JUMP,REC_IN_PAGE]=[2,CC1,CC6,20].

[0107] S1020: The service node receives the pagination query request sent by the client node, interprets the input pagination query request message, and sends it to the service node's main query service and the service node's query adder; and the service node's main query service receives the main query message, while the service node's pagination query adder input component receives the input component information of the client node's pagination query adder and interprets and assembles it.

[0108] S1030: The main query service of the business node assembles the main query message and assembles the input component information through the locally deployed pagination query adder, and sends the corresponding matching pagination query request to each data node to call the query service of each data node and send the input component information of the locally deployed pagination query adder to each data node.

[0109] S1040, the data node receives a pagination query request sent by the service node, interprets the received query request message, and sends it to the data node main query service and the data node pagination query adder; and the data node main query service assembles the input message, and after the message is assembled, the main query sends it to the query adapter of the data node.

[0110] S1050, the pagination query adapter of the data node queries the records of the pagination query request again, the data node assembles the output message, and sends it to the business node after the message assembly is completed.

[0111] For example, taking a client-side payment system as an example, to query records based on user account, payment transaction start date, and payment transaction end date, and because PAGE_UP_DOWN=2 indicates scrolling down, an additional condition is needed: the key value must be less than PAGE_END_KEY. See also... Figure 7 Taking the e-wallet data node as an example, if a query needs to find records with key values ​​less than DW7, the 20 largest records sorted in descending order by occurrence date and occurrence number are returned, which are DW8-DW27. Debit and credit card data nodes are processed similarly. Assuming that the debit and credit card data nodes each have only 10 records satisfying the query conditions, namely DC8-DC17 and CC7-CC16, that is, there are 17 debit card payment records and 16 credit card payment records respectively.

[0112] S1060, the pagination query integrator of business nodes arranges the records returned by different data nodes in reverse order according to the payment occurrence timestamp to obtain the final preset number of records.

[0113] See Figure 5 and Figure 7 Taking data nodes 1, 2, and 3 as corresponding to the e-wallet system, debit card system, and credit card system respectively as an example, DW8-DW27 are the sorted data returned by the e-wallet, DC8-DC17 are the data returned by the debit card, and CC7-CC16 are the data returned by the credit card. The number on the right of each column of data is the order of the record in the 40 records. Finally, the 20 records in the shaded area are the data records returned to the client by the business nodes.

[0114] S1070, The main query service of the business node receives the final preset record, and the paginated query adder outputs the component assembly information.

[0115] See Figure 5 and Figure 7Taking data nodes 1, 2, and 3 as corresponding to the e-wallet system, debit card system, and credit card system respectively as examples, the output component information of the pagination query add-on returned by the e-wallet, debit card, and credit card data nodes received by the business node specifically includes the following: [PAGE_STA_KEY, PAGE_END_KEY, CONV_NO_DATA, TOTAL_RE] are [DW8, DW27, N, 60], [DC8, DC17, Y, 17], and [CC7, CC16, Y, 16] respectively (for ease of explanation, data records and data keys are represented the same way). After being integrated by the business node query integrator, the three elements are changed to [DW8 DW13, N, 60], [DC8, DC17, Y, 17], and [CC7, CC10, N, 16]. The output component of the query add-on is then returned to the client.

[0116] It's worth noting that the debit card data node returns CONV_NO_DATA=Y to the business node, indicating that there are no more records to scroll down to. Since all 10 records returned by the debit card data node are in the top 20, there's no need to scroll down to the debit card data node again. Therefore, the business node returns CONV_NO_DATA=Y to the client. However, the credit card data node returns CONV_NO_DATA=Y to the business node. But since not all 10 records returned by the credit card data node are in the top 20, there's still a need to scroll down to the credit card data node. Therefore, the business node returns CONV_NO_DATA=N to the client.

[0117] S1080: The client-side main query service receives the preset number of records from the main query message of the business node and assembles and outputs the message; at the same time, the pagination query adder output component receives the updated information from the pagination query adder output component of the business node and prepares the data for the next pagination query.

[0118] As an optional option, see Figure 11 When a client node initiates another pagination query, but after the business node queries the data nodes and finds no records that meet the conditions, the query process ends. The client node has two ways to determine if the query can continue, i.e., whether there is a next page of data. Optionally, it can query the CONV_NO_DATA field of the adder output component; if there are no more records on the next page, it returns Y. Alternatively, it can query the TOTAL_REC field of the adder output component and the number of records per page to determine the total number of pages of data being queried (in the case of multiple data nodes, the total number of records is the sum of the TOTAL_REC fields of the adder output components queried by each data node).

[0119] As an optional option, see Figure 11For debit card data, the business node returns CONV_NO_DATA=Y to the client. When the user makes a subsequent query, the client's pagination query add-in will no longer assemble debit card data; it will only query data from the e-wallet and credit card data nodes. The business node requests 20 data entries from both the e-wallet and credit card data nodes, integrates them into 20 entries, and returns them to the client. After the query is complete, all credit card data is also returned. When the user scrolls down again, it will only query e-wallet data, continuing this process until all data nodes have returned data. That is, when all data nodes return CONV_NO_DATA=Y through the business node's pagination query add-in, it indicates that all data has been queried, there are no more records on the next page, and the pagination query process ends.

[0120] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0121] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A real-time data pagination query system, characterized in that, include: The system comprises a client node, a business node, and at least two distributed data nodes, wherein the business node is equipped with a pagination query integrator, and the data nodes are equipped with a pagination query adapter; wherein: The business node is configured to respond to a pagination query request sent by a client node and send a pagination query request that matches the data node to the data node. The data node is configured to respond to the pagination query request sent by the business node, retrieve the data records that meet the query conditions through the pagination query adapter, and return them to the business node; The business node is further configured to sort and integrate data records that meet the pagination query conditions returned by different data nodes through a deployed pagination query integrator, and return the pagination query result record to the client node. A pagination query add-in is deployed on the client node, business node, and data node respectively. The pagination query add-in is used to record and transmit the input component information of the pagination query start and end records and the output component information of the pagination output start and end records of the corresponding pagination query add-in node during each pagination query. The pagination query method of the pagination query add-in adopts a page-turning query. The data structure of the pagination query add-in on both the client node and the business node is an array, and the pagination query add-in on the business node corresponds to the pagination query add-in on at least two data nodes.

2. The system according to claim 1, characterized in that, The client node is configured to send user query requests to the client node's main query service and the corresponding deployed pagination query add-in; and to assemble the main query message through the local main query service and assemble the input component information through the locally deployed pagination query add-in, and send the corresponding pagination query request to the business node.

3. The system according to claim 1, characterized in that, The specific configuration of the business node is as follows: it sends the main query message carried in the pagination query request to the main query service of the business node and transmits the input component information of the pagination query add-on on the client node carried in the pagination query request to the input component of the pagination query add-on on the business node. Additionally, it assembles subject query messages through the local subject query service and assembles input component information through the locally deployed pagination query adder, and sends corresponding matching pagination query requests to each data node.

4. The system according to claim 1, characterized in that, The data node is specifically configured to send the main query message carried in the pagination query request to the main query service of the data node and to pass the input component information of the pagination query add-on on the business node carried in the pagination query request to the input component of the pagination query add-on on the data node. The data node is further configured to send the main query message carried in the pagination query request to the pagination query adapter of the data node through the local main query service and to send the input component information of the locally deployed pagination query add-on to the pagination query adapter; and to control the pagination query adapter to retrieve data records that meet the pagination query conditions based on the main query message and the input component information of the locally deployed pagination query add-on.

5. The system according to any one of claims 2-4, characterized in that, The main query message includes the user account to be queried, the start date of the record to be queried, and the end date of the record to be queried.

6. The system according to claim 1, characterized in that, The input component information of the pagination query adder includes the following parameters: page jump direction indicator parameter, start and end record key value parameter of pagination query, and start and end record quantity parameter of pagination query; The output component information of the pagination query adder includes the following parameters: the start and end record key value parameters of the pagination output and the down-scrolling end flag parameter of the pagination query.

7. The system according to claim 6, characterized in that, When the pagination query request sent by the client node is the first query, the page jump direction indicator parameter and the start and end record key value parameter of the pagination query in the input component information of the pagination query adder are empty, the start and end record quantity parameter of the pagination query is a preset quantity value, and the parameter values ​​in the input component information of the pagination query adder of different nodes are the same during a pagination query process.

8. The system according to claim 6, characterized in that, When the pagination query request sent by the client node is not the first query, the input component information of the pagination query adder is determined based on the start and end record key value parameters of the pagination output in the output component information of the client node's pagination query adder during the previous pagination query.

9. The system according to claim 6, characterized in that, When the page jump direction indicator parameter is set to "upward query", the input component information of the pagination query adder is used to indicate the data record whose key value is greater than the key value of the starting record in the input component information; when the page jump direction indicator parameter is set to "downward query", the input component information of the pagination query adder is used to indicate the data record whose key value is less than the key value of the starting record in the input component information.

10. The system according to claim 4, characterized in that, Retrieve data records that meet the pagination query criteria, including: Find the data records that match the query conditions indicated by the main query message, and extract the preset number of data records that match the query conditions indicated by the input component information of the pagination query add-in on the data node from the found data records.

11. The system according to claim 4, characterized in that, Sort and integrate the data records returned by different data nodes that meet the pagination query conditions, including: The data records that meet the pagination query conditions returned by different data nodes are integrated and sorted according to the preset sorting fields agreed upon with the data nodes, and the preset number of data records that are sorted first are extracted from the integrated and sorted data records. For a preset number of data records that have been retrieved and sorted, obtain the start record key and end record key of the data records retrieved from each data node after sorting.

12. The system according to claim 11, characterized in that, For a single data node, the sorting result of the data records retrieved from the single data node by the business node is consistent with the sorting result of the key value when querying data records on each data node separately.

13. The system according to claim 1, characterized in that, The pagination query adder, the pagination query adapter, and the pagination query integrator are implemented using software algorithms.

14. A method for paginated query of real-time data, characterized in that, The method includes: The business node responds to the pagination query request sent by the client node and sends a pagination query request that matches the data node to the data node; a pagination query integrator is deployed on the business node, and a pagination query adapter is deployed on the data node; The data node responds to the pagination query request sent by the business node, retrieves the data records that meet the query conditions through the locally pre-deployed pagination query adapter, and returns them to the business node; The business node also sorts and integrates the data records that meet the pagination query conditions returned by different data nodes through the deployed pagination query integrator, and returns the pagination query result records to the client node. The client node, business node, and data node are each equipped with a pagination query adder. Each pagination query adder records and transmits the input component information of the pagination start and end records and the output component information of the pagination output start and end records for each pagination query process. The pagination query method of the pagination query adder is page-turning query. The data structure of the pagination query adders on both the client node and business node is an array, and the pagination query adder on the business node corresponds to at least two pagination query adders on the data nodes.

Images