Data query method and device and storage medium

Abstract

The invention discloses a data query method and device and a storage medium, which can be applied to various scenes such as cloud technology, artificial intelligence, smart traffic, Internet of Vehicles and the like, and the method comprises the following steps: obtaining a starting line number and a target number of to-be-queried data in a database; determining the sum of the starting line number and the target number to obtain a target termination line number of the to-be-queried data; if the target termination line number is greater than or equal to a preset threshold value, obtaining a data query interval corresponding to the to-be-queried data; splitting the data query interval into at least two sub-intervals to obtain a data volume corresponding to each sub-interval; traversing the at least two sub-intervals, and determining a target sub-interval corresponding to the initial row number; and querying the to-be-queried data in the target sub-interval based on the target number and the sum of the number of the preorder sub-intervals. According to the method and the device, the target subinterval to which the to-be-queried data belongs is positioned during paging query, so that the data of the random page number can be quickly and accurately queried in the target subinterval.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data query method, device, and storage medium. Background Art

[0002] Currently, there are three ways to perform paging queries on Elasticsearch: from+size, scroll, and search after.

[0003] The from+size method retrieves and sorts data that meets the conditions based on the request parameters, finds the initial data based on the from offset, and then retrieves the corresponding amount of data based on the size. The scroll method creates a scroll context in Elasticsearch, which stores a snapshot of the search results. The scroll_id identifies the scroll context and is used to return data in subsequent scroll operations. The scroll context has a lifecycle and cannot be used after it expires. The search after method uses a unique value field in the document as the arbiter for the sort specification, starting from the cursor's starting position and retrieving the next page of data based on the last document in the previous page of results. When the from value is 100,000 or 1,000, the resource requirements for a single node are very high, making this method unsuitable for deep paging. The scroll method requires Elasticsearch to store a snapshot of the results, which consumes server resources and only allows sequential viewing, not random or forward paging. The search after method uses a cursor to page backward sequentially, not random or forward paging. Summary of the Invention

[0004] The present application provides a data query method, device and storage medium, which can locate the sub-interval to which data belongs during paging query, thereby quickly and accurately querying data on random pages.

[0005] In one aspect, the present application provides a data query method, the method comprising:

[0006] Get the starting row number and target number of the data to be queried in the database;

[0007] Determine the sum of the starting number of rows and the target number to obtain the target ending number of rows of the data to be queried;

[0008] If the target end row number is greater than or equal to a preset threshold, obtaining a data query interval corresponding to the data to be queried;

[0009] Splitting the data query interval into at least two subintervals, and obtaining the data volume corresponding to each subinterval;

[0010] Traversing the at least two subintervals to determine a target subinterval corresponding to the starting row number;

[0011] The target subinterval is searched for the data to be queried based on the target number and the sum of the numbers of preceding subintervals; the sum of the numbers of preceding subintervals is the sum of the data amounts of the subintervals that are sorted before the target subinterval.

[0012] Another aspect provides a data query device, the device comprising:

[0013] The target quantity acquisition module is used to obtain the starting row number and target quantity of the data to be queried in the database;

[0014] A target end row number module is used to determine the sum of the starting row number and the target number to obtain the target end row number of the data to be queried;

[0015] A data query interval acquisition module, configured to acquire a data query interval corresponding to the data to be queried if the target termination row number is greater than or equal to a preset threshold;

[0016] A data volume determination module, configured to split the data query interval into at least two sub-intervals and obtain the data volume corresponding to each sub-interval;

[0017] a traversal module, configured to traverse the at least two subintervals and determine a target subinterval corresponding to the starting row number;

[0018] A data query module is used to query the data to be queried in the target sub-interval based on the target number and the sum of the number of preceding sub-intervals; the sum of the number of preceding sub-intervals is the sum of the data amounts of each sub-interval sorted before the target sub-interval.

[0019] In an exemplary embodiment, the data amount determination module includes:

[0020] A bucket quantity threshold obtaining unit, configured to obtain a bucket quantity threshold of the database;

[0021] A partition interval determining unit, configured to determine a partition interval based on a bucket quantity threshold of the database and two endpoint values corresponding to the data query interval;

[0022] The bucketing unit is configured to bucket the data query interval according to the partition interval to obtain the at least two sub-intervals; and determine the data volume corresponding to each sub-interval.

[0023] In an exemplary embodiment, the traversal module includes:

[0024] a sorting unit, configured to sort the at least two subintervals according to the splitting order to obtain a sorting result;

[0025] a traversal unit, configured to traverse the at least two subintervals in sequence according to the sorting result;

[0026] a current sub-interval determining unit, configured to perform the following operations on the traversed current sub-interval: according to the sorting result, taking each sub-interval located before the current sub-interval as the currently traversed sub-interval;

[0027] a current data amount sum determination unit, configured to determine the sum of the data amounts corresponding to the currently traversed subintervals to obtain the current data amount sum;

[0028] The target sub-interval determining unit is configured to determine a target sub-interval corresponding to the starting row number according to the current data amount sum and the starting row number.

[0029] In an exemplary embodiment, the target subrange determining unit includes:

[0030] a first determining unit configured to, if the total amount of current data is less than the starting number of rows, re-set the first subinterval after the current subinterval as the current subinterval, and repeat the operation performed on the traversed current subinterval;

[0031] The second determining unit is configured to determine the current sub-interval as the target sub-interval if the total current data amount is greater than or equal to the starting number of rows.

[0032] In an exemplary embodiment, the data query module includes:

[0033] a preceding subinterval determining unit, configured to determine, according to the sorting result, a subinterval preceding the target subinterval as a preceding subinterval;

[0034] a preamble subinterval quantity sum determining unit, configured to determine the sum of the data amounts corresponding to the respective preamble subintervals to obtain the preamble subinterval quantity sum;

[0035] a target quantity sum obtaining unit, configured to obtain the sum of the data amounts corresponding to the target subintervals to obtain a target quantity sum;

[0036] a comprehensive quantity sum determining unit, configured to determine the sum of the target quantity sum and the preceding subinterval quantity sum to obtain a comprehensive quantity sum;

[0037] The data query unit is configured to query the data to be queried in the target subinterval if the sum of the comprehensive quantities is greater than or equal to the target termination row number.

[0038] In an exemplary embodiment, the apparatus further comprises:

[0039] A screening module is configured to obtain, if the sum of the comprehensive numbers is less than the target end row number, the first subinterval located after the target subinterval according to the sorting result to obtain a screening subinterval;

[0040] an updating module, configured to merge the target subinterval with the screening subinterval to obtain an updated target subinterval;

[0041] The update data query module is used to query the data to be queried in the update target sub-interval.

[0042] In an exemplary embodiment, the update data query module includes:

[0043] An update data volume acquisition unit, configured to acquire the data volume corresponding to the update target subinterval to obtain the update data volume;

[0044] A step jump unit is used to obtain the data interval corresponding to the update target sub-interval if the updated data amount is greater than the preset threshold; and jump to the step of splitting the data interval into at least two sub-intervals to obtain the data amount corresponding to each sub-interval.

[0045] In an exemplary embodiment, the apparatus further comprises:

[0046] an offset determination module, configured to determine, if the amount of updated data is less than or equal to the preset threshold, a difference between the starting row number and the sum of the numbers of the preceding sub-intervals, and obtain an offset of the starting row number in the update target sub-interval;

[0047] The module for determining data to be queried is configured to extract the target number of rows of data from the update target subinterval starting from the number of rows corresponding to the offset to obtain the data to be queried.

[0048] In an exemplary embodiment, the target quantity acquisition module includes:

[0049] A primary parameter receiving unit, configured to receive an initial data query request sent by a terminal; the initial data query request carries a primary request parameter;

[0050] A parameter parsing unit, configured to parse the request parameters to obtain the starting row number and target number of the data to be queried in the database;

[0051] In an exemplary embodiment, the apparatus further comprises:

[0052] A data total amount acquisition module is used to obtain the total amount of data corresponding to the data query interval;

[0053] A data storage module, configured to generate a preset hash code based on the one-time request parameter and store the preset hash code in a hash code repository if the total amount of data is greater than a preset amount threshold;

[0054] A bucket group generating module, configured to generate a bucket group based on the at least two subintervals;

[0055] An offset array determination module, configured to determine an offset corresponding to each bucket in the bucket group to obtain an offset array;

[0056] The database construction module is used to construct a cache database using the preset hash code as a key and the bucket group and the offset array as values.

[0057] In an exemplary embodiment, the apparatus further comprises:

[0058] A secondary request receiving module, configured to receive a secondary data query request sent by the terminal; the secondary data query request carries a secondary request parameter;

[0059] A target hash code generation module, configured to generate a target hash code based on the secondary request parameters;

[0060] A hash code matching module, configured to search the hash code repository for a hash code that matches the target hash code, and obtain a matching hash code;

[0061] a target offset array determination module, configured to search the cache database for a target bucket group and a target offset array corresponding to the matching hash code;

[0062] A data search module is used to search for data corresponding to the secondary data query request based on the target bucket group and the target offset array.

[0063] In an exemplary embodiment, the data search module includes:

[0064] A secondary parsing unit, configured to parse the secondary request parameters to obtain a secondary starting row number and a secondary target number corresponding to the secondary query data;

[0065] a screening offset determining unit, configured to determine a position of the secondary starting row number in the target offset array to obtain a screening offset;

[0066] A bucket matching unit is used to filter buckets that match the filter offset from the target bucket group to obtain matching buckets;

[0067] A data search unit is used to search for data corresponding to the secondary data query request from the matching bucket based on the secondary target quantity.

[0068] On the other hand, a data query device is provided, which includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by the processor to implement the data query method described above.

[0069] On the other hand, a computer storage medium is provided, wherein the computer storage medium stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by a processor to implement the data query method described above.

[0070] Another aspect provides a computer program product or computer program, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the data query method described above.

[0071] The data query method, device, and storage medium provided in this application have the following technical effects:

[0072] The present application obtains the starting number of rows and the target number of the data to be queried in the database; determines the sum of the starting number of rows and the target number to obtain the target ending number of rows of the data to be queried; if the target ending number of rows is greater than or equal to a preset threshold, obtains the data query interval corresponding to the data to be queried; splits the data query interval into at least two subintervals to obtain the data volume corresponding to each subinterval; traverses at least two subintervals to determine the target subinterval corresponding to the starting number of rows; and queries the data to be queried in the target subinterval based on the target number and the sum of the numbers of the preceding subintervals; the sum of the numbers of the preceding subintervals is the sum of the data volumes of the subintervals sorted before the target subinterval. The present invention can reduce the data volume of the subinterval to an acceptable range for Elasticsearch through multiple divisions, locate the target subinterval to which the data to be queried belongs during paging queries, and thus quickly and accurately query data of random page numbers in the target subinterval. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] In order to more clearly illustrate the technical solutions and advantages of the embodiments of this specification or the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0074] Figure 1 This is a schematic diagram of a data query system provided by an embodiment of this specification;

[0075] Figure 2 This is a flow chart of a data query method provided in an embodiment of this specification;

[0076] Figure 3 This is a flowchart of a method for splitting a data query interval into at least two sub-intervals and obtaining the data volume corresponding to each sub-interval, provided by an embodiment of this specification;

[0077] Figure 4 This is a flowchart of a method for traversing at least two subintervals and determining a target subinterval corresponding to a starting row number, provided by an embodiment of this specification;

[0078] Figure 5 1 is a flowchart of a method for searching for data to be queried in a target subinterval based on the target number and the sum of the number of preceding subintervals, provided in an embodiment of this specification;

[0079] Figure 6 This is a flowchart of a method for determining an update target subinterval and querying data to be queried provided by an embodiment of this specification;

[0080] Figure 7 1 is a flow chart of a method for querying to-be-queried data in an update target subinterval provided by an embodiment of this specification;

[0081] Figure 8 This is a flowchart of another method for searching for data to be queried in an update target subinterval provided by an embodiment of this specification;

[0082] Figure 9 This is a flowchart of a method for searching for data corresponding to a secondary data query request based on a target bucket group and a target offset array, provided by an embodiment of this specification;

[0083] Figure 10 This is a flowchart of another data query method provided by an embodiment of this specification;

[0084] Figure 11 This is a schematic diagram of a method for splitting a data query interval provided in an embodiment of this specification;

[0085] Figure 12 This is a diagram of a display interface for performing data query in a preset application based on the method of this embodiment provided in the embodiment of this specification;

[0086] Figure 13 This is a structural diagram of a data query device provided in an embodiment of this specification;

[0087] Figure 14 This is a structural diagram of a server provided in an embodiment of this specification. DETAILED DESCRIPTION

[0088] The following will be combined with the drawings in the embodiments of this specification to clearly and completely describe the technical solutions in the embodiments of this specification. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0089] First, some nouns or terms that appear in the description of the embodiments of this specification are explained as follows:

[0090] Elasticsearch is a distributed, highly scalable, and highly real-time search and data analysis engine. It easily enables the search, analysis, and exploration of large amounts of data. Leveraging Elasticsearch's horizontal scalability makes data more valuable in production environments. Elasticsearch's implementation primarily involves the following steps: First, users submit data to the Elasticsearch database. The word segmentation controller then segments the corresponding sentences, storing the weights and segmentation results in the database. When users search for data, the results are ranked and scored based on the weights, and the returned results are presented to the user.

[0091] It should be noted that the terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0092] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program that has a predetermined function and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0093] See also Figure 1 , Figure 1 This is a schematic diagram of a data query system provided by an embodiment of this specification. Figure 1 As shown, the data query system may include at least a server 01 and a client 02 .

[0094] Specifically, in the embodiments of this specification, the server 01 may include an independently running server, or a distributed server, or a server cluster composed of multiple servers. It may also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. Server 01 may include a network communication unit, a processor, a memory, and the like. Specifically, the server 01 can be used to split the data query interval into at least two sub-intervals to obtain the amount of data corresponding to each sub-interval; traverse the at least two sub-intervals to determine the target sub-interval corresponding to the starting number of rows; based on the target number and the sum of the number of preceding sub-intervals, query the data to be queried in the target sub-interval.

[0095] Specifically, in the embodiments of this specification, the client 02 may include a physical device such as a smartphone, desktop computer, tablet computer, laptop computer, digital assistant, smart wearable device, smart speaker, in-vehicle terminal, smart TV, etc. It may also include software running on the physical device, such as a web page provided by a service provider to a user, or an application provided by the service provider to a user. Specifically, the client 02 may be used to query the data to be queried online based on the starting row number and the target number.

[0096] The following describes a data query method of this application. Figure 2It is a flow chart of a data query method provided by an embodiment of this specification. This specification provides method operation steps as described in the embodiment or flow chart, but may include more or fewer operation steps based on conventional or non-creative labor. The order of steps listed in the embodiment is only one way of executing the order of many steps and does not represent the only execution order. When the actual system or server product is executed, it can be executed in sequence or in parallel according to the method shown in the embodiment or the accompanying drawings (for example, in a parallel processor or multi-threaded processing environment).

[0097] Specific examples Figure 2 As shown, the method may include:

[0098] S201: Obtain the starting row number and target number of the data to be queried in the database.

[0099] In the embodiments of this specification, the database may be Elasticsearch, which can be used for data storage and search. Elasticsearch stores data in pages. The data to be queried is the data stored in the database, and the data to be queried typically includes multiple data or documents. The starting number of rows is from, and the target number is size. From represents the starting number of rows in the database; size represents the number of documents to be queried, and size defaults to 10. From = 10000, size = 10 indicates that the query starts from the 10000th data item and the total 10 data items after the 10000th data item are the query data.

[0100] S203: Determine the sum of the starting number of rows and the target number to obtain the target ending number of rows of the data to be queried.

[0101] In the embodiments of this specification, the target ending number of rows for the data to be queried can be obtained by calculating the sum of the starting number of rows and the target number. That is, the target ending number of rows for the data to be queried can be obtained by calculating the sum of from and size. For example, when from = 10000 and size = 10, the formula for calculating the target ending number of rows is as follows:

[0102] 10000+10=10010.

[0103] S205: If the target ending row number is greater than or equal to the preset threshold, obtain the data query interval corresponding to the data to be queried.

[0104] In an embodiment of the present specification, after calculating the target end row number, it can be further determined whether the target end row number is greater than or equal to a preset threshold; wherein the preset threshold can be determined based on the data paging parameters of Elasticsearch; exemplarily, the preset threshold can be 10,000; it can also be set to other data according to actual needs; when the target end row number is greater than or equal to the preset threshold, the data query interval corresponding to the data to be queried can be obtained; the data to be queried can be the data requested by the terminal, and exemplarily, a data query request sent by the terminal can be received, the request carrying request parameters; the request parameters include the target number of starting rows corresponding to the data to be queried, the data query interval, and the data query time.

[0105] In some embodiments, the above method further comprises:

[0106] If the target end row number is less than the preset threshold, the start row number in the database is queried based on the start row number;

[0107] Query the data from the first row starting from the starting row number to the target number of rows to obtain the data to be queried.

[0108] In an embodiment of the present specification, when the target ending row number is less than a preset threshold, the starting row number in the database can be queried based on the starting row number; the data from the first row starting from the starting row number to the target number of rows are queried to obtain the data to be queried; exemplarily, when the target ending row number is less than the preset threshold, that is, from is small, the data can be directly queried in the database based on from plus size.

[0109] For example, assuming that the paginated data with a time range of (start, end) is to be queried, if from+size≤10000, then use from and size to query the data in Elasticsearch and return it, otherwise continue to execute the subsequent steps.

[0110] S207: Split the data query interval into at least two sub-intervals, and obtain the data volume corresponding to each sub-interval.

[0111] In some embodiments, as Figure 3 As shown, the data query interval is divided into at least two sub-intervals, and the data volume corresponding to each sub-interval is obtained, including:

[0112] S2071: Obtaining the database bucket quantity threshold;

[0113] S2073: Determine a partition interval based on a bucket quantity threshold of the database and two endpoint values corresponding to the data query interval;

[0114] S2075: Bucket the data query interval according to the partition interval to obtain at least two sub-intervals; and determine the data volume corresponding to each sub-interval.

[0115] In the embodiment of this specification, if the target end row number is greater than or equal to the preset threshold, the data query interval can be split into at least two sub-intervals; for example, the data query interval is (start, end), the database bucket number threshold can be obtained and the partition interval (interval) can be determined based on the bucket number threshold, and the data query interval can be partitioned according to the partition interval; wherein, the database bucket number threshold can be determined based on the configuration parameters of the database, which is usually a pre-set value in the database. Evaluate the interval value based on the business data volume, and make the data volume in the interval as small as possible as less than 10,000 to avoid re-dividing the interval, and pay attention to The number of buckets must be less than the max_buckets configured in Elasticsearch.

[0116] The data query interval can be bucketed according to the partition interval, starting from the smallest endpoint value of the data query interval and partitioned according to the partition interval to obtain at least two sub-intervals; the difference between the two endpoint values of each sub-interval is the same; after partitioning, the amount of data in each sub-interval can be determined and a new sub-interval can be constructed, for example (s i , e i , hum i ), where si represents the starting data of the interval, ei represents the maximum data of the interval, that is, the last number of the interval, and numi represents the total number of data in the interval; this makes it easy to quickly determine the data to be queried.

[0117] S209: Traverse at least two sub-intervals to determine a target sub-interval corresponding to the starting row number.

[0118] In the embodiment of the present specification, after the data query interval is divided into at least two sub-intervals, a target sub-interval corresponding to the starting number of rows of the data to be queried may be determined.

[0119] In some embodiments, as Figure 4 As shown, traversing at least two subintervals and determining the target subinterval corresponding to the starting row number includes:

[0120] S2091: Sort at least two subintervals according to the splitting order to obtain a sorting result;

[0121] S2093: According to the sorting result, traverse at least two subintervals in sequence, and perform the following operations on the current subinterval traversed:

[0122] S2095: According to the sorting result, take each sub-interval before the current sub-interval as the currently traversed sub-interval;

[0123] S2097: Determine the sum of the data volumes corresponding to each currently traversed sub-interval to obtain the current total data volume;

[0124] S2099: Determine the target sub-interval corresponding to the starting row number according to the current total data volume and the starting row number.

[0125] In the embodiments of this specification, at least two sub-intervals can be sorted according to the splitting order, and (s0, e0, num0), …… (s i , e i , num i ) …… can be obtained from left to right in sequence; then traverse at least two sub-intervals in sequence according to the sorting result. For the currently traversed sub-interval, according to the sorting result, take each sub-interval before the current sub-interval as the currently traversed sub-interval; determine the sum of the data volumes corresponding to each currently traversed sub-interval to obtain the current total data volume offsei; where offset0 = 0, Determine the target sub-interval corresponding to the starting row number according to the current total data volume and the starting row number; find the first sub-interval (s i ≥ from from left to right, (e i , e i , num i ) as the target sub-interval.

[0126] In some embodiments, determining the target sub-interval corresponding to the starting row number according to the current total data volume and the starting row number includes:

[0127] If the current total data volume is less than the starting row number, take the first sub-interval after the current sub-interval as the current sub-interval again, and repeat the operations performed on the currently traversed sub-interval;

[0128] If the current total data volume is greater than or equal to the starting row number, determine the current sub-interval as the target sub-interval.

[0129] In the embodiments of this specification, if the current total data volume is less than the starting row number, take the first sub-interval after the current sub-interval as the current sub-interval again, and repeat the operations performed on the currently traversed sub-interval, that is, repeat steps S2095 to step S2097; if the current total data volume is greater than or equal to the starting row number, determine the current sub-interval as the target sub-interval. Exemplarily, if offset i + num i < from + size, then take the interval (s i , ei , num i ) and (s i+1 , e i+1 , num i+1 ) are merged to obtain a new (start, end) interval, start = s i ,end=e i+1 , num=num i +num i+1 , that is (s i , e i+1 ), if offset i +num i ≥from+size, then use the interval i directly, that is, (s i , e i ).

[0130] S2011: Based on the target number and the sum of the numbers of preceding subintervals, search for the data to be queried in the target subinterval; the sum of the numbers of preceding subintervals is the sum of the data volumes of the subintervals that are sorted before the target subinterval.

[0131] In an embodiment of the present specification, the sum of the number of preceding subintervals can be calculated based on the sum of the data amounts of the subintervals sorted before the target subinterval; then, the target subinterval is searched for data based on the target number and the sum of the number of preceding subintervals.

[0132] In some embodiments, as Figure 5 As shown, based on the target number and the sum of the number of preceding subintervals, the target subinterval is searched for the data to be queried, including:

[0133] S20111: According to the sorting result, the subinterval before the target subinterval is determined as the preceding subinterval;

[0134] S20113: Determine the sum of the data amounts corresponding to each pre-order sub-interval to obtain the sum of the number of pre-order sub-intervals;

[0135] S20115: Obtain the sum of the data volumes corresponding to the target subintervals to obtain the target sum;

[0136] S20117: Determine the sum of the target quantity and the sum of the quantity of the preceding subintervals to obtain the comprehensive quantity sum;

[0137] S20119: If the total number is greater than or equal to the target end row number, query the target sub-interval for the data to be queried.

[0138] In an embodiment of the present specification, the sum of the data amounts corresponding to the preceding sub-intervals before the target sub-interval can be calculated to obtain the sum of the preceding sub-interval quantities; then the sum of the data amounts corresponding to the target sub-interval can be obtained to obtain the target sum; then the sum of the target sum and the sum of the preceding sub-interval quantities can be calculated to obtain the comprehensive sum; and the comprehensive sum can be compared with the target termination row number; when the comprehensive sum is greater than or equal to the target termination row number, the data to be queried can be quickly and accurately queried in the target sub-interval.

[0139] In some embodiments, as Figure 6 As shown, the above method also includes:

[0140] S601: If the total number is less than the target end row number, obtain the first sub-interval after the target sub-interval according to the sorting result to obtain the screening sub-interval;

[0141] S603: Merge the target sub-interval and the screening sub-interval to obtain an updated target sub-interval;

[0142] S605: Query the data to be queried in the update target sub-interval.

[0143] In an embodiment of the present specification, when the combined number is less than the target end row number, the first sub-interval after the target sub-interval can be obtained based on the sorting result to obtain a filtered sub-interval; then the target sub-interval and the filtered sub-interval are merged to obtain an updated target sub-interval; thereby, fast and accurate query of the data to be queried can be achieved in the updated target sub-interval.

[0144] In some embodiments, as Figure 7 As shown, querying the data to be queried in the update target subinterval includes:

[0145] S6051: Obtain the data volume corresponding to the update target sub-interval to obtain the updated data volume;

[0146] S6053: If the updated data volume is greater than the preset threshold, obtain the data interval corresponding to the updated target subinterval; and jump to the step of splitting the data interval into at least two subintervals to obtain the data volume corresponding to each subinterval.

[0147] In an embodiment of the present specification, the data volume of all data in the update target sub-interval can be obtained to obtain the updated data volume; then the updated data volume is compared with the preset threshold value. If the updated data volume is greater than the preset threshold value, the data interval corresponding to the updated target sub-interval is obtained; and the process jumps to the step of splitting the data interval into at least two sub-intervals to obtain the data volume corresponding to each sub-interval, thereby further splitting the data interval; so as to improve data query efficiency.

[0148] In some embodiments, as Figure 8 As shown, the above method also includes:

[0149] S801: If the updated data volume is less than or equal to a preset threshold, determine the difference between the starting row number and the sum of the numbers of the previous sub-intervals, and obtain the offset of the starting row number in the update target sub-interval;

[0150] S803: In the update target sub-interval, starting from the number of rows corresponding to the offset, extract the target number of rows of data to obtain the data to be queried.

[0151] In an embodiment of the present specification, if the amount of updated data is less than or equal to a preset threshold, the difference between the starting number of rows and the sum of the numbers of the preceding sub-intervals can be calculated to obtain the offset of the starting number of rows in the update target sub-interval; then, in the update target sub-interval, starting from the number of rows corresponding to the offset, the target number of rows of data are taken out to obtain the data to be queried.

[0152] For example, after bucketing, each sub-interval corresponds to a bucket, and the data of each bucket can be obtained:

[0153] buckets=

[0154] [(start0, end0, num0)…(start i , end i , num i )…(start n-1 , end n-1 , num n-1 )], assuming the data is [(2023-10-0100:00:00, 2023-10-0101:00:00, 20000), (2023-10-0101:00:00, 2023-10-0102:00:00, 20000), (2023-10-0102:00:00, 2023-10-0103:00:00, 2000)]

[0155] Starting from the first bucket, traverse to the right and find the bucket where from is located:

[0156] a) Assuming from = 30000 and size = 10, in the second bucket, reset (start, end) to (2023-10-0101:00:00, 2023-10-0102:00:00), reset from = 30000 - 20000 = 10000, re-evaluate interval, and re-bucket;

[0157] b) Assuming from = 39990 and size = 20, merge the second and third buckets, reset (start, end) to (2023-10-01 01:00:00, 2023-10-01 03:00:00), reset from = 30000 - 20000 = 10000, re-evaluate interval, and re-execute the bucketing task;

[0158] c) Assuming from = 41000, size = 10, and the data falls into the third bucket, reset from = 41000 - 40000 = 1000, and query the data in Elasticsearch based on from = 1000, size = 10, and other query conditions.

[0159] In some embodiments, obtaining the starting row number and target number of the data to be queried in the database includes:

[0160] The receiving terminal sends an initial data query request; the initial data query request carries a request parameter;

[0161] Parse the request parameters once to obtain the business attribute parameters of the data to be queried, the starting row number of the data to be queried in the database, and the target number.

[0162] In an embodiment of the present specification, an initial data query request is sent by a receiving terminal; the initial data query request carries a single request parameter; when performing the data query for the first time, the single request parameter can be parsed to obtain the starting row number and target number of the data to be queried in the database; the business attribute parameters of the data to be queried can also be obtained; if from is too large, the interval splitting operation can be performed in real time.

[0163] In some embodiments, after splitting the data query interval into at least two sub-intervals, the method further includes:

[0164] Get the total amount of data corresponding to the data query interval;

[0165] If the total amount of data is greater than a preset quantity threshold, a preset hash code is generated based on a request parameter and the preset hash code is stored in a hash code repository;

[0166] Generate bucket groups based on at least two subintervals;

[0167] Determine the offset corresponding to each bucket in the bucket group and obtain the offset array;

[0168] Build a cache database using the preset hash code as the key and the bucket group and offset array as the value.

[0169] In an embodiment of the present specification, whether to cache multiple sub-intervals obtained from the data query interval can be determined based on the total amount of data in the data query interval; if the total amount of data is greater than a preset quantity threshold, a preset hash code is generated based on a single request parameter and the preset hash code is stored in a hash code repository; the preset quantity threshold can be set according to actual conditions, for example, it can be set to tens of millions; then, based on at least two sub-intervals, a bucket group is generated; the offset corresponding to each bucket in the bucket group is determined to obtain an offset array; finally, the preset hash code is used as a key, and the bucket group and the offset array are used as values to construct a cache database, thereby facilitating subsequent rapid data queries.

[0170] In some embodiments, as Figure 9 As shown, the above method also includes:

[0171] S901: receiving a secondary data query request sent by a terminal; the secondary data query request carries a secondary request parameter;

[0172] S903: Generate a target hash code based on the secondary request parameters;

[0173] S905: Searching for a hash code that matches the target hash code in the hash code repository to obtain a matching hash code;

[0174] S907: Searching the cache database for the target bucket group and the target offset array corresponding to the matching hash code;

[0175] S909: Based on the target bucket group and the target offset array, search for data corresponding to the secondary data query request.

[0176] In the embodiment of the present specification, after the cache database is built, after receiving the secondary data query request sent by the terminal, the data can be directly queried in the cache database to avoid splitting the interval again, and the hash code matching the target hash code is searched in the hash code repository to obtain the matching hash code;

[0177] Then, the target bucket group and target offset array corresponding to the matching hash code are searched in the cache database; finally, the data corresponding to the secondary data query request is searched in the target bucket group according to the target offset array, thereby improving the data query efficiency.

[0178] In some embodiments, searching for data corresponding to the secondary data query request based on the target bucket group and the target offset array includes:

[0179] Parse the secondary request parameters to obtain the secondary starting row number and secondary target number corresponding to the secondary query data;

[0180] Determine the position of the secondary starting row number in the target offset array to obtain the filter offset;

[0181] Filter the buckets that match the filter offset from the target bucket group to obtain matching buckets;

[0182] Based on the number of secondary targets, the data corresponding to the secondary data query request is searched from the matching buckets.

[0183] In the embodiment of this specification, after determining the target bucket group and the target offset array, the matching buckets (target sub-intervals) can be traversed according to the aforementioned data query method to quickly find the data corresponding to the secondary data query request.

[0184] In some embodiments, when a user request to query the first page is received, from+size is first used to query and return the first page of data and the unique identifier hash_code of the request parameter. If the total amount of data requested is greater than 10,000, a background bucketing task is created and bound to hash_code.

[0185] Assume that the range to be queried is (start, end) and the total amount of data is total;

[0186] If the total is large (for example, 1,000,000), calculate the interval and use the scale to adjust the interval to ensure that the amount of data in the bucket is less than 10,000. This scale can be adjusted according to actual conditions.

[0187]

[0188] if If the number of buckets is greater than 10,000, recalculate Ensure that the number of buckets is less than 10,000;

[0189] Use interval, start, and end as query conditions to count the data volume in buckets. Iterate through the buckets. If the data volume in a bucket is greater than 10,000, reset the start, end, and from parameters and jump to step 3. Continue until the data volume in all intervals is less than 10,000. If the data volume in adjacent intervals is smaller, merge them.

[0190] Get the buckets array from the previous step:

[0191] buckets

[0192] = [(start0, end0, num0)…(start i , end i , num i )…(start n-1 , end n-1 , num n-1 )]

[0193] Create an array offset, where the values in the array correspond to the left boundary quantity values of the current sub-interval, that is, the offset of the ordered field values:

[0194] offset[0] = 0,

[0195]

[0196] Cache the arrays buckets and offset into redis, and set the key as the hash code.

[0197] Exemplarily, the specific process of accessing data is as follows:

[0198] Receive the access request from the Client, including from and size;

[0199] Use the request parameters to calculate the hash_code, and obtain the bucket information buckets from redis according to the hash_code; find the position of from in the offset array, satisfying offset[i] < from < buckets[i].num. The search method can use sequential traversal or binary search for data query to determine the target sub-interval.

[0200] Exemplarily, for data pagination with a volume of less than ten million, real-time statistical data can be used for pagination, and more accurate pagination data can be obtained when data is inserted, updated, or deleted. For data volumes greater than ten million, pre-calculated bucket data can be used, and the cached bucket metadata can be directly used to locate the bucket and search for data during query. When there are real-time insertions, updates, or deletions, there are errors in the queried pagination data.

[0201] In the embodiments of this specification, Figure 10 is a schematic flow diagram of another data query method, as Figure 10 shown, the above data query method includes:

[0202] 1. If the from value is relatively small (from <= 10000), directly access the data in the from + size manner. If from is relatively large, then proceed to the next step;

[0203] 2. Obtain the value range of the sorting field, assumed to be (start, end);

[0204] 3. As Figure 11 shown, Figure 11 it is a schematic diagram of a method for splitting a data query range; the range (start, end) is split into n sub-ranges according to the interval (interval) and the data volume of each sub-range is obtained. The data volume of the sub-range is represented by (s i , e i , num i );

[0205] 4. Locate the interval where from is located, starting from (s0, e0, num0), offset0 = 0, find the first sub-range (s i ≥ from from left to right i , e i , num i );

[0206] 5. If offSet i + num i < from + size, then merge the intervals (s i , e i , num i ) and (s i+1 , e i+1 , num i+1 ) to obtain a new (start, end) interval, start = s i , end = e i+1 , num = num i + num i+1 , that is, (s i , e i+1 ). If offset i + num i ≥ from + size, then directly use the interval i, that is, (s i , e i );

[0207] 6. If the num of the new interval is small (num ≤ 10000), then calculate the offset of from in the current interval from new = from - offset i , and then retrieve size data starting from from new . If num is still relatively large, then jump to step 3 and continue to split and execute using the new (start, end).

[0208] In some embodiments, the present invention can be applied to the scenario of random paging access of massive data, such as randomly viewing millions or tens of millions of data in chronological order, and users can jump to access data. Figure 12 As shown, Figure 12 This is a display interface diagram for performing data query in a preset application based on the method of this embodiment; the total amount of accessed data is 1049.6w, from=10000001, size=10; the partition query method according to this embodiment can quickly find the data to be queried.

[0209] It can be seen from the technical solutions provided by the above embodiments of this specification that the embodiments of this specification obtain the starting number of rows and the target number of the data to be queried in the database; determine the sum of the starting number of rows and the target number to obtain the target ending number of rows of the data to be queried; if the target ending number of rows is greater than or equal to a preset threshold, obtain the data query interval corresponding to the data to be queried; split the data query interval into at least two subintervals to obtain the amount of data corresponding to each subinterval; traverse at least two subintervals to determine the target subinterval corresponding to the starting number of rows; based on the target number and the sum of the number of preceding subintervals, query the data to be queried in the target subinterval; the sum of the number of preceding subintervals is the sum of the amount of data of each subinterval sorted before the target subinterval. The present invention can reduce the amount of data in the subinterval to an acceptable range for Elasticsearch through multiple divisions, locate the target subinterval to which the data to be queried belongs during paging queries, and thus quickly and accurately query data with random page numbers in the target subinterval.

[0210] The embodiment of this specification also provides a data query device, such as Figure 13 As shown, the device includes:

[0211] The target number acquisition module 1310 is used to obtain the starting row number and target number of the data to be queried in the database;

[0212] A target ending row number module 1320 is configured to determine the sum of the starting row number and the target number to obtain the target ending row number of the data to be queried;

[0213] The data query interval acquisition module 1330 is configured to acquire the data query interval corresponding to the data to be queried if the target termination row number is greater than or equal to a preset threshold;

[0214] A data volume determination module 1340 is configured to split the data query interval into at least two sub-intervals and obtain the data volume corresponding to each sub-interval;

[0215] A traversal module 1350 is configured to traverse the at least two subintervals to determine a target subinterval corresponding to the starting row number;

[0216] The data query module 1360 is configured to query the data to be queried in the target subinterval based on the target number and the sum of the numbers of preceding subintervals; the sum of the numbers of preceding subintervals is the sum of the data volumes of the subintervals that are sorted before the target subinterval.

[0217] In an exemplary embodiment, the data amount determination module includes:

[0218] A bucket quantity threshold obtaining unit, configured to obtain the bucket quantity threshold of the above database;

[0219] A partition interval determination unit, configured to determine a partition interval based on a bucket quantity threshold of the database and two endpoint values corresponding to the data query interval;

[0220] The bucketing unit is configured to bucket the data query interval according to the partition interval to obtain the at least two sub-intervals; and determine the data volume corresponding to each sub-interval.

[0221] In an exemplary embodiment, the traversal module includes:

[0222] a sorting unit, configured to sort the at least two subintervals according to the splitting order to obtain a sorting result;

[0223] a traversal unit, configured to traverse the at least two subintervals in sequence according to the sorting result;

[0224] The current sub-interval determining unit is configured to perform the following operations on the traversed current sub-interval: according to the sorting result, taking each sub-interval located before the current sub-interval as the currently traversed sub-interval;

[0225] a current data amount sum determination unit, configured to determine the sum of the data amounts corresponding to the currently traversed subintervals to obtain the current data amount sum;

[0226] The target sub-interval determining unit is configured to determine a target sub-interval corresponding to the starting row number according to the total current data volume and the starting row number.

[0227] In an exemplary embodiment, the target subrange determining unit includes:

[0228] a first determining unit configured to, if the total amount of current data is less than the starting number of rows, re-set the first subinterval after the current subinterval as the current subinterval and repeat the operation performed on the traversed current subinterval;

[0229] The second determining unit is configured to determine the current sub-interval as the target sub-interval if the total amount of current data is greater than or equal to the starting number of rows.

[0230] In an exemplary embodiment, the data query module includes:

[0231] a preceding subinterval determining unit, configured to determine, based on the sorting result, a subinterval preceding the target subinterval as a preceding subinterval;

[0232] a preamble subinterval quantity sum determining unit, configured to determine the sum of the data amounts corresponding to the respective preamble subintervals, and obtain the preamble subinterval quantity sum;

[0233] a target quantity sum obtaining unit, configured to obtain the sum of the data amounts corresponding to the target subintervals to obtain the target quantity sum;

[0234] a comprehensive quantity sum determining unit, configured to determine the sum of the target quantity sum and the preceding subinterval quantity sum to obtain a comprehensive quantity sum;

[0235] The data query unit is configured to query the data to be queried in the target subinterval if the sum of the comprehensive quantities is greater than or equal to the target termination row number.

[0236] In an exemplary embodiment, the apparatus further comprises:

[0237] A screening module is configured to obtain, if the sum of the comprehensive number is less than the target end row number, the first subinterval after the target subinterval according to the sorting result to obtain a screening subinterval;

[0238] an updating module, configured to merge the target subinterval with the screening subinterval to obtain an updated target subinterval;

[0239] The update data query module is used to query the above-mentioned data to be queried in the above-mentioned update target sub-interval.

[0240] In an exemplary embodiment, the update data query module includes:

[0241] An update data volume acquisition unit is used to acquire the data volume corresponding to the update target subinterval to obtain the update data volume;

[0242] The step jump unit is used to obtain the data interval corresponding to the above-mentioned update target sub-interval if the above-mentioned updated data amount is greater than the above-mentioned preset threshold; and jump to the above-mentioned step of splitting the above-mentioned data interval into at least two sub-intervals to obtain the data amount corresponding to each sub-interval.

[0243] In an exemplary embodiment, the apparatus further comprises:

[0244] an offset determination module, configured to determine, if the amount of updated data is less than or equal to the preset threshold, a difference between the starting row number and the sum of the numbers of the preceding subintervals, and obtain an offset of the starting row number in the update target subinterval;

[0245] The module for determining data to be queried is used to extract the target number of rows of data from the update target subinterval starting from the number of rows corresponding to the offset to obtain the data to be queried.

[0246] In an exemplary embodiment, the target quantity acquisition module includes:

[0247] A primary parameter receiving unit, configured to receive an initial data query request sent by a terminal; the initial data query request carries a primary request parameter;

[0248] A parameter parsing unit is used to parse the above request parameters to obtain the starting row number and target number of the above data to be queried in the database;

[0249] In an exemplary embodiment, the apparatus further comprises:

[0250] A data total amount acquisition module is used to obtain the total amount of data corresponding to the above data query interval;

[0251] A data storage module, configured to generate a preset hash code based on the one-time request parameters and store the preset hash code in a hash code repository if the total amount of the data is greater than a preset quantity threshold;

[0252] A bucket group generation module, configured to generate bucket groups based on the at least two subintervals;

[0253] The offset array determination module is used to determine the offset corresponding to each bucket in the above bucket group and obtain the offset array:

[0254] The database construction module is used to construct a cache database using the preset hash code as a key and the bucket group and the offset array as values.

[0255] In an exemplary embodiment, the apparatus further comprises:

[0256] A secondary request receiving module, configured to receive a secondary data query request sent by the terminal; the secondary data query request carries a secondary request parameter;

[0257] A target hash code generation module, configured to generate a target hash code based on the secondary request parameters;

[0258] A hash code matching module is used to search the hash code repository for a hash code that matches the target hash code to obtain a matching hash code;

[0259] A target offset array determination module is used to search the cache database for a target bucket group and a target offset array corresponding to the matching hash code.

[0260] The data search module is used to search for data corresponding to the secondary data query request based on the target bucket group and the target offset array.

[0261] In an exemplary embodiment, the data search module includes:

[0262] A secondary parsing unit is used to parse the secondary request parameters to obtain the secondary starting row number and the secondary target number corresponding to the secondary query data;

[0263] A screening offset determination unit, configured to determine the position of the secondary starting row number in the target offset array to obtain a screening offset;

[0264] A bucket matching unit is used to filter buckets that match the filter offset from the target bucket group to obtain matching buckets;

[0265] The data search unit is used to search for data corresponding to the secondary data query request from the matching bucket based on the secondary target quantity.

[0266] The device and method embodiments in the above-mentioned device embodiments are based on the same inventive concept.

[0267] An embodiment of this specification provides a data query device, which includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by the processor to implement the data query method provided in the above method embodiment.

[0268] An embodiment of the present application also provides a computer storage medium, which can be set in a terminal to store at least one instruction or at least one program related to implementing a data query method in a method embodiment. The at least one instruction or at least one program is loaded and executed by the processor to implement the data query method provided by the above method embodiment.

[0269] Embodiments of the present application also provide a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to implement the data query method provided in the above method embodiment.

[0270] Optionally, in the embodiments of this specification, the storage medium may be located in at least one of the multiple network servers of the computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media that can store program code, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0271] The memory described in the embodiments of this specification can be used to store software programs and modules, and the processor executes various functional applications and data processing by running the software programs and modules stored in the memory. The memory may mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, application programs required for functions, etc.; the data storage area can store data created according to the use of the device, etc. In addition, the memory may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. Accordingly, the memory may also include a memory controller to provide the processor with access to the memory.

[0272] The data query method embodiment provided in the embodiment of this specification can be executed in a mobile terminal, a computer terminal, a server or a similar computing device. Taking running on a server as an example, Figure 14 This is a hardware structure diagram of a server of a data query method provided in an embodiment of this specification. Figure 14As shown, the server 1400 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPUs) 1410 (the central processing unit 1410 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 1430 for storing data, and one or more storage media 1420 (such as one or more mass storage devices) for storing application programs 1423 or data 1422. Among them, the memory 1430 and the storage medium 1420 can be temporary storage or permanent storage. The program stored in the storage medium 1420 may include one or more modules, each module may include a series of instruction operations on the server. Furthermore, the central processing unit 1410 can be configured to communicate with the storage medium 1420 to execute a series of instruction operations in the storage medium 1420 on the server 1400. The server 1400 may also include one or more power supplies 1460, one or more wired or wireless network interfaces 1450, one or more input and output interfaces 1440, and / or one or more operating systems 1421, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0273] The input / output interface 1440 can be used to receive or send data via a network. Specific examples of the aforementioned network may include a wireless network provided by the communication provider of the server 1400. In one embodiment, the input / output interface 1440 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In one embodiment, the input / output interface 1440 can be a radio frequency (RF) module for wireless communication with the Internet.

[0274] It can be understood by those skilled in the art that Figure 14 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 14 More or fewer components than shown, or with Figure 14 Different configurations shown.

[0275] As can be seen from the embodiments of the data query method, device, equipment or storage medium provided by the present application, the present application obtains the starting row number and target number of the data to be queried in the database; determines the sum of the starting row number and the target number to obtain the target ending row number of the data to be queried; if the target ending row number is greater than or equal to a preset threshold, obtains the data query interval corresponding to the data to be queried; splits the data query interval into at least two sub-intervals to obtain the data volume corresponding to each sub-interval; traverses at least two sub-intervals to determine the target sub-interval corresponding to the starting row number; based on the target number and the sum of the number of preceding sub-intervals, queries the data to be queried in the target sub-interval; the sum of the number of preceding sub-intervals is the sum of the data volume of each sub-interval sorted before the target sub-interval. The present invention can reduce the data volume of the sub-interval to an acceptable range for Elasticsearch through multiple divisions, locate the target sub-interval to which the data to be queried belongs during paging queries, and thus quickly and accurately query data with random page numbers in the target sub-interval.

[0276] It should be noted that the order in which the embodiments of this specification are presented is for illustrative purposes only and does not represent the superiority or inferiority of the embodiments. Furthermore, the foregoing descriptions are of specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in an order different from that in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0277] The various embodiments in this specification are described in a progressive manner. Similar portions between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from the other embodiments. In particular, the device, equipment, and storage medium embodiments are generally similar to the method embodiments, so their descriptions are relatively simplified. For relevant portions, refer to the descriptions of the method embodiments.

[0278] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or by a program to instruct the relevant hardware. The program may be stored in a computer storage medium, and the above-mentioned storage medium may be a read-only memory, a disk, or an optical disk, etc.

[0279] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application.

Claims

1. A data query method, characterized in that: The method comprises: Get the starting row number and target number of the data to be queried in the database; Determine the sum of the starting number of rows and the target number to obtain the target ending number of rows of the data to be queried; If the target end row number is greater than or equal to a preset threshold, obtaining a data query interval corresponding to the data to be queried; Splitting the data query interval into at least two subintervals, and obtaining the data volume corresponding to each subinterval; Traversing the at least two subintervals to determine a target subinterval corresponding to the starting row number; The target subinterval is searched for the data to be queried based on the target number and the sum of the numbers of preceding subintervals; the sum of the numbers of preceding subintervals is the sum of the data amounts of the subintervals that are sorted before the target subinterval.

2. The method according to claim 1, characterized in that Splitting the data query interval into at least two sub-intervals to obtain the data volume corresponding to each sub-interval includes: Obtaining a bucket quantity threshold for the database; Determine a partition interval based on a bucket quantity threshold of the database and two endpoint values corresponding to the data query interval; The data query interval is bucketed according to the partition interval to obtain the at least two sub-intervals; and the amount of data corresponding to each sub-interval is determined.

3. The method according to claim 1, characterized in that The traversing the at least two subintervals to determine the target subinterval corresponding to the starting row number includes: Sorting the at least two subintervals according to the splitting order to obtain a sorting result; According to the sorting result, the at least two subintervals are traversed in sequence, and the following operations are performed on the current subinterval traversed: According to the sorting result, each subinterval located before the current subinterval is regarded as the currently traversed subinterval; Determine the sum of the data volumes corresponding to each currently traversed subinterval to obtain the current total data volume; According to the current data amount sum and the starting row number, a target subinterval corresponding to the starting row number is determined.

4. The method according to claim 3, characterized in that The determining, based on the current data amount sum and the starting row number, a target subinterval corresponding to the starting row number includes: If the total amount of the current data is less than the starting number of rows, the first subinterval after the current subinterval is used as the current subinterval again, and the operation performed on the traversed current subinterval is repeated; If the total amount of current data is greater than or equal to the starting number of rows, the current sub-interval is determined as the target sub-interval.

5. The method according to claim 4, characterized in that The step of searching for the data to be queried in the target subinterval based on the target number and the sum of the number of preceding subintervals includes: According to the sorting result, determining the subinterval before the target subinterval as the preceding subinterval; Determine the sum of the data amounts corresponding to each preamble subinterval to obtain the sum of the preamble subinterval quantities; Obtaining the sum of the data amounts corresponding to the target subintervals to obtain a target sum; Determine the sum of the target quantity sum and the sum of the quantity of the preceding subintervals to obtain a comprehensive quantity sum; If the sum of the comprehensive quantities is greater than or equal to the target end row number, the target subinterval is searched for the data to be queried.

6. The method according to claim 5, characterized in that The method further comprises: If the sum of the comprehensive numbers is less than the target end row number, obtaining the first subinterval after the target subinterval according to the sorting result to obtain a screening subinterval; Merging the target subinterval with the screening subinterval to obtain an updated target subinterval; The to-be-queried data is searched in the update target subinterval.

7. The method according to claim 6, characterized in that The querying the to-be-queried data in the update target subinterval includes: Obtaining the data volume corresponding to the update target subinterval to obtain the updated data volume; If the updated data volume is greater than the preset threshold, the data interval corresponding to the updated target subinterval is obtained; and the process jumps to the step of splitting the data interval into at least two subintervals to obtain the data volume corresponding to each subinterval.

8. The method according to claim 7, characterized in that The method further comprises: If the updated data amount is less than or equal to the preset threshold, determining the difference between the starting row number and the sum of the numbers of the preceding subintervals, and obtaining the offset of the starting row number in the update target subinterval; In the update target subinterval, starting from the number of rows corresponding to the offset, the target number of rows of data are retrieved to obtain the data to be queried.

9. The method according to any one of claims 1 to 8, characterized in that The step of obtaining the starting number of rows and the target number of rows of the data to be queried in the database includes: Receiving an initial data query request sent by a terminal; the initial data query request carries a primary request parameter; Parsing the request parameters to obtain the starting row number and target number of the data to be queried in the database; After splitting the data query interval into at least two sub-intervals, the method further includes: Obtaining the total amount of data corresponding to the data query interval; If the total amount of data is greater than a preset quantity threshold, generating a preset hash code based on the one-time request parameter and storing the preset hash code in a hash code repository; generating bucket groups based on the at least two subintervals; Determine the offset corresponding to each bucket in the bucket group to obtain an offset array; A cache database is constructed using the preset hash code as a key and the bucket group and the offset array as values.

10. The method according to claim 9, characterized in that The method further comprises: receiving a secondary data query request sent by the terminal, wherein the secondary data query request carries a secondary request parameter; Based on the secondary request parameters, generating a target hash code; Searching for a hash code that matches the target hash code in the hash code repository to obtain a matching hash code; Searching the cache database for a target bucket group and a target offset array corresponding to the matching hash code; Based on the target bucket group and the target offset array, data corresponding to the secondary data query request is searched.

11. The method according to claim 10, characterized in that The searching for data corresponding to the secondary data query request based on the target bucket group and the target offset array includes: Parsing the secondary request parameters to obtain the secondary starting row number and the secondary target number corresponding to the secondary query data; Determine the position of the secondary starting row number in the target offset array to obtain a screening offset; Filtering buckets that match the filter offset from the target bucket group to obtain matching buckets; Based on the secondary target quantity, data corresponding to the secondary data query request is searched from the matching buckets.

12. A data query device, characterized in that: The device comprises: The target quantity acquisition module is used to obtain the starting row number and target quantity of the data to be queried in the database; A target end row number module is used to determine the sum of the starting row number and the target number to obtain the target end row number of the data to be queried; A data query interval acquisition module, configured to acquire a data query interval corresponding to the data to be queried if the target termination row number is greater than or equal to a preset threshold; A data volume determination module, configured to split the data query interval into at least two sub-intervals and obtain the data volume corresponding to each sub-interval; a traversal module, configured to traverse the at least two subintervals and determine a target subinterval corresponding to the starting row number; A data query module is used to query the data to be queried in the target sub-interval based on the target number and the sum of the number of preceding sub-intervals; the sum of the number of preceding sub-intervals is the sum of the data amounts of each sub-interval sorted before the target sub-interval.

13. An electronic device, characterized in that: The device includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the data query method according to any one of claims 1-11.

14. A computer storage medium, characterized in that The computer storage medium stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the data query method according to any one of claims 1 to 11.

15. A computer program product comprising computer instructions, characterized in that When the computer instructions are executed by a processor, the data query method according to any one of claims 1 to 11 is implemented.

Images