Data query method and device, electronic equipment and storage medium

Abstract

The invention relates to a data query method and device, electronic equipment and a storage medium, and the method comprises the steps: receiving a data query request from a client, and carrying out the iteration processing through the following steps until a preset iteration stop condition is met: constructing a database query statement, and executing the database query statement, a batch of query results are queried from the database, the number of records in the query results of each batch does not exceed the set number of returned records, and the query results of different batches are different; loading the query results of the current batch to a service layer, and screening the query results of the current batch in the service layer according to a preset service scene rule; and returning a screening result to the client. The method can improve database query efficiency and reduce resource consumption so as to better adapt to complex and changeable business scene requirements.

Description

Technical Field

[0001] The present application relates to the field of databases, and in particular to a data query method, device, electronic device and storage medium. Background Art

[0002] In current e-commerce and diversified transaction systems, querying the order database is one of the core functions of the system business. Faced with complex and ever-changing business scenarios, although SQL (Structured Query Language) provides basic data query functions, it is difficult to meet the refined screening requirements under specific business logic. In this regard, the following query method is adopted in related technologies: first, a large amount of original order data is queried from the database through SQL statements, and then the business layer performs more detailed and refined screening based on complex business rules.

[0003] Although the above method can cope with complex and ever-changing business scenarios, it is an inefficient query method. Its inefficiency is mainly reflected in the following three aspects: First, when the order database is large in scale, even if the user or business logic only needs to query and process a small part of the order records, the above method still retrieves and returns a large amount of data from the database indiscriminately. This "carpet-style" query method is not only inefficient, but also greatly consumes the query processing capacity of the database, resulting in a waste of resources. Secondly, when the business layer processes a large amount of data, it needs to occupy a large amount of memory resources as temporary storage, which may lead to a shortage of memory resources, especially when there are many concurrent queries, the business layer may become a bottleneck restricting system performance when processing a large amount of data. Furthermore, in a distributed system, the database and the business layer may be deployed on different physical nodes, which means that the data queried from the database needs to be transmitted to the business layer through the network for processing. Then, when the amount of data queried from the database is large, it will occupy a large amount of network bandwidth, resulting in a longer response time, which in turn affects the stability and reliability of the entire system. Summary of the invention

[0004] The present application provides a data query method, device, electronic device and storage medium to solve the technical problems of low database query efficiency and waste of resources in the prior art when dealing with complex and changing business scenarios.

[0005] In a first aspect, the present application provides a data query method, the method comprising:

[0006] Receive data query requests from the client and perform iterative processing using the following steps until the preset iteration stop condition is met:

[0007] Constructing a database query statement and executing the database query statement to query a batch of query results from the database, wherein the number of records in each batch of query results does not exceed a set limit on the number of records returned, and query results of different batches are different;

[0008] Loading the query results of the current batch into the business layer, and filtering the query results of the current batch in the business layer according to preset business scenario rules;

[0009] The screening result is returned to the client.

[0010] In a possible implementation, the preset iteration stop condition includes:

[0011] The number of iterations reaches the set threshold;

[0012] Alternatively, the total number of records in the currently obtained screening results reaches a set total number threshold, wherein the total number threshold is greater than the limited number of records to be returned.

[0013] In a possible implementation, constructing a database query statement includes:

[0014] A database query statement is constructed according to the data query request and the current iteration round.

[0015] In a possible implementation, constructing a database query statement according to the data query request and the current iteration round includes:

[0016] In the case of the first round of iteration, a database query statement is constructed according to the data query request and the limit on the number of records to be returned;

[0017] When the current iteration is not the first round, a database query statement is constructed based on the data query request, the limit on the number of records returned, and the current query starting position identifier, wherein the current query starting position identifier points to the last record queried at the end of the previous round of iterative processing.

[0018] In a possible implementation, constructing a database query statement according to the data query request and the current iteration round includes:

[0019] According to the current iteration round and the number of records returned by the limit, the query scope of the current iteration processing is determined from the database;

[0020] A database query statement is constructed according to the data query request and the query scope.

[0021] In a possible implementation, the limiting number of returned records is determined in the following manner:

[0022] Determine the number of records in the screening results obtained in the previous round of iterative processing;

[0023] When the number of records determined above meets the set conditions, the default value is determined as limiting the number of records returned;

[0024] When the number of records determined above does not satisfy the set condition, the default value is increased to obtain the limited number of records to be returned.

[0025] In a possible implementation, returning the screening result to the client includes:

[0026] After the current round of iterative processing is completed, the screening result obtained by the current round of iterative processing is returned to the client;

[0027] Alternatively, after the entire iterative processing process is completed, all screening results obtained in the entire iterative processing process are returned to the client.

[0028] In a second aspect, the present application provides a database query device, the device comprising:

[0029] A receiving module, used for receiving a data query request from a client;

[0030] The iterative query module is used to perform iterative processing using the following steps until a preset iteration stop condition is met:

[0031] Constructing a database query statement and executing the database query statement to query a batch of query results from the database, wherein the number of records in each batch of query results does not exceed a set limit on the number of records returned, and query results of different batches are different;

[0032] Loading the query results of the current batch into the business layer, and filtering the query results of the current batch in the business layer according to preset business scenario rules;

[0033] The result returning module is used to return the screening result to the client.

[0034] In a possible implementation, the preset iteration stop condition includes:

[0035] The number of iterations reaches the set threshold;

[0036] Alternatively, the total number of records in the currently obtained screening results reaches a set total number threshold, wherein the total number threshold is greater than the limited number of records to be returned.

[0037] In a possible implementation, the iterative query module includes:

[0038] The query statement building unit is used to build database query statements, including:

[0039] A database query statement is constructed according to the data query request and the current iteration round.

[0040] In a possible implementation manner, the query statement building unit is specifically used to:

[0041] In the case of the first round of iteration, a database query statement is constructed according to the data query request and the limit on the number of records to be returned;

[0042] When the current iteration is not the first round, a database query statement is constructed based on the data query request, the limit on the number of records returned, and the current query starting position identifier, wherein the current query starting position identifier points to the last record queried at the end of the previous round of iterative processing.

[0043] In a possible implementation manner, the query statement building unit is specifically used to:

[0044] According to the current iteration round and the number of records returned by the restriction, the query scope of the current iteration processing is determined from the database;

[0045] A database query statement is constructed according to the data query request and the query scope.

[0046] In a possible implementation, the device further includes:

[0047] The limit number adjustment module is used to determine the limit number of returned records in the following way:

[0048] Determine the number of records in the screening results obtained in the previous round of iterative processing;

[0049] When the number of records determined above meets the set conditions, the default value is determined as limiting the number of records returned;

[0050] When the number of records determined above does not satisfy the set condition, the default value is increased to obtain the limited number of records to be returned.

[0051] In a possible implementation manner, the result returning module is specifically used to:

[0052] After the current round of iterative processing is completed, the screening result obtained by the current round of iterative processing is returned to the client;

[0053] Alternatively, after the entire iterative processing process is completed, all screening results obtained in the entire iterative processing process are returned to the client.

[0054] In a third aspect, the present application provides an electronic device, comprising: a processor and a memory, wherein the processor is used to execute a database query program stored in the memory to implement the data query method described in any one of the first aspects.

[0055] In a fourth aspect, the present application provides a storage medium storing one or more programs, wherein the one or more programs can be executed by one or more processors to implement the data query method described in any one of the first aspects.

[0056] The above technical solution provided by the embodiment of the present application has the following advantages over the prior art: the method provided by the embodiment of the present application, by receiving a data query request from the client, performs iterative processing using the following steps until a preset iteration stop condition is met: constructing a database query statement, and executing the database query statement, querying a batch of query results from the database, wherein the number of records in the query results of each batch does not exceed the set limit on the number of records returned, and the query results of different batches are different; loading the query results of the current batch into the business layer, and filtering the query results of the current batch in the business layer according to preset business scenario rules, and returning the filtered results to the client, thereby realizing a dynamic on-demand multiple query processing mechanism, under which the entire query task is completed through multiple small batch queries, the number of records returned in a query process has an upper limit, which can avoid the waste of data query capabilities caused by a one-time large-scale data query, and no longer needs to occupy a large amount of memory resources as temporary storage at the business layer, thereby reducing the risk of affecting system performance, and no longer needs to transfer a large number of records to the business layer for processing at one time, thereby improving the query response speed and improving the stability and reliability of the entire system. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0058] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0059] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0060] Figure 1 A flowchart of an embodiment of a data query method provided in an embodiment of the present application;

[0061] Figure 2 A flowchart of another data query method provided in an embodiment of the present application;

[0062] Figure 3 An exemplary system architecture diagram provided for an embodiment of the present application;

[0063] Figure 4 A block diagram of an embodiment of a data query device provided in an embodiment of the present application;

[0064] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0066] The disclosure below provides many different embodiments or examples to realize the different structures of the present application. In order to simplify the disclosure of the present application, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present application. In addition, the present application can repeat reference numbers and / or letters in different examples. This repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed.

[0067] In order to solve the technical problems in the prior art that database query efficiency is low and resources are wasted when dealing with complex and changing business scenarios, the present application provides a data query method, device, electronic device and storage medium. The method can improve database query efficiency and reduce resource consumption to better meet the needs of complex and changing business scenarios.

[0068] Figure 1The flowchart of a data query method provided in the embodiment of the present application is shown in FIG. In one embodiment, the execution subject of the method is a server. Figure 1 As shown, the process includes the following steps:

[0069] Step 101: Receive a data query request from a client.

[0070] The client refers to an application that provides services to users (such as e-commerce services, transaction services, etc.), which can be an application running in an independent process, a sub-application (applet) independent of the main page embedded in the client, a function running on a web browser (also known as WebApp), a mini-program embedded in an email, etc. The above-mentioned server refers to the backend server corresponding to the client, which provides backend services to the client. The backend services include but are not limited to: data query services, data transmission services, data processing services, etc.

[0071] When a user has a data query requirement, he can enter query conditions through the user interaction interface provided by the client, and then the client encapsulates the data query request according to these query conditions, and then sends the encapsulated data query request to the server. Then, the server can receive the data query request from the client.

[0072] For example, if a user needs to query order records, the user can enter query conditions through the user interaction interface provided by the client, such as order date, transaction status (such as transaction completed, transaction cancelled, etc.), product name, payment type (such as free order, paid order), etc. One or more of the query conditions. Then the client generates a data query request based on the query conditions entered by the user and the user's identity (such as user ID) and sends it to the server.

[0073] It should be noted here that although the SQL query language provides basic retrieval functions, it is often difficult to meet the refined screening requirements under specific business logic. This has also led to the background of the technical solution of the embodiment of the present application: first, a large amount of original record data is captured from the database through SQL statements, and then the business layer performs secondary screening based on complex business rules. For example, at the business level, the user needs to screen out orders that meet the following conditions: paid orders completed by user A in the past 30 days and with a total order amount of more than 200 yuan. Since the SQL query language itself is difficult to directly express complex business logic, all order records of user A in the past 30 days can be queried from the database through simple SQL query statements, and then secondary screening is performed at the business layer according to complex business rules to obtain paid orders completed by user A in the past 30 days and with a total order amount of more than 200 yuan.

[0074] Those skilled in the art will appreciate that the above are merely exemplary business scenarios. In actual applications, there may be complex and varied business scenarios, and the business rules in different business scenarios are different, which is not limited in the embodiments of the present application.

[0075] Step 102: perform iterative processing using the following steps until a preset iteration stop condition is met:

[0076] Step 1021, construct a database query request, and execute the database query request to query a batch of query results from the database, wherein the number of records in each batch of query results does not exceed the set limit on the number of records returned, and query results of different batches are different.

[0077] Step 1022: Load the query results of the current batch into the business layer, and filter the query results of the current batch in the business layer according to preset business scenario rules.

[0078] In the technical solution provided in the embodiment of the present application, the server receives a data query request from the client, and performs iterative processing using the above-mentioned steps 1021 and 1022. In general, the iterative processing process includes: the server receives a data query request from the client, starts the first round of iterative processing, and executes steps 1021 and 1022 during the first round of iterative processing. After the first round of iterative processing is completed, it is determined whether the preset iteration stop condition is currently met. If the preset iteration stop condition is not currently met, the second round of iterative processing is continued to be started, and steps 1021 and 1022 are executed during the second round of iterative processing. If the preset iteration stop condition is currently met, the iterative processing is stopped.

[0079] Assuming that the second round of iterative processing is started, after the second round of iterative processing is completed, it is further determined whether the preset iterative stop condition is currently satisfied. If the preset iterative stop condition is not currently satisfied, the third round of iterative processing is continued to be started, and step 1021 and step 1022 are performed in the third round of iterative processing. If the preset iterative stop condition is currently satisfied, the iterative processing is stopped.

[0080] When the third round of iterative processing is started, the same processing process is repeated until the preset iterative stopping condition is met, and then the iterative processing is stopped.

[0081] When the iterative processing stops, it means that the server has completed the data query task.

[0082] According to the description of step 1021 and step 1022, the server constructs a database query statement and executes the database query statement in each round of iterative processing to query a batch of query results from the database. After that, the query results of the current batch are loaded into the business layer, and the query results of the current batch are filtered in the business layer according to the preset business scenario rules. It can be seen that in each round of iterative processing, a general query is first performed at the database level, and then judged and filtered by the business layer logic.

[0083] It should be noted that the number of records in each batch of query results does not exceed the set limit on the number of records returned, and the query results of different batches are different. In other words, the number of records queried in each round of iterative processing is limited, and each round of iterative processing can query different records from the database.

[0084] In one embodiment, the preset iteration stop condition includes: the number of iterations reaches a set number threshold; or the total number of records of the current screening results reaches a set total number threshold, wherein the total number threshold is greater than the limit on the number of returned records. Exemplarily, the total number threshold depends on the user's query requirements and can be set by the user. Alternatively, the total number threshold depends on the business scenario. For example, since the display screen of a mobile phone is smaller than the display screen of a computer, the total number threshold of the mobile client is set to be smaller than the total number threshold of the computer client.

[0085] It can be seen from the above description that in the technical solution provided by the embodiment of the present application, after receiving the data query request from the client, the server does not query all the records that meet the query conditions from the database at one time, and load all these records into the business layer for further judgment and screening, but performs multiple small batch queries on the database, and loads the queried small batch records into the business layer for further judgment and screening after each query. If the number of filtered records is insufficient, and the number of queries has not reached the set upper limit of the number of queries, the next query will be performed until the number of filtered records meets the demand, or the number of queries reaches the set upper limit of the number of queries. This realizes a dynamic on-demand multiple query processing mechanism, which can avoid the waste of resources caused by one-time large-scale data queries through multiple small batch queries.

[0086] Step 103: Return the screening result to the client.

[0087] In one embodiment, the specific implementation of returning the screening results to the client includes: after the current round of iterative processing is completed, the screening results obtained by the current round of iterative processing are returned to the client. That is, each time the server performs a round of iterative processing and obtains a batch of screening results, the current screening results are immediately returned to the client for the client to use these screening results (for example, display the screening results). This processing method can improve the query response speed.

[0088] In another embodiment, the specific implementation of returning the screening results to the client includes: after the entire iterative processing process is completed, all the screening results obtained in the entire iterative processing process are returned to the client. That is, after the server completes the data query task this time, it returns the complete query results to the client so that the client can use these screening results (for example, display the screening results). This processing method can ensure the integrity and consistency of the query results.

[0089] The technical solution provided by the embodiment of the present application performs iterative processing by using the following steps when receiving a data query request from a client until a preset iteration stop condition is met: constructing a database query statement and executing the database query statement to query a batch of query results from the database, wherein the number of records in the query results of each batch does not exceed the set limit on the number of records returned, and the query results of different batches are different; loading the query results of the current batch into the business layer, and filtering the query results of the current batch in the business layer according to preset business scenario rules, and returning the filtered results to the client, thereby realizing a dynamic on-demand multiple query processing mechanism, under which the entire query task is completed through multiple small batch queries, the number of records returned in a query process has an upper limit, which can avoid the waste of data query capabilities caused by a one-time large-scale data query, and no longer needs to occupy a large amount of memory resources as temporary storage at the business layer, thereby reducing the risk of affecting system performance, and no longer needs to transfer a large number of records to the business layer for processing at one time, thereby improving the query response speed and improving the stability and reliability of the entire system.

[0090] In one embodiment, the specific implementation of constructing a database query statement during each round of iterative processing by the server includes: constructing a database query statement according to the data query request and the current iterative round. It can be seen that the database query statement executed during each round of iterative processing is different, which means that after the end of one round of iterative processing, if the next round of iterative processing is to be started, the database query statement is reconstructed. This mechanism ensures that the server dynamically constructs the database query statement according to the current demand in each iterative round, thereby ensuring that the query results of different batches are different.

[0091] Among them, as an optional implementation method, the specific implementation of constructing a database query statement according to the data query request and the current iteration round includes: when the current iteration is in the first round, the database query statement is constructed according to the data query request and the number of records to be returned is limited. When the current iteration is not in the first round, the database query statement is constructed according to the data query request, the number of records to be returned is limited, and the current query starting position identifier, wherein the current query starting position identifier points to the last record queried at the end of the previous round of iteration processing.

[0092] In the case of the first round of processing, you can use the LIMIT clause of SQL and the query conditions contained in the data query request to construct a database query statement. The LIMIT clause is used to specify the number of records that should be returned in the query results. The following is a simple example to show how to use the LIMIT clause in a database query statement:

[0093] Assume that there is an order table in the database, which contains the order information of each user, and assume that the query condition contained in the data query request indicates to return 10 order records of user A. The corresponding database query statement can be:

[0094] SELECT * FROM (table name) WHERE department = 'User A';

[0095] LIMIT 10;

[0096] When executing the above database query statement, based on the global order of the data table, the query is performed one by one from top to bottom, that is, starting from the first record in the data table, and searching backwards one by one until 10 order records are found, and then the query stops. The 10 order records found at this time are the query results of the first batch.

[0097] When the current iteration is not the first one, you can use the SQL LIMIT clause, OFFSET clause, and the query conditions contained in the data query request to construct a database query statement. The OFFSET clause is used to specify the current query start position identifier, which can point to the last record queried at the end of the previous iteration. The following is a simple example to show how to use the LIMIT clause and OFFSET clause in a database query statement:

[0098] Assume that there is an order table in the database, which contains the order information of each user, and assume that the query condition contained in the data query request indicates to return the 10 order records of user A after the 10th record. The corresponding database query statement can be:

[0099] SELECT * FROM (table name) WHERE department = 'User A';

[0100] LIMIT 10OFFSET 10;

[0101] When executing the above database query statement, based on the global order of the data table, the first 10 records are skipped, and the query is retrieved one by one from the 11th record until 10 order records are found, and the query is stopped.

[0102] As another optional implementation method, the specific implementation of constructing a database query statement based on the data query request and the current iteration round includes: determining the query scope of the current round of iterative processing from the database based on the current iteration round and limiting the number of records returned; and constructing a database query statement based on the data query request and the query scope.

[0103] In this implementation, the query range width is consistent in each round of iterative processing, and the difference is the query start position and query end position. This method can also ensure that the number of records in each batch of query results does not exceed the set limit on the number of records returned, and the query results of different batches are different.

[0104] For example, the query range during each round of iterative processing can be calculated as follows:

[0105] start_position=(r-1)*query_width;

[0106] end_position=start_position+query_width-1.

[0107] Where start_position indicates the query start position, end_position indicates the query end position, query_width indicates the query range width, r indicates the current iteration round, and query_width indicates the limit on the number of records returned.

[0108] When constructing a database query statement, it can be constructed according to the query scope determined in the above manner.

[0109] The above-mentioned embodiment can ensure that the server dynamically constructs database query statements according to current needs in each iteration round by constructing database query statements according to data query requests and the current iteration round during each round of iterative processing by the server, thereby ensuring that the query results of different batches are different and limiting the number of records returned by each query.

[0110] In one embodiment, the above-mentioned limit on the number of records returned is not fixed, but can be dynamically adjusted according to actual conditions during the iterative processing. Exemplarily, the limit on the number of records returned is determined in the following manner: the number of records in the screening results obtained by the previous round of iterative processing is determined, and when the above-determined number of records meets the set conditions, the default value is determined as the limit on the number of records returned; when the above-determined number of records does not meet the set conditions, the default value is increased to obtain the limit on the number of records returned.

[0111] The above embodiment provides a feedback mechanism, that is, adjusting the number of queries for the next time according to the results of the previous iteration. By dynamically adjusting the number of records returned, the query efficiency can be optimized according to the actual situation. For example, if the number of records obtained in the previous iteration is very small, it may mean that the query range needs to be increased to obtain more relevant records; if the number of records is large, it may mean that the current query conditions are relatively accurate and there is no need to expand the query range. This method of dynamically adjusting the number of queries according to the results of the previous iteration can more flexibly respond to various complex data query requirements and further improve data query efficiency.

[0112] Exemplarily, the above-mentioned increasing processing of the default value may be to increase a set value based on the default value, or to increase a set ratio, so as to limit the number of returned records.

[0113] In addition, in practical applications, the number of records to be returned may be limited differently for different business scenarios, and the present application embodiment does not limit this.

[0114] Figure 2 A flowchart of another data query method provided in an embodiment of the present application. Figure 2The process shown in the figure includes the following steps based on the above embodiment:

[0115] Step 201: Receive a data query request from a client.

[0116] Step 202 : Determine whether the current number of iterations reaches a set number threshold, if yes, execute step 206 , if no, execute step 203 .

[0117] Step 203: construct a database query request, execute the database query request, and query a batch of query results from the database.

[0118] The number of records in each batch of query results does not exceed the set limit on the number of records returned, and query results in different batches are different.

[0119] Step 204: load the query results of the current batch into the business layer, and filter the query results of the current batch in the business layer according to preset business scenario rules.

[0120] Step 205 , determine whether the total number of records in the current screening result reaches the set total number threshold, if yes, execute step 206 , if not, return to execute step 202 .

[0121] Step 206: End the query and return the screening results to the client.

[0122] Figure 2 The process shown implements a dynamic on-demand multiple query processing mechanism by combining the number of iterations and the total number of records in the current screening results after receiving the data query request from the client. Under this processing mechanism, the entire query task is completed through multiple small batch queries. The number of records returned in a query process has an upper limit, which can avoid the waste of data query capabilities caused by a one-time large-scale data query. In addition, it is no longer necessary to occupy a large amount of memory resources as temporary storage at the business layer, thereby reducing the risk of affecting system performance. It is no longer necessary to transfer a large number of records to the business layer for processing at one time, thereby improving the query response speed and improving the stability and reliability of the entire system.

[0123] Figure 3 An exemplary system architecture diagram provided for an embodiment of the present application. Figure 3 The system architecture shown includes: a query management module 31 , a database query module 32 , a business scenario judgment module 33 , and a result processing module 34 .

[0124] The query management module 31 is used to control the query process, including setting the number of iterative queries, limiting the number of returned records, and deciding whether to perform the next query. For example, whether to perform the next query is determined based on the current number of iterations and the total number of records in the current screening results.

[0125] The database query module 32 is used to perform database query of data using database query statements.

[0126] The business scenario judgment module 33 is used to filter the records returned by the database query module according to predefined business scenario rules.

[0127] The result processing module 34 is used to process (eg, return to the client) and accumulate the records screened by the business scenario determination module 32 .

[0128] As for Figure 3 The working details of each module in the system shown can be found in the relevant description in the above embodiments, which will not be repeated here.

[0129] Figure 4 This is a block diagram of an embodiment of a data query device provided in an embodiment of the present application. Figure 4 As shown, the device comprises:

[0130] The receiving module 41 is used to receive a data query request from a client;

[0131] The iterative query module 42 is used to perform iterative processing using the following steps until a preset iteration stop condition is met:

[0132] Constructing a database query statement and executing the database query statement to query a batch of query results from the database, wherein the number of records in each batch of query results does not exceed a set limit on the number of records returned, and query results of different batches are different;

[0133] Loading the query results of the current batch into the business layer, and filtering the query results of the current batch in the business layer according to preset business scenario rules;

[0134] The result returning module 43 is used to return the screening result to the client.

[0135] In a possible implementation, the preset iteration stop condition includes:

[0136] The number of iterations reaches the set threshold;

[0137] Alternatively, the total number of records in the currently obtained screening results reaches a set total number threshold, wherein the total number threshold is greater than the limited number of records to be returned.

[0138] In a possible implementation, the iterative query module 42 includes:

[0139] The query statement building unit is used to build database query statements, including:

[0140] A database query statement is constructed according to the data query request and the current iteration round.

[0141] In a possible implementation manner, the query statement building unit is specifically used to:

[0142] In the case of the first round of iteration, a database query statement is constructed according to the data query request and the limit on the number of records to be returned;

[0143] When the current iteration is not the first round, a database query statement is constructed based on the data query request, the limit on the number of records returned, and the current query starting position identifier, wherein the current query starting position identifier points to the last record queried at the end of the previous round of iterative processing.

[0144] In a possible implementation manner, the query statement building unit is specifically used to:

[0145] According to the current iteration round and the number of records returned by the limit, the query scope of the current iteration processing is determined from the database;

[0146] A database query statement is constructed according to the data query request and the query scope.

[0147] In a possible implementation, the device further includes:

[0148] The limit number adjustment module is used to determine the limit number of returned records in the following way:

[0149] Determine the number of records in the screening results obtained in the previous round of iterative processing;

[0150] When the number of records determined above meets the set conditions, the default value is determined as limiting the number of records returned;

[0151] When the number of records determined above does not satisfy the set condition, the default value is increased to obtain the limited number of records to be returned.

[0152] In a possible implementation manner, the result returning module 43 is specifically configured to:

[0153] After the current round of iterative processing is completed, the screening result obtained by the current round of iterative processing is returned to the client;

[0154] Alternatively, after the entire iterative processing process is completed, all screening results obtained in the entire iterative processing process are returned to the client.

[0155] like Figure 5 As shown, an embodiment of the present application provides an electronic device, including a processor 111, a communication interface 112, a memory 113 and a communication bus 114, wherein the processor 111, the communication interface 112, and the memory 113 communicate with each other through the communication bus 114.

[0156] Memory 113, used for storing computer programs;

[0157] In one embodiment of the present application, the processor 111 is used to execute the program stored in the memory 113 to implement the data query method provided by any of the above method embodiments, including:

[0158] Receive data query requests from the client and perform iterative processing using the following steps until the preset iteration stop condition is met:

[0159] Constructing a database query statement and executing the database query statement to query a batch of query results from the database, wherein the number of records in each batch of query results does not exceed a set limit on the number of records returned, and query results of different batches are different;

[0160] Loading the query results of the current batch into the business layer, and filtering the query results of the current batch in the business layer according to preset business scenario rules;

[0161] The screening result is returned to the client.

[0162] An embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the data query method provided in any of the aforementioned method embodiments are implemented.

[0163] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0164] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0165] It should be understood that the terms used herein are only for the purpose of describing specific example embodiments and are not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms "one", "an" and "said" as used herein may also be meant to include plural forms. The terms "include", "comprise", "contain", and "have" are inclusive, and therefore specify the existence of stated features, steps, operations, elements and / or parts, but do not exclude the existence or addition of one or more other features, steps, operations, elements, parts, and / or combinations thereof. The method steps, processes, and operations described herein are not interpreted as necessarily requiring them to be performed in the specific order described or illustrated, unless the execution order is clearly indicated. It should also be understood that additional or alternative steps may be used.

[0166] The above description is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest range consistent with the principles and novel features applied for herein.

Claims

1. A data query method, characterized in that: The method comprises: Receive data query requests from the client and perform iterative processing using the following steps until the preset iteration stop condition is met: Constructing a database query statement and executing the database query statement to query a batch of query results from the database, wherein the number of records in each batch of query results does not exceed a set limit on the number of records returned, and query results of different batches are different; Loading the query results of the current batch into the business layer, and filtering the query results of the current batch in the business layer according to preset business scenario rules; The screening result is returned to the client.

2. The method according to claim 1, characterized in that The preset iteration stop condition includes: The number of iterations reaches the set threshold; Alternatively, the total number of records in the currently obtained screening results reaches a set total number threshold, wherein the total number threshold is greater than the limited number of records to be returned.

3. The method according to claim 1, characterized in that The step of constructing a database query statement includes: A database query statement is constructed according to the data query request and the current iteration round.

4. The method according to claim 3, characterized in that The step of constructing a database query statement according to the data query request and the current iteration round includes: In the case of the first round of iteration, a database query statement is constructed according to the data query request and the limit on the number of records to be returned; When the current iteration is not the first round, a database query statement is constructed based on the data query request, the limit on the number of records returned, and the current query starting position identifier, wherein the current query starting position identifier points to the last record queried at the end of the previous round of iterative processing.

5. The method according to claim 3, characterized in that: The step of constructing a database query statement according to the data query request and the current iteration round includes: According to the current iteration round and the number of records returned by the restriction, the query scope of the current iteration processing is determined from the database; A database query statement is constructed according to the data query request and the query scope.

6. The method according to any one of claims 1 to 5, characterized in that: The limit on the number of returned records is determined in the following way: Determine the number of records in the screening results obtained in the previous round of iterative processing; When the number of records determined above meets the set conditions, the default value is determined as limiting the number of records returned; When the number of records determined above does not satisfy the set condition, the default value is increased to obtain the limited number of records to be returned.

7. The method according to claim 1, characterized in that The step of returning the screening result to the client comprises: After the current round of iterative processing is completed, the screening result obtained by the current round of iterative processing is returned to the client; Alternatively, after the entire iterative processing process is completed, all screening results obtained in the entire iterative processing process are returned to the client.

8. A database query device, characterized in that: The device comprises: A receiving module, used for receiving a data query request from a client; The iterative query module is used to perform iterative processing using the following steps until a preset iteration stop condition is met: Constructing a database query statement and executing the database query statement to query a batch of query results from the database, wherein the number of records in each batch of query results does not exceed a set limit on the number of records returned, and query results of different batches are different; Loading the query results of the current batch into the business layer, and filtering the query results of the current batch in the business layer according to preset business scenario rules; The result returning module is used to return the screening result to the client.

9. An electronic device, characterized in that: include: A processor and a memory, wherein the processor is used to execute a database query program stored in the memory to implement the data query method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the data query method according to any one of claims 1 to 7.

Images