A data query method, apparatus, device and medium

Abstract

This invention relates to the field of data processing technology, specifically providing a data query method, apparatus, device, and medium, aiming to solve the problem of slow performance in the total number of records query and sorting process during data querying, especially with large volumes of data. To this end, the method of this invention includes: calculating the number of digits *i* of the primary key element required for fuzzy matching of the primary key column in the table containing the data to be queried using the SQL LIKE operator; constructing an SQL statement based on the number of digits *i*: `select * from tablename where uuid like 'specified pattern'`; and iterating through all possible *i* digit primary key elements to execute the SQL statement k... i The For loop iterates through the data retrieved in each iteration, loading the data into memory as a separate batch. This implementation reduces performance overhead and improves work efficiency.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, specifically to a data query method, apparatus, device, and medium. Background Technology

[0002] Currently, traditional batch data query methods require querying the total number of records and paginating the data after sorting it. When there is a large amount of data in the database, this solution is slow and affects the user experience.

[0003] Accordingly, a new technical solution is needed in this field to solve the above problems. Summary of the Invention

[0004] To overcome the above-mentioned defects, the present invention is proposed to provide a data query method, apparatus, device, and medium that solves, or at least partially solves, the performance of the total number of data queries and the ordering process slows down in the case of a large number of data entries.

[0005] In a first aspect, a data query method is provided, the method comprising:

[0006] Calculate the number of bits i of the primary key elements required to perform fuzzy matching on the primary key column of the table containing the data to be queried using the SQLlike operator, wherein the primary key corresponds one-to-one with the data to be queried, and the primary key is a string consisting of L primary key elements, with each primary key element selected from a group consisting of k characters;

[0007] Construct an SQL statement based on the number of bits i: select * from tablename where uuid like 'specified pattern', where tablename represents the table name, * represents all columns of the table, uuid represents the primary key, and the specified pattern consists of i primary key elements and wildcards %.

[0008] Iterate through all possible i-bit primary key elements to execute k operations on the SQL statement. i This is the second For loop;

[0009] The data retrieved in each loop is loaded into memory as a separate batch.

[0010] In one technical solution of the above data query method, the calculation of the number of bits i of the primary key element required for fuzzy matching of the primary key column in the table containing the data to be queried using the SQL like operator includes:

[0011] Obtain the data capacity m of the database where the table is located and the number of data records n that the application is allowed to query in each batch;

[0012] Based on the formula n = m / kx, we obtain x;

[0013] The integer part of x is taken to obtain the number of bits i.

[0014] In one technical solution of the above data query method, the specified pattern consists of i primary key elements first and wildcard % last.

[0015] In one technical solution of the above data query method, loading the data retrieved in each loop into memory as a separate batch includes:

[0016] Retrieve all data corresponding to UUIDs that begin with the i-th primary key element in the current loop;

[0017] The corresponding data is loaded into memory.

[0018] In one technical solution of the above data query method, the specified pattern consists of i primary key elements followed by a wildcard % character.

[0019] In one technical solution of the above data query method, loading the data retrieved in each loop into memory as a separate batch includes:

[0020] Retrieve all data corresponding to the UUID that ends with the i-th primary key element corresponding to this loop;

[0021] The corresponding data is loaded into memory.

[0022] In one technical solution of the above data query method, the method further includes:

[0023] The data retrieved from different tables will be compared in batches.

[0024] In a second aspect, a data query device is provided, the device comprising:

[0025] The calculation module is configured to calculate the number of bits i of the primary key elements required to perform fuzzy matching of the primary key column in the table containing the data to be queried using the SQLlike operator, wherein the primary key corresponds one-to-one with the data to be queried, and the primary key is a string consisting of L primary key elements, with each primary key element selected from a group of k characters.

[0026] The building module is configured to construct an SQL statement based on the number of bits i: select * from tablename where uuid like 'specified pattern', where tablename represents the table name, * represents all columns of the table, uuid represents the primary key, and the specified pattern consists of i primary key elements and wildcards %.

[0027] The execution module is configured to iterate through all possible i-bit primary key elements, thereby executing k operations on the SQL statement. i This is the second For loop;

[0028] The loading module is configured to load the data retrieved in each loop as a separate batch into memory.

[0029] In a third aspect, an electronic device is provided, comprising a processor and a storage device, the storage device being adapted to store a plurality of program codes, the program codes being adapted to be loaded and executed by the processor to perform the data query method described in any of the above-described technical solutions.

[0030] In a fourth aspect, a computer-readable storage medium is provided, wherein a plurality of program codes are stored therein, the program codes being adapted to be loaded and run by a processor to perform the data query method described in any of the above-described technical solutions.

[0031] The present invention comprises one or more of the following technical solutions:

[0032] Beneficial effects:

[0033] In implementing the technical solution of this invention, the number of digits *i* of the primary key elements required for fuzzy matching of the primary key column in the table containing the data to be queried using the SQLlike operator is calculated. The primary key corresponds one-to-one with the data to be queried, and the primary key is a string composed of L primary key elements, each selected from a group of k characters. Based on the number of digits *i*, an SQL statement is constructed: `select * from tablename where uuid like 'specified pattern'`, where `tablename` represents the table name, `*` represents all columns of the table, `uuid` represents the primary key, and the specified pattern consists of i primary key elements and a wildcard %; all possible i-digit primary key elements are traversed, thereby executing k... i The For loop iterates through the data retrieved in each iteration, loading the data into memory as a separate batch. Compared to traditional data query methods, this approach reduces the steps of summarizing the total number of data entries, directly obtaining data in separate batches. Furthermore, it eliminates the need for an ordering process, thus reducing performance overhead and improving efficiency. Attached Figure Description

[0034] The disclosure of this invention will become more readily understood with reference to the accompanying drawings. It will be readily understood by those skilled in the art that these drawings are for illustrative purposes only and are not intended to limit the scope of protection of this invention. Furthermore, similar numbers in the drawings are used to denote similar components, wherein:

[0035] Figure 1 This is a schematic diagram of the main steps of a data query method according to an embodiment of the present invention;

[0036] Figure 2 This is a main structural block diagram of a data query device according to an embodiment of the present invention;

[0037] Figure 3 This is a schematic diagram of the main structure of an electronic device according to an embodiment of the present invention.

[0038] List of reference numerals in the attached diagram:

[0039] 201: Calculation module; 202: Construction module; 203: Execution module; 204: Loading module; 301: Processor; 302: Storage device. Detailed Implementation

[0040] Some embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0041] In the description of this invention, "module" and "processor" can include hardware, software, or a combination of both. A module can include hardware circuitry, various suitable sensors, communication ports, memory, and may also include software components, such as program code, or a combination of software and hardware. A processor can be a central processing unit, microprocessor, image processor, digital signal processor, or any other suitable processor. The processor has data and / or signal processing capabilities. The processor can be implemented in software, in hardware, or a combination of both. Non-transitory computer-readable storage media includes any suitable medium capable of storing program code, such as magnetic disks, hard disks, optical disks, flash memory, read-only memory, random access memory, etc. The term "A and / or B" means all possible combinations of A and B, such as only A, only B, or A and B. The terms "at least one A or B" or "at least one of A and B" have a similar meaning to "A and / or B" and can include only A, only B, or A and B. The singular terms "a" or "this" can also include plural forms.

[0042] Here we will first explain some of the terms involved in this invention.

[0043] SQL: Structured Query Language, is a database query and programming language used to access, query, update, and manage relational database systems; SQL statements are a language for manipulating databases.

[0044] SQLlike: refers to the like clause in SQL statements, used for fuzzy matching in the where clause. It compares a given matching pattern with a certain field, selecting the field if a match is found, otherwise not selecting it. The like clause can be used with wildcards.

[0045] Currently, traditional batch data query methods require querying the total number of records and paginating the data after sorting it. When the amount of data in the database is large, this approach is slow, affecting the user experience. To address the issue of slow performance in querying the total number of records and sorting the data when dealing with large volumes, this invention provides a data query method, apparatus, device, and medium.

[0046] See appendix Figure 1 , Figure 1 This is a schematic flowchart illustrating the main steps of a data query method according to an embodiment of the present invention. Figure 1 As shown, the data query method in this embodiment of the invention mainly includes the following steps S101 to S104.

[0047] Step S101: Calculate the number of bits i of the primary key element required to perform fuzzy matching on the primary key column of the table containing the data to be queried using the SQLlike operator.

[0048] The primary key corresponds one-to-one with the data to be queried. The primary key is a string consisting of L primary key elements, and each primary key element is selected from a group consisting of k characters.

[0049] Step S102: Construct an SQL statement based on the number of bits i: select * from tablename where uuid like 'specified pattern'.

[0050] Where tablename represents the table name, * represents all columns of the table, uuid represents the primary key, and the specified pattern consists of i primary key elements and wildcard %.

[0051] Step S103: Iterate through all possible i-bit primary key elements to execute the SQL statement k. i This is the second For loop.

[0052] Step S104: Load the data retrieved in each loop into memory as a separate batch of data.

[0053] The method described in steps S101 to S104 above, compared to traditional data query methods, reduces the step of summarizing the total number of data entries, allowing direct acquisition of individual batches of data and loading them into memory. Furthermore, it eliminates the need for an ordering process, thereby reducing performance overhead and improving work efficiency.

[0054] The following provides a further explanation of steps S101 to S104.

[0055] In one example of step S101 above, the primary key corresponds one-to-one with the data to be queried. The primary key is a string consisting of L primary key elements, and each primary key element is selected from a group consisting of k characters.

[0056] The number of digits L in the primary key is usually 32 or 64, and the k characters that make up the primary key element include numbers, letters, special characters, etc.

[0057] In a specific example, a 32-bit primary key can be used, consisting of 16 random combinations of letters and numbers, specifically the random combination of the alphanumeric characters {'0','1','2','3','4','5','6','7','8','9','a','b','c','d','e','f'}. This would result in a total of 16... 32 One of the primary keys is: 0123456789abcdef0123456789abcdef.

[0058] In another specific example, a 64-bit primary key can be used, which is a random combination of 18 letters and numbers, namely, a random combination of the letters and numbers {'a','b','c','d','e','f','g','h','0','1','2','3','4','5','6','7','8','9'}. This results in a total of 18... 64 One of the primary keys is: abcdefgh0123456789abcdefgh0123456789abcdefgh0123456789abcdefgh01.

[0059] The examples above regarding the number of digits L in the primary key and the k characters that make up the primary key element are merely illustrative. Those skilled in the art can set them according to specific scenarios in actual applications, and no limitations are imposed here.

[0060] Further, the calculation of the number of bits i of the primary key element required to perform fuzzy matching on the primary key column of the table containing the data to be queried using the SQLlike operator includes the following steps S1011 to S1013.

[0061] Step S1011: Obtain the data capacity m of the database where the table is located and the number of data rows n that the application is allowed to query in each batch.

[0062] The values ​​of m and n can be set according to the specific scenario and are not limited here. For example, if the database containing the table has a data capacity of m of 20 million records, and the application performing the data query is allowed to query n records per batch of 5000 records, it should be noted that if the number of records allowed to be queried per batch exceeds this preset value, the application will crash and become unusable.

[0063] Step S1012: Based on the formula n = m / kx, obtain x;

[0064] Step S1013: Round x to obtain the number of digits i.

[0065] The value of x calculated using the formula n = m / kx may be an irrational number or a decimal. In this case, x can be rounded to the nearest integer to obtain the number of digits i. For example, when x = 1.26, i is 1; when x = 8 / 3, i is 3.

[0066] The above is a further explanation of step S101. The following is a further explanation of step S102.

[0067] In an example of step S102 above, in the constructed SQL statement `select * from tablename where uuid like 'specified pattern'`, `tablename` represents the table name; `*` represents all columns of the table, i.e., all fields in the table, with each row of data being a row in the table; `uuid` represents the primary key; `like 'specified pattern'` represents a fuzzy search; where the specified pattern consists of i primary key elements and the wildcard % character.

[0068] In a specific example, the specified pattern can be composed of i primary key elements first and wildcard % last. For example, select * from tablename where uuid like '0%' means to query the data corresponding to uuids that start with 0 in the tablename.

[0069] In another specific example, the specified pattern can be composed of i primary key elements followed by the wildcard % before, for example, select * from tablename where uuid like '%12', which means to query the table tablename for data corresponding to uuids ending with 12.

[0070] Furthermore, data can be retrieved from the table containing the data to be queried based on the constructed SQL statement. In a specific example, the table containing the data to be queried, whose tablename is "bill," can be shown in Table 1 below.

[0071] Table 1

[0072] id username uuid Amount ... 1 Adams 0123456789abcdef0123456789abcdef 1000 ... 2 Bush 1fcdef0123456789abcdef0123456789 800 ... 3 Carter 0f1301126dc042ae5bd03189697fedc0 2020 ... 4 John 987654321abcdef0123456789abcdeff 600 ... 5 George ca75f6cd63875269defac1b926cae800 3000 ...

[0073] In the table above, username is the username, uuid is the 32-bit primary key, and each piece of data corresponds to a unique uuid.

[0074] The above is a further explanation of step S102. The following is a further explanation of step S103.

[0075] In one example of step S103 above, when the number of bits in the primary key element is 1, k is executed. 1 The For loop iterates through the remaining nodes; when the number of digits in the primary key element is 2, it executes step k. 2 The For loop iterates through the number of positions in the primary key element until the number of positions in the primary key is w, then executes k. w This is the second For loop.

[0076] In a specific example, if the primary key element has 1 bit length, using a 32-bit primary key, and the primary key consists of 16 random combinations of letters and numbers, then the SQL statement will execute 16... 1 The For loop is repeated several times, and the specific loop is shown below:

[0077] 1st time: select*fromtablenamewhereuuidlike'0%'

[0078] 2nd time: select*fromtablenamewhereuuidlike'1%' ...

[0080] Time 16: select*fromtablenamewhereuuidlike'f%'

[0081] Here, like'0%' means to query the tablename for data corresponding to UUIDs that start with 0, like'1% means to query the tablename for data corresponding to UUIDs that start with 1, and so on. The data retrieved in each For loop is the data for each batch (each page).

[0082] In another specific example, if the primary key element has 2 bits, using a 64-bit primary key, and the primary key is a random combination of 16 letters and numbers, then the SQL statement will execute 16... 2 The For loop is repeated several times, and the specific loop is shown below:

[0083] 1st time: select*fromtablenamewhereuuidlike'00%'

[0084] 2nd time: select*fromtablenamewhereuuidlike'01%' ...

[0086] 16th time: select*fromtablenamewhereuuidlike'0f%'

[0087] 17th time: select*fromtablenamewhereuuidlike'10%' ...

[0089] Time 256: select*fromtablenamewhereuuidlike'ff%'

[0090] Here, like'00%' means to specifically loop through and query the data corresponding to UUIDs that start with 00, like'ff% means to specifically loop through and query the data corresponding to UUIDs that start with ff, and so on. The data retrieved in each For loop is the data for each batch (each page).

[0091] It should be noted that the query order in the above example is only illustrative. In actual applications, once the number of digits of the primary key element is determined, the query order is uncertain. Those skilled in the art can set it according to the specific scenario, and no limitation is made here.

[0092] The above is a further explanation of step S103. The following is a further explanation of step S104.

[0093] In one example of step S104 above, if the specified pattern consists of an i-bit primary key element followed by a wildcard, the data corresponding to all UUIDs that begin with the i-bit primary key element of the current loop can be retrieved first.

[0094] For example, if the specified pattern consists of a 1-character primary key element followed by a wildcard % and the primary key is a random combination of 16 letters and numbers, then the data in Table 1 above will be processed using a 16-character wildcard. 1 The first iteration of the For loop: select * from tablename where uuid like '0%', will retrieve the data corresponding to the uuids starting with 0 in Table 1. The query results are shown in Table 2 below.

[0095] Table 2

[0096] id username uuid Amount ... 1 Adams 0123456789abcdef0123456789abcdef 1000 ... 3 Carter 0f1301126dc042ae5bd03189697fedc0 2020 ...

[0097] Continue processing the data in Table 1 for step 16. 1The second iteration of the For loop: select * from tablename where uuid like '1%', will retrieve the data corresponding to the uuids starting with 1 in Table 1. The query results are shown in Table 3 below.

[0098] Table 3

[0099] id username uuid Amount ... 2 Bush 1fcdef0123456789abcdef0123456789 800 ...

[0100] Next, the corresponding data is loaded into memory.

[0101] For example, the first For loop query retrieves data from Table 1 containing UUIDs starting with 0, forming Table 2. This Table 2 contains the first batch (first page) of retrieved data. The second For loop query retrieves data from Table 1 containing UUIDs starting with 1, forming Table 3. This Table 3 contains the second batch (second page) of retrieved data. Each batch of retrieved data is loaded into memory; for example, the data from Tables 2 and 3 are loaded into memory.

[0102] In another example of step S104 above, if the specified pattern consists of an i-bit primary key element followed by a wildcard, the data corresponding to all UUIDs ending with the i-bit primary key element corresponding to this loop can be retrieved first. 。

[0103] For example, if the specified pattern consists of a 2-character primary key followed by a wildcard % before the primary key, and the primary key is a random combination of 16 letters and numbers, then the 16-character pattern will be executed for the data in Table 1 above. 2 The first iteration of the For loop: select * from tablename where uuid like '%00', will retrieve the data corresponding to the uuids ending with 00 in Table 1. The query results are shown in Table 4 below.

[0104] Table 4

[0105] id username uuid Amount ... 5 George ca75f6cd63875269defac1b926cae800 3000 ...

[0106] Continue processing the data in Table 1 for step 16. 2 If we use the For loop 256 times, then the 256th iteration will use the query: select * from tablename where uuid like '%ff' to retrieve the data corresponding to the uuids ending with 'ff' in Table 1. The query results are shown in Table 5 below.

[0107] Table 5

[0108] id username uuid Amount ... 4 John 987654321abcdef0123456789abcdeff 600 ...

[0109] Next, the corresponding data is loaded into memory.

[0110] For example, the first For loop retrieves data corresponding to UUIDs ending in 00, forming Table 4. Table 4 contains the first batch (first page) of retrieved data. The 256th For loop retrieves data from Table 1, generating Table 5 containing UUIDs ending in ff. Table 5 contains the last batch (last page) of retrieved data. Each batch of retrieved data is loaded into memory; for example, the data from Tables 4 and 5 are loaded into memory.

[0111] The above is a further explanation of step S104.

[0112] In one example, after performing step S104 above, the data query method of the present invention further includes:

[0113] The data retrieved from different tables will be compared in batches.

[0114] For example, the query method provided by this invention can be used to query data from a table in the source database (e.g., the positive transaction database) and the corresponding table in the target database (e.g., the negative transaction database, which is synchronized with the tables in the positive transaction database), and store the data in memory in batches for batch comparison, analysis and statistics.

[0115] It should be noted that although the steps in the above embodiments are described in a specific order, those skilled in the art will understand that in order to achieve the effects of the present invention, different steps do not necessarily have to be executed in such an order. They can be executed simultaneously (in parallel) or in other orders, and these variations are all within the scope of protection of the present invention.

[0116] Furthermore, the present invention also provides a data query device.

[0117] See appendix Figure 2 , Figure 2 This is a main structural block diagram of a data query device according to an embodiment of the present invention. Figure 2As shown, the data query device in this embodiment of the invention mainly includes a calculation module 201, a construction module 202, an execution module 203, and a loading module 204. In one embodiment, the calculation module 201 can be configured to calculate the number of digits *i* of the primary key elements required for fuzzy matching of the primary key column in the table containing the data to be queried using the SQLlike operator. Here, the primary key corresponds one-to-one with the data to be queried, and the primary key is a string composed of L primary key elements, with each primary key element selected from a group of k characters. The construction module 202 can be configured to construct an SQL statement based on the number of digits *i: select * from tablename where uuid like 'specified pattern', where tablename represents the table name, * represents all columns of the table, uuid represents the primary key, and the specified pattern consists of i primary key elements and a wildcard % character. The execution module 203 can be configured to traverse all possible i-digit primary key elements, thereby executing k... i The For loop iterates through the data retrieved in each iteration. Loading module 204 can be configured to load the data retrieved in each iteration into memory as a separate batch. In one embodiment, a description of the specific functionality can be found in steps S101 to S104.

[0118] Furthermore, it should be understood that since the various modules are only provided to illustrate the functional units of the device of the present invention, the physical devices corresponding to these modules may be the processor itself, or a part of the processor's software, hardware, or a combination of software and hardware. Therefore, the number of modules shown in the figures is merely illustrative.

[0119] Those skilled in the art will understand that the various modules in the device can be adaptively split or combined. Such splitting or combining of specific modules will not cause the technical solution to deviate from the principles of the present invention; therefore, the technical solutions after splitting or combining will fall within the protection scope of the present invention.

[0120] The aforementioned data query device is used for execution Figure 1 The data query method embodiments shown are similar in technical principle, the technical problems they solve, and the technical effects they produce. Those skilled in the art can clearly understand that, for the sake of convenience and brevity, the specific working process and related descriptions of the data query device can be found in the embodiments of the data query method, and will not be repeated here.

[0121] Those skilled in the art will understand that all or part of the processes in the method of the above embodiment of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer-readable storage medium can include any entity or device capable of carrying the computer program code, a medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory, a random access memory, an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0122] Furthermore, the present invention also provides an electronic device. (See appendix.) Figure 3 , Figure 3 This is a schematic diagram of the main structure of an electronic device according to an embodiment of the present invention. Figure 3 As shown, the electronic device in this embodiment of the invention mainly includes a processor 301 and a storage device 302. The storage device 302 can be configured to store a program for executing the data query method of the above-described method embodiments. The processor 301 can be configured to execute the program in the storage device 302, which includes, but is not limited to, a program for executing the data query method of the above-described method embodiments. For ease of explanation, only the parts related to the embodiments of the present invention are shown. For specific technical details not disclosed, please refer to the method section of the embodiments of the present invention.

[0123] In some possible embodiments of the present invention, the electronic device may include multiple processors 301 and multiple storage devices 302. The program executing the data query method of the above method embodiments can be divided into multiple subroutines, each of which can be loaded and run by a processor 301 to execute different steps of the data query method of the above method embodiments. Specifically, each subroutine can be stored in different storage devices 302, and each processor 301 can be configured to execute programs in one or more storage devices 302 to jointly implement the data query method of the above method embodiments; that is, each processor 301 executes different steps of the data query method of the above method embodiments to jointly implement the data query method of the above method embodiments.

[0124] The aforementioned multiple processors 301 can be processors deployed on the same device. For example, the aforementioned electronic device can be a high-performance device composed of multiple processors, and the aforementioned multiple processors 301 can be processors configured on that high-performance device. Alternatively, the aforementioned multiple processors 301 can also be processors deployed on different devices. For example, the aforementioned electronic device can be a server cluster, and the aforementioned multiple processors 301 can be processors on different servers within the server cluster.

[0125] Furthermore, the present invention also provides a computer-readable storage medium. In one embodiment of the computer-readable storage medium according to the present invention, the computer-readable storage medium can be configured to store a program for executing the data query method of the above-described method embodiments. This program can be loaded and run by a processor to implement the above-described data query method. For ease of explanation, only the parts related to the embodiments of the present invention are shown; for specific technical details not disclosed, please refer to the method section of the embodiments of the present invention. The computer-readable storage medium can be a storage device device comprising various electronic devices. Optionally, in the embodiments of the present invention, the computer-readable storage medium is a non-transitory computer-readable storage medium.

[0126] The technical solution of the present invention has been described above with reference to one embodiment shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions resulting from such changes or substitutions will all fall within the scope of protection of the present invention.

Claims

1. A data query method, characterized in that, The method includes: Calculating the number of digits *i* of the primary key element required for fuzzy matching of the primary key column in the table containing the data to be queried using the SQL LIKE operator includes: obtaining the data capacity *m* of the database containing the table and the number of data rows *n* allowed to be queried per batch by the application performing the data query; obtaining an intermediate quantity *x* based on the data capacity *m* of the database containing the table and the number of data rows allowed to be queried per batch by the application performing the data query; and obtaining the number of digits *i* based on *x*; wherein the primary key corresponds one-to-one with the data to be queried, and the primary key is a string consisting of L primary key elements, each primary key element being selected from a group consisting of k characters; Construct an SQL statement based on the specified number of bits i: select * from tablename where uuid like 'specified pattern', where tablename represents the table name, * represents all columns of the table, uuid represents the primary key, and the specified pattern consists of i primary key elements and wildcard %; Iterate through all possible i-bit primary key elements to execute k operations on the SQL statement. i This is the second For loop; The data retrieved in each loop is loaded into memory as a separate batch.

2. The data query method according to claim 1, characterized in that, The intermediate quantity x is obtained based on the data capacity m of the database where the table is located and the number of data rows n allowed to be queried in each batch by the application for data querying, including: based on the formula We get x; Obtaining the number of bits i based on x includes: rounding the x to obtain the number of bits i.

3. The data query method according to claim 1 or 2, characterized in that, The specified pattern consists of i primary key elements first, followed by the wildcard % character.

4. The data query method according to claim 3, characterized in that, The step of loading the data retrieved in each loop into memory as a separate batch includes: Retrieve all data corresponding to UUIDs that begin with the i-th primary key element in the current loop; The corresponding data is loaded into memory.

5. The data query method according to claim 1 or 2, characterized in that, The specified pattern consists of i primary key elements followed by a wildcard % character.

6. The data query method according to claim 5, characterized in that, The step of loading the data retrieved in each loop into memory as a separate batch includes: Retrieve all data corresponding to the UUID that ends with the i-th primary key element corresponding to this loop; The corresponding data is loaded into memory.

7. The data query method according to any one of claims 1, 4, and 6, characterized in that, The method further includes: The data retrieved from different tables will be compared in batches.

8. A data query device, characterized in that, The device includes: The calculation module is configured to calculate the number of bits *i* of the primary key element required for fuzzy matching of the primary key column in the table containing the data to be queried using the SQL LIKE operator. This includes: obtaining the data capacity *m* of the database containing the table and the number of data rows *n* allowed to be queried per batch by the application performing the data query; obtaining an intermediate quantity *x* based on the data capacity *m* of the database containing the table and the number of data rows *n* allowed to be queried per batch by the application performing the data query; and obtaining the number of bits *i* based on *x*. The primary key corresponds one-to-one with the data to be queried, and the primary key is a string consisting of L primary key elements, each selected from a group of k characters. The building module is configured to construct an SQL statement based on the number of bits i: select * from tablename where uuid like 'specified pattern', where tablename represents the table name, * represents all columns of the table, uuid represents the primary key, and the specified pattern consists of i primary key elements and wildcards %. The execution module is configured to iterate through all possible i-bit primary key elements, thereby executing k operations on the SQL statement. i This is the second For loop; The loading module is configured to load the data retrieved in each loop as a separate batch into memory.

9. An electronic device comprising a processor and a storage device, said storage device being adapted to store a plurality of program codes, characterized in that, The program code is adapted to be loaded and run by the processor to perform the data query method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a plurality of program codes, characterized in that, The program code is adapted to be loaded and run by a processor to perform the data query method according to any one of claims 1 to 7.

Images