A data query method and related device
By using two-dimensional arrays to store data in the in-memory database and positioning the read data based on the upper and/or lower bounds of the query scope, the problem of inefficiency in the range query in the in-memory database is solved, and more efficient data query is achieved.
Patent Information
- Application Number
- CN202210327116.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-03-30
- Publication Date
- 2025-06-06
- Estimated Expiration
- 2042-03-30
AI Technical Summary
Key-value data structures commonly used in in-memory databases are inefficient for range query, and all data must be loaded into memory for sorting to find a result set that meets range criteria.
By obtaining the upper and/or lower bounds of the query range, the data within the query range is positioned and read in the data stored in the two-dimensional array, avoiding loading all data into memory for sorting.
Improves the efficiency of range query and saves time to load data and sort.
Smart Images

Figure CN114625785B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of databases, and in particular to a data query method and related devices. Background Art
[0002] In-memory databases have been increasingly used in the current era of big data due to their faster read and write speeds, which can greatly improve data search efficiency and the overall performance of application systems. However, compared to traditional disk databases, the key-value (KV) data structure commonly used in in-memory databases cannot support range queries well. All data must be loaded into the system's running memory and sorted according to a certain field to find the result set that meets the range condition, which makes range queries inefficient.
[0003] Therefore, a data query method is desired to improve the efficiency of range query. Summary of the invention
[0004] The present application provides a data query method and related devices.
[0005] In a first aspect, the present application provides a data query method, the method comprising: obtaining a query range, the query range including an upper bound and / or a lower bound of the data to be queried; based on the query range, determining the position of the upper bound and / or the position of the lower bound in a database; wherein the data in the database is stored in the form of a two-dimensional array and satisfies: the data of the first dimension stored in the two-dimensional array increases along a first direction, and the data of the second dimension stored in the two-dimensional array increases along a second direction; based on the position of the upper bound and / or the position of the lower bound, reading the data within the query range in the database.
[0006] Based on the above method, the upper and / or lower bounds of the query range are first located, and then the data that meets the query range is read based on the positions of the upper and / or lower bounds. In this way, it is no longer necessary to add all the data in the database to the memory for sorting before searching, which saves the time of loading and sorting data and improves the efficiency of range query.
[0007] In combination with the first aspect, in some possible implementations of the first aspect, the length of the first dimension is the same as the length of the second dimension.
[0008] In combination with the first aspect, in some possible implementations of the first aspect, the serial numbers used to identify the data of the first dimension increase along the first direction, and the serial numbers used to identify the data of the second dimension increase along the second direction.
[0009] In combination with the first aspect, in some possible implementations of the first aspect, the method also includes: determining the length N based on the number of data M to be stored in the database; storing the M data in an N×N two-dimensional array; wherein the length N and the number of data M satisfy: (N-1)2<M≤N2, and M and N are positive integers.
[0010] In combination with the first aspect, in some possible implementations of the first aspect, the serial numbers used to identify the data of the first dimension include p to p+N-1, the serial numbers used to identify the data of the second dimension include q to q+N-1, and the arrangement of the M data satisfies: the position of the mth data in the M data in the two-dimensional array is (i m , j m ), the position of the m+1th data in the two-dimensional array is (i m+1 , j m+1 ), the m+1th data is greater than the mth data; when j m =q+N-1, i m+1 =p+N-1,j m+1 =q-p+i m +1; when j m ≠q+N-1andi m = p, i m+1 =p-q+j m +1,j m+1 =q; when j m ≠q+N-1andi m ≠p, i m+1 =i m -1,j m+1 =j m +1.
[0011] In combination with the first aspect, in some possible implementations of the first aspect, the serial numbers used to identify the data of the first dimension include p to p+N, the serial numbers used to identify the data of the second dimension include q to q+N, and the arrangement of the M data satisfies: the position of the mth data in the M data is (i m , j m ), the position of the m+1th data in the M data is (i m+1 , j m+1 ), the m+1th data is greater than the mth data; when j m =q+N-1, i m+1 =i m +1,j m+1 =q; when j m When ≠q+N-1, i m+1 =i m , j m+1 =j m +1.
[0012] In combination with the first aspect, in some possible implementations of the first aspect, the serial numbers used to identify the data of the first dimension include p to p+N, the serial numbers used to identify the data of the second dimension include q to q+N, and the arrangement of the M data satisfies: the position of the mth data in the M data is (i m , j m ), the position of the m+1th data in the M data is (i m+1 , j m+1 ), the m+1th data is greater than the mth data; when i m =p+N-1, i m+1 =p,j m+1 =j m +1; when i m When ≠p+N-1, i m+1 =i m +1,j m+1 =j m .
[0013] In combination with the first aspect, in some possible implementations of the first aspect, based on the query range, the position of the upper bound and / or the position of the lower bound is determined in the database, including: taking the upper bound and / or the lower bound as target data, respectively, and determining the position of the target data through the following steps: taking a predefined position as a reference position; adjusting the reference position based on the size relationship between the data at the reference position and the target data until the data at the reference position is equal to the target data; and determining the position of the reference position as the position of the target data.
[0014] In combination with the first aspect, in some possible implementations of the first aspect, the predefined position is (i f =p+N-1,j f =q); based on the size relationship between the data at the reference position and the target data, adjusting the reference position, including: if the data at the reference position is larger than the target data, adjusting the reference position to (i f -1,j f ); if the data at the reference position is smaller than the target data, adjust the reference position to (i f , j f +1).
[0015] In combination with the first aspect, in some possible implementations of the first aspect, based on the position of the upper bound and / or the position of the lower bound, data that meets the query range is read in the database, including: reading the data in sequence within the range determined by the position of the upper bound and / or the position of the lower bound in the two-dimensional array according to the arrangement of M data; the read data is the data that meets the query range.
[0016] In a second aspect, the present application provides a data query device, which includes an acquisition module and a processing module; the acquisition module is used to acquire a query range, the query range includes the upper bound and / or lower bound of the data to be queried; the processing module is used to determine the position of the upper bound and / or the position of the lower bound in the database based on the query range; wherein the data in the database is stored in the form of a two-dimensional array and satisfies: the data of the first dimension stored in the two-dimensional array increases along a first direction, and the data of the second dimension stored in the two-dimensional array increases along a second direction; and is used to read data that satisfies the query range in the database based on the position of the upper bound and / or the position of the lower bound.
[0017] In a third aspect, the present application provides a data query device, comprising a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to call and execute the computer program, so that the device executes the first aspect and any possible implementation method of the first aspect.
[0018] In a fourth aspect, the present application provides a computer-readable storage medium, comprising a computer program, which, when executed on a computer, enables the computer to execute the method in the first aspect and any possible implementation of the first aspect.
[0019] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed on a computer, enables the computer to execute the method in the first aspect and any possible implementation manner of the first aspect.
[0020] It should be understood that the second to fifth aspects of the present application correspond to the technical solutions of the first aspect of the present application, and the beneficial effects achieved by each aspect and the corresponding feasible implementation methods are similar and will not be repeated here.
[0021] It should be noted that the data query method and related devices provided in the present application can be used for data query applications in the financial field, such as querying financial data in financial services, and can also be used for data query applications in any field other than the financial field. The present application does not limit the application field of the data query method and related devices. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] Figure 1 It is a schematic diagram of a scenario applicable to the data query method provided in the embodiment of the present application;
[0023] Figure 2 is a schematic flow chart of a data storage method provided in an embodiment of the present application;
[0024] Figure 3 is a schematic diagram of a two-dimensional array provided in an embodiment of the present application;
[0025] Figures 4 to 6 It is a schematic diagram of the arrangement of data provided in the embodiments of the present application;
[0026] Figure 7 is a schematic flow chart of a data query method provided in an embodiment of the present application;
[0027] Figure 8 is a schematic diagram of determining the location of target data provided by an embodiment of the present application;
[0028] Fig. 9 is a schematic diagram of reading data provided by an embodiment of the present application;
[0029] Fig.10 is a schematic diagram of a method for adding a data structure to a database provided in an embodiment of the present application;
[0030] Fig.11 is a schematic block diagram of a data query device provided in an embodiment of the present application;
[0031] Fig.12 It is another schematic block diagram of the data query device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0032] To make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be described clearly and completely in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of them.
[0033] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.
[0034] In the embodiments of the present application, the terms "first", "second", etc. are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. In addition, the term "includes" and any variations thereof are intended to cover non-exclusive inclusions.
[0035] Figure 1 Schematic diagram of a scenario applicable to the data query method provided in the embodiment of the present application. Figure 1 As shown, the scenario may include a server 110 and a database 120 connected thereto, and the server 110 may read data or query data from the database 120 .
[0036] Compared with traditional disk databases, in-memory databases have faster read and write speeds, which can greatly improve data search efficiency and the overall performance of application systems. Therefore, they have been increasingly widely used in the current era of big data. The KV data structure commonly used in in-memory databases cannot support range queries well. All data must be loaded into the system's running memory and sorted according to a certain field to find the result set that meets the range condition. The range query efficiency is low.
[0037] For example, the KV data structure is used to record the user's name and height data as shown in Table 1:
[0038] Table 1
[0039] Name open* plum* king* Zhao* height 180 170 165 175
[0040] If you need to query a user's height, you can directly enter the user's name to get the user's height; but if you need to query users with a height of 170 cm or above, you need to retrieve the data of all users from the database and then sort them according to the height field to get the results. This range query is relatively inefficient.
[0041] In view of this, the present application provides a data query method. According to the upper bound and / or lower bound of the query range, the data within the query range is read from the data stored in the form of a two-dimensional array. Since the data is stored in the two-dimensional array according to a predefined rule, the data stored in the first dimension increases along the first direction, and the data stored in the second dimension increases along the second direction. Therefore, the range query process can also follow this rule, first locate the upper bound and / or lower bound of the query range, and then read the data that meets the query range based on the position of the upper bound and / or lower bound. In this way, there is no need to add all the data in the database to the memory for sorting and then searching, which saves the time of loading data and sorting, and improves the efficiency of range query.
[0042] The method provided in the embodiments of the present application will be described in detail below with reference to the accompanying drawings.
[0043] In order to facilitate understanding of the data query method provided in the embodiment of the present application, the following first combines Figures 2 to 6 Describe the data storage process in detail, and then combine Figures 7 to 9 Provide a detailed description of the data query process.
[0044] Figure 2 is a schematic flow chart of a data storage method 200 provided in an embodiment of the present application. The method 200 may be executed by a data storage device, which may be, for example, Figure 1The database 120 in the embodiment of the present invention may be a component configured in the database 120, or a functional module or software for implementing part or all of the database functions. This application does not limit this.
[0045] Figure 2 The method shown includes the following steps:
[0046] Step 210, determining the dimension of the two-dimensional array according to the number M of data to be stored in the database;
[0047] Step 220, store M data in the two-dimensional array.
[0048] Each step in the method 200 is described in detail below.
[0049] In an embodiment of the present application, the data in the database can be stored in the form of a two-dimensional array and satisfy: the data of the first dimension stored in the two-dimensional array increases along the first direction, and the data of the second dimension stored in the two-dimensional array increases along the second direction.
[0050] The length of the first dimension and the length of the second dimension of the two-dimensional array may be the same or different, and this application does not limit this.
[0051] One possible design is that the length of the first dimension is the same as the length of the second dimension, that is, the number of data that can be stored in the first dimension of the two-dimensional array is the same as the number of data that can be stored in the second dimension.
[0052] Step 210 may specifically include: determining the length N according to the number M of data to be stored in the database.
[0053] It should be understood that the length of the first dimension and the length of the second dimension are N, that is, the two-dimensional array is an N×N array.
[0054] In order for the two-dimensional array to accommodate all the data, N×N should be greater than or equal to M; in order to make the size of the two-dimensional array not redundant, (N-1)×(N-1) should be less than M. That is, (N-1) 2 <M≤N 2 .
[0055] For example, if M = 25, then condition (5-1) is met if and only if N = 5 2 <25≤5 2 , then the two-dimensional array is a 5×5 two-dimensional array. There are 25 locations in a 5×5 two-dimensional array, which can hold 25 data.
[0056] Another example, if M = 26, then condition (6-1) is met if and only if N = 6 2 <26≤62 , then the two-dimensional array is a 6×6 two-dimensional array. There are 36 positions in the 6×6 two-dimensional array, which can accommodate 26 data, and the remaining 10 positions can be occupied by null values or other predefined values, which are not limited in this application.
[0057] Optionally, step 220 may specifically include: storing the M data in an N×N two-dimensional array.
[0058] As mentioned above, the storage of data in a two-dimensional array follows the rule that data increases in a first direction in the first dimension and increases in a second direction in the second dimension. The following will explain in detail how data is arranged in a two-dimensional array based on this rule.
[0059] For ease of explanation, the following description uses a two-dimensional matrix as a representation of a two-dimensional array. For example, the first dimension of the two-dimensional array is the rows of the matrix, and the second dimension is the columns of the matrix. The corresponding first direction can be from left to right or from right to left, and the second direction can be from top to bottom or from bottom to top.
[0060] For another example, the first dimension of the two-dimensional array is the columns of the matrix, and the second dimension is the rows of the matrix. The corresponding first direction can be from top to bottom or from bottom to top, and the second direction can be from left to right or from right to left.
[0061] Optionally, for a two-dimensional array, the serial numbers used to identify the data of the first dimension increase along the first direction, such as p to p+N-1; the serial numbers used to identify the data of the second dimension increase along the second direction, such as q to q+N-1, such as Figure 3 shown.
[0062] The data stored in the two-dimensional array meets the following conditions: the data itself has a size relationship, then the data stored in the first dimension of the two-dimensional array increases along the first direction, and the data stored in the second dimension of the two-dimensional array increases along the second direction, such as Figure 3 As shown in (a) in ; or, the data itself has no size relationship, but has a sequential order, such as English letters, then the data stored in the first dimension of the two-dimensional array is in the positive order along the first direction, and the data stored in the second dimension of the two-dimensional array is in the positive order along the second direction, as shown in Figure 3 As shown in (b) in .
[0063] To store M data in a two-dimensional array to satisfy the above conditions, there are many ways to arrange the M data. Three arrangements are listed below, where M=25, N=5, and p=q=0. These are examples only and are not intended to be limiting.
[0064] The first arrangement:
[0065] like Figure 4 As shown: Among the 25 data, the position of the first data is (0, 0), and the position of the mth data is (i m , j m ), the position of the m+1th data is (i m+1 , j m+1 ).
[0066] Among them, the position of the m+1th data (i m+1 , j m+1 ) and the position of the mth data (i m , j m ) satisfy the relationship between:
[0067] When m =4, i m+1 =4,j m+1 =i m +1;
[0068] When m ≠4andi m = 0, i m+1 =j m +1,j m+1 =0;
[0069] When m ≠4andi m ≠0, i m+1 =i m -1,j m+1 =j m +1.
[0070] For example, the position of the 15th data is (i 15 =0,j 15 =4), satisfying the condition j m =4, then i 16 =4,j 16 =i 15 +1=1, so the position of the 16th data is (i 16 =4,j 16 =1).
[0071] For example, the position of the 10th data is (i 10 =0,j 10 =3), satisfying the condition j m ≠4andi m =0, then i 11 =j 10 +1=4,j 11 =0, so the position of the 11th data is (i 11 =4,j 11 =0).
[0072] For example, the position of the 11th data is (i 11 =4,j 11 =0), satisfying the condition j m ≠4andi m ≠0, then i 12 =i 11 -1=3,j 12 =j 11 +1=1, so the position of the 12th data is (i 12 =3,j 12 =1).
[0073] The second arrangement:
[0074] like Figure 5 As shown: Among the 25 data, the position of the first data is (0, 0), and the position of the mth data is (i m , j m ), the position of the m+1th data is (i m+1 , j m+1 ).
[0075] Among them, the position of the m+1th data (i m+1 , j m+1 ) and the position of the mth data (i m , j m ) satisfy the relationship between:
[0076] When m =4, i m+1 =i m +1,j m+1 =0;
[0077] When m ≠4, i m+1 =i m , j m+1 =j m +1.
[0078] For example, the position of the 10th data is (i 10 =1,j 10 =4), satisfying the condition j m =4, then i 11 =i 10 +1=2,j 11 =0, so the position of the 11th data is (i 11 =2,j 11 =0).
[0079] For example, the position of the 11th data is (i 11 =2,j 11=0), satisfying the condition j m ≠4, then i 12 =i 11 =2,j 12 =j 11 +1=1, so the position of the 12th data is (i 12 =2,j 12 =1).
[0080] The third arrangement:
[0081] like Figure 6 As shown: Among the 25 data, the position of the first data is (0, 0), and the position of the mth data is (i m , j m ), the position of the m+1th data is (i m+1 , j m+1 ).
[0082] Among them, the position of the m+1th data (i m+1 , j m+1 ) and the position of the mth data (i m , j m ) satisfy the relationship between:
[0083] when i m =4, i m+1 =0,j m+1 =j m +1;
[0084] when i m ≠4, i m+1 =i m +1,j m+1 =j m .
[0085] For example, the position of the 10th data is (i 10 =4,j 10 =1), satisfying condition i m =4, then i 11 =0,j 11 =j 10 +1=2, so the position of the 11th data is (i 11 =0,j 11 =2).
[0086] For example, the position of the 11th data is (i 11 =0,j 11 =2), satisfying condition i m ≠4, then i 12 =i 11 +1=1,j 12 =j 11=2, so the position of the 12th data is (i 12 =1,j 12 =2).
[0087] Based on the several possible arrangements provided above, the M data can be stored in the database according to any one of the arrangements.
[0088] It should be understood that the three arrangements provided above are only three possible designs that meet the above rules. Based on the above rules, technicians in this field can also perform simple transformations to obtain more possible arrangements. For the sake of brevity, they are not enumerated here.
[0089] It should be understood that after the M data are stored in the database, the database may also provide an interface for modification or deletion. That is, more data may be added and stored in the database, or data may be deleted. Since the number of stored data may change, the two-dimensional array used to store data may also change.
[0090] The steps to add more data to the database are as follows: Calculate whether the number of data after adding data M still meets (N-1) 2 <M≤N 2 This condition. If (N-1) is still met 2 <M≤N 2 If this condition is met, first find the position where the data is added, then move the data after this position one position backward along the order of data arrangement, and then put the added data into the position where the data is added; if it does not meet (N-1) 2 <M≤N 2 This condition, that is, M is greater than N 2 , then we need to create a new two-dimensional array of size (N+1)×(N+1), and then add all the data after the new data is added to the new two-dimensional array in the order in which the data is arranged.
[0091] Similarly, the steps to delete data in the database are as follows: Calculate whether the number of data after deleting the data, M, still meets (N-1) 2 <M≤N 2 This condition. If (N-1) is still met 2 <M≤N 2 If this condition is met, first find the location where the data is to be deleted, delete the data, and then move the data after this location forward one position along the order of data arrangement; if it does not meet (N-1) 2 <M≤N 2 This condition, that is, M is less than or equal to (N-1) 2, then we need to create a new two-dimensional array of size (N-1)×(N-1), and then add all the data after deleting the data into the new two-dimensional array in the order in which the data is arranged.
[0092] After completing the data storage, the database can provide query services for users. Figures 7 to 9 The data query method provided in the embodiment of the present application is described in detail.
[0093] Figure 7 is a schematic flow chart of a data query method provided in an embodiment of the present application. The method 700 may include steps 710 to 730, which may be executed by a data query device. The data query device may be, for example, Figure 1 The server 110 shown may be a component configured in the database 120, or may be a functional module or software for implementing some or all of the database functions. This application does not limit this. It should be understood that the data query device and the data storage device described above may be the same device or different devices, and this application does not limit this.
[0094] Each step in method 700 is described in detail below.
[0095] In step 710, a query range is obtained, where the query range includes an upper bound and / or a lower bound of data to be queried.
[0096] The data query method provided by the present application can be used to implement range query, that is, query data within a range. Therefore, the query range can be a closed range with an upper bound and a lower bound, or an open range with only an upper bound or only a lower bound. The query range can be less than or equal to the upper bound and / or greater than or equal to the lower bound, or less than the upper bound and / or greater than the lower bound. In the embodiment of the present application, the query range is taken as an example of being less than or equal to the upper bound and / or greater than or equal to the lower bound.
[0097] In one example, the data includes integers from 1 to 25, and the data to be queried is integers from 9 to 17, then the query range includes the upper bound 17 and the lower bound 9 of the data to be queried.
[0098] For another example, the data includes integers from 1 to 25, and the data to be queried is an integer less than or equal to 17, then the query range includes the upper limit 17 of the data to be queried.
[0099] In another example, the data includes integers from 1 to 25, and the data to be queried is an integer greater than or equal to 9, then the query range includes the lower bound 9 of the data to be queried.
[0100] In step 720, based on the query range, the position of the upper bound and / or the position of the lower bound is determined in the database. As described above, the data may be stored in the database in the form of a two-dimensional array. The data may be stored in the two-dimensional array in accordance with a predefined rule: the data of the first dimension stored in the two-dimensional array increases in a first direction, and the data of the second dimension stored in the two-dimensional array increases in a second direction.
[0101] To search for data in the two-dimensional array, the upper and / or lower bounds of the query range may be located first, and then the data within the query range may be read based on the upper and / or lower bounds. Optionally, a position may be predefined as a reference position in the two-dimensional array, and then the size relationship between the data at the reference position and the target data may be compared, and the reference position may be adjusted based on the size relationship between the two until the data at the reference position is equal to the target data, and the reference position is the position of the target data.
[0102] Exemplarily, the position in the two-dimensional array that is the largest in the first dimension and the smallest in the second dimension can be predefined as the reference position, and then the size relationship between the data at the reference position and the target data is compared. If the data at the reference position is larger than the target data, the reference position is moved one position in the opposite direction of the first direction; if the data at the reference position is smaller than the target data, the reference position is moved one position in the second direction; if the data at the reference position is equal to the target data, the reference position is the position of the target data.
[0103] An example, such as Figure 8 As shown, take M=25, N=5, p=q=0, and the target data is 9.
[0104] Predefined reference positions (i f , j f ) is (4, 0), the data at the reference position is 11, 11 is greater than 9, so the reference position is moved to (i f -1,j f ), that is (3,0); next, reference position (i f , j f ) becomes (3, 0), the data at the reference position is 7, 7 is less than 9, so the reference position is moved to (i f , j f +1), that is (3, 1); next, refer to the position (i f , j f ) becomes (3, 1), the data at the reference position is 12, 12 is greater than 9, so the reference position is moved to (i f -1,j f ), that is (2, 1); next, reference position (i f , j f) becomes (2, 1), the data at the reference position is 8, 8 is less than 9, so the reference position is moved to (i f , j f +1), that is, (2, 2); next, refer to the position (i f , j f ) becomes (2, 2), the data at the reference position is 13, 13 is greater than 9, so the reference position is moved to (i f -1,j f ), that is, (1, 2); next, reference position (i f , j f ) becomes (1, 2), the data at the reference position is 9, 9 equals 9, then the reference position (1, 2) is the position of the target data.
[0105] In step 730, based on the position of the upper bound and / or the position of the lower bound, data satisfying the query range is read from the database.
[0106] Optionally, within the range given by the query range, data is read in sequence according to the arrangement of the data, and the read data is the data that satisfies the query range.
[0107] An example, such as Fig. 9 As shown, the query range is: query data in the range of 9 to 17. The upper bound is 17 and the lower bound is 9. Since the query range in the embodiment of the present application is less than or equal to the upper bound and / or greater than or equal to the lower bound, the data that ultimately meets the query range includes 9 and 17.
[0108] First, determine the position of the lower bound 9, read data 9, and then read the next data 10 according to the data arrangement, and then read the next data 11 according to the data arrangement, and so on, until the upper bound data 17 is read. The data that meets the query range is read.
[0109] Based on the above method, the upper bound and / or lower bound of the query range is located in the data stored in the form of a two-dimensional array, and then the data that meets the query range is read based on the position of the upper bound and / or lower bound. In this way, there is no need to add all the data in the database to the memory for sorting and then searching, which saves the time of loading and sorting data, and can implement range queries more efficiently.
[0110] Fig.10 1 is a schematic diagram of a method for adding a data structure to a database provided in an embodiment of the present application. The method 1000 may be Figure 1 The server 110 executes in.
[0111] Fig.10 The method shown includes the following steps:
[0112] Step 1010: define data structure;
[0113] Step 1020: define the core method of implementing the data structure;
[0114] Step 1030: Add the data structure to the database.
[0115] Each step in the method 1000 is described in detail below.
[0116] In step 1010, a data structure is defined.
[0117] For this embodiment, the data structure is a two-dimensional array, and satisfies the conditions that the data of the first dimension stored in the two-dimensional array increases along the first direction, and the data of the second dimension stored in the two-dimensional array increases along the second direction.
[0118] It should be understood that there can be many types of data structures, such as arrays of various dimensions. In addition to arrays, there are also stacks, queues, linked lists, trees, graphs, heaps, hash tables, etc.
[0119] In step 1020, define the core method for implementing the data structure.
[0120] For example, how to arrange data in a two-dimensional array, how to add data to an existing two-dimensional array, and how to delete data.
[0121] In step 1030, the data structure is added to the database.
[0122] In one example, a two-dimensional array data structure is added to a remote dictionary server (Redis) database, which is a memory database. It should be understood that the type of database that can be added with the data structure is not limited to the Redis database, but can also be a Memcached, Aerospike, Apache Ignite, etc. database, and this application does not make any limitation on this.
[0123] First, add the type definition in server.h, which can be achieved through the following code:
[0124] #define OBJ_MATRIX_TYPE 7
[0125] Then add the object creation function in object.c, which can be implemented by the following code:
[0126] robj*createMatrixObject(void){
[0127] matrix*new_matrix;
[0128] return new_node;
[0129] }
[0130] Then add the redis destruction function and add the following code to the decrRefCount function in object.c:
[0131] case OBJ_MATRIX_TYPE:destroy_matrix(o); break;
[0132] Then add the destruction function in object.c
[0133] void destroy_matrix(matrix*matrix){
[0134] zfree(matrix);
[0135] }
[0136] Next, implement the command and add the following code to the typeCommand function in db.c:
[0137] case OBJ_MATRIX_TYPE:type="matrix"; break;
[0138] Implement the command in t_hellotype.c, for example, insert the following command code:
[0139]
[0140]
[0141] Finally, write the command to redisCommandTable and add the following code to the redisCommandTable array in server.c:
[0142]
[0143] Fig.11 is a schematic block diagram of a data query device provided in an embodiment of the present application. The device can be Figure 1 The server 110 in Fig.11 As shown, the data query device 1100 can be used to implement the functions of the data storage device and / or the data query device in the above method embodiment. Exemplarily, the device 1100 may include an acquisition module 1110 and a processing module 1120.
[0144] Among them, the acquisition module 1110 can be used to obtain a query range, and the query range includes the upper bound and / or lower bound of the data to be queried; the processing module 1120 can be used to determine the position of the upper bound and / or the position of the lower bound in the database based on the query range; wherein the data in the database is stored in the form of a two-dimensional array and satisfies: the data of the first dimension stored in the two-dimensional array increases along a first direction, and the data of the second dimension stored in the two-dimensional array increases along a second direction; and is used to read data that satisfies the query range in the database based on the position of the upper bound and / or the position of the lower bound.
[0145] Fig.12 is another schematic block diagram of the data query device provided in the embodiment of the present application. Fig.12 As shown, the device 1200 may include at least one processor 1210, which is used to implement the functions of the data storage device and / or data query device in the method provided in this application. Please refer to the detailed description in the method example for details, which will not be repeated here.
[0146] The device 1200 may also include a memory 1220 for storing program instructions and / or data. The memory 1220 is coupled to the processor 1210. The coupling in this application is an indirect coupling or communication connection between devices, units or modules, which may be electrical, mechanical or other forms, for information exchange between devices, units or modules. The processor 1210 may operate in conjunction with the memory 1220. The processor 1210 may execute program instructions stored in the memory 1220. At least one of the at least one memory may be included in the processor.
[0147] The apparatus 1200 may also include a communication interface 1230 for communicating with other devices via a transmission medium, so that the device in the apparatus 1200 can communicate with other devices. The communication interface 1230 may be, for example, a transceiver, an interface, a bus, a circuit, or a device capable of implementing transceiver functions. The processor 1210 may use the communication interface 1230 to send and receive data and / or information, and to implement Figure 7 The data query method described in the corresponding embodiment.
[0148] The specific connection medium between the processor 1210, the memory 1220 and the communication interface 1230 is not limited in this application. Fig.12 The processor 1210, the memory 1220 and the communication interface 1230 are connected via a bus 1240. The bus 1240 is connected to the Fig.12 The connections between the other components are shown in bold lines, which are only for illustration and are not intended to be limiting. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.12Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0149] In the embodiments of the present application, the processor may be a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, and may implement or execute the methods, steps, and logic block diagrams disclosed in the present application. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the present application may be directly embodied as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.
[0150] According to the method provided by the present application, the present application also provides a computer-readable storage medium, which stores a program code, and when the program code is run on a computer, the computer executes Figure 2 The data storage method and / or Figure 7 The data query method in the illustrated embodiment.
[0151] According to the method provided by the present application, the present application also provides a computer program product, the computer program product comprising: computer program code. When the computer program code is run on a computer, the computer executes Figure 2 The data storage method and / or Figure 7 The data query method in the illustrated embodiment.
[0152] The technical solution provided in this application can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in this application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a network device, a terminal device or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website site, computer, server or data center through a wired, such as coaxial cable, optical fiber, digital subscriber line (DSL) or wireless (such as infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium such as a digital video disc (DVD), or a semiconductor medium, etc.
[0153] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A data query method, It is characterized in that include: Acquire a query range, where the query range includes an upper bound and / or a lower bound of data to be queried; Based on the query range, determine the position of the upper bound and / or the position of the lower bound in a database; wherein the data in the database is stored in the form of a two-dimensional array and satisfies: the data of the first dimension stored in the two-dimensional array increases along a first direction, the data of the second dimension stored in the two-dimensional array increases along a second direction, the first dimension of the two-dimensional array is the row of the matrix, the second dimension is the column of the matrix, the first direction is from left to right or from right to left, and the second direction is from top to bottom or from bottom to top; or, the first dimension of the two-dimensional array is the column of the matrix, the second dimension is the row of the matrix, the first direction is from top to bottom or from bottom to top, and the second direction is from left to right or from right to left; Based on the position of the upper bound and / or the position of the lower bound, reading data within the query range in the database; Before obtaining the query range, the method further includes: Determine the length N according to the number M of data to be stored in the database; Store M data in an N×N two-dimensional array; The length N and the number of data M satisfy: (N-1) 2 <M≤N 2 , M and N are positive integers; Determining the position of the upper limit and / or the position of the lower limit in a database based on the query range includes: The upper bound and / or lower bound are respectively used as target data, and the position of the target data is determined through the following steps: a predefined position is used as a reference position; based on the size relationship between the data at the reference position and the target data, the reference position is adjusted until the data at the reference position is equal to the target data; the position of the reference position is determined as the position of the target data.
2. The method according to claim 1, It is characterized in that The length of the first dimension is the same as the length of the second dimension.
3. The method according to claim 2, It is characterized in that The serial numbers used to identify the data of the first dimension increase along the first direction, and the serial numbers used to identify the data of the second dimension increase along the second direction.
4. The method according to claim 3, It is characterized in that The serial numbers used to identify the data of the first dimension include p to p+N-1, the serial numbers used to identify the data of the second dimension include q to q+N-1, and the arrangement of the M data satisfies: The position of the mth data in the M data in the two-dimensional array is (i m , j m ), the position of the m+1th data in the M data in the two-dimensional array is (i m+1 , j m+1 ), the m+1th data is greater than the mth data; wherein, When m =q+N-1, i m+1 =p+N-1,j m+1 =q-p+i m +1; When m ≠q+N-1andi m = p, i m+1 =p-q+j m +1,j m+1 =q; When m ≠q+N-1andi m ≠p, i m+1 =i m -1,j m+1 =j m +1.
5. The method according to claim 3, It is characterized in that The serial numbers used to identify the data of the first dimension include p to p+N, the serial numbers used to identify the data of the second dimension include q to q+N, and the arrangement of the M data satisfies: The position of the mth data in the M data is (i m , j m ), the position of the m+1th data in the M data is (i m+1 , j m+1 ), the m+1th data is greater than the mth data; When m =q+N-1, i m+1 =i m +1,j m+1 =q; When m When ≠q+N-1, i m+1 =i m , j m+1 =j m +1.
6. The method according to claim 3, It is characterized in that The serial numbers used to identify the data of the first dimension include p to p+N, the serial numbers used to identify the data of the second dimension include q to q+N, and the arrangement of the M data satisfies: The position of the mth data in the M data is (i m , j m ), the position of the m+1th data in the M data is (i m+1 , j m+1 ), the m+1th data is greater than the mth data; when i m =p+N-1, i m+1 =p,j m+1 =j m +1; This m When ≠ p+N-1, i m+1 = i m +1, j m+1 =j m .
7. The method according to claim 1, It is characterized in that The predefined position is (i f =p+N-1,j f =q); The adjusting the reference position based on the size relationship between the data at the reference position and the target data includes: if the data at the reference position is larger than the target data, adjusting the reference position to (i f -1,j f ); If the data at the reference position is smaller than the target data, adjust the reference position to (i f , j f +1).
8. The method according to claim 6, It is characterized in that The step of reading data satisfying the query range from the database based on the position of the upper bound and / or the position of the lower bound includes: According to the arrangement of the M data, data are read sequentially within the range determined by the position of the upper bound and / or the position of the lower bound in the two-dimensional array; the read data is data that satisfies the query range.
9. A data query device, It is characterized in that include: A storage module, used to determine the length N according to the number M of data to be stored in the database; M data are stored in an N×N two-dimensional array; wherein the length N and the number of data M satisfy: (N-1) 2 <M≤N 2 , M and N are positive integers; An acquisition module is used to acquire a query range, wherein the query range includes an upper bound and / or a lower bound of the data to be queried; a processing module is used to determine the position of the upper bound and / or the position of the lower bound in a database based on the query range; wherein the data in the database is stored in the form of a two-dimensional array and satisfies: the data of the first dimension stored in the two-dimensional array increases in a first direction, the data of the second dimension stored in the two-dimensional array increases in a second direction, the first dimension of the two-dimensional array is the row of the matrix, the second dimension is the column of the matrix, the first direction is from left to right or from right to left, and the second direction is from top to bottom or from bottom to top; or, the first dimension of the two-dimensional array is the column of the matrix, the second dimension is the row of the matrix, the first direction is from top to bottom or from bottom to top, and the second direction is from left to right or from right to left; The processing module is used to read data satisfying the query range in the database based on the position of the upper bound and / or the position of the lower bound; The processing module is specifically used to read data in sequence within the range determined by the position of the upper bound and / or the position of the lower bound in the two-dimensional array according to the arrangement of the M data; the read data is data that meets the query range.
10. A data query device, It is characterized in that comprising a memory and a processor, wherein: The memory is used to store computer programs; The processor is configured to call and execute a computer program so that the apparatus performs the method according to any one of claims 1 to 8.
11. A computer-readable storage medium, It is characterized in that The invention comprises a computer program, which, when being run on a computer, causes the computer to execute the method according to any one of claims 1 to 8.
12. A computer program product, It is characterized in that The invention comprises a computer program which, when being executed, causes a computer to execute the method according to any one of claims 1 to 8.
Citation Information
Patent Citations
Two-dimensional data storage system, storage method and query method
CN106528635A
Data array management method and device based on parallel RAM (Random Access Memory) and storage equipment
CN113126913A