Query cardinal number determination method and device

Abstract

Embodiments of the invention provide a query cardinal number determination method and apparatus, relate to the technical field of databases, and can reduce the space overhead when determining a query cardinal number. The method comprises the following steps: determining a maximum target suffix in suffix data of target data; and determining a query cardinal number of the target data based on the target suffix, a suffix array of the character string in the database and a target function used for indicating a character distribution condition.

Description

Technical Field

[0001] This application relates to the field of database technology, and in particular to a method and apparatus for determining the cardinality of a query. Background Technology

[0002] With the development of database technology, the amount of data that can be stored in databases is increasing. In databases, users can use query statements such as LIKE to quickly retrieve string data and other data that meet specific conditions. However, considering the varying amounts of data involved in the query results, the optimal query strategy will also differ. For example, if a query condition involves only a small number of results, nested loops can be used directly; otherwise, a more complex index structure may be necessary.

[0003] Therefore, to further optimize query performance, during query retrieval, the cardinality of the query command can be determined to estimate the number of results that meet the query conditions, thus allowing for the selection of a more efficient query method. However, to quickly determine the cardinality, the database data needs to be processed, such as creating and storing suffix arrays of the database data. This allows for the determination of the cardinality through the database's suffix arrays. However, since databases typically contain large amounts of data, these suffix arrays can consume significant amounts of space. Summary of the Invention

[0004] This application provides a method and apparatus for determining the cardinality of a query, which can reduce the space overhead of determining the cardinality to a certain extent.

[0005] To achieve the above objectives, this application adopts the following technical solution:

[0006] In a first aspect, embodiments of this application provide a method for determining the query cardinality, comprising: determining the target suffix of the target data; and determining the query cardinality of the target data based on the target suffix, the suffix array, and the target function.

[0007] Among them, the target suffix is ​​the largest suffix in the suffix data generated based on the target data, the suffix array is obtained based on the suffix data of strings in the database, the objective function is the character cumulative distribution function of the suffix array, which is used to indicate the distribution of characters in the suffix array, and the query cardinality is used to indicate the number of times the target data appears in the database.

[0008] In the above implementation, by introducing a target function to indicate the character distribution in the suffix array, and determining the query cardinality of the target data in the database based on the target suffix, it is not necessary to store the suffix data of all strings in the database. The number of occurrences of the target suffix in the database can be determined based on the suffix array and the target function, thus obtaining the query cardinality of the target data, which can save a lot of space overhead.

[0009] Optionally, the suffix array includes the first column F, and the objective function is used to indicate the relationship between the number of occurrences of each character in the last column L and its position in the L column. The F and L columns are obtained by performing the Robes-Wyler Transform (BWT) on the strings in the database, and the characters in the F column correspond one-to-one with the characters in the L column.

[0010] Optionally, the interval position of the nth character in the target suffix within the F sequence can be determined. Based on the interval position of the nth character in the F sequence, the interval position of the corresponding L sequence is determined. If n is greater than 2, then based on the objective function and the interval position of the corresponding L sequence, the interval position of the (n-1)th character in the F sequence is determined. Let n = n-1, and return the step of determining the interval position of the corresponding L sequence based on the interval position of the nth character in the F sequence. If n is less than or equal to 2, then based on the objective function, the frequency of the (n-1)th character in the interval position of the corresponding L sequence is determined, and the frequency of the (n-th)th character is determined as the query cardinality of the target data.

[0011] Where n is a positive integer, and the initial value of n is the length of the target suffix.

[0012] In the above implementation, the BWT algorithm is used to determine the frequency of the target suffix in the database using the F-series and the L-series. Furthermore, the L-series is replaced by a target function, which indicates the frequency of each character in the L-series up to different positions; that is, the relationship between the frequency of each character in the L-series and its position. In this way, there is no need to store the L-series; the frequency of the target suffix in the database can be determined based on the F-series and the target function, thus obtaining the query cardinality of the target data in the database and saving the storage overhead of the L-series.

[0013] In some possible implementations, before determining the query cardinality of the target data based on the target suffix, the suffix array, and the objective function, the distribution function of each character in the L-sequence can be determined based on the frequency of occurrence of each character. The objective function is then obtained by fitting the distribution function based on error conditions.

[0014] The distribution function is used to indicate the relationship between the frequency of a character and its position in the L-sequence.

[0015] In the above implementation, the distribution function is fitted by error conditions to obtain the objective function, thereby improving the accuracy of the objective function and thus improving the accuracy of the query cardinality of the determined target data.

[0016] Optionally, the mean squared error of the distribution function can be determined. The distribution function is then fitted based on the mean squared error, and the distribution function that minimizes the mean squared error is taken as the objective function.

[0017] Among the above implementation methods, a method for fitting the distribution function using mean square error is proposed, which can quickly achieve the fitting of the distribution function.

[0018] Optionally, the slope of the sub-function between the starting node and the i-th node of the distribution function can be determined. If the slope is less than a slope threshold and the i-th node is not the ending node of the distribution function, let i = i + 1, and return to the step of determining the slope between the starting node and the i-th node. If the slope is greater than or equal to the slope threshold and the i-th node is not the ending node of the distribution function, take the (i-1)-th node as the ending node of the sub-function and the i-th node as the starting node of the next sub-function, let i = i + 1, and return to the step of determining the slope between the starting node and the i-th node of the distribution function. All sub-functions are used as the objective function.

[0019] Where i is a positive integer, and each node of the distribution function corresponds to a position in the L sequence and the number of times a character appears.

[0020] In the above implementation, considering that the mean squared error is sensitive to outliers in the distribution function, a method for fitting the distribution function through greedy spline interpolation is also proposed. Based on the greedy difference, if a certain node causes a large change in the slope of the distribution function, the process ends directly before that node to obtain a sub-function, and uses that node as the starting node for fitting another sub-function. This avoids the fact that an outlier at a certain node will cause poor accuracy of the distribution function, making the fitting result of the distribution function more robust and ensuring the accuracy of the objective function.

[0021] In some possible implementations, it's also possible to traverse the cyclic suffixes of each string in the database and build a corresponding suffix tree. For any node in the character suffix tree, if the node satisfies the pruning condition, the node is pruned to obtain a pruned suffix tree. Replace the F sequence with the pruned suffix tree.

[0022] The suffix tree consists of multiple nodes, each node indicating a character of a cyclic suffix in the database, and each node corresponds to a position interval in the L-sequence.

[0023] In the above implementation, a pruned suffix tree is obtained by building a suffix tree of the database and pruning it. The F sequence is then replaced with the pruned suffix tree, thereby achieving a hierarchical representation of the L sequence. Furthermore, by using the pruned suffix tree, nodes of the L sequence can be pushed to their ancestor nodes, and function fitting can be performed at the ancestor nodes to obtain the objective function, without having to perform function fitting at each node, thus further reducing the space overhead of the objective function.

[0024] In some possible implementations, the pruning conditions include at least one of the following: the height of the suffix tree corresponding to the node is less than the height threshold, and the size of the position interval corresponding to the node is less than the interval threshold.

[0025] Here, the height of the suffix tree corresponding to the node refers to the height of the suffix tree in which the node is located, and the interval size is used to indicate the size of the interval of the L sequence position corresponding to the node.

[0026] In the above implementation, the suffix tree can be pruned using the tree height and / or the size of the L-sequence position interval. When pruning based on tree height, limiting the suffix tree height ensures that the query process still conforms to linear querying when determining the query cardinality based on the pruned suffix tree, thus reducing the impact of the suffix tree height on the objective function. When pruning based on the L-sequence position interval, the number of L nodes corresponding to each node in the suffix tree can be refined to avoid overly fine-grained division of L nodes, which would increase the function fitting burden and reduce the space overhead of the objective function.

[0027] In some possible implementations, the longest child suffix of the target suffix appearing in the pruned suffix tree, and the target length of the longest child suffix, can be determined. If the target length equals the length of the target suffix, the query cardinality of the target data is determined based on the node parameters of the pruned suffix tree. If the target length is less than the length of the target suffix, the interval position of the L-sequence corresponding to the longest child suffix is ​​determined based on the node parameters of the pruned suffix tree, and the query cardinality of the target data is determined based on the interval position of the L-sequence corresponding to the longest child suffix.

[0028] In the above implementation, before determining the query cardinality of the target data based on the objective function, the longest child suffix of the target data appearing in the pruned suffix tree can be determined first. If the target suffix appears completely in the pruned suffix tree, there is no need to index the query cardinality of the target data based on the objective function; the query cardinality of the target data can be determined directly based on the node parameters of the pruned suffix tree. If only a portion of the target suffix appears in the pruned suffix tree, there is no need to locate the query starting position of that portion of the suffix based on the objective function. Instead, the starting query position of that portion of the suffix can be determined based on the pruned suffix tree, thereby narrowing the query range and improving the efficiency of determining the query cardinality.

[0029] In some possible implementations, the database includes multiple strings, and the method further includes: determining at least one cyclic suffix for each string among the multiple strings, resulting in multiple cyclic suffixes. If all characters in the first cyclic suffix are different from the characters at the same positions in the second cyclic suffix, the order between the first and second cyclic suffixes is determined based on lexicographical order. The length of the second cyclic suffix is ​​greater than the length of the first cyclic suffix. If all characters in the first cyclic suffix are the same as the characters at the same positions in the second cyclic suffix, the order between the first and second cyclic suffixes is determined based on the remaining characters of the first and second cyclic suffixes compared to the first cyclic suffix. The multiple cyclic suffixes are sorted based on their order to obtain the F-sequence.

[0030] In the above implementation, considering that if the database contains multiple strings, during BWT changes, the suffixes of one string might be a partial prefix of the suffixes of another string, making it impossible to sort them. Therefore, this application defines sorting rules to solve the sorting problem between suffixes in scenarios with multiple strings, while ensuring the properties of BWT, thus enriching the application scenarios of this application.

[0031] In some possible implementations, the method also includes: identifying the modified strings in the database; determining the cyclic suffixes corresponding to the modified strings; constructing a modified suffix tree for the modified strings, where the modified suffix tree indicates the frequency of each suffix in the modified strings; and adjusting the query cardinality of the target data based on the modified suffix tree.

[0032] The modified string refers to the string that changes in the database after the target function is obtained.

[0033] Optionally, the modified string includes adding and / or deleting strings. A first query cardinality for the target data can be determined based on the target suffix, the suffix array, and the target function; a second query cardinality can be determined based on the target suffix and the modified suffix tree. When the modified string is an added string, the sum of the first and second query cardins is used as the query cardinality of the target data. When the modified string is a deleted string, the difference between the first and second query cardins is used as the query cardinality of the target data.

[0034] In the above implementation, considering that the strings in the database may dynamically change after the objective function is obtained, leading to a decrease in the accuracy of the objective function, a suffix tree corresponding to the changed strings in the database is determined. Based on this suffix tree, the frequency of each suffix in the changed strings is determined. Furthermore, given a defined cardinality pair for the target data, the cardinality result obtained through the objective function is corrected based on the change type of the changed strings. This ensures that the cardinality result adapts to the dynamic changes in the strings in the database, thereby further improving the accuracy of the cardinality.

[0035] Optionally, if the changed string meets the target condition, the target function can be redefined based on all strings in the database.

[0036] In the above implementation, to avoid the large space overhead of the modified suffix tree due to significant changes in the strings in the database, or the large fluctuations in the correction of the query cardinality affecting the accuracy of the query cardinality, the objective function is re-determined based on all the strings in the database when the changed strings meet the target conditions. This improves the accuracy of the objective function and reduces the space overhead of the modified suffix tree.

[0037] Secondly, a cardinality determination device is provided, comprising: a suffix determination module and a cardinality determination module.

[0038] The suffix determination module is used to determine the target suffix of the target data. The target suffix is ​​the largest suffix among the suffix data generated based on the target data.

[0039] The cardinality determination module is used to determine the query cardinality of target data based on the target suffix, the suffix array, and the target function.

[0040] The suffix array is obtained based on the suffix data of strings in the database. The objective function is the cumulative distribution function of the characters in the suffix array, which is used to indicate the distribution of characters in the suffix array. The query cardinality is used to indicate the number of times the target data appears in the database.

[0041] Optionally, the suffix array includes the first column F, and the objective function is used to indicate the relationship between the number of occurrences of each character in the last column L and its position in the L column. The F and L columns are obtained by performing the Robes-Wyler Transform (BWT) on the strings in the database, and the characters in the F column correspond one-to-one with the characters in the L column.

[0042] Optionally, the cardinality determination module can determine the interval position of the nth character in the target suffix within the F-sequence. Based on the interval position of the nth character in the F-sequence, the interval position of the corresponding L-sequence is determined. If n is greater than 2, then based on the objective function and the interval position of the corresponding L-sequence, the interval position of the (n-1)th character in the F-sequence is determined, and n = n-1 is set. The step of determining the interval position of the corresponding L-sequence based on the interval position of the nth character in the F-sequence is returned. If n is less than or equal to 2, then based on the objective function, the frequency of occurrence of the (n-1)th character in the interval position of the corresponding L-sequence is determined, and the frequency of occurrence of the (n-th)th character is determined as the query cardinality of the target data.

[0043] Where n is a positive integer, and the initial value of n is the length of the target suffix.

[0044] In some possible implementations, the cardinality determination module is also used to determine the distribution function of each character in the L-sequence based on the frequency of occurrence of each character. The distribution function is then fitted based on error conditions to obtain the objective function.

[0045] The distribution function is used to indicate the relationship between the frequency of a character and its position in the L-sequence.

[0046] Optionally, the cardinality determination module can determine the mean squared error of the distribution function. The distribution function is fitted based on the mean squared error, and the distribution function that minimizes the mean squared error is taken as the objective function.

[0047] Optionally, the cardinality determination module can determine the slope of the sub-function between the starting node and the i-th node of the distribution function. If the slope is less than a slope threshold and the i-th node is not the ending node of the distribution function, let i = i + 1, and return to the step of determining the slope between the starting node and the i-th node of the distribution function. If the slope is greater than or equal to the slope threshold and the i-th node is not the ending node of the distribution function, take the (i-1)-th node as the ending node of the sub-function, and take the i-th node as the starting node of the next sub-function, let i = i + 1, and return to the step of determining the slope between the starting node and the i-th node of the distribution function. All sub-functions are then used as the objective function.

[0048] Where i is a positive integer, and each node of the distribution function corresponds to a position in the L sequence and the number of times a character appears.

[0049] In some possible implementations, the cardinality determination module can also traverse the cyclic suffixes of each string in the database and build a corresponding suffix tree. For any node in the character suffix tree, if the node satisfies the pruning condition, the node is pruned to obtain a pruned suffix tree. The F-sequence is then replaced with the pruned suffix tree.

[0050] The suffix tree consists of multiple nodes, each node indicating a character of a cyclic suffix in the database, and each node corresponds to a position interval in the L-sequence.

[0051] Optionally, the pruning conditions include at least one of the following: the height of the suffix tree corresponding to the node is less than the height threshold, and the size of the position interval corresponding to the node is less than the interval threshold; the height of the suffix tree corresponding to the node refers to the height of the suffix tree in which the node is located, and the interval size is used to indicate the size of the L-sequence position interval corresponding to the node.

[0052] Optionally, the cardinality determination module can determine the longest child suffix appearing in the pruned suffix tree for the target suffix, and the target length of the longest child suffix. If the target length is equal to the length of the target suffix, the query cardinality of the target data is determined based on the node parameters of the pruned suffix tree. If the target length is less than the length of the target suffix, the interval position of the L-sequence corresponding to the longest child suffix is ​​determined based on the node parameters of the pruned suffix tree, and the query cardinality of the target data is determined based on the interval position of the L-sequence corresponding to the longest child suffix.

[0053] In some possible implementations, the database includes multiple strings. The cardinality determination module can also determine at least one cyclic suffix for each of the multiple strings, resulting in multiple cyclic suffixes. If all characters in the first cyclic suffix are different from the characters at the same positions in the second cyclic suffix, the order between the first and second cyclic suffixes is determined based on lexicographical order, with the length of the second cyclic suffix being greater than the length of the first cyclic suffix. If all characters in the first cyclic suffix are the same as the characters at the same positions in the second cyclic suffix, the order between the first and second cyclic suffixes is determined based on the remaining characters of the first and second cyclic suffixes compared to the first cyclic suffix. The multiple cyclic suffixes are sorted based on their order to obtain the F-sequence.

[0054] In some possible implementations, the cardinality determination module can also determine the modified strings in the database. Based on the cyclic suffixes corresponding to the modified strings, a modified suffix tree corresponding to the modified strings is determined. The query cardinality of the target data is then adjusted based on the modified suffix tree.

[0055] Among them, the changed string refers to the string that has changed in the database after the target function is obtained, and the changed suffix tree is used to indicate the number of times each suffix appears in the changed string.

[0056] Optionally, the modified string includes adding and / or deleting strings. The cardinality determination module can also determine the cardinality correction parameter for the target data based on the target suffix and the modified suffix tree. When the modified string is an added string, the sum of the cardinality correction parameter and the query cardinality is used as the query cardinality of the target data. When the modified string is a deleted string, the difference between the cardinality correction parameter and the query cardinality is used as the query cardinality of the target data.

[0057] Optionally, the cardinality determination module can also redetermine the target function based on all strings in the database, provided that the changed string meets the target conditions.

[0058] Thirdly, embodiments of this application provide a cardinality determination device, including: a transceiver / transceiver pin and a processor. Optionally, it also includes a memory. The transceiver / transceiver pin, the processor, and the memory communicate with each other via internal interconnection paths; the processor executes instructions to control the transceiver / transceiver pin to send or receive signals; the memory stores the instructions. When the processor executes the instructions, it performs the method of the first aspect or any possible implementation thereof.

[0059] Fourthly, embodiments of this application provide a computer-readable medium for storing a computer program, the computer program including instructions for performing the method in the first aspect or any possible implementation of the first aspect.

[0060] Fifthly, embodiments of this application provide a computer program including instructions for performing the method in the first aspect or any possible implementation thereof.

[0061] Sixthly, embodiments of this application provide a chip including a processing circuit and transceiver pins. The transceiver pins and the processing circuit communicate with each other via an internal connection path. The processing circuit executes the method in the first aspect or any possible implementation of the first aspect to control the receiving pin to receive signals and to control the transmitting pin to transmit signals.

[0062] The solutions provided in the second to sixth aspects above are used to implement or cooperate with the methods provided in the first aspect above, and therefore can achieve the same or corresponding beneficial effects as the first aspect, which will not be elaborated here. Attached Figure Description

[0063] Figure 1 This is an exemplary schematic diagram of the architecture of an implementation environment;

[0064] Figure 2 This is a flowchart illustrating an exemplary method for determining the cardinality of a query.

[0065] Figure 3 This is a schematic diagram of a suffix tree provided in one embodiment of this application;

[0066] Figure 4 This is a schematic diagram of a pruned suffix tree provided in one embodiment of this application;

[0067] Figure 5 This is a schematic diagram of function fitting combined with pruned suffix trees provided in one embodiment of this application;

[0068] Figure 6 This is a schematic block diagram of a query cardinality determination device provided in an embodiment of this application;

[0069] Figure 7 This is a schematic diagram of the structure of a query cardinality determination device provided in an embodiment of this application. Detailed Implementation

[0070] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0071] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0072] The terms "first" and "second," etc., used in the specification and claims of this application are used to distinguish different objects, not to describe a specific order of objects. For example, "first target object" and "second target object," etc., are used to distinguish different target objects, not to describe a specific order of target objects.

[0073] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0074] In the description of the embodiments in this application, unless otherwise stated, "multiple" means two or more. For example, multiple processing units means two or more processing units; multiple systems means two or more systems.

[0075] Before describing the technical solutions of the embodiments of this application, the implementation environment of the embodiments of this application will first be described with reference to the accompanying drawings. The implementation environment of the embodiments themselves involves a database system; see [link to relevant documentation]. Figure 1 This is a schematic diagram of a database system provided in an embodiment of this application. The database system includes a user interface module 101, a query module 102, and a storage module 103.

[0076] User interface module 101 is used to receive query tasks initiated by users, such as when users interact with the database through the user interface mode, or when users input Structured Query Language (SQL) into the user interface module to initiate a query task.

[0077] Optionally, users can flexibly determine the query method based on actual usage needs. For example, users can create a table containing strings and then use query statements such as LIKE statements to query the target characters in the table.

[0078] The query module is used to execute user query commands. Optional, such as... Figure 1 As shown, the query module 102 includes a query execution submodule, which is used to execute a query based on the query statement input by the user and obtain the query results.

[0079] Considering that query execution efficiency can vary in different query scenarios, such as Figure 1 As shown, the query module 102 may also include a query optimization submodule, which includes multiple query plans to adapt to different query scenarios. These multiple query plans can be flexibly configured based on actual usage needs; for example, the multiple query plans can be... Figure 1 Plan A, Plan B, etc., are used to select different query plans for different query scenarios to optimize database query efficiency.

[0080] Here, different query scenarios can refer to query cardinality of different sizes. In this scenario, in order to accurately estimate the query cardinality, such as... Figure 1 As shown, the query optimization module may include a cardinality estimation submodule, which is used to determine the query cardinality of the data to be queried, so as to select different query plans to perform data query based on the size of the query cardinality.

[0081] The storage module 103 is used to store data, such as data for storing the database, and information needed in the process of determining the query cardinality, such as the suffix array below.

[0082] In combination with the above, as follows Figure 1 The following is a schematic diagram of the implementation environment. The specific implementation scheme of this application is described below:

[0083] like Figure 2 The query base determination method provided in this application includes the following steps S201-S202.

[0084] S201: Determine the target suffix for the target data.

[0085] The target suffix is ​​the largest suffix among the suffix data generated based on the target data.

[0086] Optionally, the target data can be string data. The suffix data of the target data can be understood as the end part of the target data, that is, the data that starts from a certain position in the target data and extends to the end of the data.

[0087] For example, if the target data can be abcdef, then the suffix data of the target data includes abcdef, bcdef, cdef, def, ef, and f.

[0088] The target suffix can be understood as the longest suffix in the suffix data. For example, in the above abcdef data, the target suffix is ​​abcdef.

[0089] S202: Determine the query cardinality of the target data based on the target suffix, suffix array, and target function.

[0090] The suffix array is obtained from the suffix data in the database, the objective function is the character cumulative distribution function of the suffix array, which is used to indicate the distribution of characters in the suffix array, and the query cardinality is used to indicate the number of times the target data appears in the database.

[0091] Optionally, the suffix array can be a suffix array obtained by performing a Burrows-Wheeler Transform (BWT) on the data in the database.

[0092] In BWT, you can determine all the cyclic suffixes for each piece of data in the database and sort all the cyclic suffixes of the target data in ascending lexicographical order to obtain a suffix array. Additionally, in some scenarios, to avoid confusion with cyclic suffixes, it's necessary to add an indicator, such as $ or #, to the end of the target data before determining its cyclic suffixes to indicate the starting position of the data.

[0093] For example, the target data includes three strings: abbc, bbbca, and bcabc. Taking the string "bcabc" as an example, the data after adding the indicator is "bcabc#", and the corresponding cyclic suffixes are #bcabc, c#bcab, bc#bca, abc#bc, cabc#b, and bcabc#. After sorting in lexicographical ascending order, the resulting array of suffixes for the target data is #bcabc, abc#bc, bc#bca, bcabc#, c#bcab, and cabc#b. The results of each step in the BWT process can be shown in Table 1.

[0094] Table 1

[0095]

[0096] In some embodiments, the suffix array of BWT also includes a first column F and a last column L, where the characters in the F column correspond one-to-one with the characters in the L column.

[0097] Wherein, the F-series refers to the first column of characters in the suffix array, and the L-series refers to the last column of characters in the suffix array. Taking the suffix array described in Table 1 above as an example, the relationship between the F-series, the L-series, and the suffix array can be shown in Table 2 below:

[0098] Table 2

[0099]

[0100]

[0101] In BWT, the F and L sequences have the following properties:

[0102] Property 1: For the same row of sequences F and L, the character of sequence L is the preceding character of sequence F in the string.

[0103] Property 2: The order in which the same character appears in sequence F is the same as the order in which it appears in sequence L.

[0104] Taking row 6 of Table 2 above as an example, the data in row 6 of the suffix array is abc#bc, the F sequence is 'a', and the L sequence is 'c'. According to property 1, the 'c' in the L sequence is the character preceding the 'a' in the F sequence within 'bcabc'. According to property 2, the 'a' in the F sequence is the third 'a', which is the same character 'a' in the same string as the third 'a' in the L sequence, i.e., the 'a' in the string 'bcabc'; similarly, the 'c' in the L sequence is the fourth 'c', which should be the same character 'c' in the same string as the fourth 'c' in the F sequence, i.e., the first 'c' in the string 'bcabc'.

[0105] Thus, the cardinality of the query can be determined based on the two properties of BWT mentioned above.

[0106] For example, let's take determining the query cardinality of the data "abc" in the aforementioned database. We can first determine the target suffix of "abc", i.e., "abc", and start the query from the last character "c" of "abc" (i.e., the query cardinality of the character "c" in the database). We first determine the character "c" from the F column. Based on Table 2 above, we know that the characters in the F column from row 14 to row 17 are all "c".

[0107] Then, we query the character 'b' preceding 'c' (i.e., determine the lookup cardinality of 'bc' in the database). Since the characters in column L in the same row are the preceding characters of the characters in column F in the string, we can query the string 'bc' by querying the 'b' in rows 14 to 17 of column L. According to Table 2, the characters in rows 14 to 17 of column L are all 'b'.

[0108] Next, we need to query the character 'a' preceding 'bc'. Since the 'b's in rows 14 to 17 of sequence L are the 4th to 7th 'b's in sequence L, according to Property 2, the 4th to 7th 'b's in sequence L correspond to the same 'b's in rows 4 to 7 of sequence F. Table 2 shows that the 4th to 7th 'b's in sequence F are rows 10 to 13 of sequence F. Based on this, by querying the occurrences of 'a' in rows 10 to 13 of sequence L, we can determine the cardinality of the target data 'abc' in the database. Table 2 shows that only row 11 contains the character 'a' in rows 10 to 13 of sequence L. Therefore, the occurrence count of the data 'abc' in the database is 1, meaning its cardinality in the database is 1.

[0109] The above example only uses the database containing the three short strings "abbc", "bbbca", and "bcabc". However, databases typically contain a large amount of data. In this case, performing a BWT transformation on all the data in the database and storing the F and L sequences would consume a significant amount of space and be quite difficult to implement.

[0110] Based on this, in the embodiments of this application, a target function can be determined, that is, the target function indicates the relationship between the frequency of occurrence of each character in the L-sequence and its position in the L-sequence. In this way, there is no need to store the L-sequence; the query cardinality of the target data can be determined based on the F-sequence and the target function, thereby reducing a significant amount of space overhead.

[0111] Optionally, the relationship between the number of occurrences of each character in the L-sequence indicated by the objective function and its position in the L-sequence can refer to the total number of times the character appears in the L-sequence up to a certain position.

[0112] For example, the target function can be the function fun. c (u[i]) can be used to determine how many times the character 'c' appears up to the i-th row of the L sequence.

[0113] In some embodiments, the interval position of the nth character of the target suffix in the F sequence can be determined, where n is a positive integer and its initial value is the length of the target suffix. Based on the interval position of the nth character in the F sequence, the interval position of the corresponding L sequence is determined; if n is greater than 2, then based on the objective function and the interval position of the corresponding L sequence, the interval position of the (n-1)th character in the F sequence is determined, and n = n-1 is set, returning to the step of determining the interval position of the corresponding L sequence based on the interval position of the nth character in the F sequence; if n is less than or equal to 2, then based on the objective function, the occurrence count of the (n-1)th character in the interval position of the corresponding L sequence is determined, and the occurrence count of the (n-1)th character is determined as the query cardinality of the target data.

[0114] In this context, the interval position can be understood as the interval of the occurrence position, such as the interval position of the nth character in the F sequence, that is, the interval of the occurrence position of the nth character in the F sequence.

[0115] Optionally, since the same row of the F and L sequences contains the first and last characters of the same cyclic suffix, the interval position of the L sequence corresponding to the nth character is the same as the interval position of the nth character in the F sequence.

[0116] For example, still using the F-sequence shown in Table 2 above, if the target suffix is ​​bca, the initial value of n is the length of the target suffix, 3. First, determine the interval position of the third character of the target suffix (i.e., character a) in the F-sequence, which is row 4 to row 6. Then, the interval position of the L-sequence corresponding to the third character is also row 4 to row 6. At this time, n = 3 > 2. Based on the objective function and the interval position of the L-sequence corresponding to the third character, determine the interval position of the second character (i.e., character c) in the F-sequence.

[0117] Based on Property 1 above, since the character in column L is the preceding character of the character in column F in the same row, in this embodiment, the target suffix can be queried sequentially from back to front using suffix indexing. That is, first, the position range of the last character in the target suffix is ​​determined, and then the position range of the character preceding the last character within that position range is determined, thus sequentially indexing the target suffix.

[0118] Taking the target suffix "bca" as an example, first determine the position of character "a" in the interval of the F sequence, which is rows 4 to 6 in Table 2. Then, query the number of occurrences of character "c" in rows 4 to 6 of the L sequence. That is, query the number of occurrences of character "c" preceding character "a" in all cyclic suffixes in the database, i.e., the number of occurrences of character "bc". Based on the interval positions of these characters "c" preceding character "b" in the F sequence, query the number of occurrences of character "b" in the L sequence within these interval positions to obtain the number of occurrences of character "bca", which is the lookup cardinality of character "bca".

[0119] Based on property 2 above, since the order in which a character appears in sequence F is the same as its order in sequence L, we can determine the position of the character in sequence F based on its order of appearance in sequence L.

[0120] The order in which characters appear in the L-sequence can be understood as: the nth time each character in the L-sequence appears in the L-sequence. Taking the L-sequence shown in Table 2 above as an example, the character in the first row is 'c', which is the first occurrence of 'c' in the L-sequence; the character in the second row is 'a', which is the first occurrence of 'a' in the L-sequence; the character in the third row is 'c', which is the second occurrence of 'c' in the L-sequence; the character in the fourth row is 'c', which is the third occurrence of 'c' in the L-sequence, and so on.

[0121] Combining this with property 2 above, we know that the character c in the first row of the L sequence and the character c in the fourteenth row of the F sequence are both the first occurrences of the character c in their respective sequences, and are the same character c; the character a in the second row of the L sequence and the character a in the fourth row of the F sequence are both the first occurrences of the character a in their respective sequences, and are the same character a; the character c in the third row of the L sequence and the character c in the fifteenth row of the F sequence are both the second occurrences of the character c in their respective sequences, and are the same character c, and so on.

[0122] Based on this, we can determine the (n-1)th character's position in the L-sequence within the interval corresponding to the nth character, thus obtaining the interval position of the second character in the F-sequence. For example, if the interval position of character 'a' in the L-sequence is from row 4 to row 7, we can determine the order of character 'c''s appearance in rows 4 to 7 of the L-sequence, thereby obtaining the interval position of character 'c' in the F-sequence.

[0123] Optionally, since the objective function indicates the correspondence between the occurrence count of each character in the L-sequence and its different positions within the L-sequence, the occurrence count of the (n-1)th character in the row preceding the interval position corresponding to the nth character, and the occurrence count of the (n-1)th character in the last row of the L-sequence corresponding to the nth character, can be determined based on the objective function. This yields the occurrence order of the (n-1)th character within the interval position of the L-sequence corresponding to the nth character, and consequently, the interval position of the (n-1)th character in the F-sequence.

[0124] Taking the target suffix "bca" as an example, the third character (character 'a') is located in the range of rows 4 to 6 in array F. Therefore, the range of the L sequence corresponding to character 'a' is also from row 4 to 6. Based on the objective function, we determine the occurrence count of the (n-1)th character (character 'c') up to the row before the range of the L sequence corresponding to character 'a', i.e., up to row 3, which is 2 in this case. We also determine the occurrence count of character 'c' up to the row after the range of the L sequence corresponding to character 'a', i.e., up to row 6, which is 4 in this case.

[0125] That is, up to the 3rd row of sequence L, the character 'c' appears twice, and up to the 6th row of sequence L, the character 'c' appears four times. Therefore, it can be deduced that the character 'c' appearing in rows 4 to 6 of sequence L is the 3rd and 4th character 'c' of sequence L. Based on property 2 above, the 3rd character 'c' of sequence L and the 3rd character 'c' of sequence F are the same character 'c', and the 4th character 'c' of sequence L and the 4th character 'c' of sequence F are also the same character 'c'. Therefore, it can be further deduced that the position of the character 'c' in the target suffix 'bca' in sequence F is between the position of the 3rd character 'c' (row 16) and the position of the 4th character 'c' (row 17).

[0126] In some embodiments, at least one cyclic suffix corresponding to each of the multiple strings can be determined to obtain multiple cyclic suffixes; if all characters of the first cyclic suffix are different from the characters at the same position in the second cyclic suffix, the order between the first cyclic suffix and the second cyclic suffix is ​​determined based on lexicographical order, and the length of the second cyclic suffix is ​​greater than the length of the first cyclic suffix; if all characters of the first cyclic suffix are the same as the characters at the same position in the second cyclic suffix, the order between the first cyclic suffix and the second cyclic suffix is ​​determined based on the remaining characters of the first cyclic suffix and the second cyclic suffix compared to the first cyclic suffix; the multiple cyclic suffixes are sorted based on the order between them to obtain the F sequence.

[0127] Optionally, considering that when the database includes multiple strings, there may be cases where the cyclic suffix of one string is the prefix of the cyclic suffix of another string.

[0128] For example, Table 2 lists all cyclic suffixes generated from the strings abbc, bbbca, and bcabc, along with the lexicographically ordered results. If the database also includes the character abc, the cyclic suffixes generated from it include abc#, bc#a, and c#ab. Comparing Table 2, we can see that abc# is identical to the first four characters of abc#bc in row 6, meaning the cyclic suffix abc# of abc is a prefix of the cyclic suffix abc#bc of bcabc. This could prevent the two cyclic suffixes from being compared.

[0129] Based on this, in some embodiments, the above-mentioned cyclic suffix sorting rules can be defined to avoid the problem that cyclic suffixes cannot be sorted because the cyclic suffix of one string is the prefix of the cyclic suffix of another string.

[0130] Taking the first cyclic suffix as "abc#" as an example, if the second cyclic suffix is ​​"abbc#", we compare the characters at the same position in both cyclic suffixes. The comparison shows that the third character of the first cyclic suffix is ​​"c", while the third character of the second cyclic suffix is ​​"b". Since "b" has a lexicographical order less than "c", the second cyclic suffix "abbc#" precedes the first cyclic suffix "abc#".

[0131] If the first cyclic suffix is ​​abc# and the second cyclic suffix is ​​abc#bc, since all characters in the first cyclic suffix are the same as the characters in the same position in the second cyclic suffix, it is necessary to compare the remaining characters of the second cyclic suffix with those of the first cyclic suffix. That is, the size relationship between bc and the first cyclic suffix abc#. Since the lexicographical order of the first character 'b' in bc is greater than that of the first character 'a' in abc, the lexicographical order of abc#bc is greater than that of abc#.

[0132] Thus, by comparing all the cyclic suffixes of abc with all the cyclic suffixes of the suffix array in Table 2 above, we can obtain the suffix array as shown in Table 3, thereby obtaining the F sequence.

[0133] Table 3

[0134] Number of lines F sequence suffix array L-sequence 1 # #abbc c 2 # #abc c 3 # #bbbca a 4 # #bcabc c 5 a a#bbbc c 6 a abbc# # 7 a abc# # 8 a abc#bc c 9 b bbbca# # 10 b bbc#a a 11 b bbca#b b 12 b bc#ab b 13 b bc#a a 14 b bc#bca a 15 b bca#bb b 16 b bcabc# # 17 c c#abb b 18 c c#ab b 19 c c#bcab b 20 c ca#bbb b 21 c cabc#b b

[0135] Optionally, when performing BWT transformation on the data in the database to obtain the F and L sequences, the frequency of each character in the L sequence at different positions can be directly counted to obtain the correspondence between the frequency of each character in the L sequence and the position in the L sequence, and this correspondence can be used as the objective function.

[0136] In the methods described above, each character requires storing a corresponding relationship. However, directly storing the character statistics of the L-sequence as the corresponding relationship could consume a significant amount of storage space, leading to substantial space overhead. Therefore, in some embodiments, a target function can be obtained by fitting the character statistics of the L-sequence. This way, only one target function needs to be stored to determine the frequency of characters in the target L-sequence, thus saving considerable storage space.

[0137] Optionally, the objective function can be a linear function or other forms of function. Here, we take a linear function as an example to illustrate the distance in this application embodiment.

[0138] In some embodiments, the distribution function of each character in the L-sequence can be determined based on the frequency of occurrence of each character; and the distribution function can be fitted based on error conditions to obtain the target function. This improves the accuracy of the target function.

[0139] Optionally, the objective function can be obtained through different fitting methods depending on different usage requirements.

[0140] In some embodiments, the mean squared error of the distribution function can be determined; the distribution function is fitted based on the mean squared error, and the distribution function that minimizes the mean squared error is taken as the objective function.

[0141] Optionally, the distribution function corresponding to character c can be in the form shown in Formula 1:

[0142] fun c (u[i])=k c *u[i]+b c Formula 1

[0143] Where, k c b represents the slope of a linear function. c This represents the intercept of the distribution function. Based on the mean squared error, the loss function of this distribution function is as shown in Formula 2:

[0144]

[0145] Where k represents the total number of rows in the L sequence, fun c (u[i]) represents the total number of occurrences of character c in the i-th row of the cutoff L sequence determined by the distribution function, and u'[i] represents the total number of occurrences of character c in the i-th row of the cutoff L sequence determined by the statistical results.

[0146] The loss function L is determined using the least squares method. c When the minimum value is reached, the corresponding distribution function Fun c k c and bc The value of is used to obtain the fitted current distribution function corresponding to character c, which is the objective function corresponding to character c.

[0147] In other embodiments, the distribution function corresponding to the character can be interpolated and segmented using greedy spline interpolation to improve the robustness of the fitting results.

[0148] For example, the slope of the sub-function between the starting node and the i-th node of the distribution function can be determined, where i is a positive integer, and each node of the distribution function corresponds to a position and the number of occurrences of a character in the L-sequence. If the slope is less than the slope threshold and the i-th node is not the ending node of the distribution function, let i = i + 1, and return to the step of determining the slope between the starting node and the i-th node of the distribution function. If the slope is greater than or equal to the slope threshold and the i-th node is not the ending node of the distribution function, take the (i-1)-th node as the ending node of the sub-function and take the i-th node as the starting node of the next sub-function, let i = i + 1, and return to the step of determining the slope between the starting node and the i-th node of the distribution function. All sub-functions are taken as the objective function.

[0149] When fitting a distribution function based on mean squared error, considering the randomness of suffix data, a certain character may appear frequently in one interval of the L-sequence, while appearing less frequently in another. For example, in Table 2 above, the character 'b' appears 0 times before row 9, but appears frequently after row 9. Furthermore, when the data in the database changes, even small changes can cause drastic changes in the fitting results. In this case, fitting a linear function based on the mean squared error would lead to poor accuracy. Therefore, in some embodiments, the above method can be used to achieve function fitting based on the greedy difference between Yantai and Hong Kong / Taiwan, which can improve the robustness of the objective function.

[0150] In the example above, the distribution function can be interpolated and segmented by changing the slope of the distribution function. If introducing the next node would cause a large change in the slope of the linear function (i.e., the slope is greater than or equal to the slope threshold), the function up to the current node can be taken as a sub-function, and the next node can be taken as the starting node of another sub-function. This can save storage space of the objective function while improving the accuracy of the objective function.

[0151] In some embodiments, a suffix tree can be constructed by traversing the suffix array in the database and then pruned. The F sequence can be replaced with the pruned suffix tree. In this way, the L sequence corresponding to the pruned suffix tree node can be backtracked to the ancestor node of that node for fitting, thereby reducing the space occupied by the fitting function.

[0152] In the above method, considering the correspondence between the F-series and the L-series, the F-series can be organized, and characters with lower frequencies in the L-series belonging to the same suffix tree node can be pushed to the ancestor node of that node and fitted together with the L-series at the ancestor node, thereby further compressing the space overhead of the objective function.

[0153] For example, the cyclic suffixes of each string in the database can be traversed to build a suffix tree corresponding to the database. The suffix tree includes multiple nodes, each node is used to indicate a character of the cyclic suffix in the database, and each node corresponds to a position interval in the L sequence. For any node in the character suffix tree, if the node meets the pruning condition, the node is pruned to obtain a pruned suffix tree. The F sequence is then replaced with the pruned suffix tree.

[0154] Since each node in the suffix tree corresponds to a character of a cyclic suffix in the database, for example, the suffix tree built using the cyclic suffixes shown in Table 3 can be constructed as follows: Figure 3 As shown. Combined with Figure 3 The first node of the suffix tree, excluding the root node, corresponds to the character # in the cyclic suffix (node ​​2 in the diagram), the second node corresponds to the character a in the cyclic suffix (node ​​3 in the diagram), the third node corresponds to the character b in the cyclic suffix (node ​​4 in the diagram), and the fourth node corresponds to the character c in the cyclic suffix (node ​​5 in the diagram). These four nodes correspond to cyclic suffixes whose first character is #, a, b, and c, respectively.

[0155] The suffix tree will be explained below using node 1 as an example. Node 1 includes two child nodes, corresponding to nodes 6 and 7 in the diagram. Node 6 corresponds to the character "ab", indicating that in a cyclic suffix where the first character is "#", the character after "#" is "ab". Node 7 corresponds to the character "b", indicating that in a cyclic suffix where the first character is "#", the character after "#" is "b". Each child node also includes grandchild nodes. For example, node 6 includes child nodes 14 and 15. Node 14 indicates a cyclic suffix where the first three characters are "#ab" and the next character is "bc" (i.e., the suffix "#abbc" in Table 3). Node 15 indicates a cyclic suffix where the first three characters are "#ab" and the next character is "c" (i.e., the suffix "#abc" in Table 3).

[0156] Since node 2 corresponds to all cyclic suffixes in the suffix array whose first character is #, that is, the first row to the first row of the suffix array, it can be considered that this node corresponds to a position interval of the L sequence, that is, the first row to the fourth row of the L sequence.

[0157] Optionally, the pruning conditions for nodes can be flexibly selected based on actual usage needs, such as the amount of data in the database and the size of the database storage space.

[0158] For example, pruning conditions include at least one of the following: the height of the suffix tree corresponding to the node is less than a height threshold, and the size of the position interval corresponding to the node is less than an interval threshold.

[0159] Here, the height of the suffix tree corresponding to the node refers to the height of the suffix tree in which the node is located, and the size of the position interval corresponding to the node refers to the size of the position interval of the L sequence corresponding to the node.

[0160] The following is based on Figure 3 The suffix tree shown illustrates the concepts of position interval size and maximum number of identical characters. Based on Table 3 and Figure 3 We know that node 2 in the suffix tree is #, which represents a cyclic suffix in the suffix array whose first character is #. This corresponds to rows 1 to 4 of Table 3, so the size of the position interval for node 2 is 4. The second node is a, which represents a cyclic suffix in the suffix array whose first character is a. This corresponds to rows 5 to 8 of Table 3, so the size of the position interval for node a is 4. The third node is b, which represents a cyclic suffix in the suffix array whose first character is b. This corresponds to rows 9 to 16 of Table 3, so the size of the position interval for node b is 8. The fourth node is c, which represents a cyclic suffix in the suffix array whose first character is c. This corresponds to rows 17 to 21 of Table 3, so the size of the position interval for node c is 5.

[0161] In addition, since node 2 of the suffix tree represents all cyclic suffixes whose first character is # in the suffix array, corresponding to rows 1 to 4 of Table 3, and based on Table 3, we know that the L sequence in rows 1 to 4 includes 3 characters c and 1 character a, then the maximum number of identical characters corresponding to node 2 is 3.

[0162] For example, the height threshold can be 3, and the interval threshold can be 6. Figure 3 Prune the suffix tree shown. Combined with... Figure 3 When pruning with a height threshold, the height of the suffix tree corresponding to nodes 6-13 is 3. Therefore, nodes 6-13 should be pruned, and all child nodes of nodes 6-13 should be deleted. When pruning with an interval threshold, the interval size of the position interval corresponding to nodes 2 and 3 is 4, the interval size of the position interval corresponding to node 4 is 8, and the interval size of the position interval corresponding to node 5 is 5. Except for node 4, nodes 2, 3, and 5 all meet the pruning conditions. Therefore, nodes 2, 3, and 5 should be pruned, and all child nodes of nodes 2, 3, and 5 should be deleted, resulting in the following: Figure 4 The pruned stylist shown. Among them, Figure 4 Node 6 corresponds to Figure 3 Node 10 in the middle, Figure 4 Node 7 in the middle corresponds to Figure 3 Node 11 in the middle.

[0163] In some embodiments, each node in the suffix tree and the pruned suffix tree can be accessed via (l c , l s It is represented in the form of a 5-tuple (start, end, cnt), where l c This refers to the set of child nodes that the current node points to, used to represent the relationships between nodes. s This refers to the set of L sequences corresponding to the node. `start` indicates the starting position of the character corresponding to the node in the cyclic suffix. For example, in the cyclic suffixes shown in Table 2, the starting position of `#` is 1, indicating that the first character of the cyclic suffix is ​​`#` starting from line 1; the starting position of `a` is 4, indicating that the first character of the cyclic suffix is ​​`a` starting from line 4; the starting position of `b` is 7, indicating that the first character of the cyclic suffix is ​​`b` starting from line 7; and the starting position of `c` is 14, indicating that the first character of the cyclic suffix is ​​`c` starting from line 14. `end` indicates the ending position of the character corresponding to the node in the cyclic suffix. For example, in the cyclic suffixes shown in Table 2, the ending position of `#` is 3, indicating that the first character of the cyclic suffix is ​​no longer `#` starting from line 4; the ending position of `a` is 6, indicating that the first character of the cyclic suffix is ​​no longer `a` starting from line 7; the ending position of `b` is 13, indicating that the first character of the cyclic suffix is ​​no longer `b` starting from line 14; and the ending position of `c` is 17, indicating that the first character of the cyclic suffix is ​​no longer `c` starting from line 17. cnt represents the lookup cardinality of the characters from the root node to that node in the cyclic suffix.

[0164] For example, with Figure 4 Taking node 4 as an example, this node can be represented as ({6,7}, [L9, ..., L16], 9, 16, 4), where {6,7} indicates that node 4 includes two child nodes, node 6 and node 7. [L9, ..., L16] represents the characters from row 9 to row 16 of the L sequence corresponding to node 4. 9 indicates that the starting position of the characters from the root node to node 4 (i.e., character b) in the suffix array is row 9; that is, starting from row 9 of the suffix array, the starting character of the cyclic suffix is ​​character b. 16 indicates that the ending position of the characters from the root node to node 4 (i.e., character b) in the suffix array is row 16; that is, starting from row 17 of the suffix array, the starting character of the cyclic suffix is ​​no longer character b. 4 indicates that the lookup cardinality of the characters from the root node to node 4 (i.e., character b) in the suffix array is 4, meaning that there are a total of 4 strings in the database containing the character b.

[0165] Because each node corresponds to l c Parameters and l s There are many parameters, therefore... Figure 4The diagram only schematically shows the start, end, and cnt parameters for each node.

[0166] In some embodiments, the objective function can be fitted from bottom to top by combining each node of the pruned suffix tree. The fitting conditions for the function can be flexibly selected based on actual usage requirements. For example, if the number of occurrences of the target character in the L sequence corresponding to a node is greater than or equal to a threshold, then the objective function corresponding to the target character is fitted at that node. If the number of occurrences of the L target character corresponding to a node is less than the threshold, then the objective function corresponding to the target character is pushed up to the node preceding that node for fitting.

[0167] Still using the cyclic suffixes described in Table 3 above Figure 4 Taking the pruned suffix tree shown as an example, the fitting process can be as follows: Figure 5 As shown.

[0168] Combination Figure 5 The lowest level nodes in this pruned suffix tree are nodes 2, 3, 6, 7, and 5, none of which have any child nodes.

[0169] Node 2 corresponds to rows 1 to 4 of the L-sequence. The characters in rows 1, 2, and 4 of the L-sequence are 'c', so we can fit the function corresponding to the character 'c'. However, the character in row 3 of the L-sequence is 'a', so we cannot fit the function. Therefore, we push this character to the previous node, that is, to the root node, for fitting.

[0170] Node 3 corresponds to rows 5 to 8 of the L-sequence. The characters in rows 5 and 8 of the L-sequence are 'c', which can be used to fit the function corresponding to the character 'c'. The characters in rows 6 and 7 of the L-sequence are '#', which can be used to fit the function corresponding to the character '#'.

[0171] Node 6 corresponds to rows 9 to 11 of the L-sequence. The characters in rows 9, 10, and 11 of the L-sequence are #, a, and b, respectively. None of them can be fitted by a function. Therefore, this character is pushed to the previous node, i.e., to node 4, for fitting.

[0172] Node 7 corresponds to rows 12 to 16 of the L-sequence. The characters in rows 12 and 15 of the L-sequence are 'b', so we can fit the function corresponding to the character 'b'. The characters in rows 13 and 14 of the L-sequence are 'a', so we can also fit the function. However, the character in row 16 of the L-sequence is '#', so we cannot fit the function. Therefore, we push this character to the previous node, i.e., to node 4, for fitting.

[0173] Node 5 corresponds to rows 17 to 21 of the L-sequence, where the character is 'b'. We can fit the function corresponding to the character 'b'.

[0174] Next, function fitting is performed on the character at node 4. As mentioned earlier, node 4 includes characters derived from nodes 6 and 7. The characters derived from node 6 are the character # from row 9 of sequence L, the character a from row 10, and the character b from row 11. The character derived from node 7 is the character # from row 16 of sequence L. At this point, a function can be fitted based on the characters # from row 9 and row 16 of sequence L. However, since characters a from row 10 and b from row 11 cannot be fitted, these two characters are pushed to the previous node, i.e., to the root node, for fitting.

[0175] Next, we perform function fitting on the characters at the root node. Based on the previous steps, the root node includes characters derived from nodes 2 and 4. The character derived from node 2 is character 'a' in the 3rd row of sequence L, and the characters derived from node 4 are character 'a' in the 10th row and character 'b' in the 11th row of sequence L. Therefore, we can fit the function corresponding to character 'a' based on the characters 'a' in the 3rd and 10th rows of sequence L.

[0176] Thus, based on the above process, the function fitting for all characters in the L-sequence was completed, and the target function was obtained.

[0177] In some embodiments, the child suffixes that the target suffix appears in the pruned suffix tree, and the length of the child suffixes, can be determined. If the length of the child suffix is ​​equal to the length of the target suffix, the query cardinality of the target data is determined based on the node parameters of the pruned suffix tree. If the target length is less than the length of the target suffix, the interval position of the L-sequence corresponding to the child suffix is ​​determined based on the node parameters of the pruned suffix tree, and the query cardinality of the target data is determined based on the interval position of the L-sequence corresponding to the child suffix.

[0178] If the target length equals the target suffix length, it means the target suffix appears completely in the pruned suffix tree. Combining this with the above description, since each node in the pruned suffix tree stores the cardinality of the characters formed from the root node to that node in the cyclic suffix (i.e., cnt as mentioned above), if the characters formed from the root node to a certain node in the pruned suffix tree are the same as the target suffix, then the cnt of that node can be used as the cardinality of the target suffix.

[0179] If the child length is not equal to the target suffix, it means the target suffix is ​​not fully present in the pruned suffix tree. In this case, it is still necessary to combine the F and L sequences to determine the query cardinality of the target data. As described above, each node in the pruned suffix tree stores the interval positions of the characters from the root node to that node in the F and L sequences, i.e., the start and end positions mentioned above, and the l... sTherefore, based on the node containing the longest child suffix in the pruned suffix tree, the interval position for cardinality lookup starting from the longest child suffix can be determined. This interval position is then used as the interval position of the L-sequence corresponding to the longest child suffix. Furthermore, based on the objective function and the interval position of the L-sequence corresponding to the longest child suffix, the interval position of the character preceding the longest child suffix in the target suffix within the F-sequence can be determined. This process is repeated recursively to estimate the cardinality of the target suffix. The specific recursive lookup process can be found in the above-described process for determining the lookup cardinality of the target data based on the objective function, and will not be elaborated upon here.

[0180] In some embodiments, the sub-suffixes in which the target suffix appears in the pruned suffix tree can be determined based on a binary search method.

[0181] For example, if the target suffix is ​​p and the length of the target suffix is ​​n, first determine the search range [low:high], where the initial value of low is 0 and the initial value of high is the length n of p.

[0182] Determine the intermediate value mid = (low + high + 1) / / 2, and check if the suffix p[mid: n] of p exists in the pruned suffix tree.

[0183] If it exists, it means there may be a longer matching suffix, so a longer match can be performed. Let high = mid, and query again, that is, return the determined intermediate value mid = (low + high + 1) / / 2, and check whether the suffix p[mid: n] of p exists in the pruned suffix tree.

[0184] If it does not exist, it means that p[mid:n] does not appear in the pruned suffix tree. We can reduce the matching length to search within a wider range. Let low = mid + 1, and perform the query again. That is, return the determined intermediate value mid = (low + high + 1) / / 2. This step is to determine whether the suffix p[mid:n] of p exists in the pruned suffix tree.

[0185] Repeat the above steps until low = mid, then p[low:n] is the longest child suffix of the target suffix in the pruned suffix tree.

[0186] For example, the database shown in Table 3 and Figure 4 Taking the pruned suffix tree shown as an example, if the target suffix is ​​cabc and the length n of the target suffix is ​​4, then the initial search range [low:high] = [0:4] is determined first.

[0187] Determine the intermediate value mid = (low + high + 1) / / 2 = (0 + 4 + 1) / / 2 = 2, and determine whether the suffix p[mid:n] = p[2:4] (i.e. abc) exists in the pruned suffix tree.

[0188] Combination Figure 4 It can be seen that the suffix abc does not exist in the pruned suffix tree. Let low = mid + 1 = 2 + 1 = 3, determine the intermediate value mid = (low + high + 1) / / 2 = (3 + 4 + 1) / / 2 = 4, and determine whether the suffix p[4:n] = p[4:4] (i.e. c) of p exists in the pruned suffix tree.

[0189] Combination Figure 4 It can be seen that the suffix c exists in the pruned suffix tree. Let high = mid = 4, determine the intermediate value mid = (low + high + 1) / / 2 = (2 + 4) / / 2 = 3, and determine whether the suffix p[mid: n] = p[3: 4] (i.e. bc) exists in the pruned suffix tree.

[0190] Combination Figure 4 It is known that the suffix bc exists in the pruned suffix tree. So let high = mid = 3. Since high = low = 3 at this time, we no longer need to determine the intermediate value mid. Instead, we directly take p[low:n] = p[3:4] = bc as the longest subsuffix that the target suffix appears in the pruned suffix tree.

[0191] After determining that bc is the longest child suffix appearing in the pruned suffix tree, the interval position for radix lookup starting from the longest child suffix can be determined based on the node containing the longest child suffix in the pruned suffix tree. The node corresponding to this longest child suffix is... Figure 4 Node 7 in the middle.

[0192] Based on the above, each node in the pruned suffix tree can be represented as (l c , l s Therefore, the query cardinality of the target suffix can be determined based on the quintuples corresponding to different nodes and the objective function.

[0193] For example, the l corresponding to the node s The parameter represents the set of L sequences corresponding to this node, therefore it can be based on the l of node 7. s Determine the interval position from which the radix lookup begins, starting from the longest child suffix. Alternatively, since the start and end parameters of the node indicate the start and end positions of the character corresponding to the node in the cyclic suffix, the position from which the radix lookup begins can also be determined based on the start and end parameters of node 7.

[0194] For example, in combination Figure 4 It can be seen that the start value of node 7 is 12 and the end value is 16. Therefore, 12-16 is taken as the interval position of the L-sequence corresponding to the longest sub-suffix. Based on the objective function and the interval position 12-16, the interval position corresponding to the character preceding the longest sub-suffix in the target suffix (i.e., character 'a') is determined. In this embodiment, it is assumed that the number of times character 'a' appears up to line 11 is 2, and the number of times character 'a' appears up to line 16 is 4, as determined by the objective function. Therefore, it can be considered that the character 'a' in the target suffix 'cabc' is the 3rd to 4th character 'a' in the F-sequence. Next, the node corresponding to character 'a' in the pruned suffix tree is queried, i.e., node 3. Combined with... Figure 4 We know that the start value for node 3 is 5 and the end value is 8. Therefore, we can determine that the interval position corresponding to the third character 'a' and the fourth character 'a' in the F sequence is row 7 to row 8. Then, by using the objective function, we can determine the number of occurrences of the character 'c' preceding the character 'a' of the target suffix in row 7 to row 8 of the L sequence. This gives us the number of occurrences of the target suffix 'cabc', which is the query cardinality of the target data.

[0195] In some embodiments, the modified string in the database can be determined, and the corresponding cyclic suffix of the modified string can be determined. A modified suffix tree corresponding to the modified string can be built, and the modified suffix tree is used to indicate the number of times each suffix appears in the modified string.

[0196] In the above embodiments, considering that the data in the database is not static and will undergo dynamic changes based on actual usage needs, such as data deletion and addition, the accuracy of the objective function will gradually decrease as the data in the database changes, leading to a reduction in the accuracy of the determined query cardinality. Therefore, the accuracy of the query cardinality determined by the objective function can be improved by modifying the suffix tree.

[0197] For example, when determining the query cardinality of target data, the first query cardinality can be the target data query cardinality determined based on the target suffix, the suffix array, and the target function, and the second query cardinality can be the correction parameter determined based on the target suffix and the modified suffix tree. Then, based on the change type of the modified string, the query cardinality of the target data is determined using the first and second query cardins.

[0198] Optionally, the modified string refers to the string deleted or added to the database after obtaining the objective function. When the modified string is a newly added string, since the objective function can determine the first query cardinality of the target data when the database does not contain the newly added string, and the modified suffix tree can determine the second query cardinality of the target data when the newly added string is present, the sum of the first and second query cardinities can be used as the query cardinality of the target data in the database.

[0199] Similarly, when changing a string to a deleted string, since the objective function can determine the first query cardinality of the target data in the database including the deleted string, and changing the suffix tree can determine the second query cardinality of the target data in the deleted string, the difference between the first query cardinality and the second query cardinality can be used as the query cardinality of the target data in the database.

[0200] In addition, in some embodiments, target conditions can be set, and it can be determined whether the target function needs to be redefined based on the target conditions.

[0201] For example, if the changed string meets the target condition, the target function can be redefined based on all strings in the database.

[0202] Optionally, the target conditions can be flexibly determined based on actual usage needs.

[0203] For example, the degree of change in the modified string can be determined based on the total number of characters in the modified string. If the total number of characters in the modified string exceeds a quantity threshold, the modification is considered significant. In this case, the target condition is that the total number of characters in the modified string exceeds the quantity threshold.

[0204] For another example, the degree of change of the modified string can be determined based on the space size of the modified suffix tree. If the space occupied by the modified suffix tree corresponding to the modified string is greater than a space threshold, the modified string is considered to have undergone a significant change. In this case, the target condition is the space size of the modified suffix tree corresponding to the modified string.

[0205] As another example, the degree of change to the changed string can be determined based on the accuracy of the query cardinality. If the accuracy of the query cardinality of the target data is lower than an accuracy threshold, the change to the changed string is considered significant. In this case, the objective condition is that the accuracy of the query cardinality determined based on the changed string and the objective function is lower than the accuracy threshold.

[0206] Since a string modification satisfying the target condition usually indicates a significant change in the characters within the database, and as described above, modifying the suffix tree in this case would consume considerable space and negatively impact the accuracy of the query cardinality. Therefore, when a string modification satisfies the target condition, the target function can be redefined based on all currently available strings in the database. This not only eliminates the need to modify the suffix tree but also improves the accuracy of the target function.

[0207] Optionally, the process of determining the objective function based on the strings in the database can refer to the above process of determining the objective function, which will not be repeated here.

[0208] In some embodiments, as described above, if the objective function requires the F-series of the database to determine the query cardinality, then the corresponding F-series of the database also needs to be updated when determining the objective function.

[0209] Optionally, if the F-series in the database is replaced with a pruned suffix tree, the pruned suffix tree can also be updated based on the changed strings in the database.

[0210] In this embodiment, based on the BWT algorithm, the distribution of characters in the L-sequence of the BWT is indicated by fitting an objective function. This eliminates the need to store the L-sequence; the query cardinality of the target data can be determined based on the F-sequence and the objective function, thus saving the space overhead of the L-sequence. To ensure the accuracy of the objective function, it can also be fitted based on error conditions, such as minimizing the mean squared error or using greedy spline interpolation to fit the distribution function and obtain the objective function, thereby improving its accuracy.

[0211] Additionally, a pruned suffix tree corresponding to the database can be constructed, and the F sequence can be replaced with the pruned suffix tree to achieve hierarchical organization of the L sequence. When fitting the objective function, the L node corresponding to the node in the suffix tree can be pushed to the ancestor node of that node for fitting, thereby further reducing the space overhead of the objective function.

[0212] However, considering that the strings in the database may still change dynamically after the objective function is determined, leading to a decrease in the accuracy of the objective function, a modified suffix tree can be constructed for the modified strings in the database. Then, based on the modified suffix tree, the cardinality correction parameter for the target data in the modified strings can be determined to correct the query cardinality determined based on the objective function, further improving the accuracy of the query cardinality.

[0213] This application embodiment can divide the query cardinality determination device into functional modules according to the above method example. For example, each function can be divided into its own functional modules, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. It should be noted that the module division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0214] When each function is divided into its own modules, the remaining modules are defined according to their respective functions. Figure 6 This diagram illustrates a possible structural schematic of the query cardinality determination device 600 involved in the above embodiments, as shown below. Figure 6 As shown, the radix determination device may include a suffix determination module 601 and a radix determination module 602.

[0215] The suffix determination module 601 is used to determine the target suffix of the target data. The target suffix is ​​the largest suffix among the suffix data generated based on the target data.

[0216] The cardinality determination module 602 is used to determine the query cardinality of the target data based on the target suffix, the suffix array, and the target function.

[0217] The suffix array is obtained based on the suffix data of strings in the database. The objective function is the cumulative distribution function of the characters in the suffix array, which is used to indicate the distribution of characters in the suffix array. The query cardinality is used to indicate the number of times the target data appears in the database.

[0218] Optionally, the suffix array includes the first column F, and the objective function is used to indicate the relationship between the number of occurrences of each character in the last column L and its position in the L column. The F and L columns are obtained by performing the Robes-Wyler Transform (BWT) on the strings in the database, and the characters in the F column correspond one-to-one with the characters in the L column.

[0219] Optionally, the cardinality determination module 602 can determine the interval position of the nth character of the target suffix in the F sequence. Based on the interval position of the nth character in the F sequence, the interval position of the corresponding L sequence is determined. If n is greater than 2, then based on the objective function and the interval position of the corresponding L sequence, the interval position of the (n-1)th character in the F sequence is determined, and n = n-1 is set, returning the step of determining the interval position of the corresponding L sequence based on the interval position of the nth character in the F sequence. If n is less than or equal to 2, then based on the objective function, the frequency of occurrence of the (n-1)th character in the interval position of the corresponding L sequence is determined, and the frequency of occurrence of the (n-th)th character is determined as the query cardinality of the target data.

[0220] Where n is a positive integer, and the initial value of n is the length of the target suffix.

[0221] In some possible implementations, the cardinality determination module 602 is also used to determine the distribution function of each character in the L-sequence based on the frequency of occurrence of each character. Based on the error conditions, the distribution function is fitted to obtain the target function.

[0222] The distribution function is used to indicate the relationship between the frequency of a character and its position in the L-sequence.

[0223] Optionally, the cardinality determination module 602 can determine the mean squared error of the distribution function. The distribution function is fitted based on the mean squared error, and the distribution function that minimizes the mean squared error is taken as the objective function.

[0224] Optionally, the cardinality determination module 602 can determine the slope of the sub-function between the starting node and the i-th node of the distribution function. If the slope is less than a slope threshold and the i-th node is not the ending node of the distribution function, let i = i + 1, and return to the step of determining the slope between the starting node and the i-th node of the distribution function. If the slope is greater than or equal to the slope threshold and the i-th node is not the ending node of the distribution function, take the (i-1)-th node as the ending node of the sub-function, and take the i-th node as the starting node of the next sub-function, let i = i + 1, and return to the step of determining the slope between the starting node and the i-th node of the distribution function. All sub-functions are used as the objective function.

[0225] Where i is a positive integer, and each node of the distribution function corresponds to a position in the L sequence and the number of times a character appears.

[0226] In some possible implementations, the cardinality determination module 602 can also traverse the cyclic suffixes of each string in the database to build a corresponding suffix tree. For any node in the character suffix tree, if the node satisfies the pruning condition, the node is pruned to obtain a pruned suffix tree. The F sequence is then replaced with the pruned suffix tree.

[0227] The suffix tree consists of multiple nodes, each node indicating a character of a cyclic suffix in the database, and each node corresponds to a position interval in the L-sequence.

[0228] Optionally, the pruning conditions include at least one of the following: the height of the suffix tree corresponding to the node is less than the height threshold, and the size of the position interval corresponding to the node is less than the interval threshold; the height of the suffix tree corresponding to the node refers to the height of the suffix tree in which the node is located, and the interval size is used to indicate the size of the L-sequence position interval corresponding to the node.

[0229] Optionally, the cardinality determination module 602 can determine the longest child suffix appearing in the pruned suffix tree for the target suffix, and the target length of the longest child suffix. If the target length is equal to the length of the target suffix, the query cardinality of the target data is determined based on the node parameters of the pruned suffix tree. If the target length is less than the length of the target suffix, the interval position of the L-sequence corresponding to the longest child suffix is ​​determined based on the node parameters of the pruned suffix tree, and the query cardinality of the target data is determined based on the interval position of the L-sequence corresponding to the longest child suffix.

[0230] In some possible implementations, the database includes multiple strings. The cardinality determination module 602 can also determine at least one cyclic suffix for each of the multiple strings, resulting in multiple cyclic suffixes. If all characters in the first cyclic suffix are different from the characters at the same positions in the second cyclic suffix, the order between the first and second cyclic suffixes is determined based on lexicographical order, with the length of the second cyclic suffix being greater than the length of the first cyclic suffix. If all characters in the first cyclic suffix are the same as the characters at the same positions in the second cyclic suffix, the order between the first and second cyclic suffixes is determined based on the remaining characters of the first and second cyclic suffixes compared to the first cyclic suffix. The multiple cyclic suffixes are sorted based on their order to obtain the F-sequence.

[0231] In some possible implementations, the cardinality determination module 602 can also determine the modified strings in the database. Based on the cyclic suffixes corresponding to the modified strings, a modified suffix tree corresponding to the modified strings is determined. The query cardinality of the target data is then adjusted based on the modified suffix tree.

[0232] Among them, the changed string refers to the string that has changed in the database after the target function is obtained, and the changed suffix tree is used to indicate the number of times each suffix appears in the changed string.

[0233] Optionally, the modified string includes adding and / or deleting strings. The cardinality determination module 602 can also determine the cardinality correction parameter for the target data based on the target suffix and the modified suffix tree. When the modified string is an added string, the sum of the cardinality correction parameter and the query cardinality is used as the query cardinality of the target data. When the modified string is a deleted string, the difference between the cardinality correction parameter and the query cardinality is used as the query cardinality of the target data.

[0234] Optionally, the cardinality determination module 602 can also redetermine the target function based on all strings in the database, provided that the changed string meets the target conditions.

[0235] In this embodiment, based on the BWT algorithm, the distribution of characters in the L-sequence of the BWT is indicated by fitting an objective function. This eliminates the need to store the L-sequence; the query cardinality of the target data can be determined based on the F-sequence and the objective function, thus saving the space overhead of the L-sequence. To ensure the accuracy of the objective function, it can also be fitted based on error conditions, such as minimizing the mean squared error or using adventure spline interpolation to fit the distribution function and obtain the objective function, thereby improving the accuracy of the objective function.

[0236] Additionally, a pruned suffix tree corresponding to the database can be constructed, and the F sequence can be replaced with the pruned suffix tree to achieve hierarchical organization of the L sequence. When fitting the objective function, the L node corresponding to the node in the suffix tree can be pushed to the ancestor node of that node for fitting, thereby further reducing the space overhead of the objective function.

[0237] However, considering that the strings in the database may still change dynamically after the objective function is determined, leading to a decrease in the accuracy of the objective function, a modified suffix tree can be constructed for the modified strings in the database. Then, based on the modified suffix tree, the cardinality correction parameter for the target data in the modified strings can be determined to correct the query cardinality determined based on the objective function, further improving the accuracy of the query cardinality.

[0238] In other examples, Figure 7 A schematic block diagram of a query cardinality determination device 700 according to an embodiment of this application is shown. The query cardinality determination device 700 may include a processor 701 and a transceiver / transceiver pin 702, and optionally, a memory 703. The processor 701 can be used to execute the steps performed by the query cardinality determination device in the methods of the foregoing embodiments, and control the receive pin to receive signals, and control the transmit pin to transmit signals.

[0239] The query base determines that the various components of device 700 are coupled together via bus 704, which includes a data bus, a power bus, a control bus, and a status signal bus. However, for clarity, all buses are labeled as bus system 704 in the figure.

[0240] Optionally, the memory 703 can be used for storage instructions in the foregoing method embodiments.

[0241] It should be understood that the query base determination device 700 according to the embodiments of this application may correspond to the first device in the methods of the foregoing embodiments, and the above and other management operations and / or functions of each element in the query base determination device 700 are respectively for implementing the corresponding steps of the foregoing methods, which will not be described in detail here for the sake of brevity.

[0242] All relevant content of each step involved in the above method embodiments can be referenced from the functional description of the corresponding functional module, and will not be repeated here.

[0243] Based on the same technical concept, embodiments of this application also provide a computer-readable storage medium storing a computer program containing at least one piece of code that can be executed by a query cardinality determination device to control the query cardinality determination device to implement the above-described method embodiments.

[0244] Based on the same technical concept, this application also provides a computer program, which, when executed by a query base determination device, is used to implement the above-described method embodiments.

[0245] The program may be stored, in whole or in part, on a storage medium packaged with the processor, or in part or in whole on a memory not packaged with the processor.

[0246] Based on the same technical concept, this application also provides a processor for implementing the above-described method embodiments. The processor can be a chip.

[0247] The steps of the methods or algorithms described in conjunction with the embodiments of this application can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium well known in the art. An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and storage medium can reside in an ASIC. Additionally, the ASIC can reside in a network device. Alternatively, the processor and storage medium can exist as discrete components in the network device.

[0248] Those skilled in the art will recognize that the functions described in the embodiments of this application in one or more of the above examples can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any medium that facilitates the transfer of a computer program from one place to another. Storage media can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0249] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A method for determining the cardinality of a query, characterized in that, include: Determine the target suffix of the target data, wherein the target suffix is ​​the largest suffix among the suffix data generated based on the target data; Based on the target suffix, suffix array, and objective function, the query cardinality of the target data is determined. The suffix array is obtained based on the suffix data of strings in the database. The objective function is the character cumulative distribution function of the suffix array, used to indicate the distribution of characters in the suffix array. The query cardinality is used to indicate the number of times the target data appears in the database.

2. The method according to claim 1, characterized in that, The suffix array includes a first column F, and the objective function is used to indicate the relationship between the number of occurrences of each character in the last column L and its position in the L column. The F column and the L column are obtained by performing Robes Wheeler Transform (BWT) on the strings in the database, and the characters in the F column correspond one-to-one with the characters in the L column.

3. The method according to claim 2, characterized in that, The step of determining the query cardinality of the target data based on the target suffix, the suffix array, and the target function includes: Determine the position of the nth character of the target suffix within the interval of the F sequence, where n is a positive integer and its initial value is the length of the target suffix; Based on the interval position of the nth character in the F sequence, determine the interval position of the L sequence corresponding to the nth character; If n is greater than 2, then based on the objective function and the interval position of the L sequence corresponding to the nth character, determine the interval position of the (n-1)th character in the F sequence, let n = n-1, and return to the step of determining the interval position of the L sequence corresponding to the nth character based on the interval position of the nth character in the F sequence; If n is less than or equal to 2, then based on the objective function, determine the occurrence count of the (n-1)th character in the interval position of the L sequence corresponding to the nth character, and determine the occurrence count of the (n-th)th character as the query cardinality of the objective data.

4. The method according to claim 2, characterized in that, Before determining the query cardinality of the target data based on the target suffix, the suffix array, and the target function, the method further includes: Based on the frequency of occurrence of each character in the L-sequence, a distribution function for each character in the L-sequence is determined. The distribution function is used to indicate the relationship between the frequency of occurrence of a character and its position in the L-sequence. Based on the error condition, the distribution function is fitted to obtain the objective function.

5. The method according to claim 4, characterized in that, The process of fitting the distribution function based on the error condition to obtain the objective function includes: Determine the mean square error of the distribution function; The distribution function is fitted based on the mean square error, and the distribution function with the minimum mean square error is taken as the objective function.

6. The method according to claim 4, characterized in that, The process of fitting the distribution function based on the error condition to obtain the objective function includes: Determine the slope of the sub-function between the starting node and the i-th node of the distribution function, where i is a positive integer, and each node of the distribution function corresponds to a position and the number of occurrences of a character in the L-sequence; If the slope is less than the slope threshold and the i-th node is not the end node of the distribution function, let i = i + 1, and return to the step of determining the slope between the starting node and the i-th node of the distribution function; If the slope is greater than or equal to the slope threshold and the i-th node is not the end node of the distribution function, the (i-1)-th node is taken as the end node of the sub-function and the i-th node is taken as the start node of the next sub-function. Let i = i + 1, and return to the step of determining the slope between the start node of the distribution function and the i-th node. Use all sub-functions as the target function.

7. The method according to claim 2, characterized in that, The method further includes: Traverse the cyclic suffixes of each string in the database and build a suffix tree corresponding to the database. The suffix tree includes multiple nodes, each node is used to indicate a character of the cyclic suffix in the database, and each node corresponds to a position interval in the L sequence. For any node in the character suffix tree, if the node satisfies the pruning condition, the node is pruned to obtain a pruned suffix tree; Replace the F sequence with the pruned suffix tree.

8. The method according to claim 7, characterized in that, The pruning conditions include at least one of the following: the height of the suffix tree corresponding to the node is less than the height threshold, and the size of the position interval corresponding to the node is less than the interval threshold; the height of the suffix tree corresponding to the node refers to the height of the suffix tree to which the node is located, and the interval size is used to indicate the size of the L-sequence position interval corresponding to the node.

9. The method according to claim 7 or 8, characterized in that, The step of determining the query cardinality of the target data based on the target suffix, the suffix array, and the target function includes: Determine the longest child suffix that appears in the pruned suffix tree, and the target length of the longest child suffix; If the target length is equal to the target suffix length, then the query cardinality of the target data is determined based on the node parameters of the pruned suffix tree; If the target length is less than the target suffix length, then based on the node parameters of the pruned suffix tree, the interval position of the L sequence corresponding to the longest child suffix is ​​determined, and based on the interval position of the L sequence corresponding to the longest child suffix, the query cardinality of the target data is determined.

10. The method according to claim 2, characterized in that, The database includes multiple strings, and the method further includes: Determine at least one cyclic suffix corresponding to each of the plurality of strings to obtain a plurality of cyclic suffixes; If all characters in the first cyclic suffix are different from the characters in the same position in the second cyclic suffix, the order between the first cyclic suffix and the second cyclic suffix is ​​determined based on lexicographical order, and the length of the second cyclic suffix is ​​greater than the length of the first cyclic suffix. If all characters in the first cyclic suffix are the same as the characters in the same position in the second cyclic suffix, then the order between the first cyclic suffix and the second cyclic suffix is ​​determined based on the remaining characters of the first cyclic suffix compared to the first cyclic suffix. The multiple cyclic suffixes are sorted according to their order to obtain the F sequence.

11. The method according to claim 3, characterized in that, The method further includes: Determine the changed strings in the database, where the changed strings refer to the strings in the database that have changed after the target function is obtained; Based on the cyclic suffixes corresponding to the changed string, a change suffix tree corresponding to the changed string is determined, and the change suffix tree is used to indicate the number of times each suffix appears in the changed string; The query cardinality of the target data is corrected based on the modified suffix tree.

12. The method according to claim 11, characterized in that, The modified string includes newly added strings and / or deleted strings; the step of correcting the query cardinality of the target data based on the modified suffix tree includes: Based on the target suffix and the modified suffix tree, determine the cardinality correction parameter of the target data; When the changed string is replaced with a new string, the sum of the cardinality correction parameter and the query cardinality is used as the query cardinality of the target data; When the changed string is a deleted string, the difference between the cardinality correction parameter and the query cardinality is used as the query cardinality of the target data.

13. The method according to claim 11, characterized in that, The method further includes: If the changed string meets the target conditions, the target function is redefined based on all strings in the database.

14. A query cardinality determination device, characterized in that, include: A suffix determination module is used to determine the target suffix of the target data, wherein the target suffix is ​​the largest suffix among the suffix data generated based on the target data; The cardinality determination module is used to determine the query cardinality of the target data based on the target suffix, the suffix array, and the objective function. The suffix array is obtained based on the suffix data of strings in the database. The objective function is the character cumulative distribution function of the suffix array, which is used to indicate the distribution of characters in the suffix array. The query cardinality is used to indicate the number of times the target data appears in the database.

15. The apparatus according to claim 14, characterized in that, The suffix array includes a first column F, and the objective function is used to indicate the relationship between the number of occurrences of each character in the last column L and its position in the L column. The F column and the L column are obtained by performing Robes Wheeler Transform (BWT) on the strings in the database, and the characters in the F column correspond one-to-one with the characters in the L column.

16. The apparatus according to claim 15, characterized in that, The cardinality determination module is specifically used for: Determine the position of the nth character of the target suffix within the interval of the F sequence, where n is a positive integer and its initial value is the length of the target suffix; Based on the interval position of the nth character in the F sequence, determine the interval position of the L sequence corresponding to the nth character; If n is greater than 2, then based on the objective function and the interval position of the L sequence corresponding to the nth character, determine the interval position of the (n-1)th character in the F sequence, let n = n-1, and return to the step of determining the interval position of the L sequence corresponding to the nth character based on the interval position of the nth character in the F sequence; If n is less than or equal to 2, then based on the objective function, determine the occurrence count of the (n-1)th character in the interval position of the L sequence corresponding to the nth character, and determine the occurrence count of the (n-th)th character as the query cardinality of the objective data.

17. The apparatus according to claim 15, characterized in that, The cardinality determination module is further configured to: Based on the frequency of occurrence of each character in the L-sequence, a distribution function for each character in the L-sequence is determined. The distribution function is used to indicate the relationship between the frequency of occurrence of a character and its position in the L-sequence. Based on the error condition, the distribution function is fitted to obtain the objective function.

18. The apparatus according to claim 17, characterized in that, The cardinality determination module is specifically used for: Determine the mean square error of the distribution function; The distribution function is fitted based on the mean square error, and the distribution function with the minimum mean square error is taken as the objective function.

19. The apparatus according to claim 17, characterized in that, The cardinality determination module is specifically used for: Determine the slope of the sub-function between the starting node and the i-th node of the distribution function, where i is a positive integer, and each node of the distribution function corresponds to a position and the number of occurrences of a character in the L-sequence; If the slope is less than the slope threshold and the i-th node is not the end node of the distribution function, let i = i + 1, and return to the step of determining the slope between the starting node and the i-th node of the distribution function; If the slope is greater than or equal to the slope threshold and the i-th node is not the end node of the distribution function, the (i-1)-th node is taken as the end node of the sub-function and the i-th node is taken as the start node of the next sub-function. Let i = i + 1, and return to the step of determining the slope between the start node of the distribution function and the i-th node. Use all sub-functions as the target function.

20. The apparatus according to claim 15, characterized in that, The cardinality determination module is further configured to: Traverse the cyclic suffixes of each string in the database and build a suffix tree corresponding to the database. The suffix tree includes multiple nodes, each node is used to indicate a character of the cyclic suffix in the database, and each node corresponds to a position interval in the L sequence. For any node in the character suffix tree, if the node satisfies the pruning condition, the node is pruned to obtain a pruned suffix tree; Replace the F sequence with the pruned suffix tree.

21. The apparatus according to claim 20, characterized in that, The pruning conditions include at least one of the following: the height of the suffix tree corresponding to the node is less than the height threshold, and the size of the position interval corresponding to the node is less than the interval threshold; the height of the suffix tree corresponding to the node refers to the height of the suffix tree to which the node is located, and the interval size is used to indicate the size of the L-sequence position interval corresponding to the node.

22. The apparatus according to claim 20 or 21, characterized in that, The cardinality determination module is specifically used for: Determine the longest child suffix that appears in the pruned suffix tree, and the target length of the longest child suffix; If the target length is equal to the target suffix length, then the query cardinality of the target data is determined based on the node parameters of the pruned suffix tree; If the target length is less than the target suffix length, then based on the node parameters of the pruned suffix tree, the interval position of the L sequence corresponding to the longest child suffix is ​​determined, and based on the interval position of the L sequence corresponding to the longest child suffix, the query cardinality of the target data is determined.

23. The apparatus according to claim 15, characterized in that, The database includes multiple strings, and the cardinality determination module is further used for: Determine at least one cyclic suffix corresponding to each of the plurality of strings to obtain a plurality of cyclic suffixes; If all characters in the first cyclic suffix are different from the characters in the same position in the second cyclic suffix, the order between the first cyclic suffix and the second cyclic suffix is ​​determined based on lexicographical order, and the length of the second cyclic suffix is ​​greater than the length of the first cyclic suffix. If all characters in the first cyclic suffix are the same as the characters in the same position in the second cyclic suffix, then the order between the first cyclic suffix and the second cyclic suffix is ​​determined based on the remaining characters of the first cyclic suffix compared to the first cyclic suffix. The multiple cyclic suffixes are sorted according to their order to obtain the F sequence.

24. The apparatus according to claim 16, characterized in that, The cardinality determination module is further configured to: Determine the changed strings in the database, where the changed strings refer to the strings in the database that have changed after the target function is obtained; Based on the cyclic suffixes corresponding to the changed string, a change suffix tree corresponding to the changed string is determined, and the change suffix tree is used to indicate the number of times each suffix appears in the changed string; The query cardinality of the target data is corrected based on the modified suffix tree.

25. The apparatus according to claim 24, characterized in that, The modified string includes newly added strings and / or deleted strings; the cardinality determination module is also used for: Based on the target suffix and the modified suffix tree, determine the cardinality correction parameter of the target data; When the changed string is replaced with a new string, the sum of the cardinality correction parameter and the query cardinality is used as the query cardinality of the target data; When the changed string is a deleted string, the difference between the cardinality correction parameter and the query cardinality is used as the query cardinality of the target data.

26. The apparatus according to claim 24, characterized in that, The cardinality determination module is further configured to: If the changed string meets the target conditions, the target function is redefined based on all strings in the database.

27. A query cardinality determination device, characterized in that, include: One or more processors; Memory; And one or more computer programs, wherein the one or more computer programs are stored on the memory, and when the computer programs are executed by the one or more processors, cause the apparatus to perform the method as described in any one of claims 1-13.

28. A computer storage medium, characterized in that, Includes computer instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in any one of claims 1-13.

29. A computer program product, characterized in that, When the computer program product is run on a computer, it causes the computer to perform the method as described in any one of claims 1-13.

Images