Data query method, device, equipment, storage medium and program product

Abstract

The present application discloses a data query method, device, equipment, storage medium and program product, and relates to the field of data processing technology. The method comprises: reading a preset configuration text, determining a target database table and a target field; performing a decimal conversion on a target field content text of each target database table to obtain a first content text, storing it in a first high-efficiency bitmap to form a second high-efficiency bitmap, counting the number of counts of each character in the target field and storing it in association with the database table, and sending the second high-efficiency bitmap to a server; receiving and parsing an external operation instruction, obtaining the target character, querying the second high-efficiency bitmap corresponding to the target character in the server to obtain the target high-efficiency bitmap; determining a final target database table based on the target high-efficiency bitmap; using an nl2sql neural network model to convert the operation instruction into a query statement, querying the final target database table to obtain the target data. The present application can improve the efficiency of data query.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data query method, device, equipment, storage medium and program product. Background Art

[0002] In the reasoning process of the neural network model nl2sql (Natural Language to SQL), we need to deeply parse and disassemble the question sentence input by the user. This step is crucial because it will determine how we understand the user's query intent and help us generate accurate SQL (Structured Query Language) statements.

[0003] Specifically, the question sentence entered by the user needs to be broken down into words or characters, and each word or character will be regarded as an independent target. These targets will be queried on all business data tables in the database. This step is to analyze the candidate tables that the generative neural network needs to anchor so that the corresponding SQL can be generated later. After the full table scan query, an SQL statement will be generated based on these query results and the prediction of the neural network model. This SQL statement will accurately reflect the user's query intention and can be directly used for database query operations.

[0004] However, this method has certain defects. The questions asked by users are not all simple questions. They may contain complex logical structures such as multiple conditions and multiple sub-queries. That is, the complexity of the questions is relatively high, which will lead to the need to associate and query multiple business tables. Therefore, as the complexity of the questions increases and the data in the business tables grows, the overall query efficiency will decrease.

[0005] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0006] The main purpose of this application is to provide a data query method, device, equipment, storage medium and program product, aiming to solve the technical problem of low efficiency of data query through the nl2sql neural network model.

[0007] To achieve the above purpose, the present application provides a data query method, comprising the following steps:

[0008] Reading a preset configuration text, and determining at least one target database table and a target field in the target database table based on the configuration text;

[0009] For each target database table, determine the content text corresponding to all target fields in the target database table, convert the content text into a first content text in decimal format, and store the first content text in a preset first high-efficiency bitmap to obtain a second high-efficiency bitmap, determine the count quantity corresponding to each word in the target field, and associate the count quantity with the target database table for storage, wherein the content text includes each word in the target field;

[0010] If an operation instruction input from an external source is received, the operation instruction is parsed to obtain a target text, and based on the target text, a target efficient bitmap containing the target text is searched in each second efficient bitmap;

[0011] Determine at least one first target database table that matches the target efficient bitmap among all target database tables, determine the number of matching counts that match the target text among the number of counts associated with the at least one first target database table, and determine a final target database table in the at least one first target database table according to the number of matching counts;

[0012] Based on the preset nl2sql neural network model, the operation instructions are converted into query statements, and the query statements are used to query the final target database to obtain the target data.

[0013] In one embodiment, the step of converting the content text into a first content text in a decimal format and storing the first content text in a preset first efficient bitmap to obtain a second efficient bitmap includes:

[0014] Traversing each character in the content text, performing decimal encoding on each character, and obtaining a first content text in decimal format, wherein the first content text includes a single item text obtained by performing decimal encoding on each character;

[0015] Each single item of text in the first content text is traversed, and the single item of text is compressed and stored in the first efficient bitmap to obtain a second efficient bitmap.

[0016] In one embodiment, after the step of compressing the single-item text and storing it in the first efficient bitmap, the method further comprises:

[0017] Determine the compressed single-item text stored in the first efficient bitmap;

[0018] If the single item text is consistent with the stored compressed single item text, the number of occurrences of the stored compressed single item text is updated, wherein the number of occurrences update includes increasing the number of occurrences by a preset value;

[0019] If the single-item text is inconsistent with the stored compressed single-item text, the single-item text is used as the new stored compressed single-item text, and the number of occurrences of the new stored compressed single-item text is set to a preset value.

[0020] In one embodiment, the step of determining the number of counts corresponding to each word in the target field includes:

[0021] For each word in the target field, the number of occurrences of the single item of text corresponding to the word in the second efficient bitmap is determined as the count number of the word.

[0022] In one embodiment, the steps of determining at least one first target database table that matches the target efficient bitmap among all target database tables, determining the number of matching counts that match the target text among the number of counts associated with the at least one first target database table, and determining the final target database table in the at least one first target database table according to the number of matching counts include:

[0023] Obtaining a bitmap sequence number of a target efficient bitmap, and searching among all target database tables for at least one first target database table corresponding to the bitmap sequence number;

[0024] For each first target database table, determine the count quantity corresponding to the first target database table from the count quantities corresponding to the multiple target database tables stored in association, and determine the matching count quantity corresponding to the target text from the count quantities corresponding to the first target database table;

[0025] Counting the probability of the target word existing in each first target database table based on the number of matching counts;

[0026] The first target database table with the greatest existence probability among all the first target database tables is determined as the final target database table.

[0027] In one embodiment, the steps of receiving an external input operation instruction, parsing the operation instruction, and obtaining the target text include:

[0028] Calling a preset web page interface according to a preset nl2sql neural network model to receive an external input operation instruction, wherein the operation instruction includes a natural language sentence;

[0029] Decompose natural language sentences into individual characters to obtain the target characters.

[0030] In addition, to achieve the above purpose, the present application also proposes a data query device, which includes:

[0031] A configuration reading module, used to read a preset configuration text, and determine at least one target database table and a target field in the target database table based on the configuration text;

[0032] A data storage module is used to determine, for each target database table, content texts corresponding to all target fields in the target database table, convert the content texts into first content texts in decimal format, store the first content texts in a preset first efficient bitmap, obtain a second efficient bitmap, determine the count quantity corresponding to each word in the target field, and associate the count quantity with the target database table for storage, wherein the content text includes each word in the target field;

[0033] The data query module is used to parse the operation instruction upon receiving an external input operation instruction, obtain the target text, and based on the target text, query the target high-efficiency bitmap containing the target text in each second high-efficiency bitmap; determine at least one first target database table that matches the target high-efficiency bitmap in all target database tables, determine the number of matching counts that match the target text in the number of counts associated with at least one first target database table, and determine the final target database table in at least one first target database table based on the number of matching counts; based on a preset nl2sql neural network model, convert the operation instruction into a query statement, use the query statement to query the final target database table, and obtain the target data.

[0034] In addition, to achieve the above objectives, the present application also proposes a data query device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the data query method as described above.

[0035] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the data query method as described above are implemented.

[0036] In addition, to achieve the above-mentioned purpose, the present application also provides a program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the data query method as described above are implemented.

[0037] In the present application, by reading the preset configuration text, the target database table and the target field in the target database table are determined based on the configuration text, thereby obtaining the database table and the fields in the database table required for business queries. The target fields in each target database table are traversed to obtain the content text corresponding to the target field (the content text includes each character in the target field), the content text is converted into a decimal format, the first content text is obtained, the first content text is stored in a preset first efficient bitmap, and the second efficient bitmap is obtained. Using an efficient bitmap to store data can reduce memory usage, thereby improving data access efficiency. The count quantity corresponding to each character in the target field is determined, and the count quantity is associated with the target database table for storage, so that the final target database table can be determined based on the count quantity later. Receive an operation instruction input from the outside, parse the operation instruction, obtain the target text, query the target efficient bitmap containing the target text in each second efficient bitmap based on the target text, determine at least one first target database table that matches all target efficient bitmaps, determine the number of matching counts that match the target text in the number of counts associated with at least one first target database table, and determine the final target database table in at least one target database table according to the number of matching counts, determine the final target database table through the target efficient bitmap and the number of matching counts, optimize the data access process, avoid direct access to multiple target database tables, and cause IO (Input / Output, input / output, refers to the input and output operations of data in a computer system) congestion, and only need to query in one final target database to obtain the target data. Based on the preset nl2sql neural network model, convert the operation instruction into a query statement, use the query statement to query the target data in the final target database, and only need to access the final target database once to obtain the target data, which further improves the efficiency of data query through the nl2sql neural network model. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0039] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0040] Figure 1 A schematic diagram of a flow chart provided for the first embodiment of the data query method of the present application;

[0041] Figure 2This is a flow chart of the traditional data query method of this application;

[0042] Figure 3 This is a flowchart of the data query method for this application;

[0043] Figure 4 This is a schematic diagram of the module structure of the data query device according to an embodiment of the present application;

[0044] Figure 5 Schematic diagram of the device structure of the hardware operating environment involved in the data query method in the embodiment of the present application.

[0045] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0046] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0047] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0048] In this embodiment, for the convenience of description, the identification terminal is used as the execution subject for explanation below.

[0049] In the embodiment of the present application, by reading a preset configuration text, the target database table and the target field in the target database table are determined based on the configuration text, thereby obtaining the database table and the fields in the database table required for business query. The target field in the target database table is traversed to obtain the content text corresponding to the target field (the content text includes each character in the target field), each character in the content text is converted into a decimal format, and a first content text is obtained (the first content text includes a single text obtained by decimal encoding each character), the first content text and the content text are different formats representing the same data, and the single text in the first content text is stored in a preset first high-efficiency bitmap in turn to obtain a second high-efficiency bitmap. Using a high-efficiency bitmap to store data can reduce memory usage, thereby improving data access efficiency. In addition, after each single text is stored in the first high-efficiency bitmap, it is determined that the compressed single text has been stored in the first high-efficiency bitmap, and it is determined whether the single text is consistent with the compressed single text. If so, the number of occurrences of the compressed single text is updated. If not, the single text is used as a new compressed single text that has been stored, and the number of occurrences of the new compressed single text that has been stored is set to a preset value. The number of occurrences of the single text corresponding to each word in the target field is determined as the count number, and the count number is associated and stored with the target database table, so as to facilitate the subsequent determination of the final target database table based on the count number. If an operation instruction input from the outside is received, the operation instruction is parsed to obtain the target word, and based on the target word, the target efficient bitmap containing the target word is queried in each second efficient bitmap, so as to determine at least one first target database table matching the target efficient bitmap, and obtain the first target database table, and the matching count number of the target word matching is determined in the first database table, and the existence probability of the target word in each first target database table is counted by matching the count number, so as to determine that the first target database table with the largest existence probability is the final target database table, and the method of determining the final target database table by the efficient bitmap and the count number can optimize the data access process, and avoid directly accessing multiple target database tables when the access volume increases, causing IO congestion. Based on the preset nl2sql neural network model, the operation instruction is inferred into a query statement, and the query statement is used to query the target data in the final target database. Only one access to the final target database is required to obtain the target data, which further improves the data query efficiency.

[0050] It should be noted that the execution subject of this embodiment may be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device, a terminal, etc. capable of realizing the above functions. The following takes the terminal as an example to illustrate this embodiment and the following embodiments.

[0051] Based on this, the present application embodiment provides a data query method, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the data query method of the present application.

[0052] In this embodiment, the data query method includes steps S10 to S50:

[0053] Step S10, reading a preset configuration text, and determining at least one target database table and a target field in the target database table based on the configuration text;

[0054] It should be noted that the preset configuration text refers to a file that stores database-related information, such as a text file. The file contains definitions of the database to be accessed, the target database table and the target field, as well as which database tables and fields need to be ignored. The target database table is the database table to be accessed defined in the configuration text, and there may be multiple target database tables. The target field is the field to be accessed defined in the configuration text, and there may be multiple target fields.

[0055] Access and parse the preset configuration text through code to obtain the target database tables and fields that need to be accessed and the database tables and fields that do not need to be accessed.

[0056] Optionally, it should be noted that reading refers to the process of a program accessing and parsing a configuration text through code to extract defined configuration information. The target fields are columns in a target database table, which represent specific attributes or data types recorded in the target database table.

[0057] In a feasible implementation manner, before the step of reading the preset configuration text in step S10 and determining the target database table and the target field based on the configuration text, the step A11 is also included:

[0058] Step A11, setting a scheduled task, executing step S10 to read a preset configuration text within a preset time interval, and determining a target database table and a target field based on the configuration text.

[0059] It should be noted that scheduled tasks are tasks that are automatically executed according to a preset time plan. They can be set through a scheduler or timer mechanism, and can automatically run certain specific programs, scripts or commands at a specified time point or time interval. The preset time interval refers to a constant time period, such as 24 hours, 5 minutes, etc.

[0060] Specifically, setting a timer task to execute step S10 within a preset time interval is to ensure that after the content of the target database table is updated, the mapping between the data in the target database table and the efficient bitmap can be consistent. For example, if a database table adds a data "happy", which is converted to binary as 10010, then the corresponding efficient bitmap also needs to add the effect of 10010.

[0061] In this embodiment, by setting a timer task, the steps of reading the preset configuration text and determining the target database table and target field based on the configuration text are executed within a certain time interval, thereby ensuring that the mapping relationship between the target database table and its corresponding high-efficiency bitmap is accurate, thereby avoiding querying erroneous data.

[0062] Step S20, for each target database table, determining the content text corresponding to all target fields in the target database table, converting the content text into a first content text in decimal format, and storing the first content text in a preset first efficient bitmap to obtain a second efficient bitmap, determining the count quantity corresponding to each character in the target field, and associating the count quantity with the target database table for storage, wherein the content text includes each character in the target field;

[0063] It should be noted that the decimal system is a number system that uses ten digits (0 to 9) to represent all numbers. An efficient bitmap is a bitmap data structure used for efficient data representation and query. A bitmap is a structure that uses binary bits to represent a data set, and is often used to process large amounts of data and for efficient retrieval requirements. The count quantity refers to the number of times each word appears in the target database table.

[0064] In addition, it should be noted that all target fields in each target database table are first accessed to obtain the text content of all target fields, which can be a combination of characters in all target fields. The text content of all target fields is taken as a subset of this target database table. For example, the text content can be compared to an article, and this target database table is the title of this article. When converting the content text to decimal, the subset is actually traversed and each element in the subset is decimal-encoded. For the second efficient bitmap, the terminal stores it in the server in the terminal.

[0065] Access the target database tables in sequence. For at least one target database table, access its target fields in sequence to obtain the content text of all target fields. Take this content text as a subset of this target database table. Traverse all the characters in this subset, perform decimal encoding on each character to obtain the first content text. The first content text includes all the characters that have undergone decimal encoding. Store the first content text in the first efficient bitmap. It should be noted that when storing the first content text in the first efficient bitmap, in fact, each individual text in the first content text is stored in sequence. When storing each individual text, it is also necessary to count the number of occurrences of the individual text in the first efficient bitmap to determine the count quantity corresponding to each character in the target field corresponding to the individual text. Associate the count quantity with the target database table for storage. Until all the individual texts in the first content text are stored, store the efficient bitmap in a preset server. Optionally, one efficient bitmap corresponds to only one target database table. Until all the target database tables are accessed, map the content text in each target database table to the corresponding first efficient bitmap to obtain the second efficient bitmap. Finally, store the second efficient bitmap in the server.

[0066] Step S30, if an operation instruction is received from an external input, parse the operation instruction to obtain the target text. Based on the target text, query the target efficient bitmap containing the target text in each second efficient bitmap.

[0067] It should be noted that the operation instruction received from an external input refers to the natural language input by the user. The user asks questions by inputting natural language. Before parsing the operation instruction, this operation instruction will generate a request. Parsing actually means disassembling the request body of the request to obtain the target text, rather than directly parsing the operation instruction to obtain the target text. Exemplarily, for Chinese natural language, the target text is multiple single characters. For example, the character '东' in '广东' is a target text. For English words, the target text is the word. For example, 'in' in 'in March' is a word.

[0068] Specifically, receive the question input by the user, generate a request. The server of the terminal parses the request body, disassembles the natural language parsed from the request body into words to obtain multiple target texts, converts the target texts into the data format in the second efficient bitmap, and then matches the data in the second efficient bitmap. Sequentially query the second efficient bitmap containing the target text in the server.

[0069] Step S40: Determine at least one first target database table among all target database tables that matches the target efficient bitmap. Determine the matching count quantity that matches the target text among the count quantities associated with the at least one first target database table, and determine the final target database table from the at least one first target database table according to the matching count quantity.

[0070] It should be noted that there may be one or more target efficient bitmaps, so there may also be one or more first target database tables. When there is only one first target database table, this first target database table is the final target database table. "Determine" means to determine the corresponding first target database table through the bitmap serial number of the target efficient bitmap. Since one target database table corresponds to a second efficient bitmap, the corresponding first target database table can be anchored through the bitmap serial number of the target efficient bitmap.

[0071] Specifically, anchor the corresponding first target database table through the bitmap serial number of the target efficient bitmap. Determine the matching count quantity that matches the target text among the count quantities associated with the first target database table, and determine the first target database table with the largest matching count quantity in the first target database table as the final target database table.

[0072] Step S50: Based on the preset nl2sql neural network model, convert the operation instruction into a query statement, and use the query statement to query in the final target database to obtain the target data.

[0073] It should be noted that the preset nl2sql neural network model is a neural network technology. Its essence is to convert the user's natural language into a semantic representation that can be understood and executed by a computer, including a neural network, data preprocessing, and data output. "Convert" means that the nl2sql neural network model infers an SQL statement based on the operation instruction.

[0074] In addition, it should also be noted that converting the operation instruction into a query statement also requires based on the information of the final target database table. For example, after the above step S40, it is determined that the final target database table is Table A. The user's natural language input is "Query the sales in March in Guangdong Province". In Table A, the field corresponding to "March" is month, the field corresponding to "Guangdong Province" is province, and the field corresponding to "sales" is sum. The nl2sql neural network model infers the SQL statement as "SELECT sum FROM A WHERE month = 'March' AND province = 'Guangdong Province'" based on the final target database table and the user's natural language statement.

[0075] Specifically, the nl2sql neural network model first preprocesses the natural language in the operation instructions, including word segmentation, part-of-speech tagging, grammatical parsing and other steps, and converts it into feature vectors that can be understood by the model. Then, based on these feature vectors, the neural network model infers the semantics through a multi-layer neural network architecture (such as an encoder-decoder model), and combines the information of the final target database table to generate SQL statements corresponding to the operation instructions. Finally, the nl2sql neural network model outputs the generated SQL statements, which are used to execute queries in the final target database to obtain the target data required by the user.

[0076] In the present application, by reading the preset configuration text, the target database table and the target field in the target database table are determined based on the configuration text, thereby obtaining the database table and the fields in the database table required for business queries. The target fields in each target database table are traversed to obtain the content text corresponding to the target field (the content text includes each character in the target field), the content text is converted into a decimal format, the first content text is obtained, the first content text is stored in a preset first efficient bitmap, and the second efficient bitmap is obtained. Using an efficient bitmap to store data can reduce memory usage, thereby improving data access efficiency. The count quantity corresponding to each character in the target field is determined, and the count quantity is associated with the target database table for storage, so that the final target database table can be determined based on the count quantity later. Receive an operation instruction input from the outside, parse the operation instruction, obtain the target text, query the target efficient bitmap containing the target text in each second efficient bitmap based on the target text, determine at least one first target database table that matches all target efficient bitmaps, determine the number of matching counts that match the target text in the number of counts associated with at least one first target database table, and determine the final target database table in at least one target database table according to the number of matching counts, determine the final target database table through the target efficient bitmap and the number of matching counts, optimize the data access process, avoid direct access to multiple target database tables, and cause IO congestion (Input / Output, refers to the input and output operations of data in a computer system), and only need to query in one final target database to obtain the target data. Based on the preset nl2sql neural network model, convert the operation instruction into a query statement, use the query statement to query the target data in the final target database, and only need to access the final target database once to obtain the target data, which further improves the efficiency of data query through the nl2sql neural network model.

[0077] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the first embodiment can be referred to the above description, and will not be described in detail later. On this basis, step S20 converts the content text into a first content text in decimal format, and stores the first content text in a preset first efficient bitmap, and the step of obtaining the second efficient bitmap also includes steps A21~A22:

[0078] Step A21, traversing each character in the content text, performing decimal encoding on each character, and obtaining a first content text in decimal format, wherein the first content text includes a single item text obtained by performing decimal encoding on each character;

[0079] It should be noted that decimal encoding of each character refers to converting each character into a decimal format.

[0080] Specifically, each character in the content text is accessed in turn, and each character is converted into a decimal format. After all characters are converted into decimal values, a first content text in decimal format is obtained, and the first content text includes a single text obtained by decimal encoding each character.

[0081] Step A22, traverse each single text item in the first content text, compress the single text item and store it in the first high-efficiency bitmap, to obtain a second high-efficiency bitmap.

[0082] It should be noted that the single-item text is a decimal representation of a word. A word in the content text corresponds to a single-item text in the first content text. The compression of each single-item text in turn and its storage in the first efficient bitmap is achieved through bitmap compression technology. Bitmap compression technology is a technology used to reduce the storage space of bitmap image data. The decimal single-item text is converted into binary format using bitmap compression technology and stored in the first efficient bitmap. Traversal refers to converting the single-item text into binary format and storing it in the first efficient bitmap in the order in which the single-item text is obtained. The first efficient bitmap is a data structure similar to a train carriage, and each data occupies a carriage.

[0083] Access the first content text in sequence to obtain single-item texts, use bitmap compression technology to convert each single-item text in decimal format into binary format, and store each single-item text converted into binary format in the first efficient bitmap in sequence, that is, occupying one carriage.

[0084] In this embodiment, the first content text is obtained by performing decimal conversion on the single-word text in the natural language form in the content text, and then the decimal-coded single-item text in the first content text is converted into binary format using a bitmap compression technology. Finally, each character in the content text is compressed in the form of binary coding and stored in the first high-efficiency bitmap to obtain a second high-efficiency bitmap, thereby achieving efficient storage of data. At the same time, the number of occurrences of each single-item text is counted when storing it, so that when performing subsequent data query, the final target database table is determined by querying the first target database table determined by the second high-efficiency bitmap and the count number of single-word texts corresponding to the single-item texts in the target database table, thereby optimizing the data query process, avoiding direct access to multiple database tables when performing data query, and blocking IO, thereby improving the efficiency of data query.

[0085] Based on the first embodiment or the second embodiment of the present application, in the third embodiment of the present application, the same or similar contents as those of the first embodiment or the second embodiment can be referred to the above description, and will not be repeated in the following. On this basis, after the step A22 compresses the single text and stores it in the first efficient bitmap, it also includes steps A23 to A25:

[0086] A23, determining the compressed single-item text stored in the first efficient bitmap;

[0087] A24, if the single item text is consistent with the stored compressed single item text, updating the number of occurrences of the stored compressed single item text, wherein the number of occurrences updating includes increasing the number of occurrences by a preset value;

[0088] A25, if the single item text is inconsistent with the stored compressed single item text, the single item text is used as a new stored compressed single item text, and the number of occurrences of the new stored compressed single item text is set to a preset value.

[0089] The default value is 1, which means that the number of occurrences is increased by 1 each time a new item is added or updated. The default value is 1 to simplify the counting process while ensuring the accuracy of data statistics. If a larger value is used, statistical information may be distorted, especially in scenarios that require detailed statistics.

[0090] Determine the single item text that has been stored compressed in the first efficient bitmap, compare the single item text with the single item text that has been stored compressed, if the single item text is consistent with the stored compressed single item text, add 1 to the number of occurrences of the stored compressed single item text, if they are inconsistent, set the number of occurrences of the stored compressed single item text to 1.

[0091] Exemplarily, the compressed single-item texts A and B are stored in the first efficient bitmap, and their occurrence times are 1 and 10 respectively. When B is compressed and stored, it is determined that there is a compressed single-item text B that is consistent with the single-item text B in the first efficient bitmap, so the occurrence times of the single-item text B is increased by 1, and the occurrence times of B is now 11.

[0092] In a feasible implementation manner, the step of determining the count number corresponding to each character in the target field in step S20 further includes step A26:

[0093] A26, for each word in the target field, determining the number of occurrences of the single text corresponding to the word in the second efficient bitmap as the count number of the word.

[0094] It should be noted that because the content text includes all the characters in the target field, and each character in the content text corresponds one-to-one to a single text, the count number of characters in the target field can be determined by determining the number of occurrences of the single text corresponding to the character in the second efficient bitmap.

[0095] For example, the number of occurrences of a single text A is updated to 10 times, and the count number associated with the corresponding text A` in the target field is also updated to 10.

[0096] In this embodiment, compressing a single text and storing it in the first high-efficiency bitmap can reduce storage space usage and improve the efficiency of data storage and reading. It can also quickly locate and search data, count the number of occurrences of a single text, and then obtain the count number of corresponding text in the target field, which is convenient for subsequent determination of the final target database table.

[0097] Based on the first embodiment, the second embodiment, or the third embodiment of the present application, in the fourth embodiment of the present application, the same or similar contents as those in the first embodiment, the second embodiment, or the third embodiment can be referred to the above description, and will not be described in detail later. On this basis, step S40, determining at least one first target database table that matches the target efficient bitmap in all target database tables, determining the number of matching counts that match the target text in the number of counts associated with at least one first target database table, and determining the final target database table in at least one first target database table according to the number of matching counts also includes steps A41 to A44:

[0098] Step A41, obtaining a bitmap sequence number of a target efficient bitmap, and searching for at least one first target database table corresponding to the bitmap sequence number in all target database tables;

[0099] It should be noted that the bitmap serial number is also the serial number of the target database table. When converting and storing the content text in the target data table into the first efficient bitmap, it is carried out according to the mapping relationship that one target database table corresponds to one efficient bitmap, which is equivalent to one target database table corresponding to one efficient bitmap. When there is only one target database table, this target database table is the first target database table.

[0100] Specifically, obtain the bitmap serial number of the target efficient bitmap, and find the target database table corresponding to the bitmap serial number in the target database table to obtain the first target database table.

[0101] Step A42, for each first target database table, determine the count quantity corresponding to the first target database table among the count quantities associated with and stored for multiple target database tables, and determine the matching count quantity corresponding to the target text among the count quantities corresponding to the first target database table.

[0102] It should be noted that there may be one or more first target database tables. If there is only one, then this first target database table is the final target database table. Matching means comparing the target text with each text in all target fields of the first target database table.

[0103] Specifically, compare the target text with each text in all target fields of the first target database table, and select the count quantity of the text with a successful comparison as the matching count quantity among the count quantities corresponding to the texts of the multiple target database tables associated with and stored.

[0104] Exemplarily, the target text is the character "东" in the natural language "广东". The first target database tables containing the target text "东" are: Table A. In Table A, there is a single-character text "东", and the count quantity associated with "东" is 4. Compare the target text "东" with Table A, and the count quantity of the single-character text "东" in Table A is queried as 4. Therefore, the matching count quantity of the target text "东" in the first target database table is 4.

[0105] Step A43, based on the matching count quantity, statistically calculate the existence probability of the target text in each first target database table;

[0106] Step A44, determine the first target database table with the highest existence probability among all first target database tables as the final target database table.

[0107] It should be noted that the existence probability is the ratio of the matching count quantity to the total sum of all text count quantities in all first target database tables. The larger the matching count quantity of the target text, the greater its existence probability.

[0108] Exemplarily, there are business tables Table A, Table B, and Table C. For the target text "East", the associated count in Table A is 4, the associated count in Table B is 5, and the associated count in Table C is 10. Then the matching count of the target text "East" in Table A is 4, the matching count in Table B is 5, and the matching count in Table C is 10. It is calculated that the existence probability of "East" in Table C is the highest, and Table C is determined as the final target database table.

[0109] In this embodiment, by calculating the existence probability of the target text in each target database table, the database table most likely to contain the target text can be effectively screened out, thereby improving the accuracy and efficiency of the query.

[0110] In this embodiment, the first target database table is determined through the bitmap sequence number of the target efficient bitmap, so as to determine all target database tables containing the target text. The target text is compared with each text in all target fields in the first target database table. Among the count quantities corresponding to the texts stored in association in multiple target database tables, the count quantity of the text with a successful comparison is selected as the matching count quantity, and then the final target database table is determined, so as to facilitate the subsequent generation of an SQL statement through the nl2sql neural network model and directly query the final target database table to obtain the query result, avoiding the IO blockage caused by querying multiple database tables through an SQL statement.

[0111] Based on the first embodiment or the second embodiment or the third embodiment or the fourth embodiment of the present application, in the fifth embodiment of the present application, the content that is the same as or similar to the above first embodiment or the second embodiment or the third embodiment or the fourth embodiment can be referred to the above introduction and will not be repeated hereinafter. On this basis, the steps of receiving an external input operation instruction in step S30 and parsing the operation instruction to obtain the target text further include steps A31 to A32:

[0112] Step A31, call a preset web interface according to the preset nl2sql neural network model to receive an external input operation instruction, where the operation instruction includes a natural language statement;

[0113] It should be noted that the pre-set neural network service refers to a service based on neural network technology. It can be a trained model (such as the nl2sql neural network model) deployed on a server, which can process various inputs (such as natural language, images, etc.) and return results. For example, a natural language processing (NLP) service can convert the user's natural language input into structured data or executable commands (such as SQL queries). The web interface refers to the Web API (Web Application Programming Interface), which is a set of definitions, protocols, and tools provided by a software system for other software systems or developers to use. The API can include functions, classes, methods, data structures, etc., for realizing the interaction and communication between different software components.

[0114] Specifically, the nl2sql neural network model obtains the operation instruction input by the user (that is, the natural language statement), calls the Web API, takes the natural language statement as a parameter of the Web API, and发起 an access request to the server storing the efficient bitmap. The server receives the access request and obtains the request body of this request.

[0115] Step A32: Disassemble the natural language statement into individual characters to obtain target characters.

[0116] It should be noted that the request body refers to the data part sent by the nl2sql neural network model to the server through a network request, including the natural language statement input by the user.

[0117] Specifically, the request body may contain a natural language query sentence input by the user, such as "Query the sales amount in Guangdong Province in March". After receiving the request, the server parses the content in the request body, extracts the natural language sentence, and disassembles the sentence into individual characters or words, such as "查", "询", "3", "月", "广", "东", "省", "的", "销", "售", "额". Then each individual character (or word) is used as a target character for subsequent processing steps.

[0118] In this embodiment, parsing the request body and disassembling the natural language statement into individual characters can better refine the query conditions and help generate SQL statements or perform other operations more accurately.

[0119] Exemplarily, to help understand the implementation process of the data query method obtained by combining the above first embodiment with this embodiment and the advantages of the data query method of the present application, please refer to Figure 2 And Figure 3 , specifically:

[0120] Figure 2It is a traditional data query method. When a user inputs a source sentence, the program running on the terminal will segment the source sentence, generate a word list, traverse the word list, that is, traverse the database, and query which tables the words exist in. For these words, the number of times they appear in each table is counted, and nl2sql is used to analyze the statistical results. Based on the statistical results, SQL statements are generated, and the results are queried using SQL statements. For this traditional data query method, before generating SQL statements, the database table must be accessed multiple times to count the number of times a word appears in each table, thereby inferring the SQL statement. When SQL statements are used to query the database table later, multiple database tables still need to be accessed for query results. When the amount of data is large and the amount of queries is large, IO is easily blocked, resulting in low query efficiency.

[0121] In the embodiments of this application, refer to Figure 3, assuming that all fields in all database tables need to participate in data query, before data query, a warm-up phase is performed to parse the database table (that is, the target database table), obtain the full text of the full table (that is, obtain the text of all target fields in all target database tables), and encode the full text. When encoding the text, the text of each table is used as its subset, that is, the content text is used as a subset of the table. The content text is equivalent to an article, and the text in it can be a single Chinese word or an English word, and the corresponding target database table is the name of the article. The subset is traversed, and the single element of the subset (that is, the single word text) is decimal encoded, and then the bitmap compression technology is used to store the data. When storing, the statistical logic is executed to determine whether the high-compression bitmap already exists. Words (that is, whether compressed single-item texts already exist). If not, the word occurrence count is set to 1, and if so, the word occurrence count is increased by 1. Repeat the full text encoding operation until N tables are traversed and N high-compression bitmaps are generated. All high-compression bitmaps are stored in the service memory (that is, the server's memory). At this point, the warm-up phase has ended. In this warm-up phase, all the text in all database tables is decimal-encoded and stored in high-compression bitmaps. There is no need to directly operate the database table in subsequent data queries. Users use the neural network service to query data. First, the user asks a question. The nl2sql neural network model calls the webapi (that is, the Web API) to query the efficient bitmap containing the question word in the service memory. The service memory returns the bitmap sequence number of the efficient bitmap. Based on the bitmap sequence number, the database table containing the question word can be determined. Based on the number of word occurrences, the probability of the question word (that is, the target text) in the business table (that is, the target database table) is counted to determine the final target database table. nl2sql generates an SQL statement, and uses the SQL statement to query the final target database table to obtain the final query result. The SQL statement generated by the data query method of the present application does not need to query multiple database tables when querying the database table. Instead, a final target database table is determined by the bitmap sequence number of the efficient bitmap where the word is located and the number of occurrences of the word, and the final target database table can be directly queried without accessing multiple database tables, causing database IO congestion, thereby resulting in low query efficiency. Optionally, the warm-up phase designed in this method can synchronize data at a non-peak time point for users, and data initialization only needs to be done once. Use efficient bitmap compression technology to record database table data, reduce memory usage, and improve hardware utilization. By determining the final target database table, the same database data can be used by multiple nl2sql (that is, the abbreviation of the nl2sql neural network model), and only a shared api (that is, Web API) needs to be accessed.During the warm-up phase, it is also possible not to scan the full amount of data in the full database table, and to determine the database tables and fields that need to be scanned by configuring the text, such as the creation time, creator, update time, etc. of the record, to further optimize the data query process and improve data query efficiency. Optionally, the data query method of the present application can also use bitmap compression technology to record database table data, optimize hardware utilization, reduce memory usage of inference services, reduce memory usage time, and reduce the risk of terminal system crashes in high concurrency situations.

[0122] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the data query method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0123] This application also provides a data query device, please refer to Figure 4 , the data query device comprises:

[0124] A configuration reading module 10 is used to read a preset configuration text and determine at least one target database table and a target field in the target database table based on the configuration text;

[0125] The data storage module 20 is used to determine, for each target database table, content texts corresponding to all target fields in the target database table, convert the content texts into first content texts in decimal format, store the first content texts in a preset first efficient bitmap, obtain a second efficient bitmap, determine the count quantity corresponding to each word in the target field, and associate the count quantity with the target database table for storage, wherein the content texts include each word in the target field;

[0126] The data query module 30 is used to parse the operation instruction when receiving an external input operation instruction, obtain the target text, and based on the target text, query the target high-efficiency bitmap containing the target text in each second high-efficiency bitmap; determine at least one first target database table that matches the target high-efficiency bitmap in all target database tables, determine the number of matching counts that match the target text in the number of counts associated with at least one first target database table, and determine the final target database table in at least one first target database table based on the number of matching counts; based on the preset nl2sql neural network model, convert the operation instruction into a query statement, use the query statement to query the final target database table, and obtain the target data.

[0127] The data query device provided by the present application adopts the data query method in the above embodiment, which can solve the technical problem of low data query efficiency. Compared with the prior art, the beneficial effects of the data query device provided by the present application are the same as the beneficial effects of the data query method provided by the above embodiment, and other technical features in the data query device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0128] The present application provides a data query device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the data query method in the above-mentioned embodiment 1.

[0129] Reference below Figure 5 , which shows a schematic diagram of the structure of a data query device suitable for implementing an embodiment of the present application. The data query device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The data query device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0130] like Figure 5As shown, the data query device may include a processing device 1001 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the data query device are also stored. The processing device 1001, ROM1002 and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the data query device to communicate with other devices wirelessly or by wire to exchange data. Although the data query device with various systems is shown in the figure, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0131] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0132] The data query device provided by the present application adopts the data query method in the above embodiment to solve the technical problem of low data query efficiency. Compared with the prior art, the beneficial effects of the data query device provided by the present application are the same as the beneficial effects of the data query method provided by the above embodiment, and other technical features in the data query device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0133] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0134] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0135] The present application provides a storage medium having computer-readable program instructions (ie, computer programs) stored thereon, wherein the computer-readable program instructions are used to execute the data query method in the above-mentioned embodiment.

[0136] The storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: RandomAccess Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, system or device. The program code contained on the storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0137] The above storage medium may be included in the data query device; or may exist independently without being assembled into the data query device.

[0138] The storage medium carries one or more programs. When the one or more programs are executed by the data query device, the data query device can implement the data query method in the present application.

[0139] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0140] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0141] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0142] The readable storage medium provided in the present application is a storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned data query method, and can solve the technical problem of low data query efficiency. Compared with the prior art, the beneficial effects of the storage medium provided in the present application are the same as the beneficial effects of the data query method provided in the above-mentioned embodiment, and will not be repeated here.

[0143] The present application also provides a program product, including a computer program, which implements the steps of the above-mentioned data query method when executed by a processor.

[0144] The program product provided by this application can solve the technical problem of low data query efficiency. Compared with the prior art, the beneficial effects of the program product provided by this application are the same as the beneficial effects of the data query method provided by the above embodiment, which will not be repeated here.

[0145] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A data query method, characterized in that: The data query method comprises: Reading a preset configuration text, and determining at least one target database table and a target field in the target database table based on the configuration text; For each of the target database tables, determine the content texts corresponding to all the target fields in the target database table, convert the content texts into first content texts in decimal format, and store the first content texts in a preset first efficient bitmap to obtain a second efficient bitmap, determine the count quantity corresponding to each character in the target field, and associate the count quantity with the target database table for storage, wherein the content text includes each character in the target field; If an operation instruction input from an external source is received, the operation instruction is parsed to obtain a target text, and based on the target text, a target high-efficiency bitmap containing the target text is searched in each of the second high-efficiency bitmaps; Determine at least one first target database table that matches the target efficient bitmap among all the target database tables, determine the number of matching counts that match the target text among the number of counts associated with the at least one first target database table, and determine a final target database table in the at least one first target database table according to the number of matching counts; Based on the preset nl2sql neural network model, the operation instruction is converted into a query statement, and the query statement is used to query the final target database table to obtain the target data.

2. The data query method according to claim 1, characterized in that: The step of converting the content text into a first content text in a decimal format, and storing the first content text in a preset first efficient bitmap to obtain a second efficient bitmap comprises: Traversing each character in the content text, performing decimal encoding on each character, and obtaining a first content text in decimal format, wherein the first content text includes a single item text obtained by performing decimal encoding on each character; Each single item of text in the first content text is traversed, and the single item of text is compressed and stored in the first efficient bitmap to obtain a second efficient bitmap.

3. The data query method according to claim 2, characterized in that: After the step of compressing the single item text and storing it in the first efficient bitmap, the method further comprises: Determine that compressed single-item text is stored in the first efficient bitmap; If the single item text is consistent with the stored compressed single item text, the number of occurrences of the stored compressed single item text is updated, wherein the number of occurrences update includes increasing the number of occurrences by a preset value; If the single item text is inconsistent with the stored compressed single item text, the single item text is used as a new stored compressed single item text, and the number of occurrences of the new stored compressed single item text is set to a preset value.

4. The data query method as claimed in claim 3, characterized in that: The step of determining the number of counts corresponding to each word in the target field includes: For each character in the target field, the number of occurrences of the single item of text corresponding to the character in the second efficient bitmap is determined as the count number of the character.

5. The data query method according to claim 2, characterized in that: The steps of determining at least one first target database table that matches the target efficient bitmap among all the target database tables, determining the number of matching counts that match the target text among the number of counts associated with the at least one first target database table, and determining the final target database table in the at least one first target database table according to the number of matching counts include: Obtaining a bitmap sequence number of the target efficient bitmap, and searching among all the target database tables for at least one first target database table corresponding to the bitmap sequence number; For each first target database table, determine the count number corresponding to the first target database table from the count numbers corresponding to the multiple target database tables stored in association, and determine the matching count number corresponding to the target text from the count numbers corresponding to the first target database table; Counting the existence probability of the target text in each of the first target database tables based on the matching count number; The first target database table with the greatest existence probability among all the first target database tables is determined as the final target database table.

6. The data query method according to claim 1, characterized in that: The step of receiving an external input operation instruction, parsing the operation instruction, and obtaining the target text includes: Calling a preset web page interface according to a preset nl2sql neural network model to receive an external input operation instruction, wherein the operation instruction includes a natural language sentence; The natural language sentence is decomposed into individual characters to obtain target characters.

7. A data query device, characterized in that: The device comprises: A configuration reading module, used for reading a preset configuration text, and determining at least one target database table and a target field in the target database table based on the configuration text; a data storage module, for determining, for each of the target database tables, content texts corresponding to all target fields in the target database table, converting the content texts into first content texts in decimal format, and storing the first content texts in a preset first efficient bitmap to obtain a second efficient bitmap, determining the count quantity corresponding to each word in the target field, and associating the count quantity with the target database table for storage, wherein the content texts include each word in the target field; A data query module is used to parse the operation instruction upon receiving an external input operation instruction, obtain the target text, and based on the target text, query the target high-efficiency bitmap containing the target text in each of the second high-efficiency bitmaps; determine at least one first target database table that matches the target high-efficiency bitmap in all the target database tables, determine the number of matching counts that match the target text among the number of counts associated with at least one of the first target database tables, and determine the final target database table in at least one of the first target database tables based on the number of matching counts; based on a preset nl2sql neural network model, convert the operation instruction into a query statement, use the query statement to query the final target database, and obtain the target data.

8. A data query device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the data query method according to any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium is a storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the data query method according to any one of claims 1 to 6 are implemented.

10. A program product, characterized in that The program product comprises a computer program, and when the computer program is executed by a processor, the steps of the data query method according to any one of claims 1 to 6 are implemented.

Images