A distributed database and query method

Abstract

The application relates to the technical field of database construction and query, and relates to a distributed database and a query method. The application determines a corresponding query strategy by acquiring a corresponding communication cost in multiple query modes, acquires a corresponding connection sequence query data table according to the determined query strategy, acquires data in a corresponding master database and a corresponding sub-database based on the data table, and determines a final query result.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of database construction and query, and relates to a distributed database and a query method. BACKGROUND

[0002] Since the 1980s, through the digitization of ancient books for more than 20 years, China has made substantial progress in the management of ancient literature, but there are still many problems in the development and actual use. At present, the management of ancient literature is mainly based on a local database and is realized by establishing a data table. However, as the amount of ancient literature increases with the progress of research, the cost of local table management is high. In the process of literature retrieval, the table management is prone to cause retrieval pressure under multiple concurrent tasks, resulting in slow retrieval and network lag, low retrieval efficiency and easy database crash. SUMMARY

[0003] In order to solve the above technical problems, the application provides a distributed database and a query method, which are applied to the classified retrieval of ancient literature. The table is converted into a corresponding data structure, the retrieval efficiency is improved, and the robustness of the system under multiple concurrent conditions is realized.

[0004] In order to achieve the above purpose, the technical scheme adopted by the embodiments of the application is as follows:

[0005] In a first aspect, a distributed database is applied to multiple user terminals, and the multiple user terminals are in a relationship with the database through registration. The database comprises: a data classification module, which communicates with the multiple user terminals through a gateway, and is used for classifying query information sent by the user terminals to obtain a data type label of to-be-queried information and send the data type label to a lower module; a data analysis module, which is connected with the data classification module, determines a corresponding data analysis module based on the data type label of the to-be-queried information, and comprises a first data analysis module and a second data analysis module, wherein the first data analysis module and the second data analysis module are respectively configured with a first data query method and a second data query method, the first data query method and the second data query method convert the query information into standard data, and obtain a connection sequence query data table based on a query algorithm set based on the standard data; and multiple database servers, which are configured with multiple master data tables and multiple slave data tables corresponding to the multiple master data tables, and the master data tables and the multiple slave data tables are retrieved based on the connection sequence query data table.

[0006] In a first implementation form of the first aspect, the first database server is configured to synchronize data with the plurality of database servers through a binlog log.

[0007] In a second implementation form of the first aspect, the data analysis module is connected with a cache module, and the cache module is configured to save the SQL statement and the corresponding query result in each query process.

[0008] In a third implementation form of the first aspect, the data analysis module comprises an analysis unit and a database access unit, the analysis unit is configured to obtain a connection sequence query data table, and the database access unit is configured to call corresponding data tables based on the connection sequence query data table.

[0009] In a second aspect, a distributed database query method is provided, which is applied to the distributed database according to any one of the first aspect or the implementation forms thereof, and comprises the following steps.

[0010] Obtaining query information of the user terminal, wherein the query information comprises information entities and information tags arranged on the information entities, obtaining information categories of the query information based on the information tags, wherein the information categories comprise text categories and image categories; performing data analysis on the text category information and the image category information to obtain a connection sequence query data table; and obtaining corresponding master data tables and corresponding slave data tables based on the connection sequence query data table.

[0011] In a first implementation form of the second aspect, the data analysis on the image category information to obtain the connection sequence query data table comprises: performing image decoding on the image category information to obtain an image matrix corresponding to the image category information; performing feature extraction on the image matrix to obtain a feature vector; performing data conversion based on the feature vector to obtain a SQL statement, and obtaining the connection sequence query data table based on the SQL statement.

[0012] In a second implementation form of the second aspect, the data analysis on the text category information to obtain the connection sequence query data table comprises: obtaining a SQL statement of the text category information, and performing analysis based on the SQL statement to obtain the connection sequence query data table.

[0013] In a third implementation form of the second aspect as above, based on the SQL statement, the connected sequence query data table is obtained, including: parsing the table and the field in the SQL statement; obtaining the connected table and the connected field based on the table and the field; calling a query method based on the connected table and the connected field to obtain a query strategy according to the best connection sequence; and executing the SQL statements in the query strategy in sequence based on the query strategy to obtain the corresponding connected sequence query data table.

[0014] In a fourth possible implementation form of the third possible implementation form of the second aspect as above, the query strategy is obtained based on the connected table and the connected field by calling a query method to obtain the best connection sequence, including: establishing a plurality of query relationships based on the connected table and the connected field; encoding the plurality of query relationships based on the coding rules of the leaf nodes and the non-leaf nodes to obtain a plurality of query connection tree structures; obtaining the query cost of the plurality of query connection trees; comparing the plurality of query costs to obtain the query relationship with the minimum query cost, and obtaining the query strategy based on the query relationship; and obtaining the query cost of the plurality of query connection trees, including: obtaining the communication cost estimation value and the I / O value of the plurality of query connection trees; and obtaining the communication cost by the following formula: Wherein R is the transmission rate, the unit is second / bit, X is the data transmission amount, the unit is bit; and C0 is the time spent for communication initialization between communication network sites, which is determined by the communication system and is approximately a constant, the unit is second.

[0015] In a fifth implementation form of the second aspect as above, in the query connection structure, the array variable is configured on the leaf node. Wherein i is the number of leaf nodes.

[0016] In a third aspect, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program. When the computer program is executed by a processor, the distributed database query method described in any of the above aspects is implemented.

[0017] In the technical scheme provided by the embodiments of the present application, the corresponding query strategy is determined by obtaining the corresponding communication cost in the plurality of query modes, the corresponding connected sequence query data table is obtained according to the determined query strategy, the data in the main database and the sub-database is obtained based on the data table, and finally the query result is determined. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative effort based on these drawings.

[0019] The methods, systems, and / or programs in the drawings will be further described according to the exemplary embodiments. These exemplary embodiments will be described in detail with reference to the drawings. These exemplary embodiments are non-limiting exemplary embodiments, in which the example numbers represent similar mechanisms in each view of the drawings.

[0020] Figure 1 is a structural schematic diagram of a distributed database provided by the embodiments of the present application.

[0021] Figure 2 is a flowchart of a distributed database query method according to some embodiments of the present application.

[0022] Figure 3 is a block schematic diagram of a distributed database query method device according to the embodiments of the present application. DETAILED DESCRIPTION

[0023] In order to better understand the above technical solutions, the technical solutions of the present application will be described in detail below through the drawings and specific embodiments. It should be understood that the embodiments of the present application and the specific features in the embodiments are detailed descriptions of the technical solutions of the present application, and are not limitations of the technical solutions of the present application. In the case of no conflict, the technical features in the embodiments of the present application and the embodiments can be combined with each other.

[0024] In the following detailed description, many specific details are set forth in order to provide a thorough understanding of relevant guidance. However, it is apparent to those skilled in the art that the present application can be implemented without these details. In other cases, well-known methods, procedures, systems, components and / or circuits have been described at a relatively high level without details, in order to avoid unnecessary obscuring aspects of the present application.

[0025] The flowcharts in the present application illustrate the execution processes performed by the systems according to the embodiments of the present application. It should be explicitly understood that the execution processes of the flowcharts can not be executed in sequence. On the contrary, these execution processes can be executed in reverse order or simultaneously. In addition, at least one other execution process can be added to the flowchart. One or more execution processes can be deleted from the flowchart.

[0026] Before the embodiments of the present application are described in further detail, the terms and names involved in the embodiments of the present application are explained, and the terms and names involved in the embodiments of the present application are applicable to the following explanations.

[0027] (1) In response to, for indicating the condition or state on which the operation is dependent, when the dependent condition or state is met, one or more operations performed can be real-time or have a set delay; in the absence of special instructions, there is no restriction on the execution order of the multiple operations performed.

[0028] (2) Based on, for indicating the condition or state on which the operation is dependent, when the dependent condition or state is met, one or more operations performed can be real-time or have a set delay; in the absence of special instructions, there is no restriction on the execution order of the multiple operations performed.

[0029] (3) Distributed database, a distributed database system usually uses a smaller computer system, each computer can be placed separately in a place, each computer can have a complete copy of DBMS, or a partial copy, and has its own local database, many computers located in different places are connected through a network, and together form a complete, global, logically centralized, physically distributed large database.

[0030] The technical scheme provided by the embodiments of the present application mainly applies to the construction of a corresponding distributed database for ancient literature and data query based on the distributed database. In the prior art, the storage method for ancient literature is mainly based on the overall database in the local server, and the construction method for the overall database is based on the storage of the classification table of ancient literature into the overall database. By constructing a retrieval directory and corresponding coding, the corresponding literature under the corresponding directory is found through coding. This kind of construction is relatively simple, and the coding query has high accuracy and fast query speed. However, the overall database of the local server has the problem that the storage space has requirements, and the subsequent supplementary literature materials need to be configured with high redundancy of storage space. Moreover, under the condition of multiple concurrent retrieval, because of the single-line retrieval of the overall database, the retrieval time cost is high under the condition of a large amount of retrieval data, and the system is prone to crash due to the burst of upload and download data.

[0031] In view of this background, a distributed database is provided to realize the retrieval of ancient literature under high concurrency and high retrieval tasks.

[0032] Based on the above technical background, the embodiment of the present application provides a distributed database, which is applied to a plurality of user terminals, wherein the plurality of user terminals establish a relationship with the database through a registration mode. In the embodiment, the database comprises a data classification module. In the embodiment, the data classification module communicates with the plurality of user terminals through a gateway. The data classification module is used to classify query information sent by the user terminals to obtain a data type label of the to-be-queried information and send the data type label to a lower module. In the embodiment, the input information of the ancient literature retrieval includes text information and picture information. The logic of the query in the database is different for the text information and the picture information because of the different data structures and types. Therefore, the data type of the to-be-queried information needs to be determined before the retrieval. Therefore, the premise of the query is to determine the data classification. In the embodiment, the data classification module is determined through a query block configured in a query page of the user terminal. That is, the query interface of the user terminal comprises a text input module and a picture input module. In the embodiment, the query interface of the user terminal is not described herein.

[0033] A data analysis module is connected with the data classification module. The corresponding data analysis module is determined based on the data type label of the to-be-queried information. The data analysis module comprises a first data analysis module and a second data analysis module. The first data analysis module and the second data analysis module are respectively configured with a first data query method and a second data query method. The first data query method and the second data query method convert the query information into standard data and obtain a connection sequence query data table based on a query algorithm set based on the standard data. In the embodiment, the processing logic of the text data and the picture data is different. Therefore, different data analysis modules are configured for the two types of data. The query method is configured in the data analysis module. The corresponding connection sequence query data table can be obtained through the optimal query method through the query method. The corresponding query data is obtained by constructing the connection sequence query data table.

[0034] A plurality of database servers are configured in any one of the database servers. A plurality of master data tables and a plurality of slave data tables corresponding to the plurality of master data tables are configured in the database servers. The master data tables and the plurality of slave data tables are retrieved based on the connection sequence query data table. In the embodiment, a plurality of data slave servers are set corresponding to the plurality of database servers. The database servers and the data slave servers are synchronized through a binlog log.

[0035] And, in the embodiment, a cache module connected with the data analysis module is further arranged to save the SQL statement and the corresponding query result in each query process.

[0036] In the embodiment, the data analysis module comprises an analysis unit and a database access unit, the analysis unit is used to acquire the connection sequence query data table, and the database access unit is used to call the corresponding data table based on the connection sequence query data table.

[0037] In the embodiment, the data layer analysis program contains a multi-connection query method, and the multi-connection query is optimized, and the optimization of the connection query is transparent to the upper user end business level, so as to simplify the query logic. The connection query method is introduced without invasion by increasing the data analysis module, the decoupling of the business logic and the database connection query optimization is realized, and the maintainability of the system in the later period is improved. When the connection query method needs to be modified, only the data layer analysis program code needs to be modified, and the business logic code does not need to be modified, so that the modification of the connection query optimization algorithm is transparent to the upper business logic layer.

[0038] When the query is performed, only the multi-connection query SQL statement needs to be transmitted, and the data analysis module is used to analyze the SQL statement to obtain the data table and the connection field in the SQL statement. Then, the multi-connection query method is executed according to the data table and the connection field obtained by analysis, the best connection query sequence is obtained, the original SQL statement analysis optimization query strategy is usually decomposed into a plurality of SQL statements. The analysis module executes these SQL statements in sequence, and ensures that the connection sequence query data table obtained by the connection query method is ensured.

[0039] In the embodiment, the data analysis module comprises the upper SQL analysis optimization code and the lower Sharding-JDBC database access layer. The Shading-JDBC shields the complex database cluster details of the lower database cluster, and the upper business analysis program only needs to face the logical table for SQL programming, without needing to pay attention to the complex database and a plurality of database nodes. The upper SQL analysis program is not aware of and transparent to the database, so as to simplify the SQL analysis optimization logic code. The configuration of the database and the table is dynamically obtained from the configuration center. When the OLAP database cluster adds or reduces the database node, only the configuration of the database and the table in the configuration center needs to be modified, so that the data layer analysis program is not restarted, and the database related configuration is dynamically modified.

[0040] In the present embodiment, the system improves the availability of the database cluster through Sharding-JDBC and the database master-slave replication mechanism. First, through the binlog log of the MySQL database, the database master-slave replication is realized, and a database cluster with high availability is constructed. Then, Sharding-JDBC is configured, and the addresses, usernames, passwords, database names, data table names of each database node, and the master-slave relationship between each data node are written into the configuration file. When the database executes the SQL statement, Sharding-JDBC will automatically send the SQL statement to the responding master and slave according to a certain load balancing strategy, avoiding the problem of too much query pressure on a single master and slow query speed under high concurrency.

[0041] In the present embodiment, the centralized cache Redis is introduced to realize cache sharing and parameter data sharing among multiple data layer parsing programs. For example, if the genetic algorithm in the multi-connection query optimization algorithm is iterated online, some parameters need to be constantly iterated and modified. Placing these parameters in the Redis cache realizes the sharing of multiple data layer parsing programs. When the parameters of the genetic algorithm are modified, only the parameter copy in Redis needs to be modified, and the data layer mechanical program will automatically synchronize the latest parameter value from Redis, realizing dynamic change of algorithm parameters without restarting. At the same time, the Redis cluster realizes the sharing of the cache among the data layer programs. One data layer parsing program can obtain the program results of the previous SQL statement executed by other data layer parsing programs, reducing the number of database queries and improving the overall response speed of the system.

[0042] In the present embodiment, the settings of the database table forms are set based on the characteristics of ancient literature, which can be referred to Tables 1 and 2:

[0043]

[0044] Table 1. Design table of label codes of Jian Du account book historical material database

[0045]

[0046] Table 2. Encoding table of Jian Du publication sources

[0047]

[0048] Table 3. Example of design of label code of Jian Du account book historical material database

[0049] Referring to Tables 1-3, the corresponding table relationship is established according to the characteristics of ancient documents, and the classification retrieval of ancient documents is realized, specifically, the handed-down documents and the unearthed documents are distinguished. The unearthed documents are identified by "jianshu" or "tomb"; the handed-down documents are distinguished by classics, history, children and collection; the unearthed documents are distinguished by batch, such as Lijie Qin bamboo slips, Mawangdui Han tomb bamboo slips and Zoumalou Three Kingdoms Wu bamboo slips.

[0050] And economic data is searched from ancient documents, including handed-down documents and unearthed documents, specifically, accounting books are identified, such as total account / detailed account, income account / expenditure account (see Table 1 for details). For example, the appearance of "right", "combination", "account" and other marks in the text is the total account; the text has "its" as the detailed account; the text has "entry", "collection" and other marks as the income account; the text has "out", "pay", "give" and "branch" as the expenditure account; the text has "balance" and other marks as the balance account. Data such as price, interest rate and population are identified. The text has "straight", "city" and other marks as price data; the text has "loan", "return" and other marks as interest rate data; the text has "male", "female", "old", "household", "wife", "mouth", "brother", "father", "mother" and other marks as population data. And financial and tax data are identified. The text has "tribute", "tax", "salary" and other marks as financial and tax data.

[0051] And the above results are realized by acquiring the corresponding coding field in the data analysis module and calling the data information in the database.

[0052] In this embodiment, for multi-connection query, the database and table division strategy of vertical database and horizontal table is adopted, each data table is horizontally divided, so that the number of records in each data table does not exceed 5 million. At the same time, different data tables are stored in different database servers, which is used to ensure that the data volume on a single database server will not be too large, resulting in insufficient disk capacity and disk IO and other hardware performance bottlenecks. And the data synchronization between the master server and the slave server is realized through the binlog log. The data table of the slave database is consistent with the master database.

[0053] And, for the database server in the embodiment, the association query for the data tables on the same database server can avoid the network transmission cost, and the required time is shorter than the association query across servers, therefore, the selection of the data tables in the same database server greatly affects the speed of the multi-connection query, and the data tables with high association query frequency should be preferentially placed on the same database server. Since the distribution of the initialized data tables is not the most appropriate, the best data table distribution needs to be obtained by statistics in operation. This requires adding a data table association frequency statistics module to the data layer analysis program to record the data table association frequency. Each time the association query is performed, the data layer analysis program sends an incr command to Redis to increase the value of the corresponding key representing the association frequency of the associated table. The data table distribution management program periodically obtains the data table association information from Redis to count the data tables with high association frequency, and adjusts the distribution of the data tables on the database server when the server load is low at night, so as to optimize the speed of the data table association query.

[0054] And, in the database query in the embodiment, some data tables have high query frequency, and some data tables may be rarely used, therefore, the database server resources are as much as possible to be inclined to the frequently queried data tables, which can further improve the overall speed of the database query. Because the data tables are distributed to different database servers through the vertical database splitting strategy, it is convenient to allocate different resources to different data tables. At the same time, a table with high query frequency is often associated with tables with high query frequency, so the data tables with high association query frequency are as much as possible to be distributed to the same database server after the data table distribution optimization, and therefore, it is more convenient to allocate more server resources to the data tables with high query frequency.

[0055] Since the corresponding key value in Redis is increased by one each time the association query is performed, the query frequency of each data table can be easily obtained by counting in Redis. Therefore, it is known that which data tables are hot data tables, and more resources are allocated to the database server nodes where the hot data tables are located to improve resource utilization and query performance.

[0056] In the embodiment, the server includes a memory, a processor and a communication unit. The memory, the processor and the communication unit are directly or indirectly electrically connected to each other to realize data transmission or interaction. For example, these elements can be electrically connected to each other through one or more communication buses or signal lines. The memory is used to store specific information and programs, and the communication unit is used to send the processed information to the corresponding user end.

[0057] The embodiment divides the storage module into two storage areas, one of which is a program storage unit and the other is a data storage unit. The program storage unit is equivalent to a firmware area, and the read-write permission of the area is set to a read-only mode, and the data stored in the area cannot be erased and changed. The data in the data storage unit can be erased or read and written, and when the capacity of the data storage area is full, the newly written data will overwrite the earliest historical data.

[0058] The memory can be, but is not limited to, a random access memory (RAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), etc.

[0059] The processor can be an integrated circuit chip with a processing capability of signals. The processor described above can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; or can be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the disclosed methods, steps and logic block diagrams in the embodiments of the application. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0060] Referring to Figure 2 In the embodiment, for the distributed database query method, the working logic is to establish a corresponding code based on the characteristics of ancient literature, establish a corresponding master data report based on the code and a plurality of slave data tables corresponding to the master data table, and obtain the optimal query strategy by setting the query method to realize the acquisition of the query target.

[0061] The working logic provided in the embodiment provides a distributed database query method applied to a distributed database, which specifically includes the following methods:

[0062] Step S210. Obtain the query information of the user terminal.

[0063] In the embodiment, the query information includes an information body and an information label configured on the information body, and an information category of the query information is obtained based on the information label, and the information category includes a text category and a picture category.

[0064] Specifically, because the data input by the user end and the data processed subsequently by the database can be text data and picture data, the type of the data needs to be determined before processing, and the determination of the type of the data is based on the information label given in the query box configured by the user end, and the corresponding data category is determined based on the information label, so as to process the data of different categories.

[0065] Step S220. Data analysis is performed on the text category information and the picture category information to obtain a connection sequence query data table.

[0066] In the embodiment, this step mainly converts the input information into a corresponding data structure, and obtains a corresponding connection sequence query data table based on the converted data structure, wherein the connection sequence query data table is used to determine a main data table and a corresponding slave data table, and the final information of the query is obtained based on the main data table and the slave data table.

[0067] Specifically, this process performs corresponding processing on different data types, and the processing process for the picture category data is as follows:

[0068] Image decoding is performed on the picture category information to obtain an image matrix corresponding to the image category information. Feature extraction is performed on the image matrix to obtain a feature vector. Data conversion is performed based on the feature vector to obtain an SQL statement, and a connection sequence query data table is obtained based on the SQL statement.

[0069] The image decoding is feature extraction on the picture category information, and a deep learning model such as VGG, ResNet, MobileNet, SqueezeNet, etc. can be used as a backbone network, and then different methods are used to generate features. The classification layer of a classification model such as a VGG model can be directly used as the features of the image. The second method is to pool the features of the middle layer of the model using special methods such as RMAC, GeM and dimension reduction to obtain the features. The third method is to use a specially designed loss function to pre-train the model on the target data set. Through any one of the above feature extraction methods, the features in the picture category information can be extracted, and the features can be constructed into a matrix and a feature vector can be extracted in the matrix, and the sql data structure that can be recognized by the database can be obtained by packing through sql.

[0070] The process for processing the text category information is as follows:

[0071] The SQL statement for obtaining the text category information is parsed to obtain a connection sequence query data table.

[0072] The connection sequence query data table obtained based on the SQL statement includes: parsing the table and field in the SQL statement; obtaining the connected table and connection field based on the table and field; calling a query method based on the connected table and connection field to obtain an optimal connection sequence to obtain a query strategy; and executing the SQL statements in the query strategy in sequence based on the query strategy to obtain the corresponding connection sequence query data table.

[0073] In this embodiment, calling a query method based on the connected table and connection field to obtain an optimal connection sequence to obtain a query strategy includes: establishing a plurality of query relationships for the connected table and connection field; encoding the plurality of query relationships based on the coding rules of the leaf node and the non-leaf node to obtain a plurality of query connection tree structures; obtaining the query cost of the plurality of query connection trees; comparing the plurality of query costs to obtain the query relationship with the minimum query cost, and obtaining the query strategy based on the query relationship; and obtaining the query cost of the plurality of query connection trees includes: obtaining the communication cost estimation value and the I / O value of the plurality of query connection trees; and the communication cost is obtained by the following formula: wherein R is the transmission rate, the unit is second / bit, X is the data transmission amount, the unit is bit; C0 is the time spent on communication initialization between communication network sites, which is determined by the communication system and the value is approximately a constant, the unit is second.

[0074] In this embodiment, the array variable is configured on the leaf node of the query connection structure wherein i is the number of leaf nodes.

[0075] In the present embodiment, the data needs to be encoded before decoding, the information of the query is converted into a mathematical model by encoding, and the existing encoding methods mainly include the following: (1) binary encoding: a widely used encoding method composed of binary 0 and 1. However, binary encoding requires high precision, and long or short strings will affect search efficiency, and cannot intuitively reflect the structural characteristics of information. (2) Real number encoding: This encoding method has high precision and can better solve the spatial search problem in practical applications, which is composed of real numbers within a certain range. (3) Gray code: This encoding method is a set of binary number sequences. In a group of codes, only one binary number is different between two adjacent Gray codes. Gray code has good local search capability. (4) Symbolic encoding: Symbolic encoding is a way of encoding using symbols. The advantage of symbolic encoding is that symbols have special meanings or background knowledge.

[0076] However, the distributed database connection query provided in the present embodiment has the characteristics that the above encoding methods are not suitable and cannot better solve the problem of distributed database connection query. Therefore, for the solution of the distributed database query optimization problem, the present embodiment encodes the problem in the form of a connection query tree, and continuously traverses the leaf nodes and non-leaf nodes on the connection tree to form a corresponding relationship with the actual query order. In the process of encoding using the query tree, the non-leaf nodes are usually set to 0, and the leaf nodes are distinguished and represented by corresponding symbols or numbers. In the present embodiment, the corresponding query relationship for this structure is as follows: And the second query relationship is: .

[0077] According to the encoding rules of leaf nodes and non-leaf nodes, Query is encoded, and the connection tree of Query is obtained after encoding. According to the above connection tree encoding scheme, various transformations and access operations are performed on the query connection tree. According to the post-order traversal method of the tree, the query connection tree and the result (1, 2, 0, 3, 0, 4, 0, 5, 0, 6, 0) and the result (1, 2, 0, 3, 0, 4, 5, 0, 0, 6, 0) form a corresponding relationship, and the tree structure is converted into a linear structure and stored in the computer.

[0078] After encoding the above connection tree, in order to calculate the query cost of each query connection tree, the information obtained from the connection query encoding scheme cannot meet the basic information required in query optimization. Therefore, in order to obtain the optimal output result, some basic information needs to be additionally configured for the nodes.

[0079] Wherein, in the embodiment, an array variable S[ ] is configured on the child nodes of the query connection tree, the array subscript i = 1, 2, 3, …, n (n starts from 1, and the size is the number of the query connection table, that is, the number of the leaf nodes), and the node basic information can be better stored through the array variable S[ ]. Each element of the array S[i] has a value in the unit of the database table.

[0080] In the embodiment, the communication cost estimation formula is: Cnetwork = C0 + R * X, wherein R is the transmission rate (the inverse of the transmission speed), that is, the time for unit data transmission, and the unit is second / bit (S / bit); X is the data transmission amount, representing the amount of data transmission, and the unit is bit; C0 is the time for one communication initialization between the communication network sites, which is generally determined by the communication system, and the value is approximately a constant, and the unit is second (S).

[0081] Step S230. Obtain the corresponding master data table and the corresponding slave data table based on the connection order query data table.

[0082] In the embodiment, the corresponding query strategy is determined by obtaining the corresponding communication cost in multiple query modes, the connection order query data table is obtained based on the determined query strategy, the data in the master database and the slave database are obtained based on the data table, and thus the final query result is determined.

[0083] The distributed database and the corresponding distributed database query method provided in the embodiment realize the correspondence of data saving and searching by constructing the master database and the multiple slave databases based on the structure of the database, and realize the optimal query path by processing the data and obtaining the communication cost.

[0084] In addition, for the use scene of the ancient literature, the processed information includes not only the text information but also the picture information, the data structure based on SQL is obtained by processing the two kinds of information, the optimal query method is obtained by obtaining the corresponding query tree and the corresponding communication cost, and the final query result is obtained based on the optimal query method.

[0085] For this method, a virtual device, that is, a distributed database query device 300, is also configured, which includes: an information classification module 310, which obtains the information classification of the query information based on the information label, and the information classification includes a text category and a picture category; a data analysis module 320, which is used for data analysis on the text category information and the picture category information to obtain a connection order query data table; and a processing module 330, which obtains the corresponding master data table and the corresponding slave data table based on the connection order query data table.

[0086] It needs to be understood that the technical terms which are not explained in the above description can be determined without any doubt by the skilled in the art based on the disclosed content.

[0087] The skilled in the art can determine some preset, reference, predetermined, set and preferred label technical features / technical terms without any doubt based on the disclosed content, such as threshold, threshold interval, threshold range, etc. For some technical feature terms which are not explained, the skilled in the art can reasonably deduce without any doubt based on the logical relationship of the context, so as to clearly and completely implement the above technical solutions. The prefix of the technical feature term which is not explained, such as "first", "second", "example", "target", etc. can be deduced and determined without any doubt based on the context. The suffix of the technical feature term which is not explained, such as "set", "list", etc. can also be deduced and determined without any doubt based on the context.

[0088] The above content disclosed by the embodiments of the present application is clear and complete for the skilled in the art. It should be understood that the process of deducing and analyzing the technical terms which are not explained by the skilled in the art based on the disclosed content is based on the content recorded in the present application, so the above content is not the creative judgment of the whole scheme.

[0089] The above has described the basic concept. Obviously, for the skilled in the art, the above detailed disclosure is only as an example, and does not constitute a limitation of the present application. Although it is not explicitly stated here, the skilled in the art can make various modifications, improvements and corrections to the present application. Such modifications, improvements and corrections are suggested in the present application, so such modifications, improvements and corrections still belong to the spirit and scope of the exemplary embodiments of the present application.

[0090] At the same time, specific terms are used in the present application to describe the embodiments of the present application. As "one embodiment", "an embodiment", and / or "some embodiments" means a certain feature, structure or characteristic related to at least one embodiment of the present application. Therefore, it should be emphasized and noted that the "an embodiment" or "one embodiment" or "an alternative embodiment" mentioned in different parts of the specification twice or more does not necessarily refer to the same embodiment. In addition, some features, structures or characteristics in at least one embodiment of the present application can be properly combined.

[0091] Moreover, those skilled in the art will appreciate that the various aspects of the application can be practiced with a variety of computer-system configurations, including personal computers, desktop computers, laptop computers, tablet computers, netbooks, cellular telephones, smart-televisions, game consoles, tablet computers, smart-phones, smart- pads, smart-watches, digital-cameras, wearable computers, hand-held computers, phablets, multi-processor systems, microprocessor-based or programmable-consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, or the like. Any user-computer system should be considered suitable for use in practicing the various aspects of the application, unless specifically stated otherwise.

[0092] The computer-readable signal medium can include a propagated data signal with computer- program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated signal can take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. Computer-readable signal media can be any computer-readable media that can be selected to communicate a computer program from one place to another. A computer-readable signal medium can be any computer-readable medium that is not a computer-readable storage medium and that can communicate, propagate or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

[0093] Computer program code to carry out the various aspects of the application can be written in any combination of one or more programming languages, including an object-oriented programming language such as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C++, C#, VB.NET, Python, or the like, conventional procedural programming languages, such as the "C" programming language, Visual Basic, Fortran 2003, Perl, COBOL 2002, PHP, ABAP, dynamic programming languages such as Python, Ruby and Groovy, or other programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider), or in a cloud computing environment or as a service such as Software as a Service (SaaS).

[0094] Furthermore, the order of processing elements or steps, or the order in which they are presented, unless specifically stated in the claims, is not intended to be limiting, unless specifically stated in the application patent claims. Although the above-disclosed embodiments have shown, described, and pointed out the fundamental and novel features of the applications, it will be understood that various omissions and substitutions, changes of form, of the method of operation, of the details of equipment, and of the use of materials, as well as other changes in the specifications can be made by those skilled in the art without departing from the spirit of the applications. For example, although the system components described above can be implemented by hardware devices, they can also be implemented by software solutions, such as installing the described system on an existing server or mobile device.

[0095] It is also to be understood that the following description is only illustrative of the aspects of the present application and that no limitation of the scope of the application is intended by either the inclusion or non-inclusion of any such components or steps. Additionally, one of ordinary skill in the art will recognize that many of the components and / or steps described herein can be implemented using a variety of different platforms, software languages, systems, and / or devices, among others, and as such are intended to be illustrative only and are not limiting of the scope of the application.

Claims

1. A distributed database applied to a plurality of user terminals, a plurality of said user terminals establishing a relationship with said database by means of registration, characterized in that, The database comprises: A data classification module in communication with a plurality of user terminals through a gateway, configured to classify query information sent by the user terminals to obtain a data type label of the query information and send the data type label to a lower module; A data analysis module connected to the data classification module, configured to determine a corresponding data analysis module based on the data type label of the query information, the data analysis module comprising a first data analysis module and a second data analysis module, the first data analysis module and the second data analysis module being respectively configured with a first data query method and a second data query method, the first data query method and the second data query method converting the data type label of the query information into standard data and obtaining a connection sequence query data table based on a query algorithm set based on the standard data; the standard data being an SQL statement, the query algorithm set based on the standard data obtaining a connection sequence query data table, comprising: parsing tables and fields in the SQL statement; obtaining connected tables and connected fields based on the tables and fields; calling a query method based on the connected tables and the connected fields to obtain an optimal connection sequence to obtain a query strategy; and sequentially executing SQL statements in the query strategy based on the query strategy to obtain a corresponding connection sequence query data table; A plurality of database servers, each of which is configured with a plurality of master data tables and a plurality of slave data tables corresponding to the plurality of master data tables, the master data tables and the plurality of slave data tables being called based on the connection sequence query data table.

2. The distributed database of claim 1, wherein, Further comprising a plurality of data slave servers corresponding to the plurality of database servers, the database servers and the data slave servers being in data synchronization through binlog logs.

3. The distributed database of claim 1, wherein, Further comprising a cache module connected to the data analysis module, the cache module being configured to save SQL statements and corresponding query results in each query process.

4. The distributed database of claim 1, wherein, The data analysis module comprises an analysis unit and a database access unit, the analysis unit being configured to obtain a connection sequence query data table, and the database access unit being configured to call corresponding data tables based on the connection sequence query data table.

5. A method of distributed database querying, the method comprising: The distributed database of any one of claims 1-4, comprising: Obtaining query information of the user terminal, the query information comprising an information body and an information label configured on the information body, obtaining an information classification of the query information based on the information label, the information classification comprising a text category and a picture category; Performing data analysis on the text category information and the picture category information to obtain a connection sequence query data table; Obtaining corresponding master data tables and corresponding slave data tables based on the connection sequence query data table.

6. The distributed database query method of claim 5, wherein, Performing data analysis on the picture category information to obtain a connection sequence query data table, comprising: Performing image decoding on the picture category information to obtain an image matrix corresponding to the image category information; Performing feature extraction on the image matrix to obtain a feature vector; Data transformation is performed based on the feature vector to obtain an SQL statement, and a connection sequence query data table is obtained based on the SQL statement.

7. The distributed database query method of claim 5, wherein, The text category information is subjected to data analysis to obtain a connection sequence query data table, including: An SQL statement of the text category information is obtained, and the SQL statement is analyzed to obtain a connection sequence query data table.

8. The distributed database query method of claim 6 or 7, wherein, A connection sequence query data table is obtained based on the SQL statement, including: The tables and fields in the SQL statement are analyzed; Connected tables and connected fields are obtained based on the tables and fields; A query method is called based on the connected tables and connected fields to obtain a best connection sequence and obtain a query strategy; The SQL statements in the query strategy are executed in sequence based on the query strategy to obtain corresponding connection sequence query data tables.

9. The distributed database query method of claim 8, wherein, A query method is called based on the connected tables and connected fields to obtain a best connection sequence and obtain a query strategy, including: A plurality of query relationships are established for the connected tables and connected fields; The plurality of query relationships are coded based on coding rules of leaf nodes and non-leaf nodes to obtain a plurality of query connection tree structures; Query costs of the plurality of query connection trees are obtained; The query relationship with the minimum query cost is obtained by comparing the plurality of query costs, and a query strategy is obtained based on the query relationship; The query costs of the plurality of query connection trees are obtained, including: Communication cost estimation values and I / O value of the plurality of query connection trees are obtained; The communication cost is obtained by the following formula: where R is the transmission rate, with units of seconds / bits, X is the amount of data transmitted, with units of bits; and Co is the time taken to initialize communication between sites of a communication network, which is determined by the communication system and has a value that is approximately constant, with units of seconds.

10. The distributed database query method of claim 9, wherein, an array variable is configured on the leaf node on the query connection number structure where i is the number of leaf nodes.

Images