Data access method, device, system, equipment and medium based on virtual database

Through the virtual database layer, the access of heterogeneous databases is uniformly handled, and the deep learning algorithm generation and verification rules are used to solve the complexity of heterogeneous database connection access code, and efficient data query and maintenance improvement are achieved.

CN113688151BActive Publication Date: 2024-11-26INDUSTRIAL AND COMMERCIAL BANK OF CHINA
0 Cites 0 Cited by

Patent Information

Application Number
CN202110980369.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-08-25
Publication Date
2024-11-26
Estimated Expiration
2041-08-25

AI Technical Summary

Technical Problem

In the prior art, connection access for heterogeneous databases requires writing different code logics in the application program, resulting in large amount of code writing, complex business logic on the request side and difficult maintenance.

Method used

Using the virtual database layer, by receiving SQL query requests from the user layer, performing statement analysis and calling the preset rule database to decompose the SQL query request into multiple local query statements, routes and distributes it to the corresponding database to obtain query data, and query data Perform assembly and output, use deep learning algorithms to generate preset rules and perform manual verification, and uniformly handle access to heterogeneous databases.

Benefits of technology

Data access to the underlying heterogeneous data source is completed in the virtual database layer, avoiding different code logic writing in different databases, reducing the logic complexity and maintenance difficulty, and improving data query efficiency and scalability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113688151B_ABST
    Figure CN113688151B_ABST
Patent Text Reader

Abstract

The present disclosure provides a data access method, device, system, equipment, medium and program product based on a virtual database, and relates to the field of database technology. The method includes receiving a SQL query request from a user layer; performing statement analysis on the SQL query request, calling a preset rule in a rule library based on the result of the statement analysis to decompose the SQL query request into multiple local query statements, and routing and distributing the local query statements to the corresponding database to obtain corresponding query data; and assembling the query data and outputting the query results to the user layer. The data access method based on a virtual database in the present disclosure solves the problem in the prior art that different code logics are written for different databases to complete corresponding connection and access operations, resulting in a large amount of code writing, complex request-side business logic, and difficult maintenance.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of database technology, and more specifically to a data access method, device, system, equipment, medium and program product based on a virtual database. Background Art

[0002] The internal logic of mainstream large-scale enterprise applications is becoming increasingly complex, and the types of databases used are also diverse due to historical or upgrade reasons, such as MySQL, Oracle, Informix, etc. If the data requester wants to achieve connection access to heterogeneous databases, it is necessary to write different code logic for different databases within the application to complete the corresponding connection and access operations. The amount of code to be written is large, the business logic of the requester is complex, and maintenance is difficult. Summary of the invention

[0003] In view of the above problems, the present disclosure provides a data access method, apparatus, system, device, medium and program product for a virtual database.

[0004] According to a first aspect of the present disclosure, a data access method for a virtual database is provided, which is applied to a virtual database layer and includes: receiving an SQL query request from a user layer; performing statement analysis on the SQL query request, calling preset rules in a rule library based on the result of the statement analysis to decompose the SQL query request into multiple local query statements, and routing and distributing the local query statements to corresponding databases to obtain corresponding query data; and assembling the query data and outputting the query results to the user layer.

[0005] In some embodiments, the method further includes: generating a series of the preset rules using a deep learning algorithm; and creating a rule base based on the series of the preset rules.

[0006] In some embodiments, the use of a deep learning algorithm to generate a series of preset rules includes: creating a structured data table; extracting data information from heterogeneous data sources and storing the data information in the structured data table; using a deep learning algorithm to aggregate and match the data information in the structured data table to determine the similarity between different data tables; and defining corresponding rules based on the similarity to form the preset rules.

[0007] In some embodiments, the data information includes a library name, a table name, and table structure information, and the table structure information includes a column name, column attributes, constraints, and table data in a data table.

[0008] In some embodiments, the preset rules include operations of associating isomorphic tables with heterogeneous data sources and extracting associative operations of data with the same attributes from non-isomorphic tables.

[0009] In some embodiments, the method further includes: verifying the similarity by using a manual error marking method, wherein the method of verifying the similarity by using a manual error marking method includes:

[0010] The similarity and the relationship view corresponding to the similar fields are uploaded to a control center; and the administrator of the control center determines the accuracy of the similarity according to the relationship view corresponding to the similar fields.

[0011] In some embodiments, the further step includes: writing differentiated SQL grammar rules of heterogeneous data sources into the local rule base.

[0012] In some embodiments, calling the preset rules in the rule library based on the result of the statement analysis to decompose the SQL query request into multiple local query statements includes: matching the preset rules according to the result of the statement analysis; determining the data table in the heterogeneous data source that needs to be accessed according to the preset rules obtained by matching; determining the corresponding differentiated SQL grammar rules according to the type of the heterogeneous data source; and decomposing the SQL query request into multiple local query statements based on the table structure information of the data table and the differentiated SQL grammar rules.

[0013] In some embodiments, the outputting the query result to the user layer after assembling the query data includes: performing grouping and sorting operations on the queried data according to the constraint rules of the SQL query request to form the query result.

[0014] The second aspect of the present disclosure provides a data access device based on a virtual database, including: a receiving module, used to receive SQL query requests from a user layer; a SQL statement distribution routing module, used to perform statement analysis on the SQL query request, call preset rules in a rule library based on the results of the statement analysis to decompose the SQL query request into multiple local query statements, and route and distribute the local query statements to corresponding databases to obtain corresponding query data; a data assembly module, used to assemble the query data and output the query results to the user layer.

[0015] In some embodiments, it also includes: a wrapper module, which is used to generate a series of preset rules using a deep learning algorithm; and a rule base module, which is used to write the preset rules.

[0016] In some embodiments, it further includes: a control center module, which is used to verify the similarity using a manual error marking method.

[0017] The third aspect of the present disclosure provides a data access system based on a virtual database, comprising: a virtual database layer, used to execute any of the above-mentioned data access methods based on a virtual database; a user layer, used to issue SQL query requests to the virtual database layer; and a heterogeneous database layer, used to store data sources and provide access queries for the virtual database layer.

[0018] The fourth aspect of the present disclosure provides an electronic device, comprising: one or more processors; a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the above-mentioned data access method based on the virtual database.

[0019] The fifth aspect of the present disclosure further provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the above-mentioned data access method based on a virtual database.

[0020] The sixth aspect of the present disclosure also provides a computer program product, including a computer program, which implements the above-mentioned data access method based on the virtual database when executed by a processor.

[0021] According to the data access method based on a virtual database disclosed in the present invention, a SQL query request from the user layer is received; a statement analysis is performed on the SQL query request, and based on the result of the statement analysis, the preset rules in the rule library are called to decompose the SQL query request into multiple local query statements, and the local query statements are routed and distributed to the corresponding database to obtain the corresponding query data; and the query data is assembled and the query results are output to the user layer. In the data access method based on a virtual database in the embodiment of the present invention, data access to the underlying heterogeneous data source is completed at the virtual database layer, and the results obtained from the access are assembled and uniformly fed back to the user layer, avoiding the problem in the prior art of writing different code logics for different databases to complete the corresponding connection and access operations, resulting in a large amount of code writing, complex request-side business logic, and difficult maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The above contents and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0023] Figure 1 The application scenario diagram of the data access method, apparatus, device, medium and program product based on the virtual database according to the embodiment of the present disclosure is schematically shown;

[0024] Figure 2 A flowchart of a data access method based on a virtual database according to an embodiment of the present disclosure is schematically shown;

[0025] Figure 3 Another flow chart of the data access method based on the virtual database according to an embodiment of the present disclosure is schematically shown;

[0026] Figure 4 A structural block diagram of a data access system based on a virtual database according to an embodiment of the present disclosure is schematically shown;

[0027] Figure 5 A structural block diagram of a data access device based on a virtual database according to an embodiment of the present disclosure is schematically shown;

[0028] Figure 6 The block diagram of a computer system suitable for implementing a data access prevention method based on a virtual database according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0029] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0030] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0031] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.

[0032] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0033] The detailed background technology may include other technical problems besides the technical problems solved by the claim alone.

[0034] An embodiment of the present disclosure provides a data access method based on a virtual database, including: receiving an SQL query request from a user layer; performing statement analysis on the SQL query request, calling preset rules in a rule library based on the result of the statement analysis to decompose the SQL query request into multiple local query statements, and routing and distributing the local query statements to corresponding databases to obtain corresponding query data; and assembling the query data and outputting the query results to the user layer.

[0035] It should be noted that the data access prevention method for the virtual database provided in the embodiments of the present disclosure can be used in big data and distributed technology in microservice data transmission related aspects, and can also be used in various fields other than big data and distributed technology, such as the financial field.

[0036] The data access method based on a virtual database in the embodiment of the present disclosure is particularly suitable for performing data queries on the user side, when the queried data is stored in a heterogeneous data source and when there are multiple types of databases that need to be accessed, which can improve the efficiency of data queries.

[0037] Figure 1 The application scenario diagram of the data access method based on the virtual database according to the embodiment of the present disclosure is schematically shown.

[0038] like Figure 1 As shown, the application scenario 100 according to this embodiment may include user-layer terminal devices 101, 102, 103. A virtual database 104 is used to provide data query and output between the terminal devices 101, 102, 103 and a plurality of databases 105.

[0039] Users can use terminal devices 101, 102, 103 to interact with server 105 through virtual database 104 to query data, etc. Various communication client applications can be installed on terminal devices 101, 102, 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only examples).

[0040] The terminal devices 101 , 102 , and 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0041] The type of the database 105 may be mysql, oracle, informix, and the database 105 stores data for users to view.

[0042] It should be noted that the data access method based on the virtual database provided in the embodiment of the present disclosure can generally be executed by the virtual database layer 104. The virtual database layer receives the data query request sent by the terminal devices 101, 102, 103 of the user layer, obtains the user's query data in the multiple databases 105 by parsing the query request, and assembles the query data to form the query result and returns it to the user layer for the user to view.

[0043] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0044] The following will be based on Figure 1 The scene described by Figure 2-3 The data access method based on the virtual database according to the embodiment of the present disclosure is described in detail.

[0045] Figure 2 The flowchart of the data method based on the virtual database according to the embodiment of the present disclosure is schematically shown.

[0046] like Figure 2 As shown, the data access method based on the virtual database of this embodiment includes operations S210 to S230.

[0047] In operation S210, a SQL query request of a user layer is received.

[0048] The user layer is the initiator of data demand. Users in the user layer enter query requests on the user operation interface of the application according to the needs of data query. After the query request is processed, it sends an SQL query request based on the database management language to the virtual database layer. The SQL query request can be to query a specific type of table structure, query all columns in a specified data table, query a specified column or a specified row in a specified data table, query whether the field content in a specified data table is empty / non-empty, or sort the query results by the value of the field, etc.

[0049] It can be understood that the user layer can be a customer's personal mobile terminal device, such as a mobile phone, a laptop or a desktop computer, or a workstation of an enterprise customer. A data transmission channel is established between the user layer and the underlying heterogeneous data source based on the network. The type of network can be a wired network, a wireless communication network, etc. The relevant application programs of the user layer can access the virtual database through a unified SQL and XML interface to realize the transmission of SQL query requests to the virtual database.

[0050] In operation S220, statement analysis is performed on the SQL query request, and based on the result of the statement analysis, the preset rules in the rule library are called to decompose the SQL query request into multiple local query statements, and the local query statements are routed and distributed to corresponding databases to obtain corresponding query data.

[0051] Before executing operation S220, operations S221 to S222 need to be executed. Figure 3 shown.

[0052] In operation S221: a series of preset rules are generated using a deep learning algorithm;

[0053] In the embodiment of the present disclosure, a structured data table is created. The structured data table is used as a data table for temporarily storing data. It is a mapping of the formal data table stored in the database, and the data information in the heterogeneous data source is extracted, such as using a commonly used for loop to export the corresponding table from the database. And the data information is stored in the structured data table. The data information of the present disclosure includes the library name, table name and table structure information. The table structure information includes the column name, column attributes, constraints and table data in the data table. For example, to export the data information of table A1 from database A to the structured data table, it is necessary to copy the library name of database A, the table name of table A1, and the table structure information in table A1 to the structured data table. The structured data table is created with corresponding field names, field types, field lengths, data records, etc. to classify and record the above data information. A deep learning algorithm is used to collect and match the data information in the structured data table to determine the similarity between different data tables. As mentioned above, the data information of the data tables of different databases is stored in the structured data table, and the data relationship between different data tables can be determined through the process of collection and matching. For example: the field names, field types, field lengths, etc. of data table A1 in database A and data table B1 in database B are the same, and only the recorded data contents are different. At this time, through the process of aggregation and matching, we can get that data table A1 and data table B1 are data tables of the same type. We can create rules for associating heterogeneous data source isomorphic table operations to associate data table A1 and data table B1. When users query data tables of this table structure, they can obtain data table A1 and data table B1 at the same time. Another example: the field names, field types, field lengths, etc. of data table A1 in database A and data table B1 in database B are different, but the recorded data contents are the same. At this time, through the process of aggregation and matching, we can get that data table A1 and data table B1 have the same data records. We can create rules for associating heterogeneous data source non-isomorphic table operations to associate data table A1 and data table B1. When users query the above data records, they can obtain the data records recorded in data table A1 and data table B1 at the same time. In the above judgment process, it is necessary to match different field names, field types, data records, etc. one by one to confirm the similarity between the two data tables, and then define corresponding rules according to the similarity to form preset rules.

[0054] It can be understood that in order to ensure the accuracy of deep learning results, the embodiments of the present disclosure also include using a manual error marking method to verify the similarity, wherein the manual error marking method to verify the similarity includes: uploading the similarity and the relationship view corresponding to the similar fields to the control center; and the administrator of the control center judges the accuracy of the similarity based on the relationship view corresponding to the similar fields, and the names of the current two or more data tables and the databases they are located in will be displayed on the relationship view, and the similarity results and the similar fields matched by such results will be displayed at the same time, so that the administrator can perform manual similarity correction.

[0055] It can be understood that, as mentioned above, data is obtained by querying multiple databases at the bottom layer, and there are differences in the access grammar rules of different databases. In order to facilitate the access of the virtual database layer to the database, the differentiated SQL grammar rules of heterogeneous data sources are written into the rule base in the embodiment of the present disclosure. The grammar rules of existing databases can be counted and their corresponding grammar rules can be entered into the rule base with the database name as the identifier, so that when it is determined that the data path needs to be accessed, the corresponding grammar rules can be called to improve access efficiency.

[0056] In operation S222, a rule base is created based on a series of preset rules. It should be noted that the rule base here is equivalent to a small database built on the virtual database layer, and its creation process is consistent with the database creation process, which will not be repeated here. Its built-in rules are equivalent to data tables in the database. When calling the rules, the results of the statement analysis mentioned above are used as indexes to query the corresponding preset rules in the rule base and use them. The preset rules can be created in the rule base using a variety of commands such as the CREATE RULE command, the Enterprise Manager command, etc. The created rules include the name of the rule, the creation time, the owner, etc., to facilitate the subsequent calling of the preset rules.

[0057] Returning to operation S220, the SQL query request is analyzed, and based on the result of the analysis, the preset rules in the rule library are called to decompose the SQL query request into multiple local query statements, and the local query statements are routed and distributed to the corresponding database to obtain the corresponding query data, including:

[0058] According to the results of statement analysis, preset rules are matched. For example, if the results of statement analysis contain the field name and field type of the data to be queried, all preset rules are traversed according to the field name and field type to find the appropriate preset rule. The statement analysis tool can use the built-in SQL parser in the database, which parses the SQL statement into a parse tree and completes the preprocessing of the grammatical order, query content keywords, etc.

[0059] According to the preset rules obtained by matching, the data tables in the heterogeneous data source that need to be accessed are determined. For example, according to the preset rules, it can be determined that the data required to be queried by the SQL query request is distributed in multiple data tables. The above data tables can be in the same database or in different databases. When the above data tables exist in different databases, the library name of the database where the data table is located can be obtained according to the preset rules, and the corresponding differentiated SQL grammar rules are determined by the library name of the database. Then, based on the table structure information of the data table and the differentiated SQL grammar rules, the SQL query request is decomposed into multiple local query statements. Then the local query statements are routed and distributed to the corresponding database to obtain the corresponding query data. The process of obtaining query data is the same as the method of accessing the database using SQL statements, which will not be repeated.

[0060] In operation S230, the query data is assembled and then the query result is output to the user layer. The data queried from the database is assembled according to the constraint rules in the above SQL query request, and then the query result required by the user is output.

[0061] Based on the above data access method based on a virtual database, the present disclosure also provides a data access device for a virtual database. Figure 5 The device is described in detail.

[0062] Figure 5 The structural block diagram of a data access device based on a virtual database according to an embodiment of the present disclosure is schematically shown.

[0063] like Figure 5 As shown, the embodiment 300 includes a receiving module 301 , a SQL statement distribution routing module 302 , and a data assembly module 303 .

[0064] A receiving module 301, used for receiving a SQL query request at the user layer, and adapted to execute the above step S210;

[0065] The SQL statement distribution routing module 302 is used to perform statement analysis on the SQL query request, call the preset rules in the rule library based on the result of the statement analysis to decompose the SQL query request into multiple local query statements, and route and distribute the local query statements to the corresponding database to obtain the corresponding query data, which is suitable for executing the above step 220;

[0066] The data assembly module 303 is used to assemble the query data and output the query result to the user layer, and is suitable for executing the above step 230.

[0067] For example, the data access method based on a virtual database according to an embodiment of the present disclosure also includes: a wrapper module, which is used to generate a series of preset rules using a deep learning algorithm, suitable for executing step 221 above.

[0068] For example, the data access method based on the virtual database according to the embodiment of the present disclosure further includes: a rule base module, which is used to write preset rules and is suitable for executing the above step 222.

[0069] For example, the data access method based on the virtual database according to the embodiment of the present disclosure further includes: a control center module, which is used to verify the similarity by using a manual error marking method.

[0070] According to the data access method based on a virtual database disclosed in the present invention, a SQL query request from the user layer is received; a statement analysis is performed on the SQL query request, and based on the result of the statement analysis, the preset rules in the rule library are called to decompose the SQL query request into multiple local query statements, and the local query statements are routed and distributed to the corresponding database to obtain the corresponding query data; and the query data is assembled and the query results are output to the user layer. In the data access method based on a virtual database in the embodiment of the present invention, data access to the underlying heterogeneous data source is completed at the virtual database layer, and the results obtained from the access are assembled and uniformly fed back to the user layer, avoiding the problem in the prior art of writing different code logics for different databases to complete the corresponding connection and anti-question operations, resulting in a large amount of code writing, complex request-side business logic, and difficult maintenance. At the same time, the data access method based on the virtual database disclosed in the present invention also has the following advantages: the access channel does not need to control the connection and access strategy of each heterogeneous data source, the logic complexity is reduced, and the maintainability is improved; the data access and integration strategies are unified in the virtual database layer, which reduces the coupling degree between the channel and the downstream data sources; it realizes the unified management of data access and improves the scalability of external services. Since the virtual database layer can integrate and access the data of various heterogeneous data sources, it provides a complete basic platform for the subsequent expansion of business scenarios based on big data.

[0071] According to an embodiment of the present disclosure, any multiple modules in the receiving module 301, the SQL statement distribution routing module 302, and the data assembly module 303 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the receiving module 301, the SQL statement distribution routing module 302, and the data assembly module 303 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware, and firmware or in any appropriate combination of any of them. Alternatively, at least one of the receiving module 301, the SQL statement distribution routing module 302, and the data assembly module 303 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be executed.

[0072] Figure 4 The structural block diagram of a data access system based on a virtual database according to an embodiment of the present disclosure is schematically shown.

[0073] like Figure 4 As shown, the embodiment includes: a virtual database layer, which is used to execute the data access method based on the virtual database in the above embodiment; the virtual database layer in this embodiment includes an SQL statement distribution routing module, the SQL query request sent by the user layer is received by the SQL statement distribution routing module, the SQL statement distribution routing module performs statement analysis on the SQL query request, and calls the preset rules in the rule library according to the result of the statement analysis to decompose the SQL query into several local query statements, and accesses multiple databases through local query statements to return query data. The returned query results enter the data assembly module from the SQL statement distribution routing module, and the data assembly module sorts the query data according to the constraint rules of the SQL query request in the above, and forms query results and sends them to the user layer. The above preset rules are obtained by using the wrapper module to collect and match different underlying databases using a deep learning algorithm.

[0074] The user layer is used to issue SQL query requests to the virtual database layer;

[0075] The heterogeneous database layer is used to store data sources and provide access and query services to the virtual database layer.

[0076] Figure 6A block diagram of an electronic device suitable for implementing a data access method based on a virtual database according to an embodiment of the present disclosure is schematically shown.

[0077] like Figure 6 As shown, the electronic device 400 according to an embodiment of the present disclosure includes a processor 401, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 402 or a program loaded from a storage part 408 to a random access memory (RAM) 403. The processor 401 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 401 may also include an onboard memory for caching purposes. The processor 401 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0078] In RAM 403, various programs and data required for the operation of electronic device 400 are stored. Processor 401, ROM 402 and RAM 403 are connected to each other via bus 404. Processor 401 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 402 and / or RAM 403. It should be noted that the program can also be stored in one or more memories other than ROM 402 and RAM 403. Processor 401 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in one or more memories.

[0079] According to an embodiment of the present disclosure, the electronic device 400 may further include an input / output (I / O) interface 405, which is also connected to the bus 404. The electronic device 400 may further include one or more of the following components connected to the I / O interface 405: an input portion 406 including a keyboard, a mouse, etc.; an output portion 407 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 408 including a hard disk, etc.; and a communication portion 405 including a network interface card such as a LAN card, a modem, etc. The communication portion 405 performs communication processing via a network such as the Internet. A drive 410 is also connected to the I / O interface 405 as needed. A removable medium 411, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 410 as needed, so that a computer program read therefrom is installed into the storage portion 408 as needed.

[0080] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0081] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present disclosure, a computer-readable storage medium may include the ROM 402 and / or RAM 403 described above and / or one or more memories other than ROM 402 and RAM 403.

[0082] The embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the item recommendation method provided by the embodiment of the present disclosure.

[0083] The above functions defined in the system / device of the embodiment of the present disclosure are performed when the computer program is executed by the processor 401. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0084] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 405, and / or installed from the removable medium 411. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0085] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 405, and / or installed from the removable medium 411. When the computer program is executed by the processor 401, the above functions defined in the system of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0086] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0087] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0088] It will be appreciated by those skilled in the art that the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations and / or combinations are not explicitly described in the present disclosure. In particular, the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways without departing from the spirit and teachings of the present disclosure. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0089] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. The scope of the present disclosure is defined by the attached claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A data access method based on a virtual database, applied to the virtual database layer, It is characterized in that include: Receive SQL query requests from the user layer; Perform statement analysis on the SQL query request, call preset rules in a rule library based on the result of the statement analysis to decompose the SQL query request into multiple local query statements, and route and distribute the local query statements to corresponding databases to obtain corresponding query data; as well as After assembling the query data, outputting the query result to the user layer, Wherein, the method further comprises: Using a deep learning algorithm to generate a series of the preset rules; and Creating the rule base based on a series of the preset rules, Wherein, the use of a deep learning algorithm to generate a series of preset rules includes: Create structured data tables; Extracting data information from heterogeneous data sources and storing the data information in the structured data table; Using a deep learning algorithm to aggregate and match the data information in the structured data table to determine the similarity between different data tables; and Corresponding rule definitions are performed according to the similarity to form the preset rules.

2. The data access method based on the virtual database according to claim 1, It is characterized in that The data information includes a library name, a table name, and table structure information. The table structure information includes a column name, column attributes, constraints, and table data in a data table.

3. The data access method based on a virtual database according to claim 1, It is characterized in that The preset rules include operations for associating isomorphic tables with heterogeneous data sources and extracting associative operations for data with the same attributes from non-isomorphic tables.

4. The data access method based on a virtual database according to claim 1, It is characterized in that Also includes: The similarity is verified by using a manual error marking method, wherein the similarity is verified by using a manual error marking method includes: Uploading the similarity and the relationship view corresponding to the similar fields to a control center; and The administrator of the control center determines the accuracy of the similarity based on the relationship view corresponding to the similar fields.

5. The data access method based on virtual database according to claim 1, It is characterized in that Also includes: Write differentiated SQL syntax rules of heterogeneous data sources into the local rule base.

6. The data access method based on the virtual database according to claim 5, It is characterized in that The calling of the preset rules in the rule base based on the result of the statement analysis to decompose the SQL query request into multiple local query statements includes: Matching the preset rules according to the result of the statement analysis; Determine the data table in the heterogeneous data source that needs to be accessed according to the preset rule obtained by matching; Determine the corresponding differentiated SQL syntax rules according to the type of heterogeneous data sources; and The SQL query request is decomposed into a plurality of local query statements based on the table structure information of the data table and the differentiated SQL grammar rules.

7. The data access method based on virtual database according to claim 1, It is characterized in that The assembling of the query data and outputting the query result to the user layer comprises: The queried data is grouped and sorted according to the constraint rules of the SQL query request to form the query result.

8. A data access device based on a virtual database, It is characterized in that include: A receiving module, used for receiving SQL query requests from the user layer; The SQL statement distribution routing module is used to perform statement analysis on the SQL query request, call the preset rules in the rule library based on the result of the statement analysis to decompose the SQL query request into multiple local query statements, and route and distribute the local query statements to the corresponding database to obtain corresponding query data. The steps for creating the rule base are as follows: Using a deep learning algorithm to generate a series of the preset rules; and Creating the rule base based on a series of the preset rules, Wherein, the use of a deep learning algorithm to generate a series of preset rules includes: Create structured data tables; Extracting data information from heterogeneous data sources and storing the data information in the structured data table; Using a deep learning algorithm to aggregate and match the data information in the structured data table to determine the similarity between different data tables; and Defining corresponding rules according to the similarity to form the preset rules; A data assembly module is used to assemble the query data and then output the query result to the user layer.

9. The data access device based on the virtual database according to claim 8, It is characterized in that Also includes: A wrapper module, used to generate a series of the preset rules using a deep learning algorithm; The rule base module is used to write the preset rules.

10. The data access device based on virtual database according to claim 8, It is characterized in that Also includes: The control center module is used to verify the similarity by using a manual error marking method.

11. A data access system based on a virtual database, It is characterized in that include: A virtual database layer, used to execute the data access method based on the virtual database as described in any one of claims 1 to 7; The user layer is used to issue SQL query requests to the virtual database layer; The heterogeneous database layer is used to store data sources and provide access and query services to the virtual database layer.

12. An electronic device, include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors execute the method according to any one of claims 1 to 7.

13. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the method according to any one of claims 1 to 7.

14. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.