Metadata extraction data access mechanism of heterogeneous database in cloud side end environment
Through the metadata extraction and matching mechanism, unified access to heterogeneous databases is achieved, solving the problems of complexity and performance bottlenecks of heterogeneous database access in the existing technology, and improving data access efficiency and consistency.
Patent Information
- Application Number
- CN202411498380.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-24
- Publication Date
- 2025-03-07
AI Technical Summary
The existing technology is difficult to achieve unified access and management of heterogeneous databases, resulting in high development and maintenance costs, and poor access performance in cloud-edge distributed environments.
Through the metadata extraction mechanism, metadata of various databases is uniformly extracted and managed, and the metadata matching and query execution modules are used to realize the precise routing and conversion of SQL query statements to ensure unified access to heterogeneous databases.
It improves the efficiency of data access in cloud-edge environments, reduces performance degradation caused by network delay, and maintains high query efficiency and data consistency by dynamically managing metadata.
Smart Images

Figure CN119576962A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data technology, and specifically to a metadata extraction data access mechanism for heterogeneous databases in a cloud-edge-terminal environment, which is conducive to achieving unified access to heterogeneous databases. Background Art
[0002] With the rapid development of emerging technologies, especially the popularization of applications such as the Internet of Things, cloud computing, and big data analysis, the scale of data is growing exponentially. In these application scenarios, data is widely distributed and diverse, covering various storage forms from structured data to unstructured data. In order to meet the storage and access needs of these applications, database technology continues to evolve, especially database solutions for specific data types and scenarios have been widely used in various fields.
[0003] In the current database ecosystem, different types of databases are optimized for their specific application scenarios. For example, some relational databases are known for their stable transaction processing capabilities and data consistency guarantees, and are often used in core business systems in the fields of finance, telecommunications, etc. Databases used to process large amounts of document data are more commonly used in information retrieval and content management systems. In addition, there are some key-value storage databases, which were originally designed to meet the needs of high-speed access in high-concurrency scenarios and are widely used in social media and e-commerce platforms. At the same time, databases specifically designed for complex queries and full-text retrieval provide efficient solutions for the analysis of massive data.
[0004] Although these databases have performed well in their respective fields, unified access and management across databases have become very complicated due to the great differences in their internal structures and operating modes. Existing developers need to write code for each database separately and deal with different data models and operating interfaces, which greatly increases development and maintenance costs. As data demand increases, the demand for collaboration and data sharing between heterogeneous databases becomes stronger. How to efficiently and uniformly manage and access these heterogeneous data has become a major challenge in the database field.
[0005] Existing technologies have attempted some preliminary solutions. For example, some studies have implemented cross-data source operations by adding a unified interface to the outer layer of the database. However, such methods often rely on the support of the underlying database and are easily limited by performance bottlenecks when processing large-scale heterogeneous data sets. In addition, these methods do not fully consider the complexity of accessing heterogeneous databases in a cloud-edge distributed environment. In particular, when the network is unstable and the data is widely distributed, it is difficult for existing solutions to provide satisfactory access performance.
[0006] To address these issues, this paper proposes a cross-heterogeneous database data access mechanism based on metadata extraction. This mechanism provides data support for subsequent query operations by uniformly extracting and managing metadata from various databases. The metadata extraction module extracts key information such as table structure and field name from different databases and updates them regularly to ensure metadata consistency. Then, the system uses the metadata matching module to accurately route query requests to the appropriate database instance for processing based on the key information in the SQL query statement. Finally, the query execution module of the system converts standardized query statements into specific operations required by the database and encapsulates and processes the query results.
[0007] Compared with the traditional cross-database access mechanism, the advantage of this mechanism lies in its ability to dynamically manage metadata. Through an efficient metadata matching algorithm, the system can maintain high query efficiency in a diverse database environment while effectively reducing performance degradation caused by network delays. In addition, the system also designs a data cache and update mechanism to further reduce the response time of cross-network queries and improve the consistency of query results.
[0008] In summary, the metadata extraction mechanism proposed in this article can effectively improve the efficiency of data access in the cloud-edge environment by abstracting and unifying the structure and query process of heterogeneous databases, and is suitable for complex data operation requirements in large-scale distributed systems. Summary of the invention
[0009] The present invention proposes a metadata extraction data access mechanism for heterogeneous databases in a cloud-edge environment, which is conducive to abstracting the database access model, building a unified access method for various heterogeneous databases, and realizing accurate forwarding and execution of database operations. The main idea of this mechanism is to extract metadata from databases and operation statements (such as table names and field names of structured databases, and specialized identifiers of various databases) for efficient matching to locate database instances, and convert query statements through abstract database query conversion units, and finally send the operation statements to specific database instances, thereby realizing unified access to heterogeneous databases.
[0010] Specifically, this method consists of three components. First, this method designs a metadata extraction component to obtain metadata from the database and SQL statements; second, this method designs a metadata matching component to match the database metadata obtained by the metadata extractor with the metadata in the SQL statement; finally, this method designs a data query component, which converts the SQL statement and sends it to the specified database for execution, and finally uniformly processes and packages the returned data.
[0011] The metadata extraction data access mechanism of the heterogeneous database in the cloud-edge-device environment is characterized by comprising the following steps formed by the data access mechanism based on the metadata extraction:
[0012] Step 1: By abstracting and transforming the database, unified extraction and management of database metadata are achieved;
[0013] Step 2: Match the database metadata obtained by the metadata extractor with the metadata in the SQL statement to accurately locate the database and forward the query request;
[0014] Step three: convert the SQL statement and send it to the specified database for execution. Finally, the returned data is uniformly processed and packaged to achieve unified access to heterogeneous databases.
[0015] Includes the following components:
[0016] Metadata extraction component, used to obtain metadata from databases and SQL statements;
[0017] The metadata matching component matches the database metadata obtained by the metadata extractor with the metadata in the SQL statement;
[0018] The data query component converts SQL statements and sends them to the specified database for execution, and finally processes and packages the returned data in a unified manner.
[0019] This includes unifying access into SQL statements, using pattern matching to extract metadata from SQL statements, extracting metadata from each database at the same time, then using a matching algorithm to automatically locate the database endpoint that should be accessed, and finally converting the SQL statements into corresponding database operations and forwarding them to the corresponding database endpoint.
[0020] Includes the following components:
[0021] Metadata extraction component, used to extract metadata information of heterogeneous databases in a unified and abstract metadata format, providing a basis for unified access to heterogeneous databases;
[0022] The metadata matching component is used to extract metadata information from the unified access statement for matching, so as to accurately locate the database and provide relevant metadata information for subsequent data operation components;
[0023] The data operation component is used to unify the upper-layer operation mode and package the data returned by each database in a unified manner.
[0024] The unified processing package includes the following:
[0025] First, retain the original return information;
[0026] Second, the corresponding field information and field description will be extracted for the query request, and the data will be serialized into an object array corresponding to the field, and the data will be cached at the same time;
[0027] Third, for other requests that cause data changes, the type of operation and the scope of the operation are returned, the cache is invalidated, and the remaining nodes are notified.
[0028] The metadata extraction component includes a database metadata extractor and an SQL statement metadata extractor. The database metadata extractor connects to the corresponding database and exports the metadata of the corresponding database. For relational databases and MongoDB document databases, the database metadata extractor directly exports data from the corresponding data table through the connectors officially provided by each database. For non-relational databases, if it is a simple key-value database, only the key of the key-value pair of the database description in the table is filled in. If it is a full-text search database, the HTTP interface provided by it is accessed, and the database information is aggregated through the information of multiple interfaces. Its index Index corresponds to the entity database description, and its type Type corresponds to the data table description.
[0029] The metadata matching component includes a metadata matching algorithm, which matches the database metadata obtained by the database metadata extractor with the metadata in the SQL statement to achieve the purpose of accurately locating the database and forwarding the query request.
[0030] The validity of database metadata is ensured through two mechanisms. One is to design a configurable lease time to periodically obtain the metadata description of the database. The other is to design an error feedback mechanism. If the corresponding database is located through metadata matching, but the database query fails, the database metadata description will be expired and the database metadata description will be obtained again. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] Figure 1 The present invention relates to a data access mechanism flow chart based on metadata extraction.
[0032] Figure 2 It is a module diagram of a data access mechanism based on metadata extraction involved in the present invention. DETAILED DESCRIPTION
[0033] Combined with the attached figure ( Figure 1-Figure 2 ) The present invention is described in further detail.
[0034] Figure 1 The present invention relates to a data access mechanism flow chart based on metadata extraction. Figure 2 This is a module diagram of the data access mechanism based on metadata extraction involved in the present invention. Figure 1 to Figure 2As shown, a data access mechanism based on metadata extraction is characterized in that it includes the following steps: Specifically, the present invention is composed of three components and works in a modular and interactive manner, wherein the metadata extraction component extracts metadata information of heterogeneous databases in a unified abstract metadata format to provide a basis for unified access to heterogeneous databases, and the metadata matching component extracts metadata information from the unified access statement for matching to accurately locate the database and provide relevant metadata information for subsequent data operation components, and the data operation component unifies the upper-layer operation mode to uniformly package the data returned by each database.
[0035] The specific design and implementation process of the metadata extractor is as follows:
[0036] The metadata extraction design can support the data extraction data access mechanism, code generation expansion mechanism, and semantic analysis multi-level cache mechanism, provide necessary information for these mechanisms, and guide these modules to make correct decisions. This function consists of database metadata extractor and SQL statement metadata extractor.
[0037] The database metadata extractor connects to the corresponding database and exports the metadata of the corresponding database. For relational databases and document databases such as MongoDB that have database metadata for specific databases or collections, the database metadata extractor directly exports data from the corresponding data tables through the connectors officially provided by each database, and merges them into the format shown in Table 1. For non-relational databases, if it is a simple key-value database (such as Redis), only the key of the key-value pair of database description in the table is filled in, that is, the database address, and the value is not filled in, that is, the entity database description; if it is a full-text search database (such as ElasticSearch), the HTTP interface provided by it is accessed, and the database information is aggregated through the information of multiple interfaces and targeted. For example, for ElasticSearch, its index (Index) can correspond to the entity database description, and its type (Type) can correspond to the data table description.
[0038] By appropriately abstracting and transforming the database, this method achieves unified extraction and management of database metadata. However, it should be noted that the structure of the database may change. This method designs two mechanisms to ensure the validity of database metadata. One is to design a configurable lease time to periodically obtain the metadata description of the database; the other is to design an error feedback mechanism. If the corresponding database is located through metadata matching, but the database query fails to execute, it will mainly trigger the expiration of the database metadata description and re-acquire the database metadata description.
[0039] The specific design and implementation process of the metadata matching component is as follows:
[0040] This method designs a metadata matching algorithm to match the database metadata obtained by the metadata extractor with the metadata in the SQL statement to accurately locate the database and forward the query request. This method expands the table and database information field in the SQL statement. If this field carries specific database connection information, that is, IP address and port information, such as 127.0.0.1:3306, the system supports using database aliases to locate the database. It is worth noting that if the system does not contain complete database connection information in the metadata, the subsequent matching process is still required, but the search space of the metadata matching algorithm can be significantly narrowed.
[0041] In this method, the table and database information field is split according to the standard SQL syntax, and the elements in this field are analyzed. If the entity database description information is included, all entity database descriptions are aggregated and matching entity databases are searched in this search space. If only a unique entity database can be matched, the corresponding database connection is returned from the connection pool. If multiple entity databases are found, then the table name and field name are matched.
[0042] If multiple matching entity databases or only data table fields are found, the search space composed of data tables is searched. It is worth noting that if multiple matching entity databases are found, only the search space aggregated by the table information contained in the matched entity database needs to be searched. If only a unique data table can be matched, the corresponding database connection is returned from the connection pool. If multiple data tables are found, the field name is then matched.
[0043] If multiple matching data tables are found, the search space composed of field information is searched. It is worth noting that if multiple matching data tables are found, only the search space aggregated by the field information contained in the matched data tables needs to be searched. If all field information can be matched in the same data table, the corresponding database connection is returned from the connection pool. If fields in multiple data tables can match all field information, the match fails and the user is required to directly specify the database to be connected.
[0044] The data structure of key-value databases such as Redis is relatively simple. Although it includes structures such as sets, hashes, and message queues, it can still be abstracted as a key-value data type in essence. For this type of data, although all keys can be exported as metadata and metadata matching algorithms can be run at the same time, doing so will result in large space overhead and consistency maintenance costs. Therefore, it is recommended to directly specify the database address when using a key-value database.
[0045] It is worth noting that in order to improve the search efficiency of the algorithm, the dictionary tree is used for optimization when searching the solution space, reducing the time complexity of the search from O(n2) to O(nlogn).
[0046] The specific design and execution process of the data query component is as follows:
[0047] Through the two steps of metadata extraction and metadata matching, this method can accurately locate the database that executes the query statement. Next, the SQL statement will be converted and sent to the specified database for execution. Finally, the returned data will be uniformly processed and packaged.
[0048] In the SQL statement conversion step, for relational databases we only need to convert different parts of the SQL statement. Most data table operations do not require special processing, but the management operations of each database are slightly different. For example, Kingbase database does not have the concept of physical database, but uses the concept of "view" to manage a set of data tables, so the management of "view" in Kingbase needs to be converted. For non-relational databases, they cannot directly execute SQL statements, so SQL also needs to be converted into specific operation statements for operation.
[0049] The query conversion unit takes the metadata contained in the SQL statement as input and outputs the corresponding operation method. Specifically, this method abstracts the information contained in the SQL statement into field information, table and database information, query condition information, grouping information, paging information, etc., and combines this information according to the fixed operation methods of each heterogeneous database to generate a complete operation logic. For example, for relational databases, different keywords and special names are converted; for MongoDB, they are converted to the corresponding BSON; for ElasticSearch, they are converted to the corresponding HTTP request, and the request is completed by filling in the URL parameters and request body.
[0050] After completing the conversion from SQL statements to operation statements, this method will execute the query, obtain the corresponding database connection, and forward the data to the corresponding database. After the database returns the execution structure, this method will uniformly package the data returned by each database in order to unify the upper-level operation mode. First, retain the original return information; second, extract the corresponding field information and field description for the query request, and serialize the data into an object array corresponding to the field, and cache the data; third, for other requests that cause data changes, return the type of operation and the scope of the operation, invalidate the cache, and notify the remaining nodes.
[0051] The components are designed as follows:
[0052] (I) Metadata Extraction Component
[0053] The metadata extraction design can support the data extraction data access mechanism, code generation expansion mechanism, and semantic analysis multi-level cache mechanism, provide necessary information for these mechanisms, and guide these modules to make correct decisions. This function consists of database metadata extractor and SQL statement metadata extractor.
[0054] 1. The database metadata extractor connects to the corresponding database and exports the metadata of the corresponding database. For relational databases and document-based databases such as MongoDB that have database metadata that describes a specific database or collection, the database metadata extractor directly exports data from the corresponding data table through the connectors officially provided by each database, and merges it into the format shown in Table 1. For non-relational databases, if it is a simple key-value database (such as Redis), only the key of the key-value pair of database description in the table is filled in, that is, the database address, and the value is not filled in, that is, the entity database description; if it is a full-text search database (such as ElasticSearch), the HTTP interface provided by it is accessed, and the database information is aggregated through the information of multiple interfaces and targeted. For example, for ElasticSearch, its index (Index) can correspond to the entity database description, and its type (Type) can correspond to the data table description.
[0055] 2. By appropriately abstracting and transforming the database, this method achieves unified extraction and management of database metadata. However, it should be noted that the structure of the database may change. This method designs two mechanisms to ensure the validity of database metadata. One is to design a configurable lease time to periodically obtain the metadata description of the database; the other is to design an error feedback mechanism. If the corresponding database is located through metadata matching, but the database query fails to execute, it will mainly trigger the expiration of the database metadata description and re-acquire the database metadata description.
[0056] 2. Metadata Matching Component
[0057] This method designs a metadata matching algorithm to match the database metadata obtained by the metadata extractor with the metadata in the SQL statement, so as to achieve the purpose of accurately locating the database and forwarding the query request.
[0058] This method expands the table and database information field in the SQL statement. If this field carries specific database connection information, that is, IP address and port information, such as 127.0.0.1:3306, the system supports using database aliases to locate the database. It is worth noting that if the system does not contain complete database connection information in the metadata, the following matching process is still required, but the search space of the metadata matching algorithm can be significantly narrowed.
[0059] (III) Data Query Component
[0060] Through the two steps of metadata extraction and metadata matching, this method can accurately locate the database that executes the query statement. Next, the SQL statement will be converted and sent to the specified database for execution. Finally, the returned data will be uniformly processed and packaged.
[0061] In the SQL statement conversion step, for relational databases we only need to convert different parts of the SQL statement. Most data table operations do not require special processing, but the management operations of each database are slightly different. For example, Kingbase database does not have the concept of physical database, but uses the concept of "view" to manage a set of data tables, so the management of "view" in Kingbase needs to be converted. For non-relational databases, they cannot directly execute SQL statements, so SQL also needs to be converted into specific operation statements for operation.
[0062] like Figure 2 The present invention is composed of three components and works interactively in a modular manner. The metadata extraction component extracts metadata information of heterogeneous databases in a unified abstract metadata format to provide a basis for unified access to heterogeneous databases. The metadata matching component extracts metadata information from unified access statements for matching to accurately locate the database and provide relevant metadata information for subsequent data operation components. The data operation component unifies the upper-level operation mode to uniformly package the data returned by each database.
[0063] This method unifies all database access into SQL statement access. For relational databases, only simple adaptation conversion is needed to achieve unified database access. However, for non-relational databases, SQL statements need to be converted into the access form specified by the corresponding database, so it is necessary to understand the information contained in the SQL statement. The SQL statement metadata extractor uses regular expressions to obtain metadata in SQL statements.
[0064] For the query statements in the table, obtain field information, table and database information, query condition information, grouping information, paging information, etc. It is worth noting that this method expands the field of table and database information. Users can directly specify the database to execute this query by specifying the IP address and port. For modify statements, delete statements, and add statements, the above method is also used to extract the metadata contained in the SQL statement. Since these three statements contain less information, this article will not give examples.
[0065] In non-relational databases, the overall operation logic of document-type MongoDB is similar to that of relational databases, and the above extraction scheme can be used directly. However, for key-value databases and full-text search databases, this method needs to restrict the SQL statements they use. As shown in the table, the correspondence between the operation statements and SQL statements of key-value databases (taking Redis as an example) and full-text search databases (taking ElasticSearch as an example) is shown respectively. Among them, the operation of Redis is relatively simple, and the above rules can be followed; for conditional queries and grouping operations supported by ElasticSearch, after parsing the metadata in the SQL statement, just add parameters to the requested URL.
[0066] After extracting the metadata of the SQL statement, this method splits the table and database information field according to the standard SQL syntax and analyzes the elements in this field. If the entity database description information is included, all entity database descriptions are aggregated and matching entity databases are searched in this search space. If only a unique entity database can be matched, the corresponding database connection is returned from the connection pool. If multiple entity databases are found, then the table name and field name are matched.
[0067] If multiple matching entity databases or only data table fields are found, the search space composed of data tables is searched. It is worth noting that if multiple matching entity databases are found, only the search space aggregated by the table information contained in the matched entity database needs to be searched. If only a unique data table can be matched, the corresponding database connection is returned from the connection pool. If multiple data tables are found, the field name is then matched.
[0068] If multiple matching data tables are found, the search space composed of field information is searched. It is worth noting that if multiple matching data tables are found, only the search space aggregated by the field information contained in the matched data tables needs to be searched. If all field information can be matched in the same data table, the corresponding database connection is returned from the connection pool. If fields in multiple data tables can match all field information, the match fails and the user is required to directly specify the database to be connected.
[0069] The data structure of key-value databases such as Redis is relatively simple. Although it includes structures such as sets, hashes, and message queues, it can still be abstracted as a key-value data type in essence. For this type of data, although all keys can be exported as metadata and metadata matching algorithms can be run at the same time, doing so will result in large space overhead and consistency maintenance costs. Therefore, it is recommended to directly specify the database address when using a key-value database.
[0070] It is worth noting that in order to improve the search efficiency of the algorithm, the dictionary tree is used for optimization when searching the solution space, reducing the time complexity of the search from O(n2) to O(nlogn).
[0071] After metadata extraction and metadata matching, the query conversion unit uses the metadata contained in the SQL statement as input and outputs the corresponding operation method. Specifically, this method abstracts the information contained in the SQL statement into field information, table and database information, query condition information, grouping information, paging information, etc., and combines this information according to the fixed operation methods of each heterogeneous database to generate a complete operation logic. For example, for relational databases, different keywords and special names are converted; for MongoDB, they are converted to the corresponding BSON; for ElasticSearch, they are converted to the corresponding HTTP request, and the request is completed by filling in the URL parameters and the request body.
[0072] After completing the conversion from SQL statements to operation statements, this method will execute the query, obtain the corresponding database connection, and forward the data to the corresponding database. After the database returns the execution structure, this method will uniformly package the data returned by each database in order to unify the upper-level operation mode. First, retain the original return information; second, extract the corresponding field information and field description for the query request, and serialize the data into an object array corresponding to the field, and cache the data; third, for other requests that cause data changes, return the type of operation and the scope of the operation, invalidate the cache, and notify the remaining nodes.
[0073] The database metadata format description of the present invention is shown in Table 1. The metadata contained in the query statement of the present invention is shown in Table 2. The corresponding relationship between the key-value database operation statement and the SQL operation statement of the present invention is shown in Table 3. The corresponding relationship between the full-text search database operation statement and the SQL operation statement of the present invention is shown in Table 4. The metadata matching algorithm of the present invention is shown in Table 5.
[0074] Table 1 Description of database metadata format
[0075]
[0076] Table 2 Metadata included in the query statement
[0077]
[0078] Table 3 Correspondence between key-value database operation statements and SQL operation statements (taking Redis as an example)
[0079]
[0080]
[0081] Table 4 Correspondence between full-text search database operation statements and SQL operation statements (taking Elast icSearch as an example)
[0082]
[0083] Table 5 Metadata matching algorithm related table
[0084]
[0085]
[0086] So far, this method can complete the unified access of heterogeneous databases.
Claims
1. Metadata extraction and data access mechanism for heterogeneous databases in cloud-edge-device environments, characterized by: The following steps are involved: Step 1: By abstracting and transforming the database, unified extraction and management of database metadata are achieved; Step 2: Match the database metadata obtained by the metadata extractor with the metadata in the SQL statement to accurately locate the database and forward the query request; Step three: convert the SQL statement and send it to the specified database for execution. Finally, the returned data is uniformly processed and packaged to achieve unified access to heterogeneous databases.
2. According to claim 1, the metadata extraction data access mechanism for heterogeneous databases in a cloud-edge-device environment is characterized in that: Includes the following components: Metadata extraction component, used to obtain metadata from databases and SQL statements; The metadata matching component matches the database metadata obtained by the metadata extractor with the metadata in the SQL statement; The data query component converts SQL statements and sends them to the specified database for execution, and finally processes and packages the returned data in a unified manner.
3. According to claim 1, the metadata extraction data access mechanism for heterogeneous databases in a cloud-edge-device environment is characterized in that: This includes unifying access into SQL statements, using pattern matching to extract metadata from SQL statements, extracting metadata from each database at the same time, then using a matching algorithm to automatically locate the database endpoint that should be accessed, and finally converting the SQL statements into corresponding database operations and forwarding them to the corresponding database endpoint.
4. According to claim 1, the metadata extraction data access mechanism for heterogeneous databases in a cloud-edge-device environment is characterized in that: Includes the following components: Metadata extraction component, used to extract metadata information of heterogeneous databases in a unified abstract metadata format, providing a basis for unified access to heterogeneous databases; The metadata matching component is used to extract metadata information from the unified access statement for matching, so as to accurately locate the database and provide relevant metadata information for subsequent data operation components; The data operation component is used to unify the upper-layer operation mode and uniformly package the data returned by each database.
5. According to claim 1, the metadata extraction data access mechanism for heterogeneous databases in a cloud-edge-device environment is characterized in that: The unified processing package includes the following: First, retain the original return information; Second, the corresponding field information and field description will be extracted for the query request, and the data will be serialized into an object array corresponding to the field, and the data will be cached at the same time; Third, for other requests that cause data changes, the type of operation and the scope of the operation are returned, the cache is invalidated, and the remaining nodes are notified.
6. The metadata extraction data access mechanism for heterogeneous databases in a cloud-edge-device environment according to claim 2 or 4 is characterized in that: The metadata extraction component includes a database metadata extractor and an SQL statement metadata extractor. The database metadata extractor connects to the corresponding database and exports the metadata of the corresponding database. For relational databases and MongoDB document databases, the database metadata extractor directly exports data from the corresponding data table through the connectors officially provided by each database. For non-relational databases, if it is a simple key-value database, only the key of the key-value pair of database description in the table is filled in. If it is a full-text search database, the HTTP interface provided by it is accessed, and the database information is aggregated through the information of multiple interfaces. Its index Index corresponds to the entity database description, and its type Type corresponds to the data table description.
7. The metadata extraction data access mechanism for heterogeneous databases in a cloud-edge-device environment according to claim 2 or 4 is characterized in that: The metadata matching component includes a metadata matching algorithm, which matches the database metadata obtained by the database metadata extractor with the metadata in the SQL statement to achieve the purpose of accurately locating the database and forwarding the query request.
8. The metadata extraction data access mechanism for heterogeneous databases in a cloud-edge-device environment according to claim 1 is characterized in that: The validity of database metadata is ensured through two mechanisms. One is to design a configurable lease time to periodically obtain the metadata description of the database. The other is to design an error feedback mechanism. If the corresponding database is located through metadata matching, but the database query fails, the database metadata description will be expired and the database metadata description will be obtained again.
Citation Information
Patent Citations
Database access method and database access system based on cloud calculation
CN104090937A
Database and file system access method and apparatus
CN106897322A
Unified database management query method and device
CN108197277A
Data query method, system and device
CN110399388A
Heterogeneous database data unified access method and related equipment
CN110597844A