A method and system for schema conversion from a relational database to a document database

Abstract

The application provides a relational database to document database mode conversion method and system, and relates to the technical field of computer, which comprises the following steps: extracting data mode in the relational database; completing the conversion from the relational model to the document mode based on the interactive genetic algorithm; showing the document mode obtained by the mode conversion module and providing the user with feedback, and then continuously iterating to generate a new document mode. The method provided by the application can guarantee that all relational tables and foreign key relationships in the relational mode are converted, and the document mode evaluation algorithm is used to measure the goodness of the converted document mode, which is beneficial to simplifying the mode conversion process. For the relational mode, a series of candidate document modes are converted in an iterative manner, the user can evaluate the candidate document modes according to the domain knowledge, the method re-generates the document mode by referring to the feedback, and the document mode meeting the user's expectation is obtained.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, and in particular, to a relational database to document database mode conversion method and system. BACKGROUND

[0002] With the advent of Web2.0 and big data era, mobile Internet has developed rapidly, and the amount of data has increased dramatically. According to IDC prediction, the total amount of global data is estimated to grow from 33ZB in 2018 to 175ZB in 2025. The development of computer technology has given birth to various new application software, and the business scenarios are becoming more and more diversified and complex. A series of problems of traditional relational database begin to emerge, which are specifically manifested as follows:

[0003] (1) The data scale grows geometrically, and the requirement for concurrent query operation of the database is continuously improved. Due to the strict foreign key constraint, the connection operation between tables is inefficient when the relational database is queried, which leads to poor performance of the relational database in the high-performance concurrent read-write scenario.

[0004] (2) Due to the difficulty of dispersing data to each node, the relational database uses vertical expansion to improve the performance and capacity of the database, which is very costly, and people begin to seek a more inexpensive horizontal expansion way to expand the database to achieve high database capacity and availability.

[0005] (3) The data storage scene is no longer limited to OLTP, and is widely used in OLAP, data warehouse, data visualization, machine learning and other fields. The data storage mode of enterprises is no longer limited to structured data, and the storage demand for heterogeneous data is increasingly obvious. The relational database requires the data mode stored in the table structure to be fixed, and even a small amount of data mode inconsistency also needs to re-create a relational table, which is difficult to store heterogeneous data.

[0006] In the mode conversion aspect, the traditional heterogeneous database migration usually uses ETL method, which often needs customization. The data mode conversion strategy can usually only rely on the experience of database administrators, and a large amount of manpower and material resources are needed for development and testing, which is too high in cost and does not have universality. SUMMARY

[0007] The present application provides a relational database to document database mode conversion method and system, which solves the problem that the data mode conversion strategy can usually only rely on the experience of database administrators, and a large amount of manpower and material resources are needed for development and testing, which is too high in cost and does not have universality.

[0008] To solve the above-mentioned invention purposes, the technical scheme provided by the application is as follows: a mode conversion method from a relational database to a document database, characterized in that the steps comprise:

[0009] S1, obtaining related data in the relational database to be converted, inputting the related data into a mode extraction module, and performing data mode extraction; wherein the related data comprises a connection mode or a relationship mode of the relational database and a mapping conversion rule;

[0010] S2, executing a mode conversion module to convert the relationship mode of the relational database to a document mode, and obtaining the document mode;

[0011] S3, showing the document database to the user, judging whether there is a document mode meeting the user's expectation, if not, the user evaluates the document mode, obtains user evaluation data, and returns to S2 for re-conversion until the user's expectation is met, and the mode conversion from the relational database to the document database is completed.

[0012] Preferably, in step S1, the related data of the relational database to be converted is obtained, the related data is input into the mode extraction module, and the data mode extraction is performed, comprising:

[0013] S11, extracting relationship mode information of the relational database; wherein the extraction method comprises: obtaining the connection mode of the relational database, extracting the relationship mode information according to the connection mode; and describing the to-be-converted relationship mode information through a configuration file, wherein the format of the configuration file needs to follow a specific format, the specific format is a json file containing two list data types of elements "atoms" and "relationships";

[0014] S12, obtaining the data type mapping rule from the relational database to the document database; if there is no mapping rule, the default data type conversion rule will be used.

[0015] Preferably, in step S2, the mode extraction module is executed to convert the relationship mode of the relational database to a document mode, and the document database is obtained, comprising:

[0016] S21, reading the relationship mode of the relational database, converting the relationship mode into a graph structure through a relationship mode extraction algorithm, saving the meta-information of the relationship table through the graph structure, and obtaining a relationship graph;

[0017] S22, according to the graph structure, calling a connected mode graph extraction algorithm to separate all mode connected graphs in the relationship graph.

[0018] S23, according to the mode connected graph, using the chromosome structure in the genetic algorithm to model the mode connected graph;

[0019] S24, using a mode conversion algorithm to complete the conversion of the connected mode graph to a hypergraph structure, and evaluating the hypergraph structure;

[0020] S25, according to the hypergraph structure, using a JSON structure conversion algorithm to perform structure conversion of the document mode, using an application data type mapping rule to perform data type conversion of the document mode, and obtaining a candidate document mode.

[0021] Preferably, in step S21, the method will read the relational mode of the relational database, convert it to a graph structure through a relational mode extraction algorithm, and save the meta information of the relational table through the graph structure to obtain a relational graph, including:

[0022] The relational mode of the relational database is obtained by connecting the relational database, and the relational mode contains all table structures, indexes, primary keys, and foreign keys of the database and other information;

[0023] The relational mode is abstracted into a simple graph G rdb : G rdb =(V, E);

[0024] Wherein, the node V represents the table structure, records the primary key, field, field size, and data number; the edge E represents the foreign key relationship, and records the two tables connected by the foreign key and the degree relationship;

[0025] Determine whether the relational mode can be correctly read, if yes, perform graph construction on the relational mode, store the relational table as a vertex and the foreign key relationship as an edge, and convert it into an undirected graph;

[0026] The fields, field types, primary keys, field sizes, and record numbers of the relational table are stored in the vertex; the attributes of the foreign key connection and the degree relationship between the two tables are stored in the edge storage, and a relational graph is obtained.

[0027] Preferably, in step S23, the chromosome structure in the genetic algorithm is used to model the mode connected graph, including:

[0028] According to the genetic algorithm, each individual is composed of three integer chromosomes, and each chromosome structure represents structure division, conversion direction of the foreign key, and conversion rule; initialize the population, generate 20 individuals, and model the mode connected graph.

[0029] Preferably, in step S24, a mode conversion algorithm is used to complete the conversion of the connected mode graph to a hypergraph structure, and the hypergraph structure is evaluated, including:

[0030] S241, for the individuals in the population, execute the mode conversion algorithm to convert to a hypergraph model according to the chromosome structure;

[0031] S242, using the similarity analysis of the documents in the high-quality document library based on the adaptive value function based on the storage cost and the query cost, taking the adaptive value function as an evaluation function, evaluating the supergraph structure obtained by conversion, and taking the TopN supergraphs with high adaptive values as candidate supergraphs.

[0032] Preferably, in step S3, the document mode is shown to the user, and it is determined whether there is a document mode meeting the user's expectation. If not, the user evaluates the document mode, obtains user evaluation data, and returns to S2 for re-conversion until the user's expectation is met, and the mode conversion from the relational database to the document database is completed, including:

[0033] S31, the method shows the TopN document structures obtained by conversion to the user and the fitness, and if there is a document structure meeting the user's satisfaction, the process ends, otherwise the user can change the fitness of a certain document structure or add it to the high-quality document library;

[0034] S32, the next generation mode conversion is performed, selection, crossover, mutation and other operations are performed on the chromosomes in the population, new individuals are generated, and S24 is returned to be executed.

[0035] A mode conversion system from a relational database to a document database, the system is used for the mode conversion method from the relational database to the document database, and the system comprises:

[0036] A mode extraction module is used to obtain related data in the relational database to be converted, input the related data into the mode extraction module, and perform data mode extraction. The related data includes the connection mode or the relationship mode of the relational database and the mapping conversion rule.

[0037] A mode conversion module is used to convert the relationship mode of the relational database to the document mode, and obtain the document database.

[0038] A human-computer interaction module is used to show the document database to the user, determine whether there is a document mode meeting the user's expectation, if not, the user evaluates the document mode, obtains user evaluation data, returns to the mode conversion module for re-conversion until the user's expectation is met, and completes the mode conversion from the relational database to the document database.

[0039] Preferably, the mode extraction module is further used to extract the relationship mode information of the relational database. The extraction method includes: obtaining the connection mode of the relational database, and extracting the relationship mode information according to the connection mode; or describing the relationship mode information to be converted through a configuration file, wherein the format of the configuration file needs to follow a specific format.

[0040] Obtaining a data type mapping rule from a relational database to a document database; if the mapping rule does not exist, a default data type conversion rule will be used.

[0041] Preferably, the schema extraction module is further configured to extract a relational schema of the relational database by a relational schema extraction algorithm, and convert the relational schema into a graph structure, wherein the graph structure stores meta-information of the relational tables, and obtains a relational graph.

[0042] According to the graph structure, a connected schema graph extraction algorithm is called to separate all the connected graphs of the schema in the relational graph.

[0043] According to the connected schema graph, a chromosome structure in a genetic algorithm is used to model the connected schema graph.

[0044] A schema conversion algorithm is used to complete the conversion of the connected schema graph to a hypergraph structure, and the hypergraph structure is evaluated.

[0045] According to the hypergraph structure, a JSON structure conversion algorithm is used to convert the structure of the document schema, and an application data type mapping rule is used to convert the data type of the document schema to obtain a candidate document schema.

[0046] In one aspect, an electronic device is provided, the electronic device comprising a processor and a memory, the memory storing at least one instruction, the at least one instruction being loaded and executed by the processor to implement the above-mentioned relational database to document database schema conversion method.

[0047] In one aspect, a computer-readable storage medium is provided, the storage medium storing at least one instruction, the at least one instruction being loaded and executed by a processor to implement the above-mentioned relational database to document database schema conversion method.

[0048] The above technical solution has at least the following beneficial effects compared with the prior art:

[0049] The scheme has the advantages that the relational mode of the relational database can be quickly obtained by the mode extraction module without manual retrieval of metadata information by the user, and the operation process is simplified; the relational mode to document mode conversion is completed by the mode conversion module, the graph model and hypergraph model are used to define the relational mode and the document mode respectively, and the possible conversion rules are analyzed in detail, and the corresponding mode conversion algorithm is designed, so that the high-quality document structure can be given as much as possible for the user to select without user intervention in the design. The interactive design idea is introduced, the man-machine interaction module and the mode conversion module described in the scheme cooperate with each other, so that the user can participate in the mode conversion process by evaluating the document structure, and each selection and evaluation of the user can affect the direction of subsequent mode conversion, guide the mode converter to obtain the expected document mode, and reduce the difficulty of mode design for the user. BRIEF DESCRIPTION OF DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort on the basis of these drawings.

[0051] Figure 1 is a mode conversion method flow diagram of the relational database to the document database provided by the embodiment of the present application;

[0052] Figure 2 is a system architecture diagram of the mode converter provided by the embodiment of the present application;

[0053] Figure 3 is a relational mode extraction flow diagram of the mode converter provided by the embodiment of the present application;

[0054] Figure 4 is a flow diagram of the mode converter provided by the embodiment of the present application for completing the relational mode to document mode conversion;

[0055] Figure 5 is a mode conversion system block diagram of the relational database to the document database provided by the embodiment of the present application;

[0056] Figure 6 is a structural schematic diagram of an electronic device provided by the embodiment of the present application. DETAILED DESCRIPTION

[0057] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions of the embodiments of the present application will be described clearly and completely below with reference to the drawings of the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the described embodiments of the present application, all other embodiments obtained by a person of ordinary skill in the art without any inventive effort fall within the scope of protection of the present application.

[0058] The present application aims at the problem that the existing data mode conversion strategy can only rely on the experience of database administrators, needs a large amount of manpower and material resources for development and testing, has a high cost and is not universal, and provides a mode conversion method from a relational database to a document database.

[0059] As shown in Figure 1 The embodiments of the present application provide a mode conversion method from a relational database to a document database, which can be implemented by an electronic device. As shown in Figure 1 The mode conversion method from a relational database to a document database is shown in a flowchart, and the processing flow of the method can include the following steps:

[0060] In step S101, the related data of the relational database to be converted is obtained, and the related data is input to a mode extraction module for data mode extraction. The related data includes the connection mode or the relationship mode of the relational database and the mapping conversion rule.

[0061] In a feasible implementation, in step S101, the related data of the relational database to be converted is obtained, and the related data is input to a mode extraction module for data mode extraction, including:

[0062] In step S111, the relationship mode information of the relational database is extracted. The extraction method includes: obtaining the connection mode of the relational database, and extracting the relationship mode information according to the connection mode; or describing the relationship mode information to be converted in the form of a configuration file, wherein the format of the configuration file needs to follow a specific format. The specific format is a json file containing two list data types of elements "atoms" and "relationships".

[0063] In a feasible implementation, the user can obtain the relationship mode information in the following ways:

[0064] (2) If the relationship mode information cannot be obtained by connecting the database, the user can describe the relationship mode information to be converted in the form of a configuration file, and the file format needs to follow a specific format. That is, the configuration file replaces the mode extraction process.

[0065] In an embodiment, the specific format description information is as follows:

[0066] This file should be a json file, containing two list data types of elements "atoms" and "relationships"

[0067] The "atoms" element stores the relationship table in the relationship mode, and the key is the table name, and a dict is nested, which stores the attribute field information. The attribute with an asterisk is the primary key, and the attribute value is the attribute field size. The "relationships" element stores the foreign key relationship in the relationship mode, which is used to represent the association and degree relationship of the two table structure keys.

[0068] S112, obtain the data type mapping rule of the relational database to the document database; if the mapping rule does not exist, a default data type conversion rule will be used.

[0069] S102, executing a mode extraction module, converting the relational database to a document mode, obtaining a document mode.

[0070] In an embodiment, as shown in Figure 1 The flow chart for completing the mode conversion by the mode converter provided by the application, the specific process includes:

[0071] Obtain database meta information from the relational database, and extract an ER model

[0072] Convert the ER model to an undirected graph model of the relationship mode

[0073] Through structure division and rule conversion, the undirected graph model is converted to a hypergraph model of the document mode

[0074] Convert the hypergraph to a JSON physical model.

[0075] In an embodiment, in step S102, the mode conversion module converts the relational database to a document mode, obtaining a document mode, including:

[0076] S121, read the relationship mode of the relational database, convert the relationship mode to a graph structure through a relationship mode extraction algorithm, save the meta information of the relationship table through the graph structure, and obtain a relationship graph.

[0077] In an embodiment, in step S121, the relationship mode of the relational database is read, the relationship mode is converted to a graph structure through a relationship mode extraction algorithm, the meta information of the relationship table is saved through the graph structure, and a relationship graph is obtained, including:

[0078] The relational pattern extraction algorithm of the present application models the extracted relational pattern in the form of a graph, and the relational pattern extraction algorithm is described as follows:

[0079] The relational pattern is obtained by connecting a relational database, and the relational pattern contains information such as all table structures, indexes, primary keys and foreign keys of the database. The relational pattern can be abstracted as a simple graph G rdb =(V, E), wherein the node V represents a table structure, records a primary key, a field, a field size and a number of records; and the edge E represents a foreign key relationship, records two tables connected by the foreign key and a degree relationship.

[0080] It is judged whether the relational pattern can be correctly read, and if yes, the relational pattern is graph constructed, the relational table is stored as a vertex, and the foreign key relationship is stored as an edge to be converted into an undirected graph.

[0081] The field name, field type, primary key, field size and number of records of the relational table are stored in the vertex; the attribute of the foreign key connection and the degree relationship between the two tables are recorded in the edge storage, and a relational graph is obtained.

[0082] S122, according to the graph structure, a connected pattern graph extraction algorithm is called to separate all the connected pattern graphs in the relational graph.

[0083] In a feasible implementation, the specific steps of the S122 are as follows:

[0084] For the relational graph, all connected pattern graphs are extracted, and a certain node is randomly selected to find all nodes connected thereto by using the dfs method. If there is no edge between two nodes to connect them, the two tables do not exist in the relational model. The nodes in each connected pattern graph can query each other.

[0085] S123, according to the connected pattern graph, a chromosome structure in a genetic algorithm is used to model the connected pattern graph.

[0086] In a feasible implementation, in step S123, the connected pattern graph is modeled by using a chromosome structure in a genetic algorithm, including:

[0087] According to the genetic algorithm, each individual is composed of three integer chromosomes, and each chromosome structure represents structure division, conversion direction of the foreign key and conversion rule; a population is initialized, 20 individuals are generated, and the connected pattern graph is modeled.

[0088] S124, a pattern conversion algorithm is used to complete the conversion of the connected pattern graph to a hypergraph structure, and the hypergraph structure is evaluated.

[0089] In an embodiment, in step S124, a schema conversion algorithm is used to convert the connected schema graph into a hypergraph structure, and the hypergraph structure is evaluated, including:

[0090] S1241, for individuals in the population, according to the chromosome structure, the mode conversion algorithm is used to convert into a hypergraph model;

[0091] S1242, using the fitness value function based on the storage cost and the query cost and the similarity analysis of the documents in the high-quality document library, the converted hypergraph structure is evaluated, the fitness value function is used as the evaluation function, and the top N hypergraphs with high fitness values are used as candidate hypergraphs.

[0092] S125, according to the hypergraph structure, a JSON structure conversion algorithm is used to convert the document schema, and an application data type mapping rule is used to convert the document schema data type, to obtain a candidate document schema.

[0093] In an embodiment, the fitness value function is used as the evaluation function. The specific fitness value function (or evaluation function) is measured from two angles: 1. The query efficiency of the document schema for a group of queries 2. The data redundancy.

[0094] In an embodiment, the specific steps of S125 are:

[0095] S1251: For the candidate hypergraph, a JSON structure conversion algorithm is executed to obtain the JSON document structure represented by the candidate hypergraph;

[0096] S1252: For the data types of the attributes in different relationship tables, a data type mapping rule is applied to obtain the final candidate document schema.

[0097] In an embodiment, as shown in Figure 2 The system architecture diagram of the schema converter provided by the application comprises three parts, namely an application layer, a conversion layer, and a storage layer. The storage layer comprises a relational database and a hypergraph schema database. The relational database is used to obtain database metadata information and obtain the relational schema of the database. The hypergraph schema database is used to store the hypergraph model and perform model conversion and iterative optimization. The conversion layer is the core of the tool and completes the conversion of the relational schema to the document schema. It comprises four modules, namely a schema extraction module, a schema conversion module, a human-computer interaction module, and a log module. In the application layer, the user selects the physical model of the document schema that he is most satisfied with from the schema conversion interaction system, and then redesigns in the database software according to specific needs to obtain the final document structure.

[0098] The core of the tool is the conversion layer, in which the schema extraction module is responsible for the source of data, through the JDBC connection relational model database, parse and save the mapping information of field type, read the database metadata information and convert it into a relational schema diagram, which records the table structure, field size, data volume, foreign key and degree relationship of the database, and prepares for the next step of schema conversion. The schema conversion module is responsible for the conversion and optimization of relational schema to document schema, using the schema conversion method to generate and analyze the chromosome structure, and convert it into a hypergraph structure for evaluation. Finally, through data type mapping, the final document structure is obtained. The human-computer interaction module will display the TopN document structures with higher fitness, and the user can select high-quality individuals to add to the hypergraph database according to personal preferences for subsequent interactive genetic algorithm evaluation, or modify the fitness value of the individual, completing the process of human-computer cooperation.

[0099] For the schema extraction module, it is the input source in the conversion layer, its main function is to connect the relational database, read the database metadata information and complete the data type mapping analysis, and then convert a relational schema diagram. For example Figure 3The extraction flow chart of the relational schema is shown. Since different relational databases store database meta-information in different system table structures, and different database versions, database drivers, encoding formats, etc. will result in different logic when reading database metadata, a special schema extractor needs to be designed for different relational databases. For databases without a schema extractor, a configuration file can be used to give the relational schema graph in the tool. There are many types of relational databases, and the supported data types and semantics are not completely the same, and even the same field type may represent different semantics. For example, when representing floating point data type data, MySQL uses DECIMAL(10, 2) to represent 10 digits, including 2 decimal places, while in Orcale, NUMBER(10, 2) is used to express the above semantics; PostgreSQL and SQL Server do not completely agree on the semantics of date data type. In PostgreSQL, the date data type stores the date format as "YYYY-MM-DD". In SQL Server, the date data type stores the date format as "YYYY-MM-DD HH:MI:SS". Therefore, different data type mapping information needs to be designed for different databases. When there is no matching data type mapping information, the relational model extraction module will use the default data type mapping table given in Table 1 for mapping, and if it is still not matched, it will be mapped to String type data. If there is no edge between two nodes to connect them, then the two tables have no association in the relational model and cannot be associated by foreign key. Therefore, the relational model graph can be converted into multiple undirected connected graphs, each connected graph representing a structurally related connected sub-schema, and all the relational tables in the connected sub-schema are connected by foreign keys and can be connected for query.

[0100] Table 1. Default data type

[0101]

[0102] The mode conversion module is composed of a mode optimizer, a mode converter and a document schema builder, and is responsible for the conversion of the relational schema graph to the document structure. The mode converter determines the conversion mode of the relational schema through the chromosome structure, and the chromosome structure and coding are defined as follows: three chromosomes are used, each with a length of |E|, the number of edges in the connected graph. The chromosome is coded with an integer, which represents the sequence number of the document structure module to which the foreign key belongs, the conversion direction and the rule. The gene value range of the document structure module division is 1- |E|; the gene value range of the conversion direction is {1, 2, 3}, 1 representing right-to-left conversion, 2 representing left-to-right conversion, and 3 representing both-to-left conversion; and the gene value range of the foreign key relationship conversion rule is {1, 2}, 1 representing a reference relationship and 2 representing a nested relationship. The relational model is converted into a hypergraph model through a hypergraph construction algorithm, and the hypergraph model is saved in a hypergraph schema database. The document schema converter is responsible for converting the hypergraph model into a physical model of the document schema, such as a JSON format. The mode optimizer executes an interactive genetic algorithm to iteratively generate new individuals and evaluate the quality of the individuals through an adaptive value function. The fittest individuals in the population are retained using a survival of the fittest method, and the fittest individuals are passed to the mode converter to generate a new hypergraph structure.

[0103] S103, showing the document database to the user, judging whether there is a document schema that meets the user's expectations, if not, the user evaluates the document schema, obtains user evaluation data, returns to S2 for re-conversion, until it meets the user's expectations, completing the mode conversion from the relational database to the document database.

[0104] Preferably, in step S103, the document database is shown to the user, and it is judged whether there is a document database that meets the user's expectations, if not, the user evaluates the document schema, obtains user evaluation data, returns to S102 for re-conversion, until it meets the user's expectations, completing the mode conversion from the relational database to the document database, comprising:

[0105] S131, showing the TopN document structures obtained by conversion and the fitness, if there is a document structure that meets the user's expectations, the process ends, otherwise the user can change the fitness of a certain document structure or add it to the high-quality document library;

[0106] S132, execute the next generation mode conversion, perform selection, crossover, mutation and other operations on the chromosomes in the population to generate new individuals, and return to execute S124.

[0107] In a feasible implementation, for the human-computer interaction module, the human-computer interaction module is composed of three parts, which are candidate individual area, best individual and operation area. The candidate individual area shows the document structure of the top N high-quality individuals with the highest fitness in the current population after each iteration, and the corresponding chromosome structure and fitness value are given. The user can change the fitness value of the individual according to his own preference; or select a certain individual to add it to the high-quality individual library to increase the generation of the document mode similar to the individual structure in the subsequent iteration process; or fix part of the structure of a certain individual to guide the subsequent conversion of the document mode. The best individual shows the individual with the minimum fitness in the first N generations. If a certain individual is added to the high-quality individual library in the Nth generation, the best individual will only be generated in the N+1th generation, because the fitness values of all individuals in the population have changed. The operation area shows the current iteration number, running time and whether to perform the next iteration operation.

[0108] As shown in Figure 4 , a flowchart executed by the tool is given. For a set of relational models and query loads input by the user, the mode conversion module will derive a series of document structures, and show the top N document structures with the highest evaluation function values to the user. If the user is not satisfied with the document structure, the user will give feedback to the document structure in the form of evaluation, which will affect the document structure generated by the next generation of mode converter, realize human-computer cooperation, and obtain the user's satisfactory document structure.

[0109] In the embodiment of the application, the mode converter from the relational database to the document database can quickly obtain the relational mode of the relational database through the mode extraction module, without the need for the user to manually search metadata information, thereby simplifying the operation process. The mode conversion module completes the conversion from the relational mode to the document mode. The graph model and hypergraph model are used to define the relational mode and the document mode respectively, and the possible conversion rules are analyzed in detail, and the corresponding mode conversion algorithm is designed, which can ensure that the high-quality document structure is given as much as possible for the user to select without the user's intervention in the design, without the loss of the relational table and the relational mode. The scheme introduces the idea of interactive design. The human-computer interaction module and the mode conversion module described in the present application cooperate with each other, so that the user can participate in the mode conversion process by evaluating the document structure. Each selection and evaluation of the user can affect the direction of subsequent mode conversion, guide the mode converter to derive the expected document mode, and reduce the difficulty of mode design for the user.

[0110] Figure 5 is a relational database to document database mode conversion system schematic diagram of the application, the system 200 is used for the above-mentioned relational database to document database mode conversion, the system 200 includes:

[0111] The mode extraction module 210 is configured to obtain relevant data in the relational database to be converted, input the relevant data into the mode extraction module, and perform data mode extraction. The relevant data includes a connection mode or a relationship mode of the relational database and a mapping conversion rule.

[0112] The mode conversion module 220 is configured to convert the relationship mode of the relational database into a document mode.

[0113] The human-computer interaction module 230 is configured to display the document mode to a user, determine whether there is a document mode meeting the user's expectation, and if not, the user evaluates the document mode, obtains user evaluation data, and returns the mode conversion module for re-conversion until the user's expectation is met, and the mode conversion from the relational database to the document database is completed.

[0114] Preferably, the mode extraction module 210 is further configured to extract relationship mode information of the relational database. The extraction method includes obtaining a connection mode of the relational database, and extracting the relationship mode information according to the connection mode. Alternatively, the relationship mode information to be converted is described in a configuration file, and the format of the configuration file needs to comply with a specific format.

[0115] The data type mapping rule from the relational database to the document database is obtained. If the mapping rule does not exist, a default data type conversion rule is used.

[0116] In an available implementation, the mode extraction module includes a plurality of database mode extractors, and different relationship databases use different mode extractors. For a database that does not support automatic extraction of data mode, the user can describe the relationship mode in the form of a file.

[0117] Preferably, the mode conversion module 220 is further configured to read the relationship mode of the relational database, convert the relationship mode into a graph structure by using a relationship mode extraction algorithm, save the meta-information of the relationship table in the graph structure, and obtain a relationship graph.

[0118] According to the graph structure, a connected mode graph extraction algorithm is called to separate all the connected mode graphs in the relationship graph.

[0119] According to the connected mode graph, a chromosome structure in a genetic algorithm is used to model the connected mode graph.

[0120] The connected mode graph is converted into a hypergraph structure by using a mode conversion algorithm, and the hypergraph structure is evaluated.

[0121] According to the hypergraph structure, the document mode is converted by using a JSON structure conversion algorithm, and the document mode data type is converted by using an application data type mapping rule, to obtain a candidate document mode.

[0122] Preferably, the schema conversion module 220 is further configured to obtain a database relationship mode by connecting a relational database, and the relationship mode includes information such as all table structures, indexes, primary keys, and foreign keys of the database.

[0123] The relationship mode is abstracted as a simple graph G rdb : G rdb =(V, E).

[0124] The node V represents a table structure, and records a primary key, a field, a field size, and a number of records. The edge E represents a foreign key relationship, and records two tables connected by the foreign key and a degree relationship.

[0125] It is determined whether the relationship mode can be correctly read. If yes, the relationship mode is graph constructed, the relationship table is stored as a vertex, and the foreign key relationship is stored as an edge, to convert the relationship mode into an undirected graph.

[0126] The fields, the field types, the primary keys, the field sizes, and the numbers of records of the relationship table are stored in the vertex. The attributes of the foreign key connection and the degree relationship between the two tables are stored in the edge storage, to obtain a relationship graph.

[0127] Preferably, the schema conversion module 220 is further configured to initialize a population, generate 20 individuals, and model a schema connected graph according to a genetic algorithm, each individual being composed of three integer chromosomes, and each chromosome structure representing a structure division, a conversion direction of a foreign key, and a conversion rule.

[0128] Preferably, the schema conversion module 220 is further configured to convert the individuals in the population into hypergraph models according to the chromosome structures.

[0129] The converted hypergraph structure is evaluated by using an adaptive value function based on a storage cost and a query cost and a document similarity analysis on the high-quality document library. The adaptive value function is used as an evaluation function, and the TopN hypergraphs with high adaptive values are used as candidate hypergraphs.

[0130] In a feasible implementation, the interactive genetic algorithm in the schema conversion module 220 will use user feedback information in the human-computer interaction module to further iteratively generate a new document structure, and the structure division algorithm will divide the relationship schema using a graph division algorithm based on edge division according to the structure division chromosome; the hypergraph construction algorithm will confirm the nesting direction of the document structure and the embedding or reference rules in the document structure according to the conversion direction and conversion rule chromosome, and record them in the hyperedge, and then construct a hypergraph; the JSON structure conversion algorithm will complete the conversion of the JSON structure from bottom to top for the top-level hypergraph structure.

[0131] Preferably, the human-computer interaction module 230 is further used for S31, the method will display the TopN document structures converted by the conversion and the fitness, and if there is a document structure that satisfies the user, the method ends, otherwise the user can change the fitness of a certain document structure or add it to the high-quality document library.

[0132] The next generation of schema conversion is performed, and selection, crossover, mutation, etc. are performed on the chromosomes in the population to generate new individuals, and the mode conversion module 220 is returned.

[0133] In a feasible implementation, the human-computer interaction module 230 is divided into a candidate area, a best individual area, and an operation area. The candidate area displays the TopN document structures, the best individual area displays the document structure of the best individual in the current population, and the operation area displays the time when the interactive genetic algorithm is executed to the ith generation, and buttons for starting the interactive genetic algorithm or performing the next iteration are given.

[0134] In the embodiment of the application, the schema converter for converting the schema of a relational database to a document database can quickly obtain the relationship schema of the relational database through the schema extraction module without the need for the user to manually search for metadata information, thereby simplifying the operation process. The conversion from the relationship schema to the document schema is completed through the schema conversion module. The graph model and the hypergraph model are used to define the relationship schema and the document schema, respectively, and the possible conversion rules are analyzed in detail, and the corresponding schema conversion algorithm is designed, which can ensure that the high-quality document structure is given as much as possible for the user to select without the need for user intervention in design, under the condition that the relationship table and the relationship schema are not lost. The scheme introduces the idea of interactive design. The human-computer interaction module and the schema conversion module described in the present application cooperate with each other, so that the user can participate in the process of schema conversion by evaluating the document structure. Each selection and evaluation of the user can affect the direction of subsequent schema conversion, guide the schema converter to obtain the expected document schema, and reduce the difficulty of schema design for the user.

[0135] Figure 6is a structural schematic diagram of an electronic device 300 provided by an embodiment of the present application. The electronic device 300 can be quite different in configuration or performance, and can include one or more processors (central processing units, CPUs) 301 and one or more memories 302. The memory 302 stores at least one instruction, which is loaded and executed by the processor 301 to implement the following steps of the relational database to document database mode conversion method:

[0136] S1, obtaining relevant data in a relational database that needs to be converted, inputting the relevant data into a mode extraction module to perform data mode extraction; wherein the relevant data includes connection mode or relational mode of the relational database, and a mapping conversion rule;

[0137] S2, executing a mode conversion module to convert the relational mode of the relational database to a document mode to obtain the document mode;

[0138] S3, showing the document mode to a user, determining whether there is a document mode that meets the user's expectation, if not, the user evaluates the document mode, obtains user evaluation data, and returns to S2 for re-conversion until the user's expectation is met, and the mode conversion from the relational database to the document database is completed.

[0139] In an exemplary embodiment, a computer-readable storage medium, such as a memory including instructions, is also provided. The instructions can be executed by a processor in a terminal to complete the above-described mode conversion method from a relational database to a document database. For example, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0140] Those of ordinary skill in the art can understand that all or part of the above-described steps can be completed by hardware, or by a program instructing relevant hardware to complete. The program can be stored in a computer-readable storage medium, such as a read-only memory, a magnetic disk, or an optical disk.

Claims

1. A schema conversion method from a relational database to a document database, characterized in that, The method steps include: S1. Obtain relevant data from the relational database that needs to be converted, and input the relevant data into the pattern extraction module to extract the data pattern; wherein, the relevant data includes: the connection method or relational pattern of the relational database, and the mapping conversion rules; S2. The execution mode conversion module converts the relational database from a relational schema to a document schema to obtain a document schema; The process of converting the relational database from a relational schema to a document schema to obtain the document schema includes: S21. Read the relation schema of the relational database, transform the relation schema into a graph structure through a relation schema extraction algorithm, and use the graph structure to store the meta-information of the relation tables to obtain a relation graph; S22. Based on the graph structure, call the connected pattern graph extraction algorithm to separate all the connected pattern graphs in the relationship graph; S23. Based on the aforementioned pattern connectivity graph, model the pattern connectivity graph using the chromosome structure in a genetic algorithm, including: According to the genetic algorithm, each individual consists of three integer chromosomes, and the structure of each chromosome represents the structural partition, the direction of foreign bond transformation, and the transformation rule, respectively; the population is initialized to generate 20 individuals, and the pattern connectivity graph is modeled. S24. Use a mode transformation algorithm to transform a connected mode graph into a hypergraph structure, and evaluate the hypergraph structure, including: S241. For individuals in the population, based on chromosome structure, execute a mode conversion algorithm to convert them into a hypergraph model; S242. Using the fitness function based on storage cost and query cost and document similarity analysis in a high-quality document library, the fitness function is used as the evaluation function to evaluate the transformed hypergraph structure, and the Top N hypergraphs with high fitness values ​​are selected as candidate hypergraphs. S25. Based on the hypergraph structure, use the JSON structure conversion algorithm to perform document pattern structure conversion, and use the application data type mapping rules to perform document pattern data type conversion to obtain candidate document patterns; The mode conversion algorithm is implemented through the execution of the mode converter; the mode converter includes an application layer, a conversion layer, and a storage layer. The conversion layer is the core structure of the pattern converter; the conversion layer includes a pattern extraction module, a pattern conversion module, a human-computer interaction module, and a logging module; S3. Show the document schema to the user and determine if there is a document schema that meets the user's expectations. If not, the user evaluates the document schema, obtains the user evaluation data, and returns to S2 to re-convert until it meets the user's expectations, thus completing the schema conversion from relational database to document database.

2. The method according to claim 1, characterized in that, In step S1, relevant data from the relational database to be converted is obtained, and the relevant data is input into the pattern extraction module for data pattern extraction, including: S11. Extract relation schema information from a relational database; wherein the extraction method includes: obtaining the connection method of the relational database and extracting relation schema information according to the connection method; or describing the relation schema information to be converted through a configuration file, wherein the configuration file must follow a specific format, the specific format being a JSON file containing two list data type elements "atoms" and "relationships"; S12. Obtain the data type mapping rules from relational database to document database; if no mapping rules exist, the default data type conversion rules will be used.

3. The method according to claim 1, characterized in that, In step S21, the relational schema of the relational database is read, and a relational schema extraction algorithm is used to transform it into a graph structure. This graph structure stores the metadata of the relational tables, thus obtaining a relational graph, including: By connecting to a relational database, you can obtain the database relation schema, which contains information such as the structure of all tables in the database, indexes, primary keys, and foreign keys. Abstract the relation schema into a simple graph. : ; In this context, node V represents the table structure, recording the primary key, fields, field size, and number of data rows; edge E represents the foreign key relationship, recording the degree relationship between the two tables connected by the foreign key. Determine whether the relation schema can be read correctly. If so, construct a graph for the relation schema, storing the relation table as vertices and foreign key relations as edges, and convert it into an undirected graph. The vertices store the field names, field types, primary keys, field sizes, and number of records of each field in the relational table; the edges store the attributes of foreign key connections and the degree relationship between the two tables, thus obtaining the relational graph.

4. The method according to claim 1, characterized in that, In step S3, the document database is displayed to the user, and it is determined whether a document pattern that matches the user's expectations exists. If not, the user evaluates the document pattern, the user evaluation data is obtained, and the process returns to S2 for re-conversion until it meets the user's expectations, thus completing the conversion from a relational database to a document database. This includes: S31. The method will display the Top N document structures and their fitness scores after conversion. If there is a document structure that satisfies the user, the process will end. Otherwise, the user can change the fitness score of a document structure or add it to the high-quality document library. S32. Perform next-generation mode conversion, perform selection, crossover, and mutation operations on chromosomes in the population to generate new individuals, and return to execute S24.

5. A schema conversion system for converting relational databases to document databases, characterized in that, The system is used for the schema conversion method from relational database to document database as described in any one of claims 1 to 4, and the system comprises: The pattern extraction module is used to acquire relevant data from the relational database that needs to be transformed, and input the relevant data into the pattern extraction module to extract data patterns; wherein, the relevant data includes: the connection method or relational schema of the relational database, and the mapping transformation rules; The schema conversion module is used to convert the relational schema of the relational database into a document schema to obtain a document schema. The human-computer interaction module is used to display the document pattern to the user, determine whether there is a document pattern that meets the user's expectations, and if not, the user evaluates the document pattern, obtains the user evaluation data, and returns it to the pattern conversion module for re-conversion until it meets the user's expectations, thus completing the pattern conversion from relational database to document database.

6. The system according to claim 5, characterized in that, The pattern extraction module is further used to extract relational pattern information from a relational database; wherein the extraction method includes: obtaining the connection method of the relational database and extracting relational pattern information according to the connection method; or describing the relational pattern information to be converted through a configuration file, wherein the format of the configuration file must follow a specific format; Retrieves the data type mapping rules from relational database to document database; if no mapping rules exist, the default data type conversion rules will be used.

Images