Database pattern matching method and device, computer equipment and storage medium

Abstract

The invention relates to a database pattern matching method and device, computer equipment and a storage medium, and the method comprises the steps: obtaining pattern information and a query log of a database; the mode information comprises a source mode and a target mode; inputting the mode information into a first model which is completely trained for coding to obtain an attribute code; inputting the query log into a second model which is completely trained for coding to obtain a query statement code; determining a matching result between a source attribute in the source mode and a target attribute in the target mode according to the attribute code and the query statement code; and obtaining a pattern matching result until all attributes in the database are traversed. Through the method and the device, the problem of low matching result accuracy caused by manual rule definition is solved, the first module which is completely trained is combined with the second model to effectively integrate structured query language representation and pattern representation, and the pattern matching accuracy is improved.

Description

Technical Field

[0001] The present application relates to the fields of computer science and artificial intelligence technology, and in particular to a database pattern matching method, apparatus, computer equipment and storage medium. Background Art

[0002] Pattern matching is one of the core technologies of data integration. Its purpose is to identify the relationship between different databases or data sources and compare their pattern structures in order to determine how the data in one pattern is mapped to the data in another pattern, so that associations can be established between multiple different data sources. As a key step in data integration, pattern matching technology is widely used in various data integration scenarios, such as enterprise data integration, Web data integration, data warehouse integration, etc. These applications can effectively manage and utilize information scattered in different sources, thereby improving decision-making quality and operational efficiency.

[0003] The commonly used pattern matching method is a rule-based matching algorithm, which matches by comparing data types, field names, data formats, or document structures. For example, matching can be established based on name similarity (such as using a string similarity algorithm) or data type compatibility. The disadvantages of this method are that it requires manual definition of rules, cannot adapt to complex patterns, and cannot handle incomplete or erroneous data, resulting in low accuracy of matching results.

[0004] There is currently no effective solution to the problem that related technologies require manual definition of rules, resulting in low accuracy of matching results. Summary of the invention

[0005] In this embodiment, a database pattern matching method, apparatus, computer device and storage medium are provided to solve the problem in the related art that rules need to be manually defined, resulting in low accuracy of matching results.

[0006] In a first aspect, a database pattern matching method is provided in this embodiment, including:

[0007] Obtain database schema information and query logs; schema information includes source schema and target schema;

[0008] Inputting the pattern information into the first fully trained model for encoding to obtain attribute encoding;

[0009] Inputting the query log into a fully trained second model for encoding to obtain a query statement encoding; the network structure of the first model and the second model is a twin network structure;

[0010] Determine a matching result between a source attribute in the source schema and a target attribute in the target schema according to the attribute code and the query statement code;

[0011] Until all attributes in the database are traversed, a pattern matching result is obtained.

[0012] In some embodiments, the method further comprises:

[0013] Building a self-supervised dataset for structured query language;

[0014] The second model is pre-trained according to the self-supervised dataset, the preset masked language modeling task and the preset cloze task to obtain a fully trained second model.

[0015] In some embodiments, the second model is pre-trained according to the self-supervised dataset, the preset masked language modeling task, and the preset cloze task to obtain a fully trained second model, including:

[0016] According to the self-supervised dataset and the preset masked language modeling task, the second model is trained to obtain a pre-trained second model;

[0017] The pre-trained second model is trained according to the self-supervised data set and the preset cloze task to obtain a fully trained second model.

[0018] In some embodiments, the method further comprises:

[0019] Acquire a training set of the first model, and perform fine-tuning of pattern-level semantic matching on the training set; the training set includes training pattern information of a training database;

[0020] Based on the fine-tuned training set, the first model is trained to obtain a fully trained first model.

[0021] In some embodiments, the method further comprises:

[0022] The first model and the second model are jointly fine-tuned to obtain a fully trained first model and a fully trained second model.

[0023] In some embodiments, determining a matching result between a source attribute in the source schema and a target attribute in the target schema according to the attribute code and the query statement code includes:

[0024] The attribute code and the query statement code are merged to obtain a source fusion code of a source attribute in the source mode and a target fusion code of a target attribute in the target mode;

[0025] A similarity calculation is performed on the source fusion code and the target fusion code to determine a matching result between the source attribute and the target attribute.

[0026] In some embodiments, the attribute code includes a source attribute code and a target attribute code; the source attribute code corresponds to a source attribute in the source mode; the target attribute code corresponds to a target attribute in the target mode;

[0027] The query statement code includes a source query statement code and a target query statement code; the source query statement code corresponds to the source query log in the query log; the target query statement code corresponds to the target query log in the query log;

[0028] The attribute code and the query statement code are fused to obtain a source fused code of a source attribute in the source mode and a target fused code of a target attribute in the target mode, including:

[0029] The source attribute code and the source query statement code are fused to obtain a source fusion code of the source attribute in the source schema;

[0030] The target attribute code and the target query statement code are fused to obtain a target fusion code of the target attribute in the target mode.

[0031] In some embodiments, the attribute code and the query statement code are fused to obtain a source fused code of a source attribute in the source schema and a target fused code of a target attribute in the target schema, including:

[0032] By using a cross attention mechanism, the attribute encoding and the query sentence encoding are fused to obtain a source fused encoding of a source attribute in the source mode and a target fused encoding of a target attribute in the target mode;

[0033] The expression of the cross attention mechanism is:

[0034]

[0035] In the formula, e M represents the target fusion code; e S Indicates the attribute code; Q Indicates the query statement encoding.

[0036] In a second aspect, a database pattern matching device is provided in this embodiment, including: an acquisition module, a first encoding module, a second encoding module, a matching module, and a traversal module;

[0037] The acquisition module is used to acquire the mode information and query log of the database; the mode information includes the source mode and the target mode;

[0038] The first encoding module is used to input the pattern information into the first fully trained model for encoding to obtain attribute encoding;

[0039] The second encoding module is used to input the query log into a fully trained second model for encoding to obtain a query statement encoding; the network structure of the first model and the second model is a twin network structure;

[0040] The matching module is used to determine the matching result between the source attribute in the source mode and the target attribute in the target mode according to the attribute code and the query statement code;

[0041] The traversal module is used to complete the traversal of all attributes in the database to obtain a pattern matching result.

[0042] In a third aspect, a computer device is provided in this embodiment, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the database pattern matching method described in the first aspect when executing the computer program.

[0043] In a fourth aspect, in this embodiment, a storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the database pattern matching method described in the first aspect is implemented.

[0044] Compared with the related art, the database pattern matching method, apparatus, computer equipment and storage medium provided in this embodiment obtain the pattern information and query log of the database; the pattern information includes a source pattern and a target pattern; the pattern information is input into a first model that has been trained to encode and obtain an attribute code; the query log is input into a second model that has been trained to encode and obtain a query statement code; the network structure of the first model and the second model is a twin network structure; according to the attribute code and the query statement code, the matching result between the source attribute in the source pattern and the target attribute in the target pattern is determined; until all attributes in the database are traversed and the pattern matching result is obtained, the problem of manually defining rules and resulting in low accuracy of matching results is solved, and the structured query language representation and pattern representation are effectively integrated by combining the first module that has been trained with the second model to improve the accuracy of pattern matching.

[0045] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0047] Figure 1 It is a hardware structure block diagram of a terminal device of a database pattern matching method provided in an embodiment of the present application;

[0048] Figure 2 is a flow chart of a database pattern matching method provided in one embodiment of the present application;

[0049] Figure 3 This is a flowchart of the first model training provided by an embodiment of the present application;

[0050] Figure 4 is a schematic diagram of masked language modeling task training provided by an embodiment of the present application;

[0051] Figure 5 is a schematic diagram of cloze task training provided by an embodiment of the present application;

[0052] Figure 6 is a schematic diagram of the second model training provided in one embodiment of the present application;

[0053] Figure 7 is a schematic diagram of joint training provided by an embodiment of the present application;

[0054] Figure 8 is a schematic diagram of a pattern matching method provided in a preferred embodiment of the present application;

[0055] Fig. 9 is a schematic diagram of a source mode and a target mode in a preferred embodiment of the present application;

[0056] Fig.10 It is a structural block diagram of a database pattern matching device provided in one embodiment of the present application.

[0057] In the figure: 102, processor; 104, memory; 106, transmission device; 108, input and output device; 210, acquisition module; 220, first encoding module; 230, second encoding module; 240, matching module; 250, traversal module. DETAILED DESCRIPTION

[0058] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0059] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the", "these" and the like in this application do not represent quantitative restrictions, and they can be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether directly or indirectly. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. Usually, the character " / " indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0060] The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. For example, running on a terminal, Figure 1 FIG. 1 is a hardware structure diagram of a terminal of the database pattern matching method of this embodiment. Figure 1 As shown, the terminal may include one or more ( Figure 1 Only one is shown in the figure) processor 102 and memory 104 for storing data, wherein processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above terminal may also include a transmission device 106 and an input and output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0061] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the pattern matching method of the database in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0062] The transmission device 106 is used to receive or send data via a network. The above network includes a wireless network provided by the communication provider of the terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0063] In this embodiment, a database pattern matching method is provided. Figure 2 is a flow chart of the database pattern matching method of this embodiment, such as Figure 2 As shown, the process includes the following steps:

[0064] Step S210, obtaining schema information and query logs of the database; the schema information includes a source schema and a target schema;

[0065] Step S220, inputting the pattern information into the first fully trained model for encoding to obtain attribute encoding;

[0066] Step S230, inputting the query log into the fully trained second model for encoding to obtain the query statement encoding; the network structure of the first model and the second model is a twin network structure;

[0067] Step S240, determining a matching result between a source attribute in the source schema and a target attribute in the target schema according to the attribute code and the query statement code;

[0068] Step S250, until all attributes in the database are traversed and the pattern matching result is obtained.

[0069] It should be noted that the source schema and target schema in the schema information refer to schemas from the same or different databases. For example: Table 1 (source schema) and Table 2 (target schema) exist in a database at the same time; Table 1 contains one or more source attributes (a column or a horizontal line representing the same information in Table 1 is a source attribute) and a corresponding description; Table 2 contains one or more target attributes (a column or a horizontal line representing the same information in Table 2 is a target attribute) and a corresponding description. In this case, the query log only needs to be the query log of a single database. If the schema information comes from different databases, the query log also covers these databases.

[0070] Among them, the first model and the second model are both pre-trained; the network structures of the first model and the second model are the same, forming a twin network structure. Its network structure includes but is not limited to BERT (Bidirectional Encoder Representations from Transformers, a natural language processing (NLP) model based on deep learning, GPT (Generative Pre-Training, a generative pre-training model), XLNet (a pre-trained language model based on Transformer), etc., and there is no restriction on this. Among them, the first model that has been trained is used to encode the input pattern information for semantic matching and discrimination, that is, the attributes of the database and their descriptions can be fused and encoded to generate corresponding representations (attribute encoding) in the semantic space; the second model that has been trained is used to encode the input query log in structured query language to obtain query statement encoding. Through these two fully trained models, structured query language representation and pattern representation can be effectively integrated without the need to manually define rules.

[0071] After obtaining the attribute encoding and the query statement encoding, a matching algorithm or a cross-attention mechanism is used to determine the matching result between the source attribute in the source pattern and the target attribute in the target pattern according to the attribute encoding and the query statement encoding; the above steps S220 to S240 are repeated until all attributes in the database are traversed, thereby obtaining a pattern matching result.

[0072] In the related art, a rule-based matching algorithm is used; it matches by comparing data types, field names, data formats or document structures. It requires manual definition of rules, resulting in the problem of low accuracy of matching results. In this embodiment, by obtaining the schema information and query log of the database; the schema information includes the source schema and the target schema; the schema information is input into a first model that has been fully trained for encoding to obtain the attribute code; so as to achieve the purpose of fusion encoding of the attributes of the database and their descriptions to generate corresponding representations in the semantic space. Combined with inputting the query log into a second model that has been fully trained for encoding, the query statement code is obtained; so as to achieve the purpose of representing the structured query language. The matching result between the source attribute in the source schema and the target attribute in the target schema is determined based on the two information of the attribute code and the query statement code, which can improve the matching accuracy. Then, after traversing all the attributes in the database, the schema matching result can be obtained, thereby solving the problem of the need to manually define rules, resulting in low accuracy of matching results.

[0073] The following is a detailed description of the above steps:

[0074] In some of the embodiments, Figure 3 As shown, the database pattern matching method also includes the following steps:

[0075] Step S310, constructing a self-supervised data set of structured query language;

[0076] Step S320, pre-training the second model according to the self-supervised data set, the preset masked language modeling task and the preset cloze task to obtain a fully trained second model.

[0077] Specifically, both the first model and the second model can adopt the BERT model, and the two BERT models constitute a twin network model (TwinBERT). Among them, the first model (SchemaBERT) and the second model (SQLBERT) can be trained separately and / or jointly. In this embodiment, there is no restriction on the network structure of the BERT model.

[0078] The process of separate training of the second model can be considered to be achieved by using a self-supervised dataset for constructing a structured query language in combination with a preset training task, wherein the training task includes a masked language modeling task (MLM) and a cloze task (CLC).

[0079] Specifically, for a self-supervised data set, a large number of self-supervised data sets of structured query language can be constructed by preprocessing the text data of the database, so that there is no need to manually annotate the data set. Among them, preprocessing includes but is not limited to data cleaning, feature screening, feature extraction according to structured query statement templates (including SQL query statement templates, etc.), etc., and there is no limitation on this.

[0080] Among them, the masked language modeling task (MLM) and the cloze task (CLC) are two different training tasks, and there is no restriction on the order of training. For example: the self-supervised data set is divided into a training set, a validation set, and a test set; through the training method corresponding to the BERT model, the second model is trained based on the training set, validation set, and test set combined with the masked language modeling task (MLM) and the cloze task (CLC), so as to obtain a fully trained second model. In this embodiment, there is no restriction on the training method.

[0081] Through this embodiment, the self-supervised dataset of structured query language is used to train the second model in combination with the masked language modeling task and the cloze task, thereby enhancing the second model's ability to understand the structured language of queries such as SQL, thereby improving the accuracy of pattern matching.

[0082] In some embodiments, pre-training the second model according to the self-supervised dataset, the preset masked language modeling task, and the preset cloze task in step S230 to obtain a fully trained second model includes the following steps:

[0083] Step S321, training the second model according to the self-supervised data set and the preset masked language modeling task to obtain a pre-trained second model;

[0084] Step S322: training the pre-trained second model according to the self-supervised data set and the preset cloze task to obtain a fully trained second model.

[0085] The Masked Language Modeling (MLM) task refers to fixing all keywords in the structured query language, masking non-keyword parts (i.e., column names, table names, etc.) with a preset second probability (which can be 20% to 50%, preferably 30%) in the sequence query statement of the structured query language, and using the second model for prediction and recovery. Figure 4As shown, non-keywords include 'building_id', 'location', 'house', and 'capacity'. Then, among the non-keywords 'building_id', 'location', 'house', and 'capacity', masked non-keywords (for example, masking 'location' and 'capacity') are randomly selected according to a preset ratio, and prediction recovery is performed through the second model.

[0086] Among them, the cloze task (CLC) refers to randomly selecting some sequence query statements of the structured query language with a preset second probability (which can be 10% to 30% probability, preferably 15% probability), covering all non-keywords in the sequence query statements, and putting them at the end of the query statement in a shuffled order, and marking the end of the sentence with the [SEP] symbol. The second model will use these words as candidates and fill them back into the original query statement. Figure 5 As shown, non-keywords 'building_id', 'location', 'house', and 'capacity' are all covered and placed at the end of the sentence in a shuffled order. The original position is marked as [EMP]. The second model is used to predict the position of [EMP] and the words at the end of the sentence are backfilled to the original position.

[0087] In this embodiment, the masked language modeling task is first trained to obtain a pre-trained second model, thereby introducing some knowledge of SQL into the second model; and then the cloze task is trained to obtain a fully trained second model, so as to reduce the training difficulty of the cloze task and improve the training efficiency of the second model.

[0088] In some of the embodiments, Figure 6 As shown, the database pattern matching method also includes the following steps:

[0089] Obtaining a training set of the first model, and fine-tuning the training set for pattern-level semantic matching; the training set includes training pattern information of a training database;

[0090] Based on the fine-tuned training set, the first model is trained to obtain a fully trained first model.

[0091] Among them, the first model focuses on semantic matching at the pattern level, and its separate training process can be considered as fine-tuning of the pattern-level semantic matching on the training set, so that the first model is trained based on the fine-tuned training set to obtain a fully trained first model.

[0092] The process of fine-tuning the pattern-level semantic matching of the training set can be as follows: 1. Attribute and description extraction: extract the attribute n of the database pattern and its corresponding description d from the training set (corresponding to the training database). 2. Formatting: format the training set using the serialization operation S(·,·). 3. Start fine-tuning: input the formatted training set S(n,d) into the first model, and fine-tune the pattern-level semantic matching according to the label of the training set to obtain the first model that has been fully trained.

[0093] Among them, S(n,d)=[CLS]S(n)[SEP]S(d)[SEP].

[0094] Among them, [CLS] and [SEP] are the symbols used by the first model to represent the beginning and middle separators of sentences.

[0095] Through this embodiment, fine-tuning of pattern-level semantic matching is performed on the training set, table name and attribute name fusion encoding is proposed, and automatic encoding of structured query language is realized, which can improve the task adaptation capability of the first fully trained model.

[0096] In some of the embodiments, Figure 7 As shown, the database pattern matching method also includes the following steps:

[0097] The first model and the second model are jointly fine-tuned to obtain a fully trained first model and a fully trained second model.

[0098] Specifically, the parameters of the first model SchemaBERT and the second model SQLBERT are randomly selected for freezing; the first model with frozen parameters is merged with the second model to form a twin network model TwinBERT, relevant query statements are extracted from the query log of the training database, and combined with the training mode information of the training set, they are input into the twin network model for joint fine-tuning; after the joint fine-tuning is successful, a fully trained first model and a fully trained second model are obtained.

[0099] In this embodiment, the adaptability of the pattern matching of the two model pairs is optimized through joint fine-tuning, thereby improving the accuracy of pattern matching.

[0100] In some embodiments, determining the matching result between the source attribute in the source schema and the target attribute in the target schema according to the attribute code and the query statement code in step S230 includes the following steps:

[0101] Step S231, fusing the attribute code and the query statement code to obtain a source fusion code of the source attribute in the source schema and a target fusion code of the target attribute in the target schema;

[0102] Step S232: perform similarity calculation on the source fusion code and the target fusion code to determine a matching result between the source attribute and the target attribute.

[0103] Specifically, the attribute encoding and the query statement encoding can be fused using a cross-attention mechanism or encoding fusion algorithm to obtain a source fusion encoding of the source attribute in the source mode and a target fusion encoding of the target attribute in the target mode. Then, a similarity algorithm or similarity model is used to calculate the similarity between the source fusion encoding and the target fusion encoding to determine the matching result between the source attribute and the target attribute; thus, the matching result between the source attribute and the target attribute can be calculated efficiently and accurately.

[0104] In some embodiments, the attribute code and the query statement code are fused to obtain a source fused code of the source attribute in the source schema and a target fused code of the target attribute in the target schema, including the following steps:

[0105] The source attribute code and the source query statement code are fused to obtain the source fusion code of the source attribute in the source mode;

[0106] The target attribute encoding and the target query statement encoding are fused to obtain the target fusion encoding of the target attribute in the target mode.

[0107] Among them, the attribute coding includes source attribute coding and target attribute coding; the source attribute coding corresponds to the source attribute in the source mode; the target attribute coding corresponds to the target attribute in the target mode; the query statement coding includes source query statement coding and target query statement coding; the source query statement coding corresponds to the source query log in the query log; the target query statement coding corresponds to the target query log in the query log.

[0108] It can be considered that the source fusion code and the target fusion code are calculated separately, and each fusion code is obtained by fusing its corresponding attribute code and query code. For example, the source attribute code and the source query code are input into the cross-attention mechanism for fusion, and the source fusion code of the source attribute in the source mode can be obtained. The target attribute code and the target query code are input into the cross-attention mechanism for fusion, and the target fusion code of the target attribute in the target mode can be obtained.

[0109] Through this embodiment, more attention can be paid to the encoding features to improve the fusion effect.

[0110] The present embodiment is described and illustrated below through preferred embodiments.

[0111] Figure 8 is a schematic diagram of a preferred flow chart of the database pattern matching method of this embodiment, such as Figure 8 As shown, the pattern matching method of the database includes the following steps:

[0112] Get the schema information and query log of the database; the schema information includes the source schema and the target schema; the source schema has source attributes, and the target schema has target attributes. For example, the source schema is represented by: building(building_id, building_name, location, capacity), course(course_id, course_title, dept_name, credits, language). The target schema is represented by: Bldg(Bld, BName, Loc, Cap), CE(CEId, Title, DeptName, Credit). The query log corresponding to the source schema is the source query log, which is to query classrooms with a capacity greater than 'value' Select building_id, location From building Where capacity>'value'. The query log corresponding to the target schema is the target query log, which is to query classrooms with a capacity less than 200 Select BId, Loc From Bldg Where Cap<200.

[0113] Improve the accuracy of pattern matching through pattern recognition. Among them, pattern recognition is: recognition based on the combination of attributes, the space of the value domain, and the frequency of occurrence in the query record. For example: Identify that "building_id" in the source pattern corresponds to "Bld" in the target pattern. Identify that "building_name" in the source pattern corresponds to "BName" in the target pattern. Identify that "location" in the source pattern corresponds to "Loc" in the target pattern. Identify that "capacity" in the source pattern corresponds to "Cap" in the target pattern. Each time, a pair of source and target patterns can be input into the twin network model (TwinBERT) to determine whether the pair of patterns matches the matching structure. After all the matches are completed, the final pattern matching result can be obtained.

[0114] The pattern information is input into the first model that has been trained to encode and obtain the attribute encoding. The process can be considered as first extracting the attributes and descriptions, then formatting, and finally inputting the formatted results into the first model for encoding to obtain the attribute encoding. The encoding result for the source attribute is the source attribute encoding. The encoding result for the target attribute is the target attribute encoding

[0115] The query log is input into the second model that has been trained to encode the query statement. The encoding result for the source query log is the source query statement encoding. The encoding result of the target query log is the target query statement encoding

[0116] Using the cross attention mechanism, the attribute encoding S (represented as Q in the figure, including source attribute encoding and target attribute encoding ) and query statement encoding e Q (Indicated as K or V in the figure, including the source query sentence encoding and target query encoding ) to obtain the source fusion code of the source attribute in the source mode and the target fusion code of the target attribute in the target mode. M (including source fusion coding and target fusion coding ). The source attribute code and the source query statement code are fused to obtain the source fusion code of the source attribute in the source mode; the target attribute code and the target query statement code are fused to obtain the target fusion code of the target attribute in the target mode.

[0117] The expression of the cross attention mechanism is:

[0118]

[0119] The fusion process can be considered as: attribute encoding S and query code Q First calculate the bit-by-bit product, then divide by the attribute code e S The square root of , then normalized by softmax, and finally multiplied by the attribute code e S , to obtain the target fusion code e M .

[0120] The source fusion encoding is performed through the similarity calculation formula. and target fusion coding Perform similarity calculation to determine the matching result between the source attribute and the target attribute. The similarity calculation formula is:

[0121]

[0122] In the formula, similarity represents the similarity between the source attribute and the target attribute; Indicates the modulus length of the source fusion code; Indicates the modulus length of the target fusion code.

[0123] When the similarity meets the preset similarity threshold, it is considered a match; otherwise, it is considered a mismatch. The similarity threshold can be adjusted according to the actual scenario and is not limited to this.

[0124] For example: To identify whether the "capacity" in the source pattern matches the "Cap" in the target pattern, such as Fig. 9 As shown in the query log, we can find that the attributes appear after where, and the conditional judgment uses the value size to judge. The structure of the entire statement is similar. Therefore, it can be judged that the capacity attribute in building matches the Cap attribute in Bldg.

[0125] By giving priority to this embodiment, the structured query language representation and the pattern representation are effectively integrated to improve the accuracy and recall rate of pattern matching; the table name and attribute name fusion encoding is proposed to realize the automatic encoding of the structured query language and optimize the modeling, so that the model has good interpretability and strong generalization.

[0126] It should be noted that the steps shown in the above process or the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0127] In this embodiment, a pattern matching device for a database is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. The terms "module", "unit", "subunit" and the like used below can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware is also possible and conceivable.

[0128] Fig.10 is a structural block diagram of the pattern matching device of the database of this embodiment, such as Fig.10 As shown, the device includes: an acquisition module 210, a first encoding module 220, a second encoding module 230, a matching module 240 and a traversal module 250;

[0129] The acquisition module 210 is used to acquire the schema information and query log of the database; the schema information includes the source schema and the target schema;

[0130] A first encoding module 220, used to input the pattern information into the first fully trained model for encoding to obtain attribute code;

[0131] The second encoding module 230 is used to input the query log into the fully trained second model for encoding to obtain the query statement encoding; the network structure of the first model and the second model is a twin network structure;

[0132] A matching module 240, configured to determine a matching result between a source attribute in a source mode and a target attribute in a target mode according to the attribute code and the query statement code;

[0133] The traversal module 250 is used to complete the traversal of all attributes in the database to obtain a pattern matching result.

[0134] Through the above device, the structured query language representation and the pattern representation are effectively integrated to improve the accuracy and recall rate of pattern matching.

[0135] In some of the embodiments, the database pattern matching device further includes: a first training module;

[0136] The first training module is used to build a self-supervised dataset of structured query language;

[0137] The first model is pre-trained according to the self-supervised dataset, the preset masked language modeling task, and the preset cloze task to obtain a fully trained first model.

[0138] In some of the embodiments, the first training module is further used to train the first model according to the self-supervised data set and the preset masked language modeling task to obtain a pre-trained first model;

[0139] The pre-trained first model is trained according to the self-supervised data set and the preset cloze task to obtain a fully trained first model.

[0140] In some of the embodiments, the database pattern matching device further comprises: a second training module;

[0141] A second training module is used to obtain a training set of a second model and fine-tune the pattern-level semantic matching of the training set; the training set includes training pattern information of a training database;

[0142] Based on the fine-tuned training set, the second model is trained to obtain a fully trained second model.

[0143] In some of these embodiments, the database pattern matching device further comprises: a joint training module;

[0144] The joint training module is used to jointly fine-tune the first model and the second model to obtain a fully trained first model and a fully trained second model.

[0145] In some of the embodiments, the matching module 240 is further used to fuse the attribute code and the query statement code to obtain a source fused code of the source attribute in the source mode and a target fused code of the target attribute in the target mode;

[0146] The similarity between the source fusion code and the target fusion code is calculated to determine the matching result between the source attribute and the target attribute.

[0147] In some of the embodiments, the matching module 240 is further used to fuse the source attribute code and the source query statement code to obtain a source fusion code of the source attribute in the source mode;

[0148] The target attribute encoding and the target query statement encoding are fused to obtain the target fusion encoding of the target attribute in the target mode.

[0149] In some of the embodiments, the matching module 240 is further configured to fuse the attribute encoding and the query sentence encoding using a cross attention mechanism to obtain a source fused encoding of the source attribute in the source mode and a target fused encoding of the target attribute in the target mode;

[0150] The expression of the cross attention mechanism is:

[0151]

[0152] In the formula, e M represents the target fusion code; e S Indicates the attribute code; Q Indicates the query statement encoding.

[0153] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0154] In this embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0155] Optionally, the computer device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0156] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0157] S1, obtain the schema information and query log of the database; the schema information includes the source schema and the target schema;

[0158] S2, inputting the pattern information into the first fully trained model for encoding to obtain attribute encoding;

[0159] S3, inputting the query log into the fully trained second model for encoding to obtain the query statement encoding; the network structure of the first model and the second model is a twin network structure;

[0160] S4, determining a matching result between a source attribute in the source schema and a target attribute in the target schema according to the attribute code and the query statement code;

[0161] S5, until all attributes in the database are traversed and the pattern matching result is obtained.

[0162] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and will not be repeated in this embodiment.

[0163] In addition, in combination with the database pattern matching method provided in the above embodiments, a storage medium can also be provided in this embodiment to implement the method. The storage medium stores a computer program; when the computer program is executed by a processor, any database pattern matching method in the above embodiments is implemented.

[0164] It should be understood that the specific embodiments described herein are only used to explain the application, rather than to limit it. Based on the embodiments provided in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of this application.

[0165] Obviously, the drawings are only some examples or embodiments of the present application. For ordinary technicians in the field, the present application can also be applied to other similar situations based on these drawings without creative work. In addition, it is understandable that although the work done in this development process may be complicated and lengthy, for ordinary technicians in the field, certain changes in design, manufacturing or production based on the technical content disclosed in this application are only conventional technical means and should not be regarded as insufficient content disclosed in this application.

[0166] The term "embodiment" in this application refers to a specific feature, structure or characteristic described in conjunction with the embodiment that can be included in at least one embodiment of the present application. The appearance of this phrase in various locations in the specification does not necessarily mean the same embodiment, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. It is clearly or implicitly understood by those of ordinary skill in the art that the embodiments described in this application can be combined with other embodiments without conflict.

[0167] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of patent protection. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the attached claims.

Claims

1. A database pattern matching method, characterized in that: include: Get database schema information and query logs; The mode information includes source mode and target mode; Inputting the pattern information into the first fully trained model for encoding to obtain attribute encoding; Inputting the query log into a fully trained second model for encoding to obtain a query statement encoding; The network structure of the first model and the second model is a twin network structure; Determine a matching result between a source attribute in the source schema and a target attribute in the target schema according to the attribute code and the query statement code; Until all attributes in the database are traversed, a pattern matching result is obtained.

2. The database pattern matching method according to claim 1, characterized in that: The method further comprises: Building a self-supervised dataset for structured query language; The second model is pre-trained according to the self-supervised dataset, the preset masked language modeling task and the preset cloze task to obtain a fully trained second model.

3. The database pattern matching method according to claim 2, characterized in that: Pre-training the second model according to the self-supervised data set, the preset masked language modeling task, and the preset cloze task to obtain a fully trained second model, including: According to the self-supervised dataset and the preset masked language modeling task, the second model is trained to obtain a pre-trained second model; The pre-trained second model is trained according to the self-supervised data set and the preset cloze task to obtain a fully trained second model.

4. The database pattern matching method according to claim 1, characterized in that: The method further comprises: Acquire a training set of the first model, and perform fine-tuning of pattern-level semantic matching on the training set; the training set includes training pattern information of a training database; Based on the fine-tuned training set, the first model is trained to obtain a fully trained first model.

5. The database pattern matching method according to any one of claims 2 or 4, characterized in that: The method further comprises: The first model and the second model are jointly fine-tuned to obtain a fully trained first model and a fully trained second model.

6. The database pattern matching method according to claim 1, characterized in that: Determining a matching result between a source attribute in the source schema and a target attribute in the target schema according to the attribute code and the query statement code includes: The attribute code and the query statement code are merged to obtain a source fusion code of a source attribute in the source mode and a target fusion code of a target attribute in the target mode; A similarity calculation is performed on the source fusion code and the target fusion code to determine a matching result between the source attribute and the target attribute.

7. The database pattern matching method according to claim 6, characterized in that: The attribute code includes a source attribute code and a target attribute code; the source attribute code corresponds to the source attribute in the source mode; the target attribute code corresponds to the target attribute in the target mode; The query statement encoding includes a source query statement encoding and a target query statement encoding; The source query statement code corresponds to the source query log in the query log; The target query statement code corresponds to the target query log in the query log; The attribute code and the query statement code are fused to obtain a source fused code of a source attribute in the source mode and a target fused code of a target attribute in the target mode, including: The source attribute code and the source query statement code are fused to obtain a source fusion code of the source attribute in the source schema; The target attribute code and the target query statement code are fused to obtain a target fusion code of the target attribute in the target mode.

8. The database pattern matching method according to claim 6, characterized in that: The attribute code and the query statement code are fused to obtain a source fused code of a source attribute in the source mode and a target fused code of a target attribute in the target mode, including: By using a cross attention mechanism, the attribute encoding and the query sentence encoding are fused to obtain a source fused encoding of a source attribute in the source mode and a target fused encoding of a target attribute in the target mode; The expression of the cross attention mechanism is: In the formula, e M represents the target fusion code; e S Indicates the attribute code; Q Indicates the query statement encoding.

9. A database pattern matching device, characterized in that: include: An acquisition module, a first encoding module, a second encoding module, a matching module, and a traversal module; The acquisition module is used to acquire the mode information and query log of the database; The mode information includes source mode and target mode; The first encoding module is used to input the pattern information into the first fully trained model for encoding to obtain attribute encoding; The second encoding module is used to input the query log into a fully trained second model for encoding to obtain a query statement encoding; The network structure of the first model and the second model is a twin network structure; The matching module is used to determine the matching result between the source attribute in the source mode and the target attribute in the target mode according to the attribute code and the query statement code; The traversal module is used to complete the traversal of all attributes in the database to obtain a pattern matching result.

10. A computer device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps of the database pattern matching method according to any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the database pattern matching method according to any one of claims 1 to 8 are implemented.

Images