Methods and apparatus for intelligent and scalable pattern matching framework

Abstract

A method can include: (i) obtaining a first data record structured according to a first schema, (ii) determining, for the first schema, one or more first schema attribute values for each schema attribute in a predefined set of schema attributes, (iii) determining, for a second schema, one or more second schema attribute values for each schema attribute in the predefined set of schema attributes, (iv) providing the first and second schema attribute values to a schema matching engine, wherein the schema matching engine contains schema mapping techniques and rules, wherein each rule recommends a schema mapping technique based on schema attributes from the predefined set of schema attributes, (v) applying the rules to select a schema mapping technique, (vi) transforming the first data record according to the selected schema mapping technique, and (vii) providing the transformed first data record in a data structure according to the second schema.

Description

Technical Field

[0001] This disclosure relates to a pattern matching engine for computing systems. Background Technology

[0002] A schema is a blueprint that defines the layout and format of data records contained in a database, storage, or file. For example, a database table may contain one or more data records, and a schema may correspondingly define logical groupings of one or more database tables. A schema may include all the implementation details necessary to store the data records, such as the data type of the data records, constraints on the data records, and foreign keys or primary keys associated with the data records.

[0003] Sometimes, it may be of interest to transform data records from being constructed according to a first schema to being constructed according to a second schema. For example, existing data records may be stored in a legacy database whose data structure is constructed according to a legacy schema. If the legacy database is replaced by a new database whose data structure is constructed according to a new schema, then the existing data records in the legacy database should be transformed to be constructed according to the new schema so that the existing data records can be properly stored in the new database. Summary of the Invention

[0004] This disclosure generally relates to a pattern matching engine for a computing system. The pattern matching engine can be configured to transform data records from being constructed according to a first pattern to being constructed according to a second pattern. Depending on (i) the data records, (ii) the first pattern, and / or (iii) the second pattern, the pattern matching engine can select one or more pattern mapping techniques to perform when transforming data records.

[0005] In a first aspect, a computer-implemented method is provided. The method includes obtaining a first pattern and a first data record constructed according to the first pattern via a parser running on a computing system. The method further includes: for the first pattern, determining one or more first pattern attribute values ​​for each pattern attribute in a predefined set of pattern attributes. The method further includes: for a second pattern, determining one or more second pattern attribute values ​​for each pattern attribute in the predefined set of pattern attributes. The method additionally includes: providing the first and second pattern attribute values ​​for each pattern attribute in the predefined set of pattern attributes to a pattern matching engine running on the computing system via the parser. The pattern matching engine may include: (i) multiple pattern mapping techniques for transforming a record from an initial pattern to a target pattern, and (ii) a set of rules, wherein each rule suggests at least one pattern mapping technique from the multiple pattern mapping techniques based on at least one pattern attribute from the predefined set of pattern attributes. The method further includes: applying the set of rules to the first and second pattern attribute values ​​via the pattern matching engine to select at least one pattern mapping technique from the multiple pattern mapping techniques. The method further includes: transforming the first data record via the pattern matching engine according to the at least one selected pattern mapping technique. The method additionally includes providing a transformed first data record in a data structure according to a second pattern via a computing system.

[0006] In a second aspect, a computing system is provided. The computing system includes a parser and a pattern matching engine. The pattern matching engine includes: (i) multiple pattern mapping techniques for transforming records from an initial pattern to a target pattern, and (ii) a set of rules, wherein each rule proposes at least one pattern mapping technique from the multiple pattern mapping techniques based on at least one pattern attribute from a predefined set of pattern attributes. The computing system also includes one or more processors configured to cause the computing system to perform operations. The operations may include obtaining a first pattern and a first data record constructed according to the first pattern by the parser. The operations may also include: for the first pattern, determining one or more first pattern attribute values ​​for each pattern attribute in the predefined set of pattern attributes. The operations may also include: for a second pattern, determining one or more second pattern attribute values ​​for each pattern attribute in the predefined set of pattern attributes. The operations may also include: providing the first and second pattern attribute values ​​for each pattern attribute in the predefined set of pattern attributes to the pattern matching engine by the parser. The operations may additionally include: applying a set of rules to the first and second pattern attribute values ​​by the pattern matching engine to select at least one pattern mapping technique from the multiple pattern mapping techniques. The operation may further include: transforming the first data record using a pattern matching engine according to at least one selected pattern mapping technique. The operation may additionally include providing the transformed first data record in a data structure according to a second pattern using a computing system.

[0007] In a third aspect, an article of art is provided. The article of art may include a non-transitory computer-readable medium having program instructions stored thereon, the program instructions causing the computing system to perform operations when executed by one or more processors of a computing system. The operations may include obtaining a first pattern and a first data record constructed according to the first pattern via a parser running on the computing system. The operations may further include: for the first pattern, determining one or more first pattern attribute values ​​for each pattern attribute in a predefined set of pattern attributes. The operations may further include: for a second pattern, determining one or more second pattern attribute values ​​for each pattern attribute in the predefined set of pattern attributes. The operations may further include: providing the first and second pattern attribute values ​​for each pattern attribute in the predefined set of pattern attributes to a pattern matching engine running on the computing system via the parser. The pattern matching engine includes: (i) multiple pattern mapping techniques for transforming records from an initial pattern to a target pattern, and (ii) a set of rules, wherein each rule suggests at least one pattern mapping technique from the multiple pattern mapping techniques based on at least one pattern attribute from the predefined set of pattern attributes. The operation may further include: applying a set of rules to the first pattern attribute value and the second pattern attribute value using a pattern matching engine to select at least one pattern mapping technique from a variety of pattern mapping techniques. The operation may also include: transforming the first data record using the pattern matching engine according to at least one selected pattern mapping technique. The operation may additionally include providing the transformed first data record in a data structure according to the second pattern using a computing system.

[0008] Other aspects, embodiments, and implementations will become apparent to those skilled in the art when the following detailed description is read with appropriate reference to the accompanying drawings. Attached Figure Description

[0009] Figure 1 A computing system is illustrated according to an example embodiment.

[0010] Figure 2 Record attributes and pattern attributes are illustrated in the example embodiment.

[0011] Figure 3 The allocation rules and pattern mapping techniques are illustrated in the example embodiments.

[0012] Figure 4A , Figure 4B and Figure 4C The relationship between the parser and the pattern matching engine is illustrated in the example embodiment.

[0013] Figure 5 A message flow is described according to an example embodiment.

[0014] Figure 6The method is illustrated according to an example embodiment. Detailed Implementation

[0015] This document describes exemplary methods, devices, and systems. It should be understood that the terms “exemplary” and “illustrated” are used herein to mean “serving as an example, instance, or illustration.” Any embodiment or feature described herein as “exemplary” or “illustrated” is not necessarily to be construed as preferred or advantageous over other embodiments or features. Other embodiments may be utilized, and other changes may be made without departing from the scope of the subject matter presented herein.

[0016] Therefore, the exemplary embodiments described herein are not intended to be limiting. As generally described herein and illustrated in the accompanying drawings, aspects of this disclosure can be arranged, replaced, combined, separated, and designed in a variety of different configurations, all of which are contemplated herein.

[0017] Furthermore, unless the context otherwise suggests, each of the features shown in the accompanying drawings can be used in combination with each other. Therefore, the drawings should generally be considered as aspects of one or more overall embodiments, and it should be understood that not all features shown are necessary for every embodiment.

[0018] I. Overview

[0019] Data records often need to be transformed from being constructed according to a first schema to being constructed according to a second schema. This is also known as "schema mapping," and such transformations can occur in a wide variety of settings. For example, if a machine learning model has specific schema requirements for its input data, then the data records should be transformed to be constructed according to those specific schema requirements before being passed to the machine learning model. As another example, if a legacy database is replaced by a new database with data constructed according to a new schema, then the existing data records in the legacy database should be transformed to be constructed according to the new schema. Other settings for schema mapping can also exist.

[0020] Typically, schema mapping is performed as follows: The schema transformation system provides the user with a list of predetermined schema mapping techniques for selection. Next, the user manually selects a schema mapping technique from the list. Then, the schema transformation system applies the selected schema mapping technique to transform the data record from being constructed according to a first schema to being constructed according to a second schema.

[0021] However, one problem with the above process is that the schema transformation system does not provide the user with any context regarding which schema mapping techniques are most suitable for the data records or the first and second schemas. Instead, the user naively chooses a schema mapping technique without considering the content of the data records or the structural details of the first and second schemas. This can result in flawed transformations of the data records. For example, an ill-judged schema mapping technique may map some data records to incorrect fields or may omit some data records, and so on.

[0022] While users could manually investigate the content of data records or the structural details of the first and second patterns to select a suitable pattern mapping technique, performing such a feat could be overly complex. For example, the number of data records could be thousands or even millions, making manual inspection challenging. Furthermore, a brute-force approach of trying all pattern mapping techniques could be computationally expensive and impractical. Additionally, manually determining aspects of the content or structural details might be too difficult for users. For instance, calculating the average string length across all data records could be extremely challenging.

[0023] Some of the techniques described herein address these problems through a schema transformation system that automatically analyzes the content of data records and the structural details of a first and second schema to inform decisions regarding which schema mapping techniques to select from a library of schema mapping techniques. It will be understood that the selected schema matching technique is used to control the computing system to transform the first data record into a data structure according to the second schema. Therefore, the various embodiments described herein provide the generation of functional data (in the form of the selected schema mapping technique) that contributes to the technical effect. Advantageously, the need for manual analysis of content and structure is eliminated, allowing the schema transformation system to quickly determine which schema mapping techniques are most suitable. Furthermore, the embodiments described herein provide a scalable framework that can be incorporated into new schema mapping techniques as they are developed, thereby further improving the capabilities of the schema transformation system. Other advantages are also anticipated and will be recognized from the discussion herein.

[0024] II. Example Computing System

[0025] Figure 1A computing system 100 is illustrated according to an example embodiment. The computing system 100 may be an example pattern transformation system that automatically analyzes the content of data records and the structural details of a first and second pattern to inform decisions regarding which pattern mapping techniques to select from a library of pattern mapping techniques. In the example embodiment, the computing system 100 includes a parser 110, a pattern matching engine 120, a network interface 140, and a controller 150; however, in other embodiments, the computing system 100 may include other components. In some embodiments, the parser 110 and the pattern matching engine 120 may take the form of separate software modules executable by the computing system 100. In other embodiments, the parser 110 and the pattern matching engine 120 may be combined into a single software module executable by the computing system 100. In some embodiments, the computing system 100 may take the form of a desktop computing device, a server device, or a mobile computing device.

[0026] Network interface 140 may include one or more wireless interfaces and / or wired interfaces configurable for communication via a network. The wireless interface may include one or more wireless transmitters, receivers, and / or transceivers configurable for communication via a wireless network, such as Bluetooth. TM transceiver transceiver, Wi-Fi TM transceiver, WiMAX TM Wireless transceivers of various types, including wired and / or other similar types. Wired interfaces may include one or more wired transmitters, receivers, and / or transceivers configured to communicate via twisted-pair, coaxial, fiber optic links, or similar physical connections to a wired network, such as Ethernet transceivers, Universal Serial Bus (USB) transceivers, or similar transceivers.

[0027] In some embodiments, network interface 140 may be configured to provide reliable, secure, and / or authenticated communication. For each communication described herein, information for facilitating reliable communication (e.g., guaranteeing message delivery) may be provided, possibly as part of message headers and / or footers (e.g., packet / message sequence information, encapsulation headers and / or footers, size / time information, and transmission verification information, such as Cyclic Redundancy Check (CRC) and / or parity values). Communication may be made secure (e.g., encoded or encrypted) and / or decrypted / decoded using one or more cryptographic protocols and / or algorithms, such as, but not limited to, Data Encryption Standard (DES), Advanced Encryption Standard (AES), Rivest-Shamir-Adelman (RSA) algorithm, Diffie-Hellman algorithm, Secure Sockets Protocols such as Secure Sockets Layer (SSL) or Transport Layer Security (TLS), and / or Digital Signature Algorithm (DSA). Other cryptographic protocols and / or algorithms may also be used, or in addition to those listed herein, to make communication secure (and then decrypt / decode the communication).

[0028] Controller 150 may include one or more processors 152 and data storage device 154. The processors 152 may include one or more general-purpose processors and / or one or more special-purpose processors (e.g., digital signal processors (DSPs), tensor processing units (TPUs), graphics processing units (GPUs), application-specific integrated circuits (ASICs), etc.). The processors 152 may be configured to execute computer-readable instructions and / or other instructions as described herein contained in the data storage device 154. The data storage device 154 may include one or more non-transitory computer-readable storage media that can be read and / or accessed by the processors 152. The one or more non-transitory computer-readable storage media may include volatile and / or non-volatile storage components, such as optical, magnetic, organic, or other memory or disk storage devices, which may be integrated integrally or partially with at least one of the processors 152. In some examples, the data storage device 154 may be implemented using a single physical device (e.g., a single optical, magnetic, organic, or other memory or disk storage unit); however, in other examples, the data storage device 154 may be implemented using two or more physical devices. In an example embodiment, processor(s) 152 may be configured to run instructions stored in data storage device 154 to perform one or more operations, such as those described in method 500 or 600 below.

[0029] Parser 110 can be configured with one or more predefined record attributes and / or one or more predefined pattern attributes. During operation, parser 110 can analyze the input pattern and the input data record constructed based on the input pattern to determine the record attribute values ​​for the predefined record attributes and / or the pattern attribute values ​​for the predefined pattern attributes.

[0030] To conceptually visualize record attributes and schema attributes, the following is provided: Figure 2 . Specifically, Figure 2 The parser 110 is shown to receive input data record 200 and input pattern 210 and then analyzes the input data record 200 and input pattern 210 in response to determine the record attribute value for record attribute 230 and the pattern attribute value for pattern attribute 240.

[0031] Record attribute 230 includes attributes presented by input data record 200. Therefore, a record attribute value for a particular record attribute can indicate the degree to which the input data record 200 presents that particular record attribute. In the example, record attribute 230 may include an average string length attribute 230A, a value range attribute 230B, and a unique value attribute 230C. Record attribute values ​​associated with record attribute 230 can be character strings, numbers, boolean values, or null values, as well as objects or arrays. For example, a record attribute value associated with the average string length attribute 230A could be a number, a record attribute value associated with the value range attribute 230B could be an array, and a record attribute value associated with the unique value attribute 230C could be a boolean value.

[0032] In some examples, the user can provide record attribute 230 to parser 110 before parser 110 performs operations on input data record 200. When record attribute 230 is provided to parser 110, the user can configure parser 110 with instructions on which elements in input data record 200 to locate and what data to extract from those located elements in order to determine the record attribute value for record attribute 230.

[0033] Therefore, during operation, parser 110 can analyze input data record 200 to determine record attribute values ​​associated with record attribute 230. For example, to obtain the record attribute value for average string length attribute 230A, parser 110 can (i) locate all records in input data record 200 that have string data type and (ii) calculate the average length of all located records that have string data type. As another example, to obtain the record attribute value for value range attribute 230B, parser 110 can (i) locate all records in input data record 200 that have integer or floating-point data type and (ii) calculate the mathematical range for all located records that have integer or floating-point data type. As yet another example, to obtain the record attribute value for unique value attribute 230C, parser 110 can (i) locate all records in input data record 200 and (ii) determine whether any of the located records is a multiple record.

[0034] Note that record attribute 230 is presented for illustrative purposes and is not intended to be limiting with respect to the embodiments described herein. Other record attributes may also exist.

[0035] Pattern attribute 240 includes attributes that can be presented by input pattern 210. Therefore, a pattern attribute value for a specific pattern attribute can indicate the degree to which input pattern 210 presents that specific pattern attribute. In the example, pattern attribute 240 includes nested attribute 240A, language type attribute 240B, domain type attribute 240C, and pattern type attribute 240D. Like record attribute values, pattern attribute values ​​associated with pattern attribute 240 can be character strings, numbers, boolean values, or null values, as well as objects or arrays. For example, the pattern attribute value associated with nested attribute 240A can be a boolean value, the pattern attribute value associated with language type attribute 240B can be a string, the pattern attribute value associated with domain type attribute 240C can be a string, and the pattern attribute value associated with pattern type attribute 240D can also be a string.

[0036] In some examples, the user can provide the pattern attribute 240 to the parser 110 before the parser 110 performs operations on the input pattern 210. When the pattern attribute 240 is provided to the parser 110, the user can configure the parser 110 with instructions on which elements in the input pattern 210 should be located and what data should be extracted from those located elements in order to determine the pattern attribute value for the pattern attribute 240.

[0037] Therefore, during operation, parser 110 can analyze input pattern 210 to determine pattern attribute values ​​associated with pattern attribute 240. For example, to obtain the pattern attribute value for nested attribute 240A, parser 110 can (i) locate all fields in input pattern 210 and (ii) determine whether any of the located fields is associated with an object data type. As another example, to obtain the pattern attribute value for language type attribute 240B, parser 110 can (i) locate all fields in input pattern 210 and (ii) apply a natural language processing model to the located fields to determine the most likely language associated with input pattern 210. As yet another example, to obtain the pattern attribute value for domain type attribute 240C, parser 110 can (i) locate all fields in input pattern 210 and (ii) apply a natural language processing model to the located fields to determine the most likely subject domain associated with input pattern 210. As yet another example, in order to obtain the pattern attribute value for pattern type attribute 240D, parser 110 may (i) locate all fields in input pattern 210 and (ii) apply a trained machine learning model to the located fields to determine the most likely pattern type associated with input pattern 210.

[0038] Note that pattern attribute 240 is used for illustrative purposes and is not intended to be limiting with respect to the embodiments described herein. Other pattern attributes may also be present.

[0039] Pattern matching engine 120 may include multiple pattern mapping techniques for transforming data records from those constructed according to a first pattern to those constructed according to a second pattern. Additionally, pattern matching engine 120 may include a set of assignment rules, wherein each assignment rule in the set of assignment rules suggests at least one pattern mapping technique from multiple pattern mapping techniques based on (i) pattern attributes presented by the first and second patterns and / or (ii) record attributes presented by the first data record constructed according to the first pattern and the second data record constructed according to the second pattern.

[0040] To conceptually illustrate allocation rules and pattern mapping techniques, the following is provided: Figure 3 . Specifically, Figure 3 This illustrates how assignment rule 310 can utilize the schema attributes and record attributes determined by parser 110 to select a schema mapping technique from schema mapping techniques 300.

[0041] Pattern mapping technique 300 can transform data records from one pattern to another. Each technique in pattern mapping technique 300 can use different methods to perform this transformation. In an example embodiment, pattern mapping technique 300 includes string distance technique 300A, semantic distance technique 300B, data type technique 300C, and value distribution technique 300D. However, other pattern mapping techniques are also possible and are envisioned in the disclosure herein. Furthermore, pattern mapping technique 300 can be extensible, allowing new pattern mapping techniques to be added to it. In an example, adding a new pattern mapping technique to pattern mapping technique 300 may involve configuring assignment rule 310 such that at least some combination of pattern attributes or record attributes determined by parser 110 suggests a new pattern mapping technique.

[0042] String distance technology 300A can transform data records from one pattern to another by calculating the string similarity score between field names in a first pattern and field names in a second pattern. For example, if the first pattern has a field named "Tree" and the second pattern has a field named "Trees" and another field named "Chicago", then string distance technology 300A may be more inclined to transform the data record from "Tree" to "Trees" rather than from "Tree" to "Chicago" because "Tree" has a greater string similarity to "Trees" than to "Chicago".

[0043] Semantic distance technology 300B can transform data records from one schema to another by calculating the semantic similarity score between field names in a first schema and field names in a second schema. For example, if the first schema has a field named "Trees" and the second schema has a field named "Treasure" and another field named "Forest", then semantic distance technology 300B may be more inclined to transform the data record from "Trees" to "Forest" rather than from "Trees" to "Treasure" because "Trees" has greater semantic similarity to "Forest" than to "Treasure".

[0044] Data Type Technology 300C can transform data records from one schema to another by calculating the data type similarity score between fields in a first schema and fields in a second schema. For example, if the first schema contains a field with an integer data type and the second schema contains a field with a floating-point data type and another field with a timestamp data type, then Data Type Technology 300C may be more inclined to transform the record from the field with the integer data type to the floating-point data type rather than to the timestamp data type, because integers and floating-point have greater data type similarity than timestamps.

[0045] Value distribution technique 300D can transform a first data record from one pattern to another by calculating a value similarity score between the value of a first data record and the value of a representative second data record constructed according to a second pattern. For example, if the first data record contains a field with a mean value of 102 and the second data record contains a field with a mean value of 100 and another field with a mean value of 400, then value distribution technique 300D may be more inclined to transform the data record from the field with a mean value of 102 to the field with a mean value of 100 rather than to the field with a mean value of 400, because 102 has a greater value similarity score than 100.

[0046] Assignment rule 310 can assist pattern matching engine 120 in selecting one or more pattern mapping techniques from pattern mapping techniques 300. Specifically, a given assignment rule from assignment rule 310 can evaluate pattern attributes or record attributes determined by parser 110 to suggest one or more pattern mapping techniques from pattern mapping techniques 300. Assignment rule 310 can be evaluated independently or jointly to suggest one or more pattern mapping techniques. In an example embodiment, assignment rule 310 includes a first pattern nesting rule 310A, a second pattern type rule 310B, and a first record average string length rule 310C. However, other assignment rules are possible and are contemplated in the disclosure herein.

[0047] First pattern nesting rule 310A is an example of an independently evaluated rule. This rule can utilize the nesting attribute 240A from pattern attribute 240 to determine which pattern mapping technique from the pattern mapping techniques 300 should be suggested. For example, if a pattern attribute value associated with nesting attribute 240A indicates that the first pattern is nested (e.g., has a field associated with an object data type), then first pattern nesting rule 310A can suggest string distance technique 300A. Otherwise, if a pattern attribute value associated with nesting attribute 240A indicates that the first pattern is not nested, then first pattern nesting rule 310A can suggest semantic distance technique 300B.

[0048] The second pattern type rule 310B and the first record average string length rule 310C are examples of joint evaluation rules. These rules can utilize the average string length attribute 230A from record attribute 230 and the pattern type attribute 240A from pattern attribute 240 to determine which pattern mapping technique from the pattern mapping techniques 300 should be recommended.

[0049] For example, if the pattern attribute value associated with pattern type attribute 240A indicates that the second pattern is a JavaScript Object Notation (JSON) pattern and the record attribute value associated with average string length attribute 230A indicates that the first data record has an average string length of less than 8, then the second pattern type rule 310B and the first record average string length rule 310C can suggest string distance technique 300A. Otherwise, if the pattern attribute value associated with pattern type attribute 240A indicates that the second pattern is a JSON pattern and the record attribute value associated with average string length attribute 230A indicates that the first data record has an average string length of greater than 8, then the second pattern type rule 310B and the first record average string length rule 310C can suggest semantic distance technique 300B.

[0050] Alternatively, if the schema attribute value associated with schema type attribute 240A indicates that the second schema is a relational schema and the record attribute value associated with average string length attribute 230A indicates that the first data record has an average string length of less than 8, then the second schema type rule 310B and the first record average string length rule 310C can suggest data type technique 300C. Otherwise, if the schema attribute value associated with schema type attribute 240A indicates that the second schema is a relational schema and the record attribute value associated with average string length attribute 230A indicates that the first data record has an average string length of greater than 8, then the second schema type rule 310B and the first record average string length rule 310C can suggest value distribution technique 300D.

[0051] Note that although assignment rule 300 is represented as a decision tree, this is merely illustrative and not intended to be limiting with respect to the embodiments herein. In other examples, assignment rule 300 may take the form of a predictive model. Predictive models may include, but are not limited to: artificial neural networks, Bayesian networks, hidden Markov models, Markov decision processes, logistic regression functions, support vector machines, statistical machine learning algorithms, and / or heuristic machine learning systems. The predictive model may be designed to take record attribute values ​​for record attribute 230 and pattern attribute values ​​for pattern attribute 240 as inputs and correspondingly output one or more pattern mapping techniques from pattern mapping technique 310 suitable for transforming data records from a construction based on a first pattern to a construction based on a second pattern.

[0052] Additionally, the allocation rule 300 can be configured to suggest a pattern mapping technique from the pattern mapping technique 300 based on one or more previous pattern transformations performed by the pattern matching engine 120. For example, suppose the pattern matching engine 120 uses semantic distance technique 300B to transform a data record from being constructed according to pattern A to being constructed according to pattern B. Then, at a later point in time, suppose the pattern matching engine 120 is responsible for transforming a data record from being constructed according to pattern C to being constructed according to pattern D. If pattern matching engine 120 determines that (i) the pattern attribute associated with pattern C is similar to the pattern attribute associated with pattern A (e.g., having the same value, within a threshold distance from each other, or, if the pattern attribute is represented as a vector, within a threshold cosine similarity (greater than .9) or Jaccard similarity) and (ii) the pattern attribute associated with pattern D is similar to the pattern attribute associated with pattern B, then the assignment rule 300 within pattern matching engine 120 can be configured to suggest the same pattern mapping technique used to transform data records from pattern A to pattern B as the pattern mapping technique used to transform data records from pattern C to pattern D (in this case, semantic distance technique 300B).

[0053] Figure 4A , Figure 4B and Figure 4C The relationship between parser 110 and pattern matching engine 120 is illustrated in an example embodiment. Specifically, Figure 4A , Figure 4B and Figure 4C Provided to illustrate how pattern matching engine 120: (i) receives a first data record constructed according to a first pattern, (ii) receives record attribute values ​​and pattern attribute values ​​from parser 110 and (iii) uses the record attribute values ​​and / or pattern attribute values ​​to select one or more pattern mapping techniques to transform the first data record from being constructed according to the first pattern to being constructed according to the second pattern.

[0054] Figure 4A The diagram illustrates how the pattern matching engine 120 receives a first data record 400 and an attribute value 408 and then transforms the first data record 400 into a transformed data record 410, wherein the transformed data record 410 has the same pattern as the second data record 406 (i.e., the second pattern 404).

[0055] The first data record 400 is constructed according to the first pattern 402. For simplicity, the content of the first pattern 402 is not shown, but the first pattern 402 can be a JSON encoding pattern. Typically, JSON is a hierarchical nesting of objects and arrays. A JSON object is an unordered collection of key / value pairs that begins with a left curly brace ("{") and ends with a right curly brace ("}"), where keys can correspond to fields and values ​​can correspond to entries for those fields. Values ​​can be strings, numbers, booleans, or null values, as well as objects or arrays. A JSON array is an ordered collection of values ​​that begins with a left square bracket ("[") and ends with a right square bracket ("]"). Values ​​in an array are separated by commas.

[0056] The second data record 406 is constructed according to the second pattern 404. For simplicity, the contents of the second pattern 404 are not shown, but the second pattern 404 can also be a JSON encoding pattern. However, the second pattern 404 can differ from the first pattern 402. For example, the first data record 400 is shown as having the keys "first name" and "last name", while the second data record 406 is shown as having the key "first name". This inconsistency is an example of the difference between the first pattern 402 and the second pattern 404.

[0057] Parser 110 can be configured to analyze first data record 400 and second data record 406 to determine record attribute values ​​associated with record attribute 230. Specifically, parser 110 can analyze first data record 400 to determine a first record attribute value associated with record attribute 230, wherein the first record attribute value indicates the degree to which first data record 400 presents record attribute 230. Parser 110 can also analyze second record 406 to determine a second record attribute value associated with record attribute 230, wherein the second record attribute value indicates the degree to which second data record 406 presents record attribute 230. As an example, to obtain a first record attribute value for value range attribute 230B, parser 110 can (i) locate all records in first data record 400 that have integer or floating-point data types and (ii) calculate a mathematical range for all located records that have integer or floating-point data types.

[0058] Similarly or alternatively, parser 110 may also be configured to analyze the first pattern 402 and the second pattern 404 to determine pattern attribute values ​​associated with pattern attribute 240. Specifically, parser 110 may analyze the first pattern 402 to determine a first pattern attribute value associated with pattern attribute 240, wherein the first pattern attribute value indicates the degree to which the first pattern 402 presents pattern attribute 240. Parser 110 may also analyze the second pattern 404 to determine a second pattern attribute value associated with record attribute 240, wherein the second pattern attribute value indicates the degree to which the second pattern 404 presents pattern attribute 240. As an example, to obtain a second pattern attribute value for nested attribute 240A, parser 110 may (i) locate all fields in the second pattern 404 and (ii) determine whether any of the located fields is associated with an object data type.

[0059] After analyzing the first data record 400, the second data record 406, the first pattern 402, and the second pattern 404, the parser 110 can provide the determined record attribute value associated with the record attribute 230 and the determined pattern attribute value associated with the pattern attribute 240 to the pattern matching engine 120 in the form of attribute value 408.

[0060] Pattern matching engine 120 can receive attribute value 408 from parser 110 and can use attribute value 408 and assignment rule 310 to select one or more pattern mapping techniques to transform first data record 400 into transformed data record 410. In this example, the keys in first data record 400 have similar string names to the keys in second data record 418. For example, second data record 406 uses the keys "name", "education", and "state", and first data record 400 uses the keys "first name", "last name", "education status", and "state". Assignment rule 310 can determine the string similarity and can cause pattern matching engine 120 to select string distance technique 300A to transform first data record 400 into transformed data record 410.

[0061] Figure 4B The diagram illustrates how the pattern matching engine 120 receives a first data record 412 and an attribute value 420 and then transforms the first data record 412 into a transformed data record 422, wherein the transformed data record 422 has the same pattern as the second data record 418.

[0062] The first data record 412 is constructed according to the first schema 414. For simplicity, the contents of the first schema 414 are not shown, but the first schema 414 can be a relational schema. Typically, a relational schema organizes data records into a series of interconnected tables. Tables contain columns and rows. Rows in a table correspond to individual entries, while columns correspond to fields of those individual entries. Entries can be character strings, numbers, boolean values, or null values.

[0063] The second data record 418 is constructed according to the second schema 416. For simplicity, the contents of the second schema 416 are not shown, but the second schema 416 can also be a relational schema. However, the second schema 416 can differ from the first schema 414. For example, the first data record 412 is shown as constructed on two tables, while the second data record 418 is shown as constructed on only a single table. Furthermore, the first data record 412 is shown as having the attribute "name", while the second data record 418 is shown as having the attribute "name". These inconsistencies are examples of the differences between the first schema 414 and the second schema 416.

[0064] Similar to Figure 4A Parser 110 can be configured to analyze first data record 412 and second data record 418 to determine record attribute values ​​associated with record attribute 230. Similarly or alternatively, parser 110 can also be configured to analyze first pattern 414 and second pattern 416 to determine pattern attribute values ​​associated with pattern attribute 240. After analyzing first data record 412, second data record 418, first pattern 414, and second pattern 416, parser 110 can provide the determined record attribute values ​​associated with record attribute 230 and the determined pattern attribute values ​​associated with pattern attribute 240 to pattern matching engine 120 in the form of attribute value 420.

[0065] Pattern matching engine 120 can receive attribute values ​​420 from parser 110 and can use attribute values ​​420 and assignment rules 310 to select one or more pattern mapping techniques to transform first data record 412 into transformed data record 422. In this example, columns in first data record 412 have semantically similar names to columns in second data record 418. For example, second data record 418 uses column names “name”, “age”, and “occupation”, while first data record 412 uses column names “name”, “age”, and “job”. Assignment rule 310 can determine this semantic similarity and may cause pattern matching engine 120 to select semantic distance technique 300B to transform first data record 412 into transformed data record 422.

[0066] Figure 4CThe diagram illustrates how the pattern matching engine 120 receives a first data record 400 and an attribute value 424 and then transforms the first data record 400 into a transformed data record 426, wherein the transformed data record 426 has the same pattern as the second data record 418.

[0067] The first data record was 400 previously. Figure 4A The second data record 450 was previously discussed and constructed according to the first pattern 402. Figure 4B As discussed earlier, and constructed according to the second schema 416. The second schema 416 can differ from the first schema 402. For example, the first data record 400 is shown as constructed according to a JSON schema, while the second data record 418 is shown as constructed according to a relational schema. Furthermore, the first data record 400 is shown as having attributes "first name" and "last name", while the second data record 418 is shown as having the attribute "first name". These inconsistencies are examples of the differences between the first schema 402 and the second schema 416.

[0068] Similar to Figure 4A and Figure 4B Parser 110 can analyze the first data record 400 and the second data record 418 to determine the record attribute value associated with record attribute 230. Similarly or alternatively, parser 110 can also analyze the first pattern 402 and the second pattern 416 to determine the pattern attribute value associated with pattern attribute 240. After analyzing the first data record 400, the second data record 418, the first pattern 402, and the second pattern 416, parser 110 can provide the determined record attribute value associated with record attribute 230 and the determined pattern attribute value associated with pattern attribute 240 to the pattern matching engine 120 in the form of attribute value 424.

[0069] Pattern matching engine 120 can receive attribute value 424 from parser 110 and can use attribute value 424 and assignment rule 310 to select one or more pattern mapping techniques to transform first data record 400 into transformed data record 426. In this example, the keys in first data record 400 have semantically similar names to the columns in second data record 418. For example, second data record 418 uses column names “name”, “age”, and “occupation”, and first data record 412 uses key names “name” and “age”. Assignment rule 310 can determine this semantic similarity and can cause pattern matching engine 120 to select semantic distance technique 300B to transform first data record 400 into transformed data record 426.

[0070] III. Example Method

[0071] Figure 5Message flow 500 is depicted according to an example embodiment. Message flow 500 can be logically divided into three phases: a parsing phase 502, in which parser 110 receives and parses a first pattern and a second pattern, as well as a first data record and a second data record; a prediction phase 504, in which pattern matching engine 120 determines one or more pattern mapping techniques and then transforms the first data record according to one or more determined pattern mapping techniques; and a feedback phase 506, in which client device 508 provides feedback on the transformations made by pattern matching engine 120 in prediction phase 504. As an example, message flow 500 may utilize parser 110, pattern matching engine 120, and client device 508 during operation. However, additional components, steps, or blocks may be added to message flow 500 without departing from the scope of this disclosure.

[0072] At block 510, parser 110 receives (i) a first pattern and (ii) a first data record constructed according to the first pattern. In some embodiments, parser 110 may determine the first pattern by parsing the first data record. In some embodiments, parser 110 receives the first pattern and the first data record from client device 508. For example, parser 110 may prompt client device 508 to input appropriate data for the first pattern and the first data record. This can be done via a webpage or a series of webpages hosted by parser 110 and provided to client device 508 upon request. Alternatively, parser 110 may receive the first pattern and the first data record from another device, such as a database device or a cloud-based computing device.

[0073] At block 512, parser 110 receives (i) a second pattern and (ii) a second data record constructed according to the second pattern. As similarly described in block 510, parser 110 can determine the second pattern by parsing the second data record. As similarly described in block 510, parser 110 receives the second pattern and the second data record from client device 508. Alternatively, parser 110 can receive the second pattern and the second data record from another device, such as a database device or a cloud-based computing device.

[0074] Since the pattern matching engine 120 can be configured to transform all first data records without transforming the second data records, in some embodiments, the number of second data records can be less than the number of first data records.

[0075] At block 514, parser 110 determines record attribute values ​​for one or more predefined record attributes and pattern attribute values ​​for one or more predefined pattern attributes. For example, parser 110 may analyze a first data record to determine a first record attribute value associated with record attribute 230 and may analyze a second data record to determine a second record attribute value associated with record attribute 230. As another example, parser 110 may analyze a first pattern to determine a first pattern attribute value associated with pattern attribute 240 and may analyze a second pattern to determine a second pattern attribute value associated with pattern attribute 240.

[0076] At block 516, parser 110 sends (i) the first pattern attribute value, (ii) the second pattern attribute value, (iii) the first record attribute value, and (iv) the second record attribute value to pattern matching engine 120. Parser 110 may additionally send the first data record to pattern matching engine 120.

[0077] At block 518, pattern matching engine 120 uses the attribute values ​​received from parser 110 at block 516 to determine one or more pattern mapping techniques from a library of pattern mapping techniques included within pattern matching engine 120. (See reference...) Figure 3 As described, the pattern matching engine 120 can use one or more assignment rules / prediction models to determine one or more pattern mapping techniques. These prediction models may include, but are not limited to: artificial neural networks, Bayesian networks, hidden Markov models, Markov decision processes, logistic regression functions, support vector machines, decision trees, statistical machine learning algorithms, and / or heuristic machine learning systems.

[0078] At block 520, pattern matching engine 120 uses one or more pattern mapping techniques determined in block 518 to transform the first data record from being constructed according to a first pattern to being constructed according to a second pattern.

[0079] In some embodiments, the pattern matching engine 120 may use different pattern mapping techniques on different portions of the first data record. For example, the first data record may contain a first plurality of first data records and a second plurality of first data records, wherein the second plurality of records differs from the first plurality of records. In some cases, the second plurality of first data records may be nested within the first plurality of first data records. In such embodiments, the pattern matching engine 120 may transform the first plurality of first data records according to a first pattern mapping technique derived from one or more pattern mapping techniques determined in block 518, and may transform the second plurality of first data records according to a second pattern mapping technique derived from one or more pattern mapping techniques determined in block 518. For example, the pattern matching engine 120 may use string distance technique 300A to transform the first plurality of first data records and may use value distribution technique 300D to transform the second plurality of first data records.

[0080] In some embodiments, after transformation, the pattern matching engine 120 may store the transformed first data record into a permanent storage device, wherein the data record stored in the permanent storage device is constructed according to a second pattern.

[0081] At block 520, pattern matching engine 120 provides a transformed first data record according to the data structure of the second pattern to client device 508. Providing the transformed first data record may include, for example, generating one or more graphical user interfaces (GUIs) containing a representation of the transformed first data record and providing one or more GUIs to client device 508.

[0082] At block 524, pattern matching engine 120 can receive feedback values ​​from client device 508 regarding the transformations made at block 520. Receiving feedback can be accomplished, for example, via a GUI provided by pattern matching engine 120 to client device 508 (possibly via network interface 140). The GUI may include one or more features for providing feedback to pattern matching engine 120. For example, the GUI may include data input fields that allow client device 508 to grade the transformations made at block 520. As an example, client device 508 may provide levels from 1 to 10, where higher levels (e.g., 9 or 10) correspond to satisfactory transformations at block 520 (e.g., the transformed first data record strictly conforms to the second pattern), and lower levels (e.g., 1 or 2) correspond to unsatisfactory transformations at block 520 (e.g., the transformed first data record does not strictly conform to the second pattern).

[0083] In some embodiments, the client device 508 provides multiple feedback values ​​for different segments of the transformed first data record. For example, the transformed first data record may include a first set of transformed first data records and a second set of transformed first data records, wherein the first set is different from the second set. Therefore, the client device 508 can provide a first feedback value indicating the degree to which the first set of transformed first data records conforms to the second pattern and a second feedback value indicating the degree to which the second set of transformed first data records conforms to the second pattern. In particular, other methods of providing feedback also exist.

[0084] At block 526, pattern matching engine 120 can adjust assignment rule 300 based on feedback received at block 524. For example, if (i) based on analysis of attribute values ​​received at block 516, assignment rule 300 selects data type technique 300C as the pattern mapping technique for block 520, and (ii) the feedback provided at block 524 indicates that client device 508 is dissatisfied with the transformation at block 520, then pattern matching engine 120 can adjust assignment rule 300 such that, based on analysis of attribute values ​​received at block 516, assignment rule 300 will select an alternative pattern mapping technique to data type technique 300C. Other adjustments are also possible. In some embodiments, pattern matching engine 120 can store feedback provided by client device 508 in a permanent storage device for later use.

[0085] IV. Example Operation

[0086] Figure 6 Method 600 is illustrated according to an example embodiment. Method 600 may include various blocks or steps. Blocks or steps may be performed individually or in combination. Blocks or steps may be performed in any order and / or sequentially or in parallel. Furthermore, blocks or steps may be omitted or added to method 600. The blocks of method 600 may be defined as follows: Figure 1 The various components of the computing system 100 illustrated and described perform operations.

[0087] Block 610 may involve obtaining a first pattern and constructing a first data record based on the first pattern by a parser running on a computing system;

[0088] Block 620 may involve determining one or more first pattern attribute values ​​for each pattern attribute in a predefined set of pattern attributes, through a parser and for a first pattern.

[0089] Block 630 may involve determining one or more second pattern attribute values ​​for each pattern attribute in a predefined set of pattern attributes, through a parser and for the second pattern;

[0090] Block 640 may involve providing a first pattern attribute value and a second pattern attribute value for each pattern attribute in a predefined set of pattern attributes to a pattern matching engine running on a computing system via a parser. The pattern matching engine may include: (i) multiple pattern mapping techniques for transforming records from an initial pattern to a target pattern, and (ii) a set of rules. Each rule may suggest at least one pattern mapping technique from the multiple pattern mapping techniques based on at least one pattern attribute from the predefined set of pattern attributes.

[0091] Block 650 may involve applying a set of rules to a first pattern attribute value and a second pattern attribute value through a pattern matching engine to select at least one pattern mapping technique from a variety of pattern mapping techniques.

[0092] Block 660 may involve transforming the first data record using a pattern matching engine according to at least one selected pattern mapping technique.

[0093] Block 670 may involve providing a transformed first data record according to a data structure based on a second mode via a computing system. In some embodiments, the data structure may be a collection of data values, relationships between data values, and functions or operations that can be applied to the data values.

[0094] Some embodiments may involve: receiving, via a parser, a representative second data record constructed according to a second pattern; determining, for a first data record, one or more first record attribute values ​​for each record attribute in a predefined set of record attributes; and for a second data record, determining one or more second record attribute values ​​for each record attribute in the predefined set of record attributes. Such embodiments may also involve providing, via a parser, the first and second record attribute values ​​for each record attribute in the predefined set of record attributes to a pattern matching engine, wherein the pattern matching engine further includes a second set of rules, and wherein each second rule suggests at least one pattern mapping technique from a plurality of pattern mapping techniques based on at least one record attribute from the predefined set of record attributes. Such embodiments may further involve applying the second set of rules to the first and second record attribute values ​​via the pattern matching engine to select at least one pattern mapping technique.

[0095] In some embodiments, the number of representative second data records is less than the number of first data records.

[0096] Some embodiments involve storing a transformed first data record into a permanent storage device using a pattern matching engine, wherein the data record stored in the permanent storage device is constructed according to a second pattern.

[0097] In some embodiments, predefined record attributes include a mathematical range attribute. In some embodiments, predefined record attributes include an average length attribute.

[0098] In some embodiments, providing the transformed first data record includes: generating one or more graphical user interfaces containing a representation of the transformed first data record by a computing system; and providing one or more graphical user interfaces to a client device by the computing system.

[0099] Some embodiments may involve: receiving from a client device at least one feedback value indicating the degree to which a transformed first data record conforms to a second mode; and storing at least one feedback value in a permanent storage device.

[0100] Some embodiments may involve updating a rule set by a pattern matching engine based on at least one feedback value, such that at least one rule suggests at least one pattern mapping technique that is different from a previously suggested pattern mapping technique.

[0101] In some embodiments, the transformed first data record may include a first plurality of transformed first data records and a second plurality of transformed first data records. In such embodiments, at least one feedback value may include a first feedback value indicating the degree to which the first plurality of transformed first data records conform to the second pattern, and at least one feedback value may include a second feedback value indicating the degree to which the second plurality of transformed first data records conform to the second pattern.

[0102] In some embodiments, a first data record may include a first plurality of first data records and a second plurality of first data records. In such embodiments, transforming a first data record according to at least one selected pattern mapping technique may include transforming the first plurality of first data records according to a first pattern mapping technique among at least one selected pattern mapping technique and transforming the second plurality of first data records according to a second pattern mapping technique among at least one selected pattern mapping technique.

[0103] In some embodiments, a second plurality of first data records are nested within a first plurality of first data records.

[0104] In some embodiments, predefined schema attributes include domain attributes. In some embodiments, predefined schema attributes include schema type attributes.

[0105] In some embodiments, the multiple pattern mapping techniques include string similarity mapping techniques. In some embodiments, the multiple pattern mapping techniques include value distribution mapping techniques. In some embodiments, the multiple pattern mapping techniques include semantic similarity mapping techniques.

[0106] Some embodiments may involve: obtaining a third pattern and a third data record constructed according to the third pattern via a parser; determining one or more third pattern attribute values ​​for each pattern attribute in a predefined set of pattern attributes via the parser and for the third pattern; and determining one or more fourth pattern attribute values ​​for each pattern attribute in the predefined set of pattern attributes via the parser and for the fourth pattern. Such embodiments may also involve: providing the third and fourth pattern attribute values ​​for each pattern attribute in the predefined set of pattern attributes to a pattern matching engine via the parser; and determining via the pattern matching engine that: (i) the third pattern attribute value is similar to a first pattern attribute value; and (ii) the fourth pattern attribute value is similar to a second pattern attribute value. Such embodiments may further include: transforming the third data record according to at least one selected pattern mapping technique via the pattern matching engine; and providing the transformed third data record according to a data structure based on the fourth pattern via a computing system.

[0107] The specific arrangements shown in the accompanying drawings should not be considered limiting. It should be understood that other embodiments may include more or fewer of each element shown in the given drawings. Furthermore, some of the elements shown may be combined or omitted. Further, exemplary embodiments may include elements not shown in the drawings.

[0108] The steps or blocks representing the processing of information may correspond to circuitry that can be configured to perform specific logical functions of the methods or techniques described herein. Alternatively or additionally, the steps or blocks representing the processing of information may correspond to modules, segments, or portions of program code (including associated data). The program code may include one or more instructions executable by a processor to implement specific logical functions or actions in the method or technique. The program code and / or associated data may be stored on any type of computer-readable medium, such as disks, hard-drive storage devices, or other storage media.

[0109] Computer-readable media may also include non-transitory computer-readable media, such as computer-readable media that store data for short periods of time, like register memory, processor cache, and random access memory (RAM). Computer-readable media may also include non-transitory computer-readable media that store program code and / or data for longer periods of time. Therefore, computer-readable media may include auxiliary or permanent long-term storage devices, such as read-only memory (ROM), optical discs or magnetic disks, and compact disc read-only memory (CD-ROM). Computer-readable media may also be any other volatile or non-volatile storage system. Computer-readable media can be considered, for example, computer-readable storage media or tangible storage devices.

[0110] While various examples and embodiments have been disclosed, other examples and embodiments will be apparent to those skilled in the art. The various disclosed examples and embodiments are for illustrative purposes and are not intended to be limiting, while the true scope is indicated by the claims.

Claims

1. A computer-implemented pattern matching method, comprising: A first pattern and a first data record constructed based on the first pattern are obtained by a parser running on a computing system; The parser determines one or more first pattern attribute values ​​for each pattern attribute in the predefined pattern attribute set for the first pattern. The parser determines one or more second pattern attribute values ​​for each pattern attribute in the predefined pattern attribute set, based on the second pattern. The parser provides a first pattern attribute value and a second pattern attribute value for each pattern attribute in the predefined pattern attribute set to a pattern matching engine running on the computing system, wherein the pattern matching engine is a software module and the pattern matching engine includes: (i) multiple pattern mapping techniques for transforming records from an initial pattern to a target pattern, and (ii) a set of rules, wherein each rule suggests at least one pattern mapping technique from the multiple pattern mapping techniques based on at least one pattern attribute from the predefined pattern attribute set; The parser receives a representative second data record constructed according to the second pattern. The parser determines one or more first record attribute values ​​for each record attribute in a predefined set of record attributes, for the first data record. The parser determines one or more second record attribute values ​​for each record attribute in the predefined set of record attributes, for the second data record. The parser provides a first record attribute value and a second record attribute value for each record attribute in the predefined set of record attributes to the pattern matching engine, wherein the pattern matching engine further includes a second set of rules, wherein each second rule suggests at least one pattern mapping technique from the plurality of pattern mapping techniques based on at least one record attribute from the predefined set of record attributes. The rule set is applied to the first pattern attribute value and the second pattern attribute value by the pattern matching engine, and the second rule set is applied to the first record attribute value and the second record attribute value, so as to select at least one pattern mapping technology from the multiple pattern mapping technologies; The pattern matching engine transforms the first data record according to at least one selected pattern mapping technique; and The computing system provides a transformed first data record based on the data structure of the second mode.

2. The computer-implemented method according to claim 1, wherein, The number of representative second data records is less than the number of first data records.

3. The computer-implemented method according to claim 1, further comprising: The transformed first data record is stored in a permanent storage device by the pattern matching engine, wherein the data record stored in the permanent storage device is constructed according to the second pattern.

4. The computer-implemented method according to claim 1, wherein, The predefined record attributes include mathematical range attributes.

5. The computer-implemented method according to claim 1, wherein, The predefined record attributes include the average length attribute.

6. The computer-implemented method according to claim 1, wherein, The first data record provided after transformation includes: The computing system generates one or more graphical user interfaces containing a representation of the transformed first data record; and The computing system provides the one or more graphical user interfaces to the client device.

7. The computer-implemented method according to claim 6, further comprising: Receive from the client device at least one feedback value indicating the degree to which the transformed first data record conforms to the second pattern; as well as The at least one feedback value is stored in a permanent storage device.

8. The computer-implemented method according to claim 7, further comprising: The pattern matching engine updates the rule set based on the at least one feedback value, such that at least one rule suggests at least one pattern mapping technique that is different from the previously suggested pattern mapping technique.

9. The computer-implemented method according to claim 7 or claim 8, wherein, The transformed first data record includes a first plurality of transformed first data records and a second plurality of transformed first data records, wherein the at least one feedback value includes a first feedback value indicating the degree to which the first plurality of transformed first data records conform to the second pattern, and wherein the at least one feedback value includes a second feedback value indicating the degree to which the second plurality of transformed first data records conform to the second pattern.

10. The computer-implemented method according to claim 1, wherein, The first data record includes a first plurality of first data records and a second plurality of first data records, wherein transforming the first data record according to the at least one selected pattern mapping technique includes: transforming the first plurality of first data records according to a first pattern mapping technique among the at least one selected pattern mapping techniques, and transforming the second plurality of first data records according to a second pattern mapping technique among the at least one selected pattern mapping techniques.

11. The computer-implemented method according to claim 10, wherein, The second plurality of first data records are nested within the first plurality of first data records.

12. The computer-implemented method according to claim 1, wherein, The predefined pattern attributes include domain attributes.

13. The computer-implemented method according to claim 1, wherein, The predefined pattern attributes include pattern type attributes.

14. The computer-implemented method according to claim 1, wherein, The various pattern mapping techniques include string similarity mapping techniques.

15. The computer-implemented method according to claim 1, wherein, The various pattern mapping techniques include value distribution mapping techniques.

16. The computer-implemented method according to claim 1, further comprising: The parser is used to obtain the third pattern and the third data record constructed based on the third pattern; The parser determines one or more third pattern attribute values ​​for each pattern attribute in the predefined pattern attribute set, based on the third pattern. The parser determines one or more fourth mode attribute values ​​for each mode attribute in the predefined set of mode attributes, specifically for the fourth mode. The parser provides the third and fourth pattern attribute values ​​for each pattern attribute in the predefined pattern attribute set to the pattern matching engine. The pattern matching engine determines that: (i) the third pattern attribute value is similar to the first pattern attribute value; (ii) The fourth mode attribute value is similar to the second mode attribute value; The pattern matching engine transforms the third data record according to at least one selected pattern mapping technique; as well as The computing system provides a transformed third data record based on the data structure of the fourth mode.

17. A computing system, comprising: Parser; A pattern matching engine, which is a software module, includes: (i) multiple pattern mapping techniques for transforming records from an initial pattern to a target pattern; and (ii) a set of rules, wherein each rule suggests at least one pattern mapping technique from the multiple pattern mapping techniques based on at least one pattern attribute from a predefined set of pattern attributes; and One or more processors are configured to cause the computing system to perform operations, the operations including: The parser is used to obtain a first pattern and a first data record constructed based on the first pattern. The parser determines one or more first pattern attribute values ​​for each pattern attribute in the predefined pattern attribute set for the first pattern. The parser determines one or more second pattern attribute values ​​for each pattern attribute in the predefined pattern attribute set, based on the second pattern. The parser provides the pattern matching engine with the first and second pattern attribute values ​​for each pattern attribute in the predefined pattern attribute set. The parser receives a representative second data record constructed according to the second pattern. The parser determines one or more first record attribute values ​​for each record attribute in a predefined set of record attributes, for the first data record. The parser determines one or more second record attribute values ​​for each record attribute in the predefined set of record attributes, for the second data record. The parser provides a first record attribute value and a second record attribute value for each record attribute in the predefined set of record attributes to the pattern matching engine, wherein the pattern matching engine further includes a second set of rules, wherein each second rule suggests at least one pattern mapping technique from the plurality of pattern mapping techniques based on at least one record attribute from the predefined set of record attributes. The rule set is applied to the first pattern attribute value and the second pattern attribute value by the pattern matching engine, and the second rule set is applied to the first record attribute value and the second record attribute value, so as to select at least one pattern mapping technology from the multiple pattern mapping technologies; The pattern matching engine transforms the first data record according to at least one selected pattern mapping technique; and The computing system provides a transformed first data record.

18. An article of manufacture comprising a non-transitory computer-readable medium having stored thereon program instructions, said program instructions, when executed by one or more processors of a computing system, causing the computing system to perform operations, said operations including: A first pattern and a first data record constructed according to the first pattern are obtained by a parser running on the computing system. The parser determines one or more first pattern attribute values ​​for each pattern attribute in the predefined pattern attribute set for the first pattern. The parser determines one or more second pattern attribute values ​​for each pattern attribute in the predefined pattern attribute set, based on the second pattern. The parser provides a first pattern attribute value and a second pattern attribute value for each pattern attribute in the predefined pattern attribute set to a pattern matching engine running on the computing system, wherein the pattern matching engine is a software module and the pattern matching engine includes: (i) multiple pattern mapping techniques for transforming records from an initial pattern to a target pattern, and (ii) a set of rules, wherein each rule suggests at least one pattern mapping technique from the multiple pattern mapping techniques based on at least one pattern attribute from the predefined pattern attribute set; The parser receives a representative second data record constructed according to the second pattern. The parser determines one or more first record attribute values ​​for each record attribute in a predefined set of record attributes, for the first data record. The parser determines one or more second record attribute values ​​for each record attribute in the predefined set of record attributes, for the second data record. The parser provides a first record attribute value and a second record attribute value for each record attribute in the predefined set of record attributes to the pattern matching engine, wherein the pattern matching engine further includes a second set of rules, wherein each second rule suggests at least one pattern mapping technique from the plurality of pattern mapping techniques based on at least one record attribute from the predefined set of record attributes. The rule set is applied to the first pattern attribute value and the second pattern attribute value by the pattern matching engine, and the second rule set is applied to the first record attribute value and the second record attribute value, so as to select at least one pattern mapping technology from the multiple pattern mapping technologies; The pattern matching engine transforms the first data record according to at least one selected pattern mapping technique; and The computing system provides a transformed first data record.

Images