Regular Expression Generation Using the Longest Common Subsequence Algorithm on Combinations of Regular Expression Codes

Abstract

To provide a method, a system, and a computer-readable medium, for automated generation of regular expressions.SOLUTION: In a regular expression generation server 110, a regular expression generator 116 receives input data including one or more character sequences, converts the character sequences into sets of regular expression codes and / or span data structures, identifies a longest common subsequence shared by the sets of regular expression codes and / or spans, and generates a regular expression based on the longest common subsequence.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application is filed under 35 U.S.C. § 119(e) of a method entitled "AUTOMATED GENERATION OF REGULAR EXPRESSIONS" filed on June 13, 2018. 119(e) to U.S. Provisional Patent Application No. 62 / 684,498, filed October 22, 2018, entitled "AUTOMATED GENERATION OF REGULAR EXPRESSIONS," which claims priority to U.S. Provisional Patent Application No. 62 / 684,498, filed October 22, 2018, under 35 U.S.C. § 119(e). Priority is claimed to application Ser. No. 62 / 749,001, the entire contents of which are incorporated herein by reference for all purposes. [Background technology]

[0002] background Big data analytics systems can be used for predictive analytics, user behavior analytics, and other advanced data analytics. However, before any data analytics can be effectively performed to provide useful results, the initial dataset may need to be formatted into a clean and curated dataset. This data onboarding often presents a challenge for cloud-based data repositories and other big data systems, where data from a variety of different data sources and / or data streams may be compiled into a single data repository. Such data may include structured data in multiple different formats, semi-structured data according to different data models, and even unstructured data. Such data repositories often contain data representations in a variety of different formats and structures and may also contain duplicate and erroneous data. When these data repositories are analyzed for reporting, predictive modeling, and other analytical tasks, the low signal-to-noise ratio of the initial dataset may lead to inaccurate or unuseful results.

[0003] Many current solutions to the problem of data formatting and preprocessing involve manual and ad hoc processes for cleaning and curating data in order to manipulate the data into a common format before performing data analysis. While these manual processes may be effective for certain smaller data sets, such processes can be inefficient and impractical when attempting to preprocess and format large data sets. Summary of the Invention [Means for solving the problem]

[0004] overview Aspects described herein provide various techniques for generating regular expressions. As used herein, "regular expression" may refer to a sequence of characters that defines a pattern that can be used to search for matches within a longer input text string. In some embodiments, regular expressions may be constructed using a wildcard matching language of symbols, and the pattern defined by the regular expression may be used to match character strings and / or extract information from character strings provided as input. Various embodiments described herein use a regular expression generator implemented as a data processing system to receive and display input text data and to identify specific characters in the input text via a client user interface. The system may receive a character subset selection and then generate one or more regular expressions based on the selected character subset. After generating the one or more regular expressions, a regular expression engine may be used to match the patterns of the regular expressions against one or more data sets. In various embodiments, the data that matches the regular expressions may be extracted, reformatted, modified, or the like. In some cases, additional columns, tables, or other data sets may be created based on the data that matches the regular expressions.

[0005] According to some aspects described herein, a regular expression generator implemented via a data processing system can generate regular expressions based on a determined longest common subsequence (LCS) shared by different sets of one or more regular expression codes. The regular expression codes (which may also be referred to as category codes) may include, for example, L for letters of the English alphabet, N for numbers, Z for spaces, P for punctuation marks, and S for other symbols. Each set of one or more regular expression codes may be assigned to a user interface. The LCS may be converted from different sequences of one or more characters received as input data via an interface. Regular expression codes excluded from the LCS may be represented as options and / or alternatives. In some embodiments, a regular expression code may be associated with a minimum number of occurrences of the regular expression code. Additionally or alternatively, a regular expression code may be associated with a maximum number of occurrences of the regular expression code. For example, a set of category codes may include L<0,1> to indicate that a particular portion of the LCS contains a character at most once, if at all. As described in more detail below, generalizing input data as intermediate regular expression codes (IRECs) may provide various technical advantages, including using very little input data, which enables near-instant generation of regular expressions that do not succumb to false positive or false negative matches in unseen data.

[0006] According to additional aspects described herein, a regular expression may be generated based on input data containing three or more character sequences. When three or more character sequences are identified as input data, a regular expression generator that identifies the LCS of the character sequences may result in an exponential increase in runtime. To identify the LCS of all character sequences in a fully functional manner, the regular expression generator may perform an LCS algorithm on each distinct combination of two character sequences. Based on the results of the LCS algorithm, a fully connected graph may be generated, where each graph node represents a different character sequence and the length of each graph edge corresponds to the LCS of the node that defines the graph edge. The order in which the character sequences are selected may then be determined by performing a depth-first traversal of a minimum spanning tree on the fully connected graph.

[0007] Further aspects described herein relate to generating regular expressions based on input that includes both positive and negative character sequence examples. A positive example may refer to a sequence of characters that matches the regular expression to be generated, and a negative example may refer to a sequence of characters that does not match the regular expression to be generated. In some embodiments, when both positive and negative examples are received, the regular expression generator may identify a discriminator, i.e., a shortest subsequence of one or more characters that distinguishes the positive examples from the negative examples. The selected discriminator may be the shortest sequence (e.g., expressed in a category code) and may be either positive or negative, such that the positive examples match and the negative examples do not match. The discriminator may then be hard-coded into the regular expressions generated by the regular expression generator. In some cases, the shortest subsequence may be included in the prefix or suffix portion of the negative examples.

[0008] Further aspects described herein relate to one or more user interfaces to which input data can be provided to generate regular expressions. In some embodiments, the user interface may be displayed on a client device communicatively coupled to the regular expression generation server. The user interface may be programmatically generated by the server, the client device, or a combination of software components executed on the server and the client. Input data received via the user interface may correspond to a user selection of one or more character sequences that may represent positive or negative cases. In some cases, the user interface may support input data including a selection of a first character sequence within a second character sequence. For example, a user may highlight one or more characters within a larger, previously highlighted character sequence, and the second user selection may provide context for the larger, first user selection. This allows input data to be provided to the regular expression generator with greater specificity, providing “context” to the regular expression generator so that it can generate regular expressions that avoid false positives. In response to a user selecting a character sequence via the user interface, the regular expression generator may generate and display a regular expression. For example, when a user highlights a first sequence of characters, the regular expression generator may generate and display a regular expression that matches the first sequence of characters, as well as other similar character sequences (e.g., consistent with the user's intent for matching sequences). When the user highlights a second sequence of characters, the regular expression generator may generate an updated regular expression that encompasses both the first sequence of characters and the second sequence of characters. Then, when the user highlights a third sequence of characters (e.g., within either the first or second sequence), the regular expression generator may update the regular expression again, and so on.

[0009] According to additional aspects described herein, regular expressions may be generated based on a longest common subsequence from one or more input sequence examples, while also handling characters that are present in only some of the examples. To handle characters that are present in only some input examples, spans may be defined in which both the minimum and maximum number of occurrences of a regular expression code are tracked. If a span may not be present in all of a given input example, the minimum number of occurrences may be set to zero. These minimum and maximum numbers may then be mapped to regular expression multiplicity syntax. A longest common subsequence (LCS) algorithm may be run on spans of characters derived from input examples, including "optional" spans (e.g., with a minimum length of zero) that do not appear in all input examples. As described below, consecutive spans may be merged during the execution of the LCS algorithm. In such cases, when additional optional spans carrying the same result in consecutive occurrences, the LCS algorithm may be run recursively on those optional spans as well.

[0010] Further aspects described herein relate to combinatorial search, in which the LCS algorithm performed by the regular expression generator may be run multiple times to generate a "correct" regular expression (e.g., a regular expression that properly matches all given positive examples and properly filters out all given negative examples) and / or to generate multiple correct regular expressions from which the most desirable or optimal regular expression can be selected. In some embodiments, the LCS algorithm may generally be run from right to left on the example input to generate a regular expression. However, for comparison purposes and to find alternative regular expressions, the LCS algorithm may be separately run in the reverse direction (e.g., from left to right) on the example input. For example, an example character sequence received as user input: may be reversed before passing them through the LCS algorithm, and the results from the LCS algorithm may then be reversed and returned (including the original text fragment). Additionally, in some embodiments, the LCS algorithm may be run multiple times by the regular expression generator, in both normal character sequence order and in reverse order, with positioning at the beginning of a line, with positioning fixed at the end of a line, and without positioning at the beginning or end of a line. Thus, in some cases, the LCS algorithm may be run at least six times, and the shortest successful regular expression may be selected from these runs. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a block diagram illustrating components of an exemplary distributed system for generating regular expressions in which various embodiments may be implemented. [Figure 2] 1 is a flowchart illustrating a process for generating a regular expression based on input received via a user interface, according to one or more embodiments described herein. [Figure 3] 1 is a flowchart illustrating a process for generating regular expressions using a Longest Common Subsequence (LCS) algorithm on a set of regular expression codes, according to one or more embodiments described herein. [Figure 4] FIG. 10 is an illustrative diagram for generating a regular expression based on two example character sequences using the Longest Common Subsequence (LCS) algorithm on a set of regular expression codes, according to one or more embodiments described herein. [Figure 5] 1 is a flowchart illustrating a process for generating regular expressions using a Longest Common Subsequence (LCS) algorithm on a larger set of regular expression codes, according to one or more embodiments described herein. [Figure 6]FIG. 10 is an illustrative diagram for generating a regular expression based on five example character sequences using the Longest Common Subsequence (LCS) algorithm on a set of regular expression codes, according to one or more embodiments described herein. [Figure 7] 1 is a flowchart illustrating a process for determining the order of execution for a longest common subsequence (LCS) algorithm on a larger set of regular expression code, according to one or more embodiments described herein. [Figure 8A] 1 illustrates a fully connected graph used to determine the order of execution for a longest common subsequence (LCS) algorithm on a larger set of regular expression code, according to one or more embodiments described herein. [Figure 8B] 1 illustrates a minimum spanning tree representation of a fully connected graph used to determine the order of execution for a longest common subsequence (LCS) algorithm on a larger set of regular expression code, according to one or more embodiments described herein. [Figure 9] 1 is a flowchart illustrating a process for generating regular expressions based on example positive and negative character sequences according to one or more embodiments described herein. [Figure 10A] 10 is an exemplary user interface screen illustrating the generation of a regular expression based on an example positive character sequence, according to one or more embodiments described herein. [Figure 10B] 1 is an exemplary user interface screen illustrating the generation of a regular expression based on example positive and negative character sequences, according to one or more embodiments described herein. [Figure 11] 1 is a flowchart illustrating a process for generating a regular expression based on a user data selection received in a user interface, according to one or more embodiments described herein. [Figure 12]10 is a flowchart illustrating a process for generating regular expressions and extracting data based on capturing groups via user data selections received within a user interface, according to one or more embodiments described herein. [Figure 13] 1 is an exemplary user interface screen illustrating a tabular data display according to one or more embodiments described herein. [Figure 14] 10 is an exemplary user interface screen illustrating the generation of regular expressions and capturing groups based on the selection of data from a tabular display, according to one or more embodiments described herein. [Figure 15] 10 is an exemplary user interface screen illustrating the generation of regular expressions and capturing groups based on the selection of data from a tabular display, according to one or more embodiments described herein. [Figure 16A] 10 is an exemplary user interface screen illustrating the generation of regular expressions based on the selection of positive and negative examples from a tabular display, according to one or more embodiments described herein. [Figure 16B] 10 is an exemplary user interface screen illustrating the generation of regular expressions based on the selection of positive and negative examples from a tabular display, according to one or more embodiments described herein. [Figure 17] 10 is another example user interface screen illustrating the generation of regular expressions and capturing groups based on the selection of data from a tabular display, according to one or more embodiments described herein. [Figure 18] 1 is a flowchart illustrating a process for generating regular expressions, including optional spans, using a longest common subsequence (LCS) algorithm, according to one or more embodiments described herein. [Figure 19]FIG. 1 is an exemplary diagram for generating a regular expression, including optional spans, using the Longest Common Subsequence (LCS) algorithm, according to one or more embodiments described herein. [Figure 20] 1 is a flowchart illustrating a process for generating regular expressions based on a combinatorial implementation of the Longest Common Subsequence (LCS) algorithm, according to one or more embodiments described herein. [Figure 21] 1 is a block diagram illustrating components of an exemplary distributed system in which various embodiments of the present invention may be implemented. [Figure 22] FIG. 1 is a block diagram showing components of a system environment in which services provided by an embodiment of the present invention can be provided as cloud services. [Figure 23] FIG. 1 is a block diagram illustrating an exemplary computer system upon which embodiments of the present invention may be implemented. DETAILED DESCRIPTION OF THE INVENTION

[0012] Detailed Description In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of various embodiments of the present invention. However, it will be apparent to those skilled in the art that embodiments of the present invention may be practiced without some of these specific details. In other instances, well-known structures and devices are shown in block diagram form.

[0013] The following description provides exemplary embodiments only and is not intended to limit the scope, applicability, or configuration of the present disclosure. Rather, the following description of exemplary embodiments will provide those skilled in the art with an enabling description for implementing the exemplary embodiments. It should be understood that various changes can be made in the function and arrangement of elements without departing from the spirit and scope of the invention as set forth in the claims.

[0014] Specific details are provided in the following description to provide a thorough understanding of the embodiments. However, it will be understood by those skilled in the art that the embodiments may be practiced without these specific details. For example, circuits, systems, networks, processes, and other components may be presented in block diagram form in order to avoid obscuring the embodiments in unnecessary detail. In other instances, well-known circuits, processes, algorithms, structures and techniques may be shown without unnecessary detail in order to avoid obscuring the embodiments.

[0015] Furthermore, it is noted that particular embodiments may be described as a process that is depicted as a flowchart, a flow diagram, a data flow diagram, a structure diagram, or a block diagram. While a flowchart may describe operations as a sequential process, many of the operations may be performed in parallel or simultaneously. Additionally, the order of operations may be rearranged. A process is terminated when the operations are completed, but may include additional steps not included in the diagram. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, its termination may correspond to a return of the function to the calling function or the main function.

[0016] The term "computer-readable medium" includes, but is not limited to, non-transitory media such as portable or fixed storage devices, optical storage devices, and various other media capable of storing, containing, or carrying instructions and / or data. A code segment or computer-executable instructions may represent a procedure, a function, a subprogram, a program, a routine, a subroutine, a module, a software package, a class, or any combination of instructions, data structures, or program statements. A code segment may be coupled to another code segment or a hardware circuit by passing and / or receiving information, data, arguments, parameters, or memory contents. Information, arguments, parameters, data, etc. may be passed, forwarded, or transmitted via any suitable means including memory sharing, message passing, token passing, network transmission, etc.

[0017] Furthermore, the embodiments may be implemented in hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. When implemented in software, firmware, middleware, or microcode, the program code or code segments to perform the necessary tasks may be stored on a machine-readable medium. A processor may perform the necessary tasks.

[0018] Described herein are various techniques (e.g., methods, systems, non-transitory computer-readable storage memories storing instructions executable by one or more processors, etc.) for generating regular expressions corresponding to patterns identified in one or more example input data. In particular embodiments, in response to receiving a selection of input data, one or more patterns in the input data are automatically identified, and regular expressions (or "regex," for short) may be automatically and efficiently generated to represent the identified patterns. Such patterns may be based on sequences of characters (e.g., sequences of letters, numbers, spaces, punctuation marks, symbols, etc.). Described herein are various embodiments, including methods, systems, non-transitory computer-readable storage media storing programs, code, or instructions executable by one or more processors, etc.

[0019] In some embodiments, regular expressions may be constructed using a symbolic wildcard matching language to match character strings and / or extract information from character strings provided as input. For example, a first exemplary regular expression [A-Za-z]{3} / d? / d, / d / d / d / d may match a date (e.g., April 3, 2018), and a second exemplary regular expression [A-Za-z]{3} / d? / d, ( / d / d / d / d) may be used to match a date (e.g., April 3, 2018). The input data received by the regular expression generation system may be The data may include, for example, one or more "positive" data examples and / or one or more "negative" data examples. As used herein, a positive example may refer to a character sequence that is received as input and would be matched by a regular expression generated based on the input. Conversely, a negative example may refer to an input character sequence that is not matched by a regular expression generated based on the input.

[0020] Several technical advantages may be realized within the various embodiments and examples described herein. For example, some techniques described in this disclosure may improve the speed and efficiency of the regular expression generation process (e.g., regeX solutions may be generated in less than one second, and user interfaces may be suitable for interactive real-time use). Various techniques described herein may also be decision-based, may not require training data, may generate solutions without requiring any initial regular expression input, or may be fully automated (e.g., generate regular expressions to the extent that they require no human intervention). Furthermore, various techniques described herein need not be limited with respect to the types of data inputs that can be effectively processed, and such techniques may improve the human readability of the resulting regular expressions.

[0021] Some embodiments described herein include one or more implementations of a Longest Common Subsequence (LCS) algorithm, which in some circumstances may be used as a differencing engine (e.g., the engine behind the Unix "diff" utility) configured to determine and show the differences between two text files. In this embodiment, input data (e.g., strings and other character sequences) may be converted into abstract tokens, which may then be provided as input to the LCS algorithm. Such abstract tokens may, for example, be tokens based on regular expression codes (e.g., Loogle codes or other character class codes) that represent regular expression character classes. Various different examples of such codes are contemplated and may be referred to herein as “regular expression codes” or “intermediate regular expression codes” (IRECs). For example, the input character sequence “May 3” may be converted to the IREC code “LLLZN,” and the tokenized string may then be subjected to the LCS algorithm along with other tokenized strings. In some embodiments, IRECs (e.g., regular expression codes) that the input character sequences do not have in common may appear as options (e.g., optional spans) in the final generated regular expression. In particular embodiments, the regular expression codes may be category codes based on the Unicode category codes shown at https: / / www.regular-expressions.info / unicode.html#category. For example, the code L may represent a letter, the code N may represent a number, the code Z may represent a space, the code S may represent a symbol, and the code P may represent a punctuation mark. For example, code L corresponds to Unicode / p{L}, code N corresponds to Unicode / p{L}, etc. This means that there is a one-to-one mapping from LCS output to regular expressions. This allows for different categories to work together (e.g., / pN / pN / pZ / pL / pL can match "10 am"), which may provide benefits in human readability. Additionally, these different categories may be disjoint, or mutually exclusive. That is, in this example, categories L, N, Z, P, and S may be disjoint, such that there is no overlap between members of the categories.

[0022] Further technical advantages may be realized in various embodiments, including more efficient generation of regular expressions based on the use of regular expression codes (e.g., category codes), spans, etc. By using such code, computational resources do not need to be wasted when the LCS algorithm successfully identifies all or substantially all of the characters in the input string as different. Further technical advantages provided by various embodiments herein include improved readability of the generated regular expressions, as well as support for both positive and negative examples as input data, and various advantageous user interfaces. This includes providing user interface features (e.g., allowing the user to highlight text fragments within larger character sequences or data cells for extraction).

[0023] I. Overall Overview Various embodiments disclosed herein relate to generating regular expressions. In some embodiments, a data processing system configured as a regular expression generator can generate regular expressions by identifying a longest common subsequence (LCS) shared by different sets of regular expression codes (e.g., category codes). Each set of regular expression codes can be converted from a sequence of characters received as input data via a user interface. Among the technical advantages described herein, abstracting the input data as intermediate code (e.g., regular expression codes, spans, etc.) allows for efficient generation of regular expressions using significantly less input data.

[0024] 1 is a block diagram illustrating components of an exemplary distributed system for generating regular expressions in which various embodiments may be implemented. As shown in this example, client device 120 may communicate with regular expression generation server 110 (or regular expression generator), interact with a user interface to retrieve and display tabular data, and generate regular expressions based on selection of input data (e.g., examples) via the user interface. In some embodiments, client device 120 may communicate with regular expression generator 110 via a client web browser 121 and / or a client-side regular expression application 122 (e.g., a client-side application that receives / consumes regular expressions generated by server 110). Within regular expression generator 110, requests from client device 120 may be received at a network interface over various communication networks and processed by an application programming interface (API), such as REST API 112. User interface data model generator 114 component, along with regular expression generator 110, may provide server-side programming components and logic to generate and render the various user interface features described herein. Such features may include functionality that allows a user to retrieve and display tabular data from the data repository 130, select example input data to initiate regular expression generation, and modify and / or extract data based on the generated regular expression. In this example, the regular expression generator component 116 may be implemented to generate regular expressions, including converting input character sequences into regular expression codes and / or spans, running an algorithm (e.g., the LCS algorithm) on the input data, and generating / simplifying regular expressions.The regular expression generated by the regular expression generator 116 may be transmitted by the REST service 112 to the client device 120, where Javascript code on the client browser 121 (or corresponding client-side application component 122) can then apply the regular expression to all cells in the spreadsheet column rendered in the browser. In other cases, a separate regular expression engine component may be implemented on the server side to compare the generated regular expression within tabular data displayed on the user interface and / or other data stored in the data repository 130 to identify matching / non-matching data on the server side. In various embodiments, matching / non-matching data may be automatically selected (e.g., highlighted) within the user interface and selected for extraction, modification, deletion, etc. Based on the generation of the regular expression, any data extracted or modified via the user interface may be stored in one or more data repositories 130. Additionally, in some embodiments, the generated regular expression (and / or the corresponding input to the LCS algorithm) may be stored in the regular expression library 135 for future retrieval and use. In some embodiments, the generated regular expressions need not actually be stored in a "library" but may be incorporated into a "transformation script." For example, as described in more detail in ET.S. Patent No. 10,210,246 (incorporated herein by reference for all purposes), such a transformation script may include programs, code, or instructions that may be executable by one or more processing units to transform received data. Other possible examples of transformation scripts may include "rename string," "large string data," or "guess gender from first name and create new column with gender," etc.

[0025] 2 is a flowchart illustrating a process for generating a regular expression based on input received via a user interface, according to one or more embodiments described herein. In step 201, the regular expression generator 110 may receive a request from the client device 120 to access a regular expression generator user interface and view particular data via the user interface. The request in step 201 may be received via the REST API 112, and / or a web server, an authentication server, etc., and the user's request may be parsed and authenticated. For example, a user within a business or organization may access the regular expression generator 110 to analyze and / or modify transactional data, customer data, performance data, forecast data, and / or any other category of data that may be stored in the organization's data repository 130. In step 202, the regular expression generator 110 may retrieve and display the requested data via a user interface that supports the generation of regular expressions based on selected input data. Various embodiments and examples of such user interfaces are described in detail below.

[0026] In step 203, the user may select one or more input character sequences from data displayed in a user interface provided by regular expression generator 110. In some embodiments, the data may be displayed in a tabular format within the user interface, including labeled columns with particular data types and / or categories of data. In such cases, the selection of input data in step 203 may correspond to the user selecting a data cell or selecting (e.g., highlighting) an individual text fragment within a data cell. However, in other embodiments, regular expression generator 110 may support searching and displaying semi-structured and unstructured data via a user interface, and the user may select input data for regular expression generation by selecting character sequences from the semi-structured or unstructured data. As will be described in the following examples, a user selecting input character sequences from displayed tabular data is merely one example of a use case. In other examples, the user (e.g., perhaps using a Linux command line tool) may select input data for regular expression generation. A software developer or power user trying to build a regular expression for a program like grep, sed, or awk may want to type in an example from scratch rather than picking one from a spreadsheet.

[0027] In step 204, regular expression generator 110 may generate one or more regular expressions based on the input data selected by the user in step 203. In step 205, regular expression generator 110 may update the user interface, for example, to display the generated regular expressions and / or to highlight matching / non-matching data in the displayed data. In step 206, which may be optional in some embodiments, the user interface may support functionality that allows the user to modify the underlying data based on the generated regular expressions. For example, the user interface may allow the user to filter specific data fields from tabular data based on whether those fields match a regular expression. The repository 130 may support features that allow data to be filtered, modified, deleted, or extracted. Filtering or modifying data may include modifying the underlying data stored in the repository 130, and in some cases, the extracted data may be stored in the repository 130 as new columns and / or new tables.

[0028] While these steps provide a general, high-level overview of an exemplary user interaction with the user interface of regular expression generator 110, other embodiments may support a variety of additional features and functionality. For example, in some embodiments, a regular expression code (or category code) may be associated with a minimum number of occurrences of the code. Additionally or alternatively, a regular expression code may be associated with a maximum number of occurrences of the code. As an example, a set of regular expression codes may include the code L<0,1>, which indicates that a particular portion of an LCS contains a character either at least zero times and at most one time.

[0029] Additionally, in some embodiments, the input data may include more than two character sequences. In such embodiments, various techniques may be used to determine the order for running the LCS algorithm on the more than two character sequences so that the resulting regular expressions can be generated in a fully functional manner, avoiding the exponential increase in runtime caused by more than two input character sequences. Note that the regular expression generator 110 may instead run the LCS algorithm on two character sequences at a time and determine the order for selecting pairs of character sequences based on the graph. For example, a fully connected graph may indicate that a first run of the LCS algorithm (e.g., LCS1) should be run on sequence 1 and sequence 3, then a second run of the LCS algorithm (e.g., LCS2) should be run on LCS1 and sequence 2, and so on. The graph may be a fully connected graph, where nodes represent character sequences and edges connect the nodes and represent the length of the LCS shared by the connected nodes. Each node in the graph may be connected to every other node in the graph, and the order for selecting character sequences may be determined by performing a depth-first traversal of a minimum spanning tree on the graph.

[0030] In further embodiments, input data may be provided via the user interface in several different ways. For example, the input data may indicate a first user selection of one or more characters within a second user selection of a set of characters. For example, the user may highlight a character within a previously highlighted set of characters. Thus, the second user selection may provide context for the first user selection, which may allow the input data to be provided to regular expression generator 110 with greater specificity. In some embodiments, regular expression generator 110 may generate and display regular expressions in near real time in response to each user selection. For example, if the user highlights a first range of characters, regular expression generator 110 may display a regular expression representing the first range of characters. Then, if the user highlights a second range of characters within the first range of characters, regular expression generator 110 may update the displayed regular expression.

[0031] Additionally, in some embodiments, regular expression generator 110 can generate regular expressions based on input that includes both positive and negative examples. As discussed above, a positive example may refer to a sequence of characters that should be included in the regular expression, and a negative example may refer to a sequence of characters that should not be included in the regular expression. In such cases, regular expression generator 110 can identify the shortest subsequence of one or more characters at a particular position that distinguishes a positive example from a negative example. The shortest subsequence may be hard-coded in the regular expression generated by regular expression generator 110. In various examples, the shortest subsequence may be included in the prefix / suffix portion or the midspan within the negative example.

[0032] Further examples of automatically generating regular expressions according to particular embodiments are described below. These examples may correspond to various specific possible implementations of the general technique of FIG. 2 and may be implemented in software (e.g., code, instructions, programs, etc.) executed by one or more processing units (e.g., processors, cores) of the respective systems, hardware, or a combination thereof. The software may be stored on a non-transitory storage medium (e.g., on a memory device). The further examples described below are intended to be illustrative and non-limiting. While the examples show various processing steps occurring in a particular order or sequence, this is not intended to be limiting. In some alternative embodiments, steps may be performed in a different order, or some steps may be performed in parallel.

[0033] In some examples, user input received via a user interface (e.g., step 203) may include one or more “positive examples” that are matched by the regular expression output and zero or more “negative examples” that are not matched by the regular expression output. Optionally, one or more of the positive examples may be highlighted to select specific ranges (or subsequences) of characters. In some cases, in step 204, the positive examples received via the user interface may be converted into spans in regular expression code (e.g., character category codes such as Unicode category codes). For each positive example, a sequence of spans may be generated. In some embodiments, a graph may be generated in which each vertex corresponds to one of the sequences of spans and the edge weight is equal to the length of the output from an LCS algorithm run on those two sequences of spans that correspond to the edge endpoints. A minimum spanning tree may be determined for the graph. For example, in some embodiments, Prim's algorithm may be used to obtain the minimum spanning tree. A depth-first traversal may be performed on the minimum spanning tree to determine the traversal order, and then the LCS algorithm may be run on the first two elements of the traversal. One by one, each additional element of the traverse may then be sequentially merged into the current LCS output by again running the LCS algorithm on the output of the previous LCS iteration and the next current traverse element. The final output of the LCS algorithm, which may be a sequence of spans, may then be converted into a regular expression. This conversion may be a one-to-one conversion in some embodiments, although certain optional embodiments described herein may not correspond to a one-to-one conversion. Finally, the resulting regular expression may be tested against all positive and negative examples received via the user interface in step 203. If any of the tests fail, the above process may be repeated using all positive examples and any negative examples that failed.

[0034] II. Regular Expression Generation Using the Longest Common Subsequence Algorithm on Regular Expression Code As mentioned above, some aspects described herein relate to generating regular expressions based on computing the longest common subsequence (LCS) shared by different sets of regular expression codes corresponding to input data.

[0035] 3 is a flowchart illustrating a process for generating a regular expression using the LCS algorithm for a set of regular expression codes, according to one or more embodiments described herein. In step 301, the regular expression generator 110 may receive one or more character sequences as input data. As mentioned above, in some examples, the input data may be selected from within tabular data displayed in a user interface. It should be understood that, although the input data may correspond to positive case data received via a user interface in some embodiments, the user interface is optional and the input data may correspond to any character sequence received via any other communication channel (e.g., a non-user interface) in various examples.

[0036] In step 302, each character sequence received in step 301 may be converted to a corresponding regular expression code. In various embodiments, the regular expression code may be a Loogle code, a Unicode category code, or any other character class code that represents a regular expression character class. For example, an input character sequence "May 3" may be converted to the Loogle code "LLLZN." In some embodiments, the regular expression code may be a category code based on the Unicode category codes shown at https: / / www.regular-expressions.info / unicode.html#category. For example, the code L may represent a letter, the code N may represent a digit, and the code Z may represent a space. For example, code S may represent a symbol, code P may represent a punctuation mark, etc. For example, code L may correspond to Unicode / p{L} and code N may correspond to Unicode / p{N}.

[0037] In step 303, a longest common subsequence may be determined from among the set of regular expression codes generated in step 302. In some embodiments, an LCS algorithm may be run using two sets of regular expression codes as input. Various different characteristics of the execution of the LCS algorithm (e.g., direction of processing, positioning, pushing whitespace, coalescing low-cardinality spans, aligning on common tokens, etc.) may be used in different embodiments. In step 304, a regular expression may be generated based on the output of the LCS algorithm. In some cases, step 304 may include capturing the output of the LCS algorithm with regular expression code and converting the regular expression code to a regular expression. In step 305, the regular expression may be simplified and output, for example, by displaying the regular expression to a user via a user interface.

[0038] FIG. 4 is an exemplary diagram for generating a regular expression using a Longest Common Subsequence (LCS) algorithm on a set of regular expression codes based on two example character sequences. Accordingly, FIG. 4 illustrates an example of applying the process described above in FIG. 3. As shown in FIG. 4, the example regular expression is generated based on two input strings: "iPhone 5" and "iPhone X." Each sequence in this example can be converted to a respective set of regular expression codes. Thus, iPhone 5 may be converted to "LLLLLLZN" and iPhone X may be converted to "LLLLLLZL." As shown in FIG. 4, these category codes are then provided as input to the LCS algorithm, which determines that both sets of IRECs (or category codes) contain six Ls and one Z. The Z category codes that are excluded from the LCS are optional and / or alternative. Therefore, a regular expression that encompasses both character sequences can be expressed as / pL{6} / pZ / pN? / pL?. In this example, the regular expression code (e.g., / pL for letters, / pZ for spaces, / pN for numbers) The braces containing the number 6 indicate six instances of the character, and the question mark indicates the last number indicates that the / character is optional. Finally, a simplification process can be performed by the regular expression generator, during which the regular expression is simplified by inserting the common text fragment "iPhone" back into the final regular expression, replacing the wider " / pL{6} / " portion of the regular expression. Replace.

[0039] As shown in this example, the input strings received by the regular expression generator 110 may be converted into "regular expression codes" that represent regular expression broad categories (which may also be called "category codes"), and the LCS algorithm generates those regular expression codes. In some embodiments, Unicode category codes may be used for the regular expression codes. For example, an input text string may be converted to a code representing a Regex Unicode broad category (e.g., / pL for letters, / pP for punctuation, etc.). This approach, illustrated by Figures 3 and 4, may be referred to as an indirect approach. However, in other embodiments, a direct approach may be used in which the LCS algorithm is run directly on the character sequence received as input.

[0040] In some embodiments, the indirect approach may provide an additional technical advantage in that it does not require large amounts of training data and may generate valid regular expressions with a relatively smaller number of input examples. This is because the indirect approach uses heuristics to reduce uncertainty in regular expression generation and to eliminate potential false positives and false negatives. For example, based on the input strings "May 3" and "Apr 11", When generating a regular expression based on the date pattern, the direct approach may require at least one example per month to generate a valid regular expression that matches the date pattern. Relying on only those two examples, the direct approach would generate the regular expression "[AM][ap][yr]

[13] 1?" In contrast, an indirect approach may generate a more effective regular expression of " / pL{3} / d{1,2}" based on Unicode broad categories. Additionally, as mentioned above, one of the technical advantages described herein includes efficiently generating regular expressions using very little input data, possibly even from a single example. For example, with respect to generating a regular expression from the single example "am," a heuristic may determine whether to generate "am" or " / pL / pL" for the regular expression. While either is likely to be correct, the programmed heuristic may implement user preferences and / or criteria to determine how to generate the optimal regular expression (e.g., whether it should also match "pm").

[0041] In addition, the indirect approach can further simplify the generated regular expression " / pL{3} / d{1,2}" ~ "[A-Za-z]{3} / d{1,2}" to make it more human-readable. This can be beneficial in some embodiments, for example, when outputting to non-sophisticated regular expression users who may not be familiar with Unicode representations for regular expressions.

[0042] Additionally, in some embodiments, instead of treating each character independently when performing the LCS algorithm, sequential and equivalent regular expression codes may be converted into a span data structure (which may also be referred to as a span). In some cases, a span may include a representation of a single regular expression code (e.g., a Unicode broad category code) along with a repeat count range (e.g., minimum and / or maximum). The conversion from regular expression codes to spans can facilitate several various additional features described below, such as recognizing alternations (e.g., disjunctions), and can also facilitate the merging of adjacent optional spans to further simplify the generated regular expression.

[0043] As mentioned above, the LCS algorithm can be configured to remember and retain underlying text fragments within an input character sequence that can potentially be inserted back into a final regular expression, such as the string "iPhone" in Figure 4. By tracking the text fragment that originally resulted in the category code assigned to that span, such an embodiment can allow literal text (e.g., am and pm) to be included directly in the generated regular expression, reducing false positives and making the regular expression output more human-readable.

[0044] III. Using the Longest Common Subsequence Algorithm on Regular Expression Code Combinations Regular Expression Generation Further aspects described herein relate to generating regular expressions based on input data containing three or more strings (e.g., three or more distinct character sequences). When three or more strings are identified as input data, the regular expression generator 110 may use a performance optimization feature in which an optimal order is determined for a sequence of LCS algorithm executions. As described below, the performance optimization function for three or more strings may include constructing a graph with vertices corresponding to each string and edge lengths / weights between each string and all other strings that may be based on the size of the LCS output. A minimum spanning tree may then be derived using those edge weights, and a depth-first traversal may be performed to determine the order of the input strings. Finally, a series of LCS algorithms may be performed using the determined order of the input strings.

[0045] FIG. 5 is a flowchart illustrating a process for generating regular expressions using the Longest Common Subsequence (LCS) algorithm for a larger set of regular expression codes (e.g., three or more character sequences). Accordingly, steps 502-505 in this example may correspond to step 303 described above in FIG. 3. However, because this example concerns generating regular expressions based on three or more input character sequences, the LCS algorithm may be run multiple times. For example, to avoid exponential growth in runtime for three or more input strings, the LCS algorithm may be run multiple times, with each run running on only two input strings. For example, regular expression generator 110 may run a first run of the LCS algorithm on two strings (e.g., two input character sequences or two transformed regular expression codes), then run a second run of the LCS algorithm on the output of the first LCS algorithm and a third string, then run a third run of the LCS algorithm on the output of the second LCS algorithm and a fourth string, and so on.

[0046] To improve and / or optimize the performance of such embodiments, it may be desirable to determine the optimal order of an input string (e.g., an input character sequence or regular expression code) in which to execute a sequence of LCS algorithms. For example, a good order for capturing the input string can affect the readability of the generated regular expression, such as by minimizing the number of optional spans. To keep the generated regex concise, it is preferable that additional strings that are LCSed into the current regex already be somewhat similar to the current regex (the intermediate result from LCSing already-seen strings).

[0047] Thus, in step 501, multiple (e.g., three or more) input character sequences are converted into regular expression codes. In step 502, the order in which to process the regular expression codes is determined using the LCS algorithm. The determination of the order in step 502 is described below with reference to FIG. 7. In step 503, the first two regular expression codes in the determined order are selected (for the first iteration of step 503), or the next regular expression code in the determined order is selected (for a subsequent iteration of step 503). In step 504, the LCS algorithm is run on two input strings corresponding to the format of the regular expression codes. In the first iteration of step 504, the LCS algorithm is run on the first two regular expression codes in the determined order, and in subsequent iterations of step 504, the LCS algorithm is run on the next regular expression code in the determined order and the output of the previous LCS algorithm (which may be in the same regular expression code format). In step 505, the regular expression generator 110 determines whether there are any additional regular expression codes in the determined order that have not yet been provided as input to the LCS algorithm. If there are such additional regular expression codes, processing returns to step 503 for another run of the LCS algorithm. Otherwise, in step 506, a regular expression is generated based on the output of the last run of the LCS algorithm.

[0048] Figure 6 is an exemplary diagram for generating a regular expression based on five example input character sequences. In this example, each input character sequence is converted to a regular expression code, and then the LCS algorithm is run iteratively based on the determined order of the regular expression codes. Thus, Figure 6 illustrates one example of applying the process described above in Figure 5. In this example, the determined order for the five regular expression codes is Code #1 through Code #5, and each code is input to the LCS algorithm in the determined order to generate a regular expression output. The final regular expression output (Reg Ex #4) corresponds to the final regular expression generated based on all five of the input character sequences.

[0049] FIG. 7 is a flowchart illustrating a process for determining the order of execution for a longest common subsequence (LCS) algorithm on a larger set (e.g., three or more) of regular expression codes. Thus, as shown in this example, steps 701-704 may correspond to the order determination in step 502 described above. In step 701, the LCS algorithm may be run on each distinct pair of regular expression codes corresponding to input data, and the resulting output LCS may be stored for each run. Thus, for k input data, this may represent all (k(k-1)) / 2 possible pairings of strings to be run through the LCS algorithm, or k(k-1) in some embodiments. For example, if an input character sequence of k=3 is received, the LCS algorithm may be run three times in step 701; if an input character sequence of k=4 is received, the LCS algorithm may be run six times in step 701; if an input character sequence of k=5 is received, the LCS algorithm may be run ten times in step 701, and so on. In step 702, a fully connected graph may be constructed from k nodes where the edge weight of (k(k-1)) / 2 edges represents a string with the length of the raw LCS output between the two nodes. In step 703, a minimum spanning tree may be derived from the fully connected graph in step 702. In step 704, a depth-first traversal may be performed on the minimum spanning tree. The output of this traversal may correspond to the order in which the regular expression codes are input into a sequence of LCS algorithm executions.

[0050] Referring briefly to Figures 8A and 8B, Figure 5 shows an example of a fully connected graph generated based on a received input character sequence for k=5, and Figure 8B shows a minimum spanning tree representation for the fully connected graph.

[0051] In some embodiments, the approaches described in Figures 5-8B may provide additional technical advantages in terms of performance. For example, certain conventional implementations of the LCS algorithm require O(n 2 ), where n is the length of the string. Extending such an implementation to k strings instead of just 2 results in exponential runtime performance of O(n k ), because the LCS algorithm may be required to search a k-dimensional space. Such traditional implementations of the LCS algorithm may not perform well or may not be well suited for a real-time online user experience.

[0052] As mentioned above, the LCS algorithm may be run (k(k-1)) / 2 times, in which case the overlap is sometimes exactly the same as seen before, since the LCS algorithm may be the case where raw input examples from the user are converted into Regex category codes. Thus, in some cases, memorization may be implemented, and the cache may be used. Using the mesh, previously seen LCS problems can be mapped to previously worked LCS solutions.

[0053] IV. Generating Regular Expressions Based on Positive and Negative Pattern Matching Examples Further aspects described herein relate to generating regular expressions based on input data corresponding to both positive and negative examples. As described above, a positive example may refer to an input data character sequence designated as an example string that should match a regular expression generated by the regular expression generator. Conversely, a negative example may refer to an input data character sequence designated as an example string that should not match a regular expression generated by the regular expression generator. As described below, in some embodiments, the regular expression generator 110 may be configured to identify a position and the shortest subsequence of characters at that position that distinguishes a positive example from a negative example. The shortest subsequence may then be hard-coded into the generated regular expression, such that positive examples will match the regular expression and negative examples will be excluded (e.g., not matched) by the regular expression.

[0054] 9 is a flowchart illustrating a process for generating regular expressions based on example positive and negative character sequences. In step 901, regular expression generator 110 may receive one or more input data character sequences corresponding to the positive examples. In step 902, regular expression generator 110 may generate regular expressions based on the received positive examples. Thus, steps 901-902 may include some or all of the steps performed in FIG. 3 or FIG. 5 discussed above to generate regular expressions based on input data character sequences.

[0055] In step 903, the regular expression generator 110 may receive one additional input data character sequence corresponding to a negative example. Thus, the negative example is specifically designated to not match the regular expression generated in step 902. In some embodiments, the negative example received in step 903 may first be tested against the regular expression generated in step 902, and if it is determined that the negative example does not match the regular expression, no further action is taken. However, in this example, it may be assumed that at least one of the negative examples received in step 903 matches the regular expression generated in step 902. Thus, in step 904, a disambiguation position may be determined within the regular expression generated in step 902. In some embodiments, the disambiguation position may be selected as either a prefix position (e.g., at the beginning of the regular expression) or a suffix position (e.g., at the end of the regular expression). For example, the regular expression generator 110 may determine a first number of characters that would be required in a prefix to distinguish positive cases from negative cases and a second number of characters that would be required in a suffix to distinguish positive cases from negative cases. The regular expression generator 110 may then select a suffix or prefix based on the shortest number of replacement characters required. In some cases, using a prefix as a disambiguation position may be preferred (e.g., weighted) for readability. In yet other examples, the disambiguation position may be a mid-span position that does not correspond to a prefix or suffix of the regular expression.

[0056] In step 905, the regular expression generator 110 may determine a replacement sequence of the custom character class that, when inserted into the regular expression at the determined location, can distinguish the positive examples from the negative examples. In some embodiments, in step 905, the regular expression generator 110 may retrieve text fragments from each of the positive and negative examples that correspond to the disambiguation location (or replacement location), and then use those text fragments to determine a discriminator to be used as a replacement sequence that distinguishes the positive examples from the negative examples. Furthermore, the discriminator replacement sequence determined in step 905 may be a character replacement sequence. A regular expression can contain multiple different replacement sequences of a custom character class, which can be substituted either in the same position or in different positions within the regular expression.

[0057] As mentioned above, in some cases, determining the replacement sequence in step 905 may be performed in conjunction with determining the disambiguation position (or replacement position) in step 904. For example, regular expression generator 110 may determine one or more replacement sequences that may distinguish positive cases from negative cases at a first possible replacement position. Regular expression generator 110 may also determine one or more other replacement sequences that may distinguish positive cases from negative cases at a second, different possible replacement position. In this example, when selecting between different possible replacement positions and corresponding replacement sequences, regular expression generator 110 may apply a heuristic formula to make the selection based on one or more of the size of the characters at the replacement position and the number and / or size of the corresponding replacement sequences. Finally, in step 906, the regular expression may be modified by inserting one or more determined replacement sequences at the determined positions to replace previous portions of the regular expression. Optionally, following modification of the regular expression in step 906, the positive and / or negative examples may be tested against the modified regular expression to verify that the positive examples match the regular expression and that the negative examples do not match the regular expression.

[0058] 10A and 10B are exemplary user interface screens illustrating the generation of a regular expression based on example positive and negative character sequences. Accordingly, the examples shown in FIGS. 10A and 10B may correspond to the user interface displayed during the execution of the process of FIG. 9 described above. In FIG. 10A, a user provides three positive examples 1001 of data input character sequences, and regular expression generator 110 generates a regular expression 1002 that matches each of the positive examples. Next, in FIG. 4B, a user provides one negative example 1004, and regular expression generator 110 generates a revised regular expression 1005 that is based on both the current set of positive examples 1003 and the current set of negative examples 1004.

[0059] As noted above, in some embodiments, once both positive and negative examples are received, the regular expression generator 110 may identify a discriminator, i.e., the shortest subsequence of one or more characters that distinguishes positive from negative examples. The selected discriminator may be the shortest sequence (e.g., expressed in a category code) and may be either positive or negative, resulting in a match for positive examples and a match for negative examples. In some cases, the discriminator may correspond to a substitution subsequence that may then be hard-coded into the regular expression in step 905. As an example, in “[AL][az]+”, [AL] may distinguish between “Alley”, “Avenue”, etc., assuming it is applied to a street suffix. " and "Lane", but will not match (or allow) anything else. As another example, in "[BC][o][az]+", [BC][o] is a positive discriminator consisting of a sequence of two character classes that will match "Boulevard" and "Court". As yet another example, in "[^A][az]+", [LA] is a positive discriminator consisting of a sequence of two character classes that will match "Alley". And it may be a negative discriminator that does not tolerate "Avenue". In some cases, the algorithm may generate a negative look - behind to correctly discriminate. For example, (?<!Av)[A - Za - z]+ will exclude "Avenue" but will admit "Alley".

[0060] As another example, if the user supplies positive examples "202 - 456 - 7800" and "313 - 678 - 8900" and negative examples "404 - 765 - 9876" and "515 - 987 - 6570", in some embodiments, the regular expression generator 110 may generate the regular expression " / d / d / d - / d / d / d - / d / d00". That is, the replacement character subsequence can be identified for the suffix of the regular expression (assuming, for example, that the goal is a regular expression that matches business phone numbers) based on the determination that phone numbers ending with 00 distinguish positive examples from negative examples. This is an example of dealing with negative examples by suffix (more specifically, an example of dealing with negative examples by using a positive suffix), but various other embodiments may support replacements at any of the prefix, suffix, or mid - span positions. In an example of a replacement at the mid - span position, characters offset within the span can be tracked and split at the mid - span point.

[0061] To determine whether to use a prefix or a suffix, in some embodiments, k a and a heuristic is used where the minimum score is selected over all combinations of k

[0062]

Number

[0063] k a = The number of characters considered to remove the ambiguity of the affix (prefix or suffix) |Fp |= number of unique text fragments from positive examples required to disambiguate the affix |F n |= number of unique text fragments from negative examples required to disambiguate the affix |E p |=Number of (complete) positive cases provided by the user |E n |= number of (perfect) negative examples provided by the user In the above example, the heuristic will try to find the longer disambiguated text fragment (so, for example, k a The heuristic is designed to favor shorter disambiguating text fragments over prefixes (hence, e.g., multiplication by ). The heuristic is also designed to favor prefixes over suffixes (hence, e.g., a penalty of 0.1 for suffixes), to improve readability. Finally, the heuristic is designed to favor disambiguating (e.g., substituting) longer prefixes or suffixes over disambiguating by using a larger number of string fragments (hence, e.g., squaring the number of string fragments to be replaced).

[0064] As mentioned above, some embodiments may also support negative midspan cases, as well as negative look-behind and negative look-ahead cases.

[0065] Once the prefix / suffix and k (the number of characters to disambiguate) Once determined, regular expression generator 110 can further determine how to express that disambiguation in the generated regular expression. The generated regular expression may be permissive for affixes (e.g., prefixes or suffixes) that appear to be positive examples, or may exclude affixes that appear to be negative examples.

[0066]

number

[0067] If usePermissive is greater than zero, what appears to be a positive case is In other cases, the regular expression generator 110 may take an approach that disallows what appear to be negative examples by generating, one by one (for each character position), regular expressions that disallow characters taken from negative examples.

[0068] As another example, the regular expression generated for positive cases at 8am and negative cases at 9pm is / d[^p]m It uses caret syntax. In some cases, the regular expression generator 110 may be configured to favor shorter regular expressions, which may not only be more readable to the user, but may also be more likely to be correct. The principle is that characters that occur frequently in the future are more likely to occur again in the future, and therefore emphasis should be placed on frequently occurring characters. The unique character |F p If | is less (something that occurs more frequently and is therefore less unique), this is rewarded in the heuristic by having it in the denominator.

[0069] Referring again to the heuristic in the usePermissive example above, one unique positive Determining a fix is ​​not a big deal when there is only one positive example from the user. Therefore, this heuristic requires a low |E p | is penalized by having it in the numerator (i.e., high |E p | is rewarded in this heuristic).

[0070] Furthermore, in some embodiments, negative examples may be based on look - behind and / or look - ahead. For example, a user provides a positive example of "323 - 1234" and a negative example of "202 - 754 - 9876", in which case it involves the use of a regex look - behind syntax (?<!) to exclude phone numbers with area codes.

[0071] In some cases, negative examples may be based on an optional span. For example, a user may provide positive examples of "ab" and "a2b" and a negative example of "a3b". In this case, an exemplary implementation may fail because it may attempt to discriminate based only on the required span, and the digit "2" is in the optional span. In this example, Failure may refer to a situation where the generated regular expression (correctly) matches all of the positive examples and (incorrectly) matches one or more of the negative examples. In such a case, the user can be warned about the failure and provided with options via a user interface to manually repair the generated regular expression and / or remove some of the negative examples.

[0072] V. User Interface for Regular Expression Generation Additional aspects described herein include several different features and functions within a graphical user interface related to the generation of regular expressions. As described below, some of these features may include various options for user selection, and highlighting of positive and negative examples, color - coding of positive and negative examples, as well as multiple overlapping / nested highlighting within data cells.

[0073] FIG. 11 is a flowchart illustrating a process for generating a regular expression based on user data selections received within a user interface. The exemplary process of FIG. 11 may correspond to any of the previously described examples of generating a regular expression based on an input data character sequence. However, FIG. 11 describes the process in terms of a user interface that may be generated and displayed on client device 120. In step 1101, in response to a request from a user via the user interface, regular expression generator 110 generates a regular expression (e.g., The system may retrieve data (e.g., from the data repository 130) and render / display the data in a tabular format within the graphical user interface. Note that while tabular data is used in this example, it should be understood that tabular data need not be used and displayed in other examples. For example, in some cases, a user may type raw data directly (rather than selecting data from a user interface). Furthermore, when data is presented via a user interface, the data need not be in a tabular format but may be unstructured (e.g., a document) or semi-structured (e.g., a spreadsheet of unformatted / unstructured data items such as tweets or posts). In various examples, the tabular data may correspond to transactional data, customer data, performance data, forecast data, and / or any other category of data that may be stored in the data repository 130 for a business or other organization. In step 1102, a user selection of input data may be received via the user interface. The selected input data may correspond, for example, to an entire data cell or a subsequence of characters within a data cell selected by the user. In step 1103, regular expression generator 110 may generate a regular expression based on the input data (e.g., a data cell or portion thereof) received in step 1102. In step 1104, a user interface may be updated in response to generating the regular expression. In some cases, the user interface may simply be updated to display the generated regular expression to the user, while in other cases, the user interface may be updated in various other ways, as described below. As shown in this example, a user may select multiple different input data character sequences via the user interface, and in response to each new input data received, regular expression generator 110 may generate an updated regular expression that encompasses both the first and second (positive) examples of the character sequence.Then, if the user highlights a third sequence of characters (e.g., outside of both character sequences, or within the first or second character sequences), regular expression generator 110 may update the regular expression again, etc. In some embodiments, regular expression generator 110 may execute an algorithm in real time (or near real time), and an entirely new regular expression may be generated in response to each new keystroke or each new highlighted section made by the user.

[0074] 11 , in response to a user selection of a character sequence via the user interface, regular expression generator 110 may generate and display a regular expression. For example, when a user highlights a first sequence of characters, the regular expression generator may generate and display a regular expression representing the first sequence of characters. When a user highlights a second sequence of characters, the regular expression generator may generate an updated regular expression that encompasses both the first sequence of characters and the second sequence of characters. Then, when a user highlights a third sequence of characters (e.g., within either the first or second sequence), the regular expression generator may again update the regular expression, and so on.

[0075] 12 is another flowchart illustrating a process for generating regular expressions and extracting data based on capturing groups via user data selections received within a user interface. In step 1201, regular expression generator 110 can retrieve data (e.g., from data repository 130) and render / display the data in a tabular format within a graphical user interface, as described above in step 1101. In step 1202, regular expression generator 110 can receive a user highlight selection of a text fragment within a particular data cell. In step 1203, regular expression generator 110 can generate a regular expression based on the positive examples of the selected data cell, and in step 1204, the regular expression generator can generate a regular expression based on the positive examples of the selected data cell and the highlighted text fragment within the cell. In step 1205, the regular expression generator 110 can determine one or more additional cells in the displayed tabular data that match the generated regular expression, and in step 1206, extract the corresponding text fragments in the additional cells that match the generated regular expression.

[0076] Thus, in addition to providing a positive example, a user may select a text fragment within any of the selected positive examples (e.g., via mouse text highlighting). In response, the regular expression generator 110 may extract that text fragment from the example and create a regular expression capturing group to extract corresponding fragments from all other matches in the text to which the regular expression is applied. Extracting text fragments from matching data cells may also include deletion and modification, and in some cases may be used to create new columns of data from existing columns of semi-structured or unstructured text.

[0077] Using the example where the user selects positive data examples, if the user highlights the year, the regular expression generator 110 generates the regular expression (?:[AZ]{3} / s+ / d / d, / s+| / d / d / / d / d)( / d / d / d / d) As shown in this example, regular expression generator 110 has added parentheses around the year and also converted the old parentheses around the month and day (used for substitution) into "non-capturing" groups by using the ?:regex syntax. In some embodiments, extraction / capture groups may be required to be on span boundaries; in such embodiments, regular expression generator 110 may take the highlighted character range as input and expand it to encompass the nearest locator span boundary. However, in other examples, mid-span extraction / capture may be supported by the user interface.

[0078] In some embodiments, the user interface may support input data from a user that includes a selection of a first character sequence within a second character sequence. For example, a user may highlight one or more characters within a larger, previously highlighted character sequence, and the second user selection may provide context for the larger, first user selection. Such embodiments may allow input data to be provided to regular expression generator 110 with greater specificity.

[0079] Additionally, in some examples, an action can be initiated and a dialog can open in response to a user making a selection (e.g., highlighting text) within the user interface. In some cases, the dialog can be a non-model dialog, such as a floating toolbox window that does not interfere with user interaction with the main screen. The dialog can also change appearance and / or functionality depending on what primary operation the user is performing. Thus, in such cases, the user does not need to search for additional menu items after highlighting selected text to initiate capture group text fragment modification, extraction, etc. Furthermore, in certain embodiments, the user interface provided for generating regular expressions may include three highlighting modes: nested auto, nested manual, and single level. In some cases, the default operating mode may be that the entire cell is identified as the highlighted region, and the user may further highlight one or more additional subsequences within the highlighted cell. In other modes, the user may be permitted to manually specify both highlighting within a data cell of a tabular data display. In yet other modes, the user may be permitted to manually specify outer highlighting without inner highlighting. These other modes are semi-structured data, e.g., tweets or other long strings of data such as browser "user agent" strings. "Semi-structured" data refers to data that can be displayed in a table format within a user interface, but where the columns within the table consist of unstructured text.

[0080] In some such embodiments, the selection (e.g., highlighting) of the inner and outer regions by the user via the user interface may be distinguished by color coding. For example, the outer highlighting of positive cases may be shown in a first text / background color combination, and the inner highlighting of positive cases may be shown in a different, contrasting text / background color combination.

[0081] As described above, a user can specify a capture group selection via selection of a character subsequence. A GUI may be used to facilitate user selection via highlighting (or other indication). An example is shown in FIG. 13, where an exemplary user interface screen is shown along with a tabular data display. In this example, FIG. 13 illustrates highlighting within a column value, caused, for example, by a user dragging a mouse across one or more desired elements of the column value. Note that the "cell" in which user highlighting is performed may exhibit a color change indicating selection of the column value. This color change may be interpreted as automated highlighting in response to user highlighting.

[0082] 14 and 15 are exemplary user interface screens illustrating the generation of regular expressions and capturing groups based on the selection of data from a tabular display. In these examples, FIGS. 14 and 15 show an additional user interface window that is automatically displayed upon detection of user highlighting 1401 in the tabular data display. The window includes a field 1402 for displaying positive cases, a field for displaying negative cases, and a field for displaying a regular expression that is dynamically (and nearly instantly) generated in response to the selection of a positive case from the tabular data display. In these examples, user highlighting in column values ​​1401 may be equivalent to user highlighting in automated highlighting. Thus, user highlighting of an area code populates positive case field 1402 with not only the user-highlighted area code 1401 but also the remainder of the phone number.

[0083] However, it should be understood that user highlighting is not limited to performance within automatic highlighting. For example, user highlighting may alternatively be performed within other user highlighting. As another example, user highlighting may alternatively be performed without inner highlighting (e.g., further highlighting within the highlighted text). These alternatives are particularly suited to semi-structured data, such as "tweets" or columns of data containing other long strings (e.g., browser "user agent" strings).

[0084] Additionally, once a corresponding regular expression is generated, other column values ​​1402 that match the regular expression may be identified based on additional automated highlighting. In the examples shown in Figures 14 and 15, the additional automated highlighting indicates elements of these other column values ​​that match the capturing groups of the generated regular expression. The additional automated highlighting may be performed using colors that are different from the colors used for user highlighting.

[0085] As shown in FIG. 15, additional user highlighting is shown to indicate user selection of other examples. Additional user highlighting may be performed in a manner similar to that described above. Thus, the user interface of FIG. 15 shows a population of other examples in field 1502 for displaying positive examples. This may occur in response to detection of additional user highlighting. Furthermore, the generated regular expression 1503 may be dynamically and nearly instantaneously updated so that it matches all of the positive examples 1502. In response to generation of the updated regular expression, automated highlighting of other column values ​​1504 that match the updated regular expression may also occur. In some implementations, dynamic color-coding may also be used. For example, matches may be color-coded using a first color (e.g., blue), positive cases may be color-coded using a second color (e.g., green), and negative cases may be color-coded using a third color (e.g., red).

[0086] 16A and 16B are example user interface screens illustrating the generation of regular expressions based on the selection of positive and negative examples from a table-like display. In FIGS. 16A-16B, individual examples from positive example field 1602 can be removed from positive example field 1603 and / or moved to negative example field 1603. Within the user interface, this may be performed, for example, by the user clicking (e.g., right-clicking) on ​​one of the examples to select it. The selection can cause the user interface to display menu 1602 with delete and modify options. Clicking an option then performs the corresponding function.

[0087] 16A and 16B, user selection of the change option results in moving the selected example to negative example field 1603 and updating regular expression 1601 to regular expression 1604, which may be generated dynamically and nearly instantaneously (e.g., in one embodiment, between 30 ms and 9000 ms). In response to generating updated regular expression 1604, automated highlighting of other column values ​​that match the updated regular expression may also be updated within the tabular data display. Additionally, automated highlighting may be performed for some or all of the negative examples, including any column values ​​that correspond to negative examples, which may be highlighted using a color different from any of the colors used above or otherwise distinguished within the user interface using other visual techniques.

[0088] In some embodiments, designating a negative example via a user interface need not require first designating the example as a positive example and then converting it to a negative example, as shown in Figures 16A and 16B. Rather, a negative example may be designated in a variety of ways. For example, a user may select (e.g., right-click) a column value via a user interface (e.g., one of the other column values ​​on which automated highlighting has been performed to show that it matches the generated regular expression), thereby causing the display of a menu containing options (e.g., "Create a new counterexample") to designate the selected column value as a negative example.

[0089] 16A and 16B, in response to generating the updated regular expression 1604, the automated highlighting of other column values ​​that match the updated regular expression may also be updated. In these examples, the updated regular expression specifies a telephone number that ends with a "9."

[0090] 14 and 15 , when the “Extract” button is clicked or otherwise selected by the user, an operation may be initiated to extract highlighted text fragments in all cells that match the current regular expression 1403 or 1503. While not shown in FIGS. 14 and 15 , in some embodiments, the user interface may provide other selectable buttons in addition to or instead of the “Extract” button. For example, a “Replace” button may be presented as an option to replace the user-highlighted element with a user-specified element. Additionally or alternatively, one or more “Delete” buttons may be presented as options that effectively replace the user-highlighted element with nothing. For example, one or both of a “Delete Fragment” operation and / or a “Delete Line” operation may be implemented, which may remove either the user-highlighted text fragment or any line, respectively. In various embodiments, additional operations may be implemented, such as a "keep lines" operation, a "split" operation (e.g., highlighting commas and then extracting the comma-separated components into separate new columns), and an "obfuscate" operation (e.g., replacing highlighted text / capture groups with a sequence of "#" or other symbols). In this example, in response to selection of the "Extract" button, the extract operation may be added to a list of transformation scripts to be executed by downstream operations. In some embodiments, the list of transformation scripts may be displayed in a portion of the user interface for review / modification by the user. Alternatively, the extract operation may be performed in-place to generate a new column containing the contents of the regeX capture group (e.g., elements corresponding to the user-highlighted portions of the positive cases). In the example shown in FIGS. 14 and 15, a new column and / or a new table of area codes may be generated in response to selection of the "Extract" button.

[0091] FIG. 17 is another example user interface screen illustrating the generation of regular expressions and capturing groups based on the selection of data from a tabular display, according to one or more embodiments described herein.

[0092] VI. Regular Expression Generation Using the Longest Common Subsequence Algorithm on Spans Further aspects described herein relate to the generation of regular expressions based on the LCS algorithm from one or more data input character sequences, although the regular expression generator 110 can also handle characters that are present in only some of the examples. To handle characters that are present in only some of the input examples, a span can be defined in which both the minimum and maximum number of occurrences of a regular expression code are tracked. For example, for the character sequence inputs "9pm" and "9 pm," an optional "min" character can be inserted between the digits and the "pm" text. There are spaces. In such cases, all of the given input examples have a certain span (e.g., "9"). If a single space between "" and "pm" may not be present, the minimum number of occurrences may be set to zero. These minimum and maximum numbers may then be mapped to regular expression multiplicity syntax. A longest common subsequence (LCS) algorithm may be run on spans of characters derived from the input examples, including "optional" spans that do not appear in all input examples (e.g., a minimum length of zero). As described below, consecutive spans may be merged during the execution of the LCS algorithm. In such cases, when additional optional spans that are carried together end up occurring consecutively, the LCS algorithm may be run recursively on those optional spans as well. That is, while the execution of the LCS algorithm is recursive in nature, in these cases the entire LCS algorithm may be run recursively (e.g., recursively running a recursive LCS algorithm). Among other technical advantages, this may enable shorter, cleaner, and more readable regular expression generation. For example, (am|am) (i.e., with optional whitespace before am) may be generated without recursively running the LCS algorithm, while recursively running the LCS algorithm may result in the regular expression being generated as the shorter and cleaner (?am).

[0093] FIG. 18 is a flowchart illustrating a process for generating regular expressions, including optional spans, using a Longest Common Subsequence (LCS) algorithm, according to one or more embodiments described herein. In step 1801, regular expression generator 110 may receive as input one or more character sequences corresponding to positive regular expression examples. In step 1802, regular expression generator 110 may convert the character sequences into regular expression code. Thus, steps 1801 and 1802 may be similar to or identical to the previous corresponding examples described above. Then, in step 1802, the regular expression code may be further converted into a span data structure (or spans). As described above, each span may be associated with a character class code (e.g., a leger The regular expression generator 110 may include a data structure that stores the set of spans, the minimum repeat count range (e.g., minimum count and / or maximum count), and the minimum repeat count range (e.g., minimum count and / or maximum count). In step 1804, the regular expression generator 110 may execute the LCS algorithm to provide a set of spans as input to the algorithm. The output of the LCS algorithm in this example may include an output set of spans that includes at least one span with a minimum repeat count range equal to zero that corresponds to an optional span in the output of the LCS algorithm. Finally, in step 1805, the regular expression generator 110 may generate a regular expression based on the output of the LCS algorithm that includes the optional span.

[0094] 19 is an exemplary diagram illustrating the generation of a regular expression using the Longest Common Subsequence (LCS) algorithm, where the generated regular expression includes an optional span. In this example, the two input data character sequences are "8 am" and "9 pm." Input Data The character sequence is first converted to a regular expression code (step 1802) and then converted to a span (step 1803), as described above. The span may be provided as input to the LCS algorithm (step 1804), and the LCS output is an optional span Z. <0, 1> to indicate that an optional single space can be a digit and a two-character text sequence. That is, the superscript notation in this example indicates that the preceding code (e.g., Z= The LCS algorithm may include two numbers, a minimum repeat count range (e.g., 0) and a maximum repeat count range (e.g., 1), that apply to the specified string (e.g., whitespace). Finally, a regular expression may be generated based on the output span of the LCS algorithm, with the optional span being represented by the corresponding regular expression code "pZ * " may be converted to "

[0095] In some embodiments, the rendering and use of optional whitespace by regular expression generator 110 during the execution of the LCS algorithm may provide further technical advantages with respect to performance and readability. For example, when generating a regular expression, it is sometimes desirable to be able to handle both characters that are common among all given examples and characters that are present in only some of those examples.

[0096] In one embodiment, for each span data structure, both the minimum number of occurrences of the category code and the maximum number of occurrences of the category code may be tracked. If no spans are present in one or more of the given examples, the minimum is set to zero. As another example, to generate a regular expression for handling spelled months, the minimum and maximum numbers may then be mapped to a regular expression multiplicity syntax with curly braces (e.g., [A-Za-z]{3,9}).

[0097] In some embodiments, regular expression generator 110 may track the minimum and maximum number of occurrences for each span, but may handle additional implementation details. For example, as a result of the combination of handling optional spans and performing LCS on spans of characters, regular expression generator 110 may be configured to detect and merge consecutive spans through the execution of the LCS algorithm. In addition, it may be desirable for any additional optional spans carried together to sometimes appear consecutively and for the LCS algorithm to be run recursively on them as well. For example, in some cases, regular expression generator 110 modifies and / or extends the LCS algorithm to favor (or weight) fewer transitions between optional and required sequence elements (e.g., spans). For example, grouping optional spans together can minimize the number of grouping parentheses that must be used within a regular expression, thus improving the human readability of the generated regular expression. In some cases, if the resulting lengths are equal even after taking into account the optional span, regular expression generator 110 may indicate a preference for the alternative with fewer transitions between the optional and required spans. For example, in some cases, a standard LCS algorithm may be implemented to favor the selection of the longer sequence at its decision point. However, At decision points where the options are of equal length, a configuration preference may be programmed into regular expression generator 110. One such configuration preference may be, for example, to favor shorter sequences (when optional spans are taken into account). Thus, a customized LCS within this configuration can simultaneously optimize longer sequences (of the required span) and shorter sequences (of the sum of the required span and optional spans).

[0098] In some embodiments, a generated regular expression may be more readable if it begins with a required span (which may also serve as a mental anchor for a human reader) rather than an optional span. Thus, in some cases, if the resulting choices have an equal number of transitions, the choice with the earlier non-optional span may be selected. Additionally, the LCS algorithm performed by regular expression generator 110 may, in some embodiments, be configured to push all whitespace (including optional spans corresponding to whitespace) to the right within the regular expression. Pushing all whitespace to the right may increase the chance that whitespace spans can be merged together, which may simplify the resulting regular expression and improve readability. Thus, during execution of the LCS algorithm, if it is determined that two sets of substrings have the same LCS, instead of arbitrarily selecting one of the two sets of substrings, the set that facilitates improved readability may be selected. Furthermore, in some embodiments, the LCS algorithm may be configured to favor a greater number of required spans and / or a fewer number of optional spans to improve readability.

[0099] As mentioned above, in some cases, negative examples may be based on an optional span. For example, a user may provide positive examples of "ab" and "a2b" and a negative example of "a3b." In this case, one example implementation may fail because it may attempt to distinguish based only on the required span, and the "2" digit is in the optional span. In such cases, the user can be alerted to the failure and can be provided with options via the user interface to manually repair the generated regular expression and / or remove some of the negative examples.

[0100] In some embodiments, there may be an isSuccess returned as part of the JSON coming back from the REST service. In some embodiments, the generated regeex may be a different color (e.g., red) when isSuccess=false.

[0101] VII. Regular Expression Generation Using the Combinatric Longest Common Subsequence Algorithm Further aspects described herein relate to combinatorial searches in which the LCS algorithm performed by regular expression generator 110 may be run multiple times to generate a "correct" regular expression (e.g., a regular expression that properly matches all given positive examples and properly filters out all given negative examples) and / or to generate multiple correct regular expressions from which the most desirable or optimal regular expression may be selected. For example, during a combinatorial search, the entire LCS algorithm and regular expression generation process may be run multiple times, including different combinations / permutations of text processing direction, different positioning, and other different characteristics of the LCS algorithm.

[0102] 20 is a flowchart illustrating a process for generating a regular expression based on a combinatorial implementation of the Longest Common Subsequence (LCS) algorithm. In step 2001, regular expression generator 110 may receive an input data character sequence corresponding to a positive example. In step 2002, regular expression generator 110 may iterate over various different combinations of implementation techniques for the LCS algorithm. As shown in this example, during each iteration of step 2002, regular expression generator 110: Different combinations of the following LCS algorithm execution parameters (or characteristics) may be selected: locator (i.e., no positioning, positioning at the beginning of a line, positioning at the end of a line), processing direction (i.e., right-to-left order, left-to-right order), whitespace pushing (i.e., with or without whitespace pushing), and span collapse (i.e., with or without span collapse). In step 2003, the LCS algorithm is run on the input data character sequence (or on the regular expression code if the input character sequence was first transformed), the LCS algorithm being configured based on the parameters / characteristics selected in step 2002. In step 2004, the output of the LCS algorithm may be stored by regular expression generator 110 and may include data such as whether the algorithm successfully identified an LCS and the length of the corresponding regular expression. In step 2005, the process may iterate until the LCS algorithm has been run with all possible combinations of combinatorial search parameters / characteristics. Finally, in step 2006, a particular output from one of the LCSs may be selected as the optimal output (eg, based on success and regular expression length), and a regular expression may be generated based on the selected LCS algorithm output.

[0103] In various embodiments, combinatorial searches such as those described above with reference to Figure 20 may be performed for a variety of different combinations of parameters / characteristics. For example, in some embodiments, the LCS algorithm uses the caret symbol ^ to locate the regular expression to the beginning of the text and / or to locate the regular expression to the end of the text. You may use the dollar sign $ to position instead. Locators can result in generating shorter regular expressions. Locators can be particularly useful when a user wants to find a particular pattern at the beginning and / or end of a string. For example, a user might want a product name at the beginning. To avoid confusing the LCS algorithm with the varying number of words that describe the product name, a caret can be used to locate the regular expression to the beginning of the string, as shown in the image below.

[0104] Additionally, in some embodiments, the LCS algorithm may be run with input data that is either forward or reverse (or equivalently, the LCS algorithm may be configured to receive input data in normal order and then reverse the order before running the algorithm). Thus, in some embodiments, a combinatorial search of the LCS algorithm that may be run against an input character sequence or code pair may be as follows:

[0105] 1. Normal (right-to-left) order, no positioning relative to the start or end 2. Normal (right-to-left) order, using the caret ^ to position relative to the beginning of the line R 3. Normal (right-to-left) order, using dollar $ to position relative to the end of the line 4. Reverse (left-to-right) order, no positioning relative to the beginning or end 5. Reverse (left to right) order, use caret ^ to position relative to the beginning of the line 6. In reverse (left to right) order, use dollar $ to position to the end of the line In this example, of the six runs of the LCS, the shortest resulting regular expression may be selected (step 2006).

[0106] In some embodiments, the combinatorial search of the LCS algorithm may iterate over the greedy quantifier "?" and the non-greedy quantifier "??". For example, the default In the example, if there is an optional span, a single question mark is issued, e.g., [AZ]+(?: for a first name and last name with an optional middle initial. [AZ] / .)? [AZ]+. A satisfactory regular expression is found when using greedy quantifiers. If not, combinatorial search can try to replace all question mark quantifiers with double question mark quantifiers (e.g., [AZ]+(?: [AZ] / .)?? [AZ]+). The double question mark corresponds to a non-greedy quantifier, which can instruct downstream regular expression matchers to enter backtracking mode to find a match.

[0107] Additionally, in some embodiments, the combinatorial search of the LCS algorithm can also iterate over whether to favor whitespace on the right. For example, in some embodiments that push whitespace to the right, as described above, a strategy may be used in the hope that, for example, when the LCS algorithm is faced with an arbitrary choice of otherwise equal options, the whitespace spans will be merged together, resulting in a smaller number of overall spans. This feature adds another option to the combinatorial search, i.e., either push whitespace to the right or perform according to the traditional LCS approach, leaving the decision arbitrary.

[0108] Additionally, in some embodiments, the LCS algorithm's combinatorial search may also iterate by running LCS on the original string to scan / unscan for literals common to all examples. In such embodiments, the LCS algorithm may be configured to identify and align common words. As used herein, a "common word" may refer to a word that appears in all positive examples. Once a common word is identified, its span type may be converted from LETTER to WORD, and then subsequent runs through the LCS algorithm may naturally align to it.

[0109] Thus, in the following example, the combinatorial search may be iterated over several parameters / properties, amounting to 96 runs of the full LCS algorithm. The various parameters / properties to be iterated over in this example are: Positional specifier (3) (value = ^, $, or neither) Push Blank (2) (value = Yes or No) ·Merge low density spans into wildcards (2) (value = Yes or No) Greedy quantifier? (2) (value = Yes or No) Alignment on common tokens in LCS algorithm (2) (value = Yes or No) Use " / w" to represent alphanumeric characters (2) (value = Yes or No) as opposed to keeping letters " / pL" and numbers " / pN" treated as separate spans As mentioned above, in this example the complete LCS algorithm is executed 96 times (eg, 3*2*2*2*2*2=96).

[0110] However, in other embodiments, regular expression generator 110 may provide a performance improvement whereby only the first three properties in the above list (locators, pushing whitespace, and coalescing low-cardinality spans into wildcards) may participate in the combinatorial search. This may result in far fewer executions of the full LCS algorithm (e.g., 3*2*2=12 times). In such an embodiment, the last three properties in the above list (greedy quantifiers, the alignment on common tokens in the LCS algorithm, and the use of " / w" to represent alphanumeric characters versus keeping letters " / pL" and numbers " / pN" treated as separate spans) do not participate in the combinatorial search, but these properties may be tested individually and sequentially at the end. In such an embodiment, a technical advantage may be realized because splitting the search whitespace in this way may still result in a satisfactory regular expression being found with approximately an eight-fold speedup in performance.

[0111] To illustrate, the following example of a combinatorial search may provide a performance advantage over the previous example. In this example, the combinatorial search may be performed based on the following parameters / characteristics to be iterated over: Positioning (3): BEGINNING_OF_LINE_MODE (beginning of line mode), END_OF_LINE_MODE (line ending mode), NO_EOL_MODE (no line ending mode) Order / Direction (2): Right-to-Left (Normal) LCS vs. Left-to-Right (Reverse) LCS Push (2): Whether to try to push whitespace to the right in the LCS algorithm Compress to Wildcards (2): Wildcard long sequences of only occasional spans. * Whether to try to compress to ? The combinatorial trick in this example may result in running the full algorithm 3*2*2*2=24 times. Regular expression generator 110 may then take the best of the 24 results of the LCS algorithm, where "best" may mean that (a) the LCS algorithm was successful, and (b) the shortest regular expression was generated. Regular expression generator 110 may then perform three additional tasks: 1. Attempt to compress sequences of letters and numbers uninterrupted by whitespace, punctuation, or symbols into a new span type I called ALPHANUMERIC, which corresponds to the generated relex of / w. This can be useful for hexadecimal digits found in IPv6 addresses from clickstream logs (see Novelty 64 from April 2019).

[0112] 2. Try using the non-greedy quantifier ?? instead of the greedy quantifier ?. 3. Try to align on the literal.

[0113] Hardware Overview 21 shows a simplified diagram of a distributed system 2100 for implementing an embodiment. In the illustrated embodiment, the distributed system 2100 includes one or more client computing devices 2102, 2104, 2106, and 2108 coupled to a server 2112 via one or more communication networks 2110. The client computing devices 2102, 2104, 2106, and 2108 may be configured to run one or more applications.

[0114] In various embodiments, server 2112 may be adapted to execute one or more services or software applications that enable automated generation of the regular expressions described in this disclosure. For example, in particular embodiments, server 2112 may receive user input data sent from a client device, the user input data being received by the client device via a user interface displayed on the client device. Server 2112 may then convert the user input data into a regular expression that is sent to the client device for display via the user interface.

[0115] In particular embodiments, server 2112 may also provide other services or software applications, which may include non-virtualized and virtualized environments. In some embodiments, these services may be provided as web-based services or cloud services, such as in a Software as a Service (SaaS) model, to users of client computing devices 2102, 2104, 2106, and / or 2108. A user operating a client computing device 2102, 2104, 2106, and / or 2108 may utilize one or more client applications to interact with the server 2112 to utilize the services provided by these components.

[0116] 21, server 2112 may include one or more components 2118, 2120, and 2122 that implement the functions performed by server 2112. These components may include software components that may be executed by one or more processors, hardware components, or a combination thereof. It should be appreciated that a wide variety of system configurations are possible that may differ from distributed system 2100. Thus, the embodiment illustrated in FIG. 21 is an example of a distributed system for implementing the system of the embodiments and is not intended to be limiting.

[0117] A user may use client computing devices 2102, 2104, 2106, and / or 2108 to run one or more applications that generate regular expressions according to the teachings of this disclosure. The client devices may provide an interface that allows a user of the client device to interact with the client device. The client devices may also output information to the user via this interface. Although FIG. 21 shows only four client computing devices, any number of client computing devices may be supported.

[0118] Client devices may include various types of computing systems, such as portable handheld devices, general-purpose computers such as personal computers and laptops, workstation computers, wearable devices, gaming systems, thin clients, various messaging devices, sensors or other sensing devices, etc. These computing devices may include various types and versions of software applications and operating systems (e.g., Microsoft Windows®, Apple Macintosh®, UNIX® or UNIX-like operating systems, Linux® or Linux-like operating systems, various mobile operating systems (e.g., Microsoft Windows Mobile®, iOS®, Windows Phone®, Android®, BlackBerry®, Google Chrome® OS, including Palm OS®). Portable handheld devices may include cellular phones, smartphones (e.g., iPhone®), tablets (e.g., iPad®), personal digital assistants (PDAs), etc. Wearable devices may include Google Glass® head-mounted displays and other devices. The gaming systems may include various handheld gaming devices, internet-enabled gaming devices (e.g., Microsoft Xbox® gaming consoles with or without Kinect® gesture input devices, Sony PlayStation® systems, various gaming systems offered by Nintendo®, etc.) The client devices may be capable of running a wide variety of applications, such as various internet-related applications, communication applications (e.g., email applications, short message service (SMS) applications), and may use a variety of communication protocols.

[0119] Network 2110 may be any type of network known to those skilled in the art that is capable of supporting data communications using any of a variety of available protocols, including, but not limited to, TCP / IP (Transmission Control Protocol / Internet Protocol), SNA (Systems Network Architecture), IPX (Internet Packet Exchange), AppleTalk®, etc. By way of example only, network 2110 may be a local area network (LAN), an Ethernet-based network, a token ring, a wide area network, etc. The network may include a wide area network (WAN), the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a public switched telephone network (PSTN), an infrared network, a wireless network (e.g., a wireless network operating under any of the Institute of Electrical and Electronics Engineers (IEEE) 802.11 protocol suites, Bluetooth® and / or any other wireless protocol), and / or any combination of these and / or other networks.

[0120] The servers 2112 may be comprised of one or more general-purpose computers, dedicated server computers (including, by way of example, PC (personal computer) servers, UNIX servers, mid-range servers, mainframe computers, rack-mounted servers, etc.), server farms, server clusters, or other suitable configurations and / or combinations. The servers 2112 may include one or more virtual machines running a virtual operating system or other computing architecture involving virtualization, such as one or more flexible pools of logical storage that can be virtualized to maintain virtual storage for the servers. In various embodiments, the servers 2112 may be adapted to run one or more services or software applications that provide the functionality described in the above disclosure.

[0121] The computing systems within server 2112 may run one or more operating systems, including any of the operating systems described above, as well as commercially available server operating systems. Server 2112 may also run any of a variety of other server and / or middle-tier applications, including an HTTP (Hypertext Transfer Protocol) server, an FTP (File Transfer Protocol) server, a CGI (Common Gateway Interface) server, a JAVA server, a database server, etc. Exemplary database servers are commercially available from Oracle®, Microsoft®, Sybase®, IBM® (International Business Machines), etc. This includes, but is not limited to:

[0122] In some implementations, server 2112 may include one or more applications for parsing and consolidating data feeds and / or event updates received from users of client computing devices 2102, 2104, 2106, and 2108. By way of example, the data feeds and / or event updates may include, but are not limited to, Twitter® feeds, Facebook® updates, or real-time updates received from one or more third-party sources and continuous data streams that may include real-time events related to sensor data applications, financial stock tickers, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, automobile traffic monitoring, etc. Server 2112 may also include one or more applications for displaying the data feeds and / or real-time events via one or more display devices of client computing devices 2102, 2104, 2106, and 2108.

[0123] The distributed system 2100 may also include one or more data repositories 2114, 2116. In particular embodiments, these data repositories may be used to store data and other information. For example, one or more of the data repositories 2114, 2116 may be used to store information such as new strings of data that match regular expressions generated by the system. The data repositories 2114, 2116 may be located in a variety of locations. For example, the data repository used by the server 2112 may be local to the server 2112 or may be located remotely from the server 2112. The data repositories 2114, 2116 may be of different types, communicating with the server 2112 via a network-based or dedicated connection. The data repositories 2114, 2116 may be of different types. In particular embodiments, the data repositories used by the server 2112 may be databases, for example, relational databases such as those provided by Oracle Corporation® and other manufacturers. One or more of these databases may be adapted to allow data to be stored, updated, and retrieved from the database in response to SQL-formatted commands.

[0124] In particular embodiments, one or more of the data repositories 2114, 2116 may be used by an application to store application data. The data repositories used by an application may be of various types, such as, for example, a key-value store repository, an object store repository, or a general-purpose storage repository supported by a file system.

[0125] In particular embodiments, the functionality described in this disclosure may be provided as a service via a cloud environment. FIG. 22 is a simplified block diagram of a cloud-based system environment that may provide various services as cloud services, according to a particular example. In the example shown in FIG. 22, cloud infrastructure system 2202 may provide one or more cloud services that users may request using one or more client computing devices 2204, 2206, and 2208. Cloud infrastructure system 2202 may include one or more computers and / or servers, which may include those described above with respect to server 2112. The computers in cloud infrastructure system 2202 may be organized as general-purpose computers, dedicated server computers, server farms, server clusters, or any other suitable arrangement and / or combination.

[0126] Network 2210 may facilitate communication and exchange of data between clients 2204, 2206, and 2208 and cloud infrastructure system 2202. Network 2210 may include one or more networks. The networks may be of the same or different types. Network 2210 may support one or more communication protocols, including wired and / or wireless protocols, to facilitate communication.

[0127] The example shown in Figure 22 is merely one example of a cloud infrastructure system and is not intended to be limiting. It should be understood that in other examples, cloud infrastructure system 2202 may have more or fewer components than those shown in Figure 22, may combine two or more components, or may have components in a different configuration or arrangement. For example, while Figure 22 shows three client computing devices, in alternative examples, any number of client computing devices may be supported.

[0128] The term cloud service is generally used to refer to services made available to users on demand over a communications network, such as the Internet, by a service provider's system (e.g., cloud infrastructure system 2202). Typically, in a public cloud environment, the servers and systems that make up the cloud service provider's system are distinct from the customer's own on-premise servers and systems. The cloud service provider's systems are managed by the cloud service provider. Thus, customers can access the cloud service provider's services without having to purchase separate licenses, support, or hardware and software resources for the services. Cloud services can be used by companies such as Oracle Corporation, a leading provider of cloud computing services. For example, a cloud service provider's system may host an application, and users can order and use the application on demand over the Internet without having to purchase infrastructure resources to run the application. Cloud services are designed to provide easy and scalable access to applications, resources, and services. Several providers offer cloud services. For example, several cloud services, such as middleware services, database services, and Java cloud services, are offered by Oracle Corporation, Redwood Shores, California.

[0129] In particular embodiments, cloud infrastructure system 2202 may provide one or more cloud services using a variety of models, such as a Software as a Service (SaaS) model, a Platform as a Service (PaaS) model, an Infrastructure as a Service (IaaS) model, etc., including hybrid service models. Cloud infrastructure system 2202 may include a suite of applications, middleware, databases, and other resources that enable the provision of various cloud services.

[0130] The SaaS model allows applications or software to be delivered as a service to customers over a communications network such as the Internet, without the customer having to purchase the underlying application hardware or software. For example, the SaaS model may be used to provide customers with access to on-demand applications hosted by cloud infrastructure system 2202. Examples of SaaS services offered by Oracle Corporation® include, but are not limited to, various services for human resource / capital management, customer relationship management (CRM), enterprise resource planning (ERP), supply chain management (SCM), enterprise performance management (EPM), analytics services, and social applications.

[0131] The IaaS model is commonly used to provide flexible computing and storage capabilities by providing infrastructure resources (e.g., servers, storage, hardware, and networking resources) to customers as cloud services. Various IaaS services are offered by Oracle Corporation (registered trademark).

[0132] The PaaS model is commonly used to provide platform and environment resources as a service that enable customers to develop, run, and manage applications and services without having to procure, build, or manage the environment resources. Examples of PaaS services offered by Oracle Corporation are Oracle This includes, but is not limited to, Java Cloud Service (JCS), Oracle Database Cloud Service (DBCS), data management cloud services, and various application development solution services.

[0133] Cloud services are generally provided on an on-demand, self-service basis, on a subscription basis, and in a flexible, scalable, reliable, highly available, and secure manner. For example, a customer may order one or more services provided by cloud infrastructure system 2202 via a subscription order. Cloud infrastructure system 2202 then performs processing to provide the services requested in the customer's subscription order. The structure system 2202 may be configured to provide one or more cloud services.

[0134] Cloud infrastructure system 2202 may provide cloud services through a variety of deployment models. In a public cloud model, cloud infrastructure system 2202 may be owned by a third-party cloud service provider, and cloud services are offered to general public customers. These customers may be individuals or businesses. In a private cloud model, cloud infrastructure system 2202 may function within an organization (e.g., within a corporate organization), and services are offered to customers within the organization. For example, these customers may be various departments within a company, such as the human resources department, payroll department, or individuals within the company. In a community cloud model, cloud infrastructure system 2202 and the services offered may be shared among various organizations within an associated community. Various other models, including hybrids of the above models, may also be used.

[0135] Client computing devices 2204, 2206, and 2208 may be of different types (e.g., devices 2102, 2104, 2106, and 2108 shown in FIG. 21) and may be capable of operating one or more client applications. Users may use the client devices to interact with cloud infrastructure system 2202, such as to request services provided by cloud infrastructure system 2202.

[0136] In some embodiments, the processing performed by cloud infrastructure system 2202 to provide management-related services may include big data analytics. This analysis may involve using large data sets, analyzing, and processing them to detect and visualize various trends, behaviors, relationships, etc. within this data. This analysis may be performed by one or more processors, possibly processing the data in parallel, running simulations with the data, etc. For example, big data analytics may be performed by cloud infrastructure system 2202 to determine regular expressions in an automated manner. The data used for this analysis may include structured data (e.g., data stored in a database or structured according to a structural model) and / or unstructured data (e.g., data blobs (binary large objects)). It may contain a large object.

[0137] 22, cloud infrastructure system 2202 may include infrastructure resources 2230 utilized to facilitate the provision of various cloud services offered by cloud infrastructure system 2202. Infrastructure resources 2230 may include, for example, processing resources, storage or memory resources, networking resources, etc.

[0138] In particular embodiments, to facilitate efficient provisioning of these resources to support the various cloud services offered by cloud infrastructure system 2202 to different customers, resources may be organized into resource sets or resource modules (also referred to as "pods"). Each resource module or pod may include a pre-integrated, optimized combination of one or more types of resources. In particular embodiments, different pods may be pre-provisioned for different types of cloud services. For example, a first set of pods may be provisioned for database services, and a second set of pods may be provisioned for Java services, etc., which may include a different combination of resources than the pods in the first set of pods. For some services, the resources may be pre-provisioned to provision those services. Allocated resources may be shared among services.

[0139] Cloud infrastructure system 2202 itself may use services 2232 internally that are shared by different components of cloud infrastructure system 2202 and that facilitate the provisioning of services by cloud infrastructure system 2202. These internal shared services may include, but are not limited to, security and identity services, integration services, enterprise repository services, enterprise manager services, virus scanning and whitelist services, high availability, backup and recovery services, services enabling cloud support, email services, notification services, file transfer services, etc.

[0140] Cloud infrastructure system 2202 may include multiple subsystems. These subsystems may be implemented in software, hardware, or a combination thereof. As shown in FIG. 22 , the subsystems may include a user interface subsystem 2212 that enables users or customers of cloud infrastructure system 2202 to interact with cloud infrastructure system 2202. User interface subsystem 2212 may include a variety of different interfaces, such as a web interface 2214, an online store interface 2216 through which cloud services offered by cloud infrastructure system 2202 are advertised and available for consumer purchase, and other interfaces 2218. For example, a customer may use a client device to request one or more services (service request 2234) offered by cloud infrastructure system 2202 using one or more of interfaces 2214, 2216, and 2218. For example, a customer may access an online store, browse cloud services offered by cloud infrastructure system 2202, and place a subscription order for one or more services offered by cloud infrastructure system 2202 and for which the customer wishes to subscribe. The service request may include information identifying the customer and one or more services for which the customer wishes to subscribe. For example, a customer may place an order for a regular expression auto-generation related service provided by cloud infrastructure system 2202.

[0141] 22, cloud infrastructure system 2202 may include an order management subsystem (OMS) 2220 configured to process new orders. As part of this processing, OMS 2220 may be configured to create an account for the customer if not already created, receive billing and / or account information from the customer to use for billing the customer for providing the requested services to the customer, verify the customer information, and, once verified, reserve the order for the customer and prepare the order for provisioning by coordinating various workflows.

[0142] Upon proper validation, OMS 2220 may invoke order provisioning subsystem (OPS) 2224, which is configured to provision resources for the order, including processing, memory, and networking resources. Provisioning may include allocating resources for the order and configuring the resources to facilitate the services requested by the customer order. The manner in which resources are provisioned for the order and the types of resources provisioned may depend on the type of cloud service the customer has ordered. For example, one workflow may invoke OPS 2224 to determine the specific cloud service being requested and identify the number of pods that may be pre-configured for this specific cloud service. The number of pods allocated for an order may depend on the size / amount / level / scope of the service requested. For example, the number of pods to allocate may be determined based on the number of users the service is to support, the duration for which the service is requested, etc. The allocated pods may then be customized to the particular requesting customer to provide the requested service.

[0143] Cloud infrastructure system 2202 may send a response or notification 2244 to the requesting customer to indicate when the requested service will be available for use. In some examples, the customer may be sent information (e.g., a link) that enables the customer to begin using and utilizing the benefits of the requested service. In particular embodiments, for a customer requesting a regular expression auto-generation related service, the response may include instructions that, when executed, cause a user interface to be displayed.

[0144] Cloud infrastructure system 2202 may provide services to multiple customers. For each customer, cloud infrastructure system 2202 manages information related to one or more subscription orders received from the customer, maintains customer data related to the orders, and is responsible for providing the requested services to the customer. Cloud infrastructure system 2202 may also collect usage statistics regarding the customer's use of the subscribed services. For example, statistics may be collected about the amount of storage used, the amount of data transferred, the number of users, and the amount of system uptime and downtime. This usage information may be used to bill the customer. Billing may be on a monthly basis, for example.

[0145] Cloud infrastructure system 2202 may provide services to multiple customers in parallel. Cloud infrastructure system 2202 may store information about these customers, possibly including copyright information. In particular embodiments, cloud infrastructure system 2202 includes an identity management subsystem (IMS) 2228 configured to manage customer information and separate the managed information so that information about one customer is not accessible from information about another customer. IMS 2228 may be configured to provide various security-related services, such as identity services, information access management, authentication and authorization services, services for managing customer identities and roles and associated capabilities, etc.

[0146] 23 illustrates an example of a computer system 2300. In some embodiments, the computer system 2300 may be used to implement any of the systems described above. As shown in FIG. 23, the computer system 2300 includes various subsystems, including a processing subsystem 2304 that communicates with several other subsystems via a bus subsystem 2302. These other subsystems may include a processing acceleration unit 2306, an I / O subsystem 2308, a storage subsystem 2318, and a communication subsystem 2324. The storage subsystem 2318 may include non-transitory computer-readable storage media, including a storage medium 2322 and a system memory 2310.

[0147] Bus subsystem 2302 provides a mechanism for allowing the various components and subsystems of computer system 2300 to communicate with each other as intended. While bus subsystem 2302 is shown schematically as a single bus, alternative embodiments of the bus subsystem may utilize multiple buses. Bus subsystem 2302 may be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, a local bus, etc., using any of a variety of bus architectures. For example, such architectures may be Industry Standard Architectures (ISAs). Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, Video Electronics Standards Association (VESA) local bus, and buses conforming to the IEEE P1386.1 standard. The bus may include a Peripheral Component Interconnect (PCI) bus, which may be implemented as a mezzanine bus manufactured by

[0148] The processing subsystem 2304 controls the operation of the computer system 2300 and may include one or more processors, application specific integrated circuits (ASICs), or field programmable gate arrays (FPGAs). The processors may include single-core or multi-core processors. The processing resources of the computer system 2300 may be organized into one or more processing units 2332, 2334, etc. The processing units may include one or more processors, one or more cores from the same or different processors, a combination of cores and processors, or other combinations of cores and processors. In some embodiments, the processing subsystem 2304 may include one or more dedicated coprocessors, such as a graphics processor, a digital signal processor (DSP), etc. In some embodiments, some or all of the processing units of the processing subsystem 2304 may use customized circuitry, such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

[0149] In some embodiments, processing units within processing subsystem 2304 may execute instructions stored in system memory 2310 or computer-readable storage medium 2322. In various examples, the processing units may execute various program or code instructions and maintain multiple programs or processes running simultaneously. At any given time, some or all of the program code to be executed may reside in system memory 2310 and / or computer-readable storage medium 2322, potentially including one or more storage devices. Through appropriate programming, processing subsystem 2304 may provide the various functions described above. In examples where computer system 2300 is running one or more virtual machines, one or more processing units may be assigned to each virtual machine.

[0150] In particular embodiments, a processing acceleration unit 2306 may optionally be provided to accelerate the overall processing performed by the computer system 2300, to perform customized processing, or to offload portions of the processing performed by the processing subsystem 2304.

[0151] The I / O subsystem 2308 can include devices and mechanisms for inputting information into the computer system 2300 and / or outputting information from or through the computer system 2300. In general, use of the term "input device" is intended to include all conceivable types of devices and mechanisms for inputting information into the computer system 2300. User interface input devices may include, for example, keyboards, pointing devices such as mice or trackballs, touchpads or touchscreens integrated into displays, scroll wheels, click wheels, dials, buttons, switches, keypads, voice input devices with voice command recognition systems, microphones, and other types of input devices. User interface input devices may also include motion-sensing and / or gesture-based devices such as Microsoft Kinect® motion sensors, Microsoft Xbox® 360 game controllers, and devices that provide an interface for receiving input using gestures and voice commands, allowing a user to control and interact with the input device. The user interface input devices may also include an eye gesture recognition device, such as a Google Glass® blink detector, that detects eye movements from the user (e.g., "blinks" while taking a picture and / or making a menu selection) and translates the eye gestures as input to the input device (e.g., Google Glass®). The user interface input devices may also include a voice recognition sensing device that allows the user to interact with a voice recognition system (e.g., Siri® navigator) via voice commands.

[0152] Other examples of user interface input devices may include, but are not limited to, three-dimensional (3D) mice, joysticks or pointing sticks, gamepads, and graphic tablets, as well as audio / visual devices such as speakers, digital cameras, digital camcorders, portable media players, webcams, image scanners, fingerprint scanners, barcode readers, 3D scanners, 3D printers, laser range finders, and eye-tracking devices. User interface input devices may also include medical imaging input devices such as, for example, computed tomography, magnetic resonance imaging, position emission tomography, and medical ultrasound devices. User interface input devices may also include audio input devices such as, for example, MIDI keyboards, digital musical instruments, and the like.

[0153] In general, the use of the term output device(s) is intended to include all conceivable types of devices and mechanisms for outputting information from computer system 2300 to a user or to another computer. User interface output devices may include display subsystems, indicator lights, or non-visual displays such as audio output devices. Display subsystems may be flat panel devices such as those using cathode ray tubes (CRTs), liquid crystal displays (LCDs), or plasma displays, plotting devices, touch screens, etc. For example, user interface output devices may include, but are not limited to, various display devices that visually convey text, graphics, and audio / visual information, such as monitors, printers, speakers, headphones, automobile navigation systems, plotters, audio output devices, and modems.

[0154] The storage subsystem 2318 provides a repository or data store for storing information and data used by the computer system 2300. The storage subsystem 2318 provides a tangible, non-transitory, computer-readable storage medium for storing the basic programming and data constructs that provide some example functionality. Software (e.g., programs, code modules, instructions) that, when executed by the processing subsystem 2304, provide the functionality described above may be stored in the storage subsystem 2318. The software may be executed by one or more processing units of the processing subsystem 2304. The storage subsystem 2318 may also provide a repository for storing data used in accordance with the teachings of the present disclosure.

[0155] The storage subsystem 2318 may include one or more non-transitory memory devices, including volatile and non-volatile memory devices. As shown in Figure 23, the storage subsystem 2318 includes a system memory 2310 and a computer-readable storage medium 2322. The system memory 2310 may include several memories, including volatile primary random access memory (RAM) for storing instructions and data during program execution, and non-volatile read-only memory (ROM) or flash memory in which fixed instructions are stored. In some implementations, a basic input / output (BIO) memory, which contains basic routines that help transfer information between elements within the computer system 2300, such as during start-up. The basic input / output system (BIOS) is typically stored in ROM. Typically, RAM contains data and / or program modules currently operated on and executed by the processing subsystem 2304. In some implementations, the system memory 2310 may include several different types of memory such as static random access memory (SRAM), dynamic random access memory (DRAM), etc.

[0156] 23, system memory 2310 may load running application programs 2312, program data 2314, and operating system 2316, which may include various applications such as a web browser, a middle-tier application, a relational database management system (RDBMS), etc. By way of example, operating system 2316 may include Microsoft Windows®, Apple Macintosh®, and / or Linux operating systems, various commercially available UNIX® or UNIX-like operating systems (including, but not limited to, various GNU / Linux operating systems, Google Chrome® OS, etc.), and / or various versions of mobile operating systems such as iOS®, Windows® Phone, Android® OS, BlackBerry® OS, Palm® OS operating systems, etc.

[0157] The computer-readable storage medium 2322 may store programming and data constructs that provide some example functionality. The computer-readable storage medium 2322 may provide storage of computer-readable instructions, data structures, program modules, and other data for the computer system 2300. Software (programs, code modules, instructions) that, when executed by the processing subsystem 2304, provide the above-described functionality may be stored in the storage subsystem 2318. By way of example, the computer-readable storage medium 2322 may include non-volatile memory such as a hard disk drive, a magnetic disk drive, a CD-ROM, a DVD, an optical disk drive such as a Blu-Ray® disk, or other optical media. The computer-readable storage medium 2322 may include, but is not limited to, a Zip® drive, a flash memory card, a Universal Serial Bus (USB) flash drive, a Secure Digital (SD) card, a DVD disk, a digital video tape, etc. The computer-readable storage medium 2322 may also include solid-state drives (SSDs) based on non-volatile memory such as flash memory-based SSDs, enterprise flash drives, solid-state ROM, etc., SSDs based on volatile memory such as solid-state RAM, dynamic RAM, static RAM, DRAM-based SSDs, magnetoresistive RAM (MRAM) SSDs, and hybrid SSDs that use a combination of DRAM and flash memory-based SSDs.

[0158] In particular embodiments, storage subsystem 2318 may also include a computer-readable storage medium reader 2320 that may be further connected to a computer-readable storage medium 2322. Reader 2320 may be configured to receive and read data from a memory device such as a disk, flash drive, or the like.

[0159] In particular embodiments, computer system 2300 may support virtualization techniques, including, but not limited to, virtualization of processing and memory resources. For example, computer system 2300 may provide support for running one or more virtual machines. In particular embodiments, computer system 2300 may support the configuration and operation of virtual machines. The virtual machine may run a program such as a hypervisor that facilitates the management and administration of the virtual machine. Each virtual machine may be assigned memory, computing (e.g., processors, cores), I / O, and networking resources. Each virtual machine typically runs independently from other virtual machines. A virtual machine typically runs its own operating system, which may be the same as or different from the operating systems run by other virtual machines run by computer system 2300. Thus, potentially multiple operating systems may be running simultaneously by computer system 2300.

[0160] The communications subsystem 2324 provides an interface to other computer systems and networks. The communications subsystem 2324 serves as an interface for sending and receiving data between other systems and the computer system 2300. For example, the communications subsystem 2324 may enable the computer system 2300 to establish a communications channel over the Internet to one or more client devices to send information to and receive information from the one or more client devices.

[0161] The communications subsystem 2324 may support both wired and / or wireless communication protocols. In an embodiment, the communications subsystem 2324 may include a radio frequency (RF) transceiver component for accessing wireless voice and / or data networks (e.g., using cellular telephone technology, advanced data network technologies such as 3G, 4G, or EDGE (High Data Rates for Global Evolution), WiFi (IEEE 802.XX family of standards, or other mobile communications technologies, or any combination thereof), a global positioning system (GPS) receiver component, and / or other components. In some embodiments, the communications subsystem 2324 may provide a wired network connection (e.g., Ethernet) in addition to or instead of a wireless interface.

[0162] The communications subsystem 2324 may receive and transmit data in a variety of formats. In some embodiments, the communications subsystem 2324 may receive incoming communications in the form of structured and / or unstructured data feeds 2326, event streams 2328, event updates 2330, etc., among other formats. For example, the communications subsystem 2324 may be configured to receive (or transmit) data feeds 2326 in real time from users of social media networks and / or other communications services, such as web feeds such as Twitter® feeds, Facebook® updates, Rich Site Summary (RSS) feeds, and / or real-time updates from one or more third-party sources.

[0163] In particular embodiments, communications subsystem 2324 may be configured to receive data in the form of a continuous data stream, which may include an event stream 2328 of real-time events and / or event updates 2330, which may be continuous or infinite in nature without a clear end. Examples of applications that generate continuous data may include, for example, sensor data applications, financial stock ticker boards, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, automobile traffic monitoring, etc.

[0164] Communications subsystem 2324 may be configured to communicate data from computer system 2300 to other computer systems or networks. This data may be transmitted to computer system 2300 in a variety of different formats, such as structured and / or unstructured data feeds 2326, event streams 2328, event updates 2330, etc. The data may be transmitted to one or more databases that may be in communication with one or more streaming data source computers coupled to 300 .

[0165] Computer system 2300 may be one of a variety of types, including a handheld portable device (e.g., an iPhone® cellular phone, an iPad® computing tablet, a PDA), a wearable device (e.g., a Google Glass® head-mounted display), a personal computer, a workstation, a mainframe, a kiosk, a server rack, or other data processing system. Due to the constantly changing nature of computers and networks, the description of computer system 2300 shown in FIG. 23 is intended only as a specific example. Many other configurations are possible, having more or fewer components than the system shown in FIG. 23. Those skilled in the art will recognize other aspects and / or methods for implementing the various examples based on the disclosure and teachings herein.

[0166] While specific examples have been described, various variations, modifications, alternative configurations, and equivalents are possible. The examples are not limited to operation in a particular data processing environment, but may freely operate in multiple data processing environments. Furthermore, while the examples have been described using a particular sequence of transactions and steps, it should be apparent to those skilled in the art that this is not intended to be limiting. While some flowcharts describe operations as a sequential process, many of these operations may be performed in parallel or simultaneously. Additionally, the order of operations may be re-specified. A process may have additional steps not included in the figures. Various features and aspects of the above examples may be used individually or together.

[0167] Additionally, while particular examples have been described using particular combinations of hardware and software, it should be understood that other combinations of hardware and software are possible. Particular examples may be implemented exclusively in hardware, exclusively in software, or using a combination thereof. The various processes described herein may be implemented on the same processor or any combination of different processors.

[0168] Where a device, system, component, or module is described as being configured to perform a particular operation or function, such configuration may be achieved, for example, by designing an electronic circuit to perform the operation, by programming a programmable electronic circuit (such as a microprocessor) to perform the operation, by executing, for example, computer instructions or code, or a processor or core programmed to execute code or instructions stored on a non-transitory memory medium, or any combination thereof. Processes may communicate using a variety of techniques, including, but not limited to, conventional techniques for inter-process communication, and different pairs of processes may use different techniques, and the same pair of processes may use different techniques at different times.

[0169] In this disclosure, specific details are provided to ensure a thorough understanding of the examples. However, the examples may be practiced without these specific details. For example, well-known circuits, processes, algorithms, structures, and techniques are shown without unnecessary detail so as not to obscure the examples. This specification provides illustrative examples only and is not intended to limit the scope, applicability, or configuration of other examples. Rather, the above description of the examples provides one skilled in the art with an enabling description for implementing various examples. Various changes are possible within the function and configuration of elements.

[0170] Accordingly, the specification and drawings are to be regarded in an illustrative rather than a restrictive sense. It will be apparent, however, that additions, subtractions, deletions, and other modifications and alterations may be made thereto without departing from the broader spirit and scope as set forth in the claims. Thus, while specific examples have been described, they are not intended to be limiting. Various modifications and equivalents are within the scope of the appended claims.

[0171] While the foregoing specification describes aspects of the disclosure with reference to specific examples thereof, those skilled in the art will recognize that the disclosure is not limited thereto. Various features and aspects of the above disclosure may be used individually or together. Moreover, the examples can be utilized in a variety of environments and applications beyond those described herein without departing from the broader spirit and scope of the specification. Accordingly, the specification and drawings should be regarded as illustrative rather than restrictive.

[0172] In the above description, the methods are described in a particular order for purposes of illustration. It should be understood that in alternative examples, the methods may be performed in an order different from that described. It should also be understood that the methods described above may be performed by hardware components or embodied in a sequence of machine-executable instructions that, when used, may cause a machine, such as a general-purpose or special-purpose processor or logic circuitry programmed with such instructions, to perform the method. These machine-executable instructions may be stored on one or more machine-readable media, such as a CD-ROM or other type of optical disk, floppy disk, ROM, RAM, EPROM, EEPROM, magnetic or optical card, flash memory, or other type of machine-readable medium suitable for storing electronic instructions. Alternatively, the methods may be performed by a combination of hardware and software.

[0173] Where a component is described as being configured to perform particular operations, such configuration may be achieved, for example, by designing electronic circuitry or other hardware to perform the particular operations, by programming a programmable electronic circuitry (e.g., a microprocessor or other suitable electronic circuitry) to perform the particular operations, or any combination thereof.

[0174] While illustrative examples of the present application have been described in detail herein, it is to be understood that the concepts of the present invention may be variously embodied and employed in other forms, and that the claims are intended to be construed to include such variations except insofar as limited by the prior art.

[0175] Where a component is described as being "configured to" perform particular operations, such configuration may be achieved, for example, by designing electronic circuitry or other hardware to perform the particular operations, by programming a programmable electronic circuitry (e.g., a microprocessor or other suitable electronic circuitry) to perform the particular operations, or any combination thereof.

Claims

1. 1. A method for generating regular expressions using a longest common subsequence (LCS) algorithm, comprising: a regular expression generator comprising one or more processors receiving input data via an interactive user interface identifying three or more character sequences; the regular expression generator converting each of the three or more character sequences into a corresponding set of regular expression codes to obtain three or more sets of regular expression codes; the regular expression generator performs multiple runs of the longest common subsequence (LCS) algorithm, the LCS algorithm being run on unique two-set combinations of the three or more sets of regular expression codes, the method further comprising: The regular expression generator stores data defining a fully connected graph, the data comprising: a plurality of nodes, each node of the fully connected graph corresponding to one of the three or more sets of regular expression codes, and the data further comprising: a plurality of edges connecting each unique pair of the plurality of nodes, wherein an edge length between each of the unique pairs of nodes is defined by the output of the LCS algorithm executed on regular expression code corresponding to the unique pair of nodes, the method further comprising: the regular expression generator determining a minimum spanning tree for the fully connected graph; the regular expression generator traversing the minimum spanning tree for the fully connected graph to determine an order for identifying a first longest common subsequence within the three or more character sequences using the LCS algorithm.

2. identifying the first longest common subsequence within the three or more character sequences; using the LCS algorithm to identify the first longest common subsequence between a first set of regular expression codes and a second set of regular expression codes corresponding to a first character sequence and a second character sequence in the input data; using the LCS algorithm to identify a second longest common subsequence between the first set of regular expression codes and a third set of regular expression codes corresponding to the first character sequence and a third character sequence in the input data; using the LCS algorithm to identify a third longest common subsequence between the second set of regular expression codes and the third set of regular expression codes corresponding to the second character sequence and the third character sequence in the input data; selecting the first longest common subsequence based on the order determined by the traversal of the minimum spanning tree of the fully connected graph.

3. The method of claim 1 or 2, wherein traversing the minimum spanning tree of the fully connected graph comprises performing a depth-first traversal on the minimum spanning tree.

4. The method further comprises: the regular expression generator storing in a memory a plurality of pairs of regular expression codes provided as input to the LCS algorithm and the corresponding output of the LCS algorithm; the regular expression generator generating one or more regular expressions based on the output of the multiple runs of the LCS algorithm; 4. The method of claim 1, wherein the plurality of pairs of regular expression codes provided as input to the LCS algorithm and the corresponding output of the LCS algorithm are retained in the memory after generation of the one or more regular expressions.

5. The method further comprises: receiving, via the interactive user interface, input data by the regular expression generator identifying a plurality of additional character sequences; the regular expression generator converting each of the plurality of additional character sequences into a corresponding set of regular expression codes, resulting in a plurality of additional regular expression codes; identifying pairs of regular expression codes within the plurality of additional regular expression codes that match pairs of regular expression codes stored and maintained in the memory; 5. The method of claim 4, further comprising: responsive to identifying a matching pair of regular expression codes provided as inputs to the LCS algorithm, retrieving the corresponding output of the LCS algorithm from the memory.

6. generating a regular expression based on the multiple executions of the LCS algorithm, wherein generating the regular expression comprises: determining first two sets of the three or more sets of regular expression codes based on the order determined by traversing the minimum spanning tree; and performing a first additional execution of the LCS algorithm, the first additional execution including providing the first two sets of regular expression code as inputs to an execution of the LCS algorithm and capturing a first output of the LCS algorithm, wherein generating the regular expression further comprises: determining a third set of the three or more sets of regular expression codes based on the order determined by traversing the minimum spanning tree; and performing a second additional execution of the LCS algorithm, the second additional execution comprising providing the first output of the LCS algorithm and the third set of regular expression code as inputs to the second additional execution of the LCS algorithm, and capturing a second output of the LCS algorithm.

7. The three or more sets of regular expression codes include at least four sets of regular expression codes, and generating the regular expressions further comprises: determining a fourth set of the at least four sets of regular expression codes based on the order determined by traversing the minimum spanning tree; 7. The method of claim 6, further comprising: performing a third additional execution of the LCS algorithm, the third additional execution comprising providing the second output of the LCS algorithm and the fourth set of regular expression code as inputs to the third additional execution of the LCS algorithm; and capturing a third output of the LCS algorithm.

8. 1. A system for generating regular expressions using a longest common subsequence (LCS) algorithm, comprising: a processing unit including one or more processors; and a memory storing instructions that, when executed by the processing unit, cause the system to: receiving input data via an interactive user interface identifying three or more character sequences; converting each of the three or more character sequences into a corresponding set of regular expression codes to obtain three or more sets of regular expression codes; and causing multiple executions of the longest common subsequence (LCS) algorithm, wherein the LCS algorithm is performed for unique two-set combinations of the three or more sets of regular expression codes. The instructions, when executed by the processing unit, further cause the system to: storing data defining a fully connected graph, said data comprising: a plurality of nodes, each node of the fully connected graph corresponding to one of the three or more sets of regular expression codes, and the data further comprising: a plurality of edges connecting each unique pair of the plurality of nodes, wherein an edge length between each of the unique pair of nodes is defined by an output of the LCS algorithm executed on a regular expression code corresponding to the unique pair of nodes, and the instructions, when executed by the processing unit, further cause the system to: determining a minimum spanning tree for the fully connected graph; traversing the minimum spanning tree for the fully connected graph to determine an order for identifying a first longest common subsequence within the three or more character sequences using the LCS algorithm.

9. the memory storing further instructions that, when executed by the processing unit, cause the system to identify the first longest common subsequence within the three or more character sequences; using the LCS algorithm to identify the first longest common subsequence between a first set of regular expression codes and a second set of regular expression codes corresponding to a first character sequence and a second character sequence in the input data; using the LCS algorithm to identify a second longest common subsequence between the first set of regular expression codes and a third set of regular expression codes corresponding to the first character sequence and a third character sequence in the input data; using the LCS algorithm to identify a third longest common subsequence between the second set of regular expression codes and the third set of regular expression codes corresponding to the second character sequence and the third character sequence in the input data; and selecting the first longest common subsequence based on the order determined by the traversal of the minimum spanning tree of the fully connected graph.

10. 10. The system of claim 8 or 9, wherein traversing the minimum spanning tree of the fully connected graph comprises performing a depth-first traversal on the minimum spanning tree.

11. The memory stores further instructions that, when executed by the processing unit, cause the system to: storing in a memory a plurality of pairs of regular expression codes provided as input to the LCS algorithm and the corresponding output of the LCS algorithm; generating one or more regular expressions based on the output of the multiple runs of the LCS algorithm; 11. The system of claim 8, wherein the plurality of pairs of regular expression codes provided as inputs to the LCS algorithm and the corresponding outputs of the LCS algorithm are retained in the memory after generation of the one or more regular expressions.

12. The memory stores further instructions that, when executed by the processing unit, cause the system to: receiving input data via the interactive user interface identifying a plurality of additional character sequences; converting each of the plurality of additional character sequences into a corresponding set of regular expression codes, resulting in a plurality of additional regular expression codes; identifying pairs of regular expression codes within the plurality of additional regular expression codes that match pairs of regular expression codes stored and maintained in the memory; 12. The system of claim 11, further comprising: responsive to identifying a matching pair of regular expression codes provided as inputs to the LCS algorithm, retrieving the corresponding outputs of the LCS algorithm from the memory.

13. The memory storing further instructions that, when executed by the processing unit, cause the system to generate a regular expression based on the multiple executions of the LCS algorithm, generating the regular expression comprising: determining first two sets of the three or more sets of regular expression codes based on the order determined by traversing the minimum spanning tree; and performing a first additional execution of the LCS algorithm, the first additional execution including providing the first two sets of regular expression code as inputs to an execution of the LCS algorithm and capturing a first output of the LCS algorithm, wherein generating the regular expression further comprises: determining a third set of the three or more sets of regular expression codes based on the order determined by traversing the minimum spanning tree; and performing a second additional execution of the LCS algorithm, the second additional execution including providing the first output of the LCS algorithm and the third set of regular expression code as inputs to the second additional execution of the LCS algorithm, and capturing a second output of the LCS algorithm.

14. The three or more sets of regular expression codes include at least four sets of regular expression codes, and generating the regular expressions further comprises: determining a fourth set of the at least four sets of regular expression codes based on the order determined by traversing the minimum spanning tree; and performing a third additional execution of the LCS algorithm, the third additional execution including providing the second output of the LCS algorithm and the fourth set of regular expression code as inputs to the third additional execution of the LCS algorithm and capturing a third output of the LCS algorithm.

15. A program for causing a computer to execute the method according to any one of claims 1 to 7.

Images