User interface for regular expression generation

Abstract

Disclosed herein are technologies related to the automatic generation of regular expressions. In some embodiments, a regular expression generator may receive input data comprising one or more character sequences. The regular expression generator may convert the character sequence into a set of regular expression codes and / or span data structures. The regular expression generator may identify the longest common subsequence shared by the set of regular expression codes and / or spans, and may generate a regular expression based on the longest common subsequence.

Description

[0001] Cross-reference to related applications

[0002] This application claims priority under 35 USC §119(e) to U.S. Provisional Patent Application No. 62 / 684,498, filed on June 13, 2018, entitled “AUTOMATED GENERATION OF REGULAR EXPRESSIONS,” and this application also claims priority under 35 USC §119(e) to U.S. Provisional Patent Application No. 62 / 749,001, filed on October 22, 2018, entitled “AUTOMATED GENERATION OF REGULAR EXPRESSIONS.” The entire contents of U.S. Provisional Patent Application Nos. 62 / 684,498 and 62 / 749,001 are incorporated herein by reference for all purposes. Background Art

[0003] Big data analytics systems can be used for predictive analytics, user behavior analytics, and other advanced data analytics. However, before any data analysis can be effectively performed to provide useful results, the initial data set may need to be formatted into a clean and curated data set. This data loading often poses challenges to cloud-based data repositories and other big data systems, in which data from a variety of different data sources and / or data streams can be compiled into a single data repository. This data can include structured data in a variety of different formats, semi-structured data according to different data models, and even unstructured data. Repositories of this data typically include data representations in a variety of different formats and structures, and may also include duplicate data and erroneous data. When analyzing these data repositories for reporting, predictive modeling, and other analytical tasks, the poor signal-to-noise ratio of the initial data set may lead to inaccurate or useless results.

[0004] Many current solutions to the problem of data formatting and preprocessing involve manual and adhoc processes to clean and organize the data into a common format before performing data analysis. While these manual processes may be effective for certain smaller datasets, such processes can be inefficient and impractical when attempting to preprocess and format large-scale datasets. Summary of the invention

[0005] Aspects described herein provide various techniques for generating regular expressions.As used herein, "regular expression" may refer to a character sequence defining a pattern, which may be used to search for matches in a longer input text string.In certain embodiments, a symbol wildcard matching language may be used to form a regular expression, and the pattern defined by the regular expression may be used to match a string and / or extract information from a string provided as input.In various embodiments described herein, a regular expression maker implemented as a data processing system may be used to receive and display input text data, receive a selection of a specific character subset of input text via a client user interface, and then generate one or more regular expressions based on the selected character subset.After generating one or more regular expressions, a regular expression engine may be used to match the pattern of a regular expression with one or more data sets.In various embodiments, the data matched with the regular expression may be extracted, reformatted or modified, etc. In some cases, additional columns, tables or other data sets may be created based on the data matched with the regular expression.

[0006] According to certain aspects described herein, a regular expression generator implemented via a data processing system can generate a regular expression based on a determined longest common subsequence (LCS) shared by different sets of one or more regular expression codes. A regular expression code (also referred to as a category code) may include, for example, L represents letters in the English alphabet, N represents numbers, Z represents spaces, P represents punctuation marks, and S represents other symbols. Each set of one or more regular expression codes may be converted from different sequences of one or more characters received as input data through a user interface. Regular expression codes excluded from LCS may be represented as optional and / or alternative. In some embodiments, a regular expression code may be associated with a minimum number of occurrences of the regular expression code. Additionally or alternatively, a regular expression code may be associated with a maximum number of occurrences of a regular expression code. For example, a category code set may include L<0,1> to indicate that a particular portion of an LCS includes a letter at most once (if any). As discussed in more detail below, summarizing input data into an intermediate regular expression code (IREC) can provide various technical advantages, including using very little input data, enabling near-instantaneous generation of regular expressions that are not affected by false positive matches or false negative matches in data not yet seen.

[0007] According to additional aspects described herein, a regular expression can be generated based on input data including three or more character sequences. When three or more character sequences are identified as input data, a regular expression generator that identifies the LCS of the character sequence may cause an exponential increase in running time. In order to identify the LCS of all character sequences in a high-performance manner, the regular expression generator can perform an LCS algorithm for each different combination of two character sequences. A fully connected graph can be generated based on the result of the LCS algorithm, wherein each graph node represents a different character sequence, and the length of each graph edge corresponds to the LCS of the node defining the graph edge. Then, the order of selecting the character sequence can be determined by performing a depth-first traversal on the minimum spanning tree of the fully connected graph.

[0008] Other aspects described herein relate to generating a regular expression based on inputs including both positive character sequence examples and negative character sequence examples. A positive example may refer to a character sequence that matches the regular expression to be generated, and a negative example may refer to a character sequence that does not match the regular expression to be generated. In some embodiments, when both the positive example and the negative example are received, a regular expression generator may identify a discriminator or the shortest subsequence of one or more characters that distinguish the positive example from the negative example. The selected discriminator may be the shortest sequence (e.g., represented by a category code), and may be positive or negative, so that the positive example will match and the negative example will not match. Then, the discriminator may be hard-coded into the regular expression generated by the regular expression generator. In some cases, the shortest subsequence may be included in the prefix or suffix portion of the negative example.

[0009] Additional aspects described herein relate to one or more user interfaces through which input data can be provided to generate regular expressions. In some embodiments, a user interface can be displayed at a client device that is communicatively coupled to a regular expression generator server. The user interface can be generated programmatically by a server, by a client device, or by a combination of software components executed on a server and a client. The input data received by the user interface can correspond to a user's selection of one or more character sequences, which can represent positive or negative examples. In some cases, the user interface can support input data including a selection of a first character sequence within a second character sequence. For example, a user can highlight one or more characters in a larger previously highlighted character sequence, and the second user selection can provide context for the larger first user selection. This enables the input data to be provided to the regular expression generator in a more targeted manner, and provides a "context" for the regular expression generator so that it can generate a regular expression that avoids false positives. In response to a user selecting a character sequence via a user interface, the regular expression generator can generate and display a regular expression. For example, when a user highlights a first character sequence, the regular expression generator can generate and display a list of matches that match the first character sequence and other similar character sequences (e.g., matches that match the user's intent for the matching sequence). Figure 1 When the user highlights a second character sequence, the regular expression generator can generate an updated regular expression that includes the first and second character sequences. Then, when the user highlights a third character sequence (e.g., within the first or second sequence), the regular expression generator can update the regular expression again, and so on.

[0010] According to additional aspects described herein, regular expressions can be generated based on the longest common subsequence from one or more input sequence examples, and characters that only appear in some examples can also be processed. In order to process characters that only appear in some input examples, spans can be defined, wherein the minimum and maximum occurrences of regular expression codes are tracked. In the case where spans may not exist in all given input examples, the minimum occurrences can be set to zero. These minimum and maximum numbers can then be mapped to regular expression heavy grammars. The longest common subsequence (LCS) algorithm can be run on the spans of characters obtained from the input examples, including "optional" spans (e.g., minimum length is zero) that do not appear in each input example. As discussed below, continuous spans can be merged during the execution of the LCS algorithm. In this case, when the extra optional spans carried no longer appear continuously, the LCS algorithm can also be recursively run on these optional spans.

[0011] Other aspects described herein relate to combinatorial search, wherein the LCS algorithm executed by the regular expression generator can be run multiple times to generate a "correct" regular expression (e.g., a regular expression that correctly matches all given positive examples and properly excludes all given negative examples), and / or generates multiple correct regular expressions, from which the most desired or optimal regular expression can be selected. In some embodiments, the LCS algorithm can usually be executed from right to left on the input example to generate a regular expression. However, for the purpose of comparison and in order to find alternative regular expressions, the LCS algorithm can be executed backward (e.g., in a left-to-right direction) on the input example, respectively. For example, the example character sequence received as user input can be reversed before they run the LCS algorithm, and then the result from the LCS algorithm can be reversed back (including the original text fragment). In addition, in some embodiments, the LCS algorithm can be run multiple times by the regular expression generator, with the usual character sequence order and the reverse order, anchored at the beginning of the line, anchored at the end of the line, and not anchored at the beginning or end of the line. Thus, in some cases, the LCS algorithm may be executed at least six times like this, and the shortest successful regular expression may be selected from these executions. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 is a block diagram illustrating components of an exemplary distributed system for generating regular expressions in which various embodiments may be implemented.

[0013] Figure 2 is a flow chart illustrating a process of generating a regular expression based on input received via a user interface, according to one or more embodiments described herein.

[0014] Figure 3 is a flow chart illustrating a process of generating a regular expression using a longest common subsequence (LCS) algorithm on a regular expression code set according to one or more embodiments described herein.

[0015] 4 is an example diagram of generating a regular expression using a longest common subsequence (LCS) algorithm on a regular expression code set based on two character sequence examples according to one or more embodiments described herein.

[0016] Figure 5 is a flow chart illustrating a process of generating a regular expression using a longest common subsequence (LCS) algorithm on a larger set of regular expression codes according to one or more embodiments described herein.

[0017] Figure 6An example diagram of generating a regular expression using the Longest Common Subsequence (LCS) algorithm on a regular expression code set based on five character sequence examples according to one or more embodiments described herein.

[0018] Figure 7 is a flow chart illustrating a process for determining an execution order for a Longest Common Subsequence (LCS) algorithm on a larger set of regular expression codes, according to one or more embodiments described herein.

[0019] Fig. 8A and 8B According to one or more embodiments described herein, a fully connected graph and a minimum spanning tree representation of the fully connected graph are shown for determining an execution order for a longest common subsequence (LCS) algorithm on a larger set of regular expression codes.

[0020] Fig. 9 is a flow chart illustrating a process of generating a regular expression based on positive and negative character sequence examples according to one or more embodiments described herein.

[0021] Fig. 10A and 10B is an example user interface screen illustrating generating a regular expression based on positive and negative character sequence examples according to one or more embodiments described herein.

[0022] Fig.11 is a flow chart illustrating a process for generating a regular expression based on user data selections received within a user interface, according to one or more embodiments described herein.

[0023] Fig.12 A flow chart illustrating a method for generating a regular expression and extracting data based on capture groups via user data selection received within a user interface, according to one or more embodiments described herein.

[0024] Fig.13 is an example user interface screen illustrating a display of tabular data according to one or more embodiments described herein.

[0025] Fig.14 and 15 is an example user interface screen illustrating the generation of regular expressions and capture groups based on data selections from a tabular display according to one or more embodiments described herein.

[0026] Fig.16A and 16B is an example user interface screen illustrating generating a regular expression based on positive and negative example selections from a tabular display according to one or more embodiments described herein.

[0027] Fig.17 is another example user interface screen illustrating generating regular expressions and capture groups based on data selections from a tabular display according to one or more embodiments described herein.

[0028] Fig.18 is a flow chart illustrating a process of generating a regular expression including an optional span using a longest common subsequence (LCS) algorithm according to one or more embodiments described herein.

[0029] Fig.19 is an example graph of generating a regular expression including an optional span using a longest common subsequence (LCS) algorithm according to one or more embodiments described herein.

[0030] Fig. 20 is a flow chart illustrating a process for generating a regular expression based on a combinatorial execution of a Longest Common Subsequence (LCS) algorithm, according to one or more embodiments described herein.

[0031] Fig.21 is a block diagram illustrating components of an exemplary distributed system in which various embodiments of the invention may be implemented.

[0032] Fig. 22 is a block diagram illustrating components of a system environment that may provide services provided by embodiments of the present invention as cloud services.

[0033] Fig.23 is a block diagram illustrating an exemplary computer system upon which embodiments of the invention may be implemented. DETAILED DESCRIPTION

[0034] In the following description, for the purpose of explanation, many specific details are presented in order to provide a thorough understanding of various embodiments of the present invention. However, it is obvious to those skilled in the art that embodiments of the present invention can be implemented without these specific details. In other cases, well-known structures and devices are shown in block diagram form.

[0035] The following description provides only exemplary embodiments and is not intended to limit the scope, applicability or configuration of the present disclosure. On the contrary, the following description of the exemplary embodiments will provide a description of the implementation for implementing the exemplary embodiments to those skilled in the art. It should be understood that various changes may be made in the functions and arrangements of the elements without departing from the spirit and scope of the present invention as set forth in the appended claims.

[0036] Specific details are given in the following description to provide a thorough understanding of the embodiments. However, it will be appreciated by those of ordinary skill in the art that the embodiments may be implemented without these specific details. For example, circuits, systems, networks, processes, and other components may be shown as components in block diagram form so as not to obfuscate the embodiments in unnecessary detail. In other cases, in order to avoid obfuscating the embodiments, well-known circuits, processes, algorithms, structures, and techniques may be shown without unnecessary detail.

[0037] It is also noted that various embodiments may be described as a process, which is depicted as a flow chart, a flowchart, a data flow diagram, a structure diagram, or a block diagram. Although a flow chart may describe operations as a sequential process, many operations may be performed in parallel or concurrently. In addition, the order of the operations may be rearranged. A process terminates when its operations are completed, but there may be other steps not included in the figure. A process may correspond to a method, a function, a process, a subroutine, a subprogram, etc. When a process corresponds to a function, its termination may correspond to the function returning to the calling function or the main function.

[0038] The term "computer-readable medium" includes, but is not limited to, non-transient media such as portable or fixed storage devices, optical storage devices, and various other media capable of storing, containing or carrying instructions and / or data. A code segment or computer executable instruction can represent any combination of a process, function, subroutine, program, routine, subroutine, module, software package, class or instruction, data structure or program statement. A code segment can be coupled to another code segment or hardware circuit by passing and / or receiving information, data, independent variables, parameters or memory contents. Information, independent variables, parameters, data, etc. can be delivered, forwarded or transmitted via any suitable means including memory sharing, message passing, token passing, network transmission, etc.

[0039] In addition, the embodiments may be implemented by hardware, software, firmware, middleware, microcode, hardware description language, or any combination thereof. When implemented in software, firmware, middleware, or microcode, program code or code segments for performing the necessary tasks may be stored in a machine-readable medium. A processor may perform the necessary tasks.

[0040] Various techniques (e.g., methods, systems, non-transitory computer-readable storage storing a plurality of instructions executable by one or more processors, etc.) for generating regular expressions corresponding to patterns identified in one or more input data examples are described herein. In certain embodiments, in response to receiving a selection of input data, one or more patterns in the input data are automatically identified, and a regular expression (or "regex" for short) can be automatically and efficiently generated to represent the identified patterns. Such patterns can be based on character sequences (e.g., sequences of letters, numbers, spaces, punctuation marks, symbols, etc.). Various embodiments are described herein, including methods, systems, non-transitory computer-readable storage media storing programs, codes, or instructions executable by one or more processors, etc.

[0041] In some embodiments, a symbolic wildcard matching language can be used to compose a regular expression to match a string and / or extract information from a string provided as input. For example, a first example regular expression [A-Za-z]{3}\d? \d,\d\d\d\d can match certain dates (e.g., April 3, 2018), and a second example regular expression [A-Za-z]{3}\d? \d,(\d\d\d\d) can be used to extract the year from the matching date. The input data received by the regular expression generator system may include, for example, one or more "positive" data examples, and / or one or more "negative" data examples. As used herein, a positive example may refer to a character sequence received as input to be matched with a regular expression generated based on the input. In contrast, a negative example may refer to an input character sequence that is not matched with a regular expression generated based on the input.

[0042] Many technical advantages can be achieved within the various embodiments and examples described herein. For example, certain techniques described in the present disclosure can improve the speed and efficiency of the regular expression generation process (e.g., regex solutions can be generated in less than a second, and the user interface can be suitable for interactive real-time use). The various techniques described herein can also be deterministic, can require no training data, can produce solutions without requiring any initial regular expression input, and can be fully automated (e.g., regular expressions are generated without requiring any human intervention). In addition, the various techniques described herein do not require restrictions on the types of data inputs that can be effectively processed, and such techniques can improve the human readability of the resulting regular expressions.

[0043] Certain embodiments described herein include one or more executions of the longest common subsequence (LCS) algorithm. The LCS algorithm can be used as a difference engine (e.g., the engine behind the Unix "diff" utility) in certain contexts, which is configured to determine and display the differences between two text files. In some embodiments, input data (e.g., strings and other character sequences) can be converted into abstract tokens, which can then be provided as input to the LCS algorithm. Such abstract tokens can be, for example, tokens based on regular expression codes (e.g., Loogle codes or other character class codes) representing regular expression character classes. Various different examples of such codes are possible, and may be referred to herein as "regular expression codes" or "intermediate regular expression codes" (IRECs). For example, the input character sequence "May 3" can be converted into the IREC code "LLLZN", after which the tokenized string and other tokenized strings can be provided to the LCS algorithm. In some embodiments, the IREC (e.g., regular expression code) that the input character sequence does not have in common can appear as an optional (e.g., an optional span) in the final generated regular expression. In some embodiments, the regular expression code can be a category code based on the UNICODE category code shown in https: / / www.regular-expressions.info / unicode.html#category. For example, code L can represent letters, code N can represent numbers, code Z can represent spaces, code S can represent symbols, code P can represent punctuation, and so on. For example, code L can correspond to Unicode\p{L}, and code N can correspond to Unicode\p{N}. This allows a one-to-one mapping from LCS output to regular expressions (for example, \pN\pN\pZ\pL\pL can match "10am"), which may be beneficial to human readability. In addition, these different categories may be disjoint or mutually exclusive. That is, in this example, categories L, N, Z, P, and S can be disjoint so that there may be no overlap between members of the categories.

[0044] Additional technical advantages can be achieved in various embodiments, including more efficient generation of regular expressions based on the use of regular expression codes (e.g., category codes), spans, etc. By using such codes, computing resources do not need to be wasted when the LCS algorithm successfully identifies all or substantially all characters in an input string as different. Other technical advantages provided by various embodiments of the present invention include improved readability of generated regular expressions, support for positive and negative examples as input data, and providing various advantageous user interface features (e.g., allowing a user to highlight text fragments within a larger character sequence or data unit for extraction).

[0045] I. General Overview

[0046] Various embodiments disclosed herein relate to the generation of regular expressions. In some embodiments, a data processing system configured as a regular expression generator can generate a regular expression by identifying the longest common subsequence (LCS) shared by a set of different regular expression codes (e.g., category codes). Each regular expression code set can be converted from a character sequence received as input data through a user interface. In the technical advantages described herein, abstracting input data into intermediate codes (e.g., regular expression codes, spans, etc.) can achieve efficient generation of regular expressions using very little input data.

[0047] Figure 11 is a block diagram showing components of an exemplary distributed system for generating regular expressions, in which various embodiments can be implemented. As shown in this example, a client device 120 can communicate with a regular expression generator server 110 (or a regular expression generator), and interact with a user interface to retrieve and display table data, and generate regular expressions based on the selection of input data (e.g., examples) via a user interface. In some embodiments, the client device 120 can communicate with the regular expression generator 110 via a client web browser 121 and / or a client-side regular expression application 122 (e.g., a client application that receives / consumes regular expressions generated by the server 110). Within the regular expression generator 110, requests from the client device 120 can be received at a network interface through various communication networks and processed by an application programming interface (API) such as a REST API 112. A user interface data model generator 114 component with a regular expression generator 110 can provide server-side programming components and logic to generate and present various user interface features described herein. Such features may include the following functionality: allowing a user to retrieve and display tabular data from the data repository 130, select an input data example to initiate the generation of a regular expression, and modify and / or extract data based on the generated regular expression. In this example, the regular expression generator component 116 may be implemented to generate a regular expression, including converting an input character sequence into a regular expression code and / or span, executing an algorithm (e.g., an LCS algorithm) on the input data, and generating / simplifying the regular expression. The regular expression generated by the regular expression generator 116 may be transmitted to the client device 120 by the REST service 112, where the Javascript code on the client browser 121 (or a corresponding client application component 122) may then apply the regular expression to each cell in the electronic table column presented in the browser. In other cases, a separate regular expression engine component may be implemented on the server side to compare the generated regular expression with the tabular data displayed on the user interface and / or other data stored in the data repository 130 to identify matching data / non-matching data on the server side. In various embodiments, matching / non-matching data may be automatically selected (e.g., highlighted) within the user interface and may be selected for extraction, modification, deletion, etc. Based on the generation of the regular expression, any data extracted or modified via the user interface may be stored in one or more data repositories 130. Additionally, in some embodiments, the generated regular expressions (and / or corresponding inputs to the LCS algorithm) may be stored in the regular expression library 135 for future retrieval and use. In some embodiments, the generated regular expressions do not actually need to be stored in a "library", but may be incorporated into a "transformation script".For example, as described in more detail in U.S. Pat. No. 10,210,246 (which is incorporated herein by reference for all purposes), such a transformation script may include a program, code, or instruction executable by one or more processing units to transform received data. Other possible examples of transformation script actions may include "rename column", "uppercase column data", or "infer gender from name and create a new column with gender", etc.

[0048] Figure 2 2 is a flowchart showing a process of generating a regular expression based on input received via a user interface according to one or more embodiments described herein. In step 201, a regular expression generator 110 may receive a request from a client device 120 to access a regular expression generator user interface and view specific data via the user interface. The request in step 201 may be received via a REST API 112 and / or a web server, an authentication server, etc., and the user's request may be parsed and authenticated. For example, a user within an enterprise or organization may access the regular expression generator 110 to analyze and / or modify transaction data, customer data, performance data, forecast data, and / or any other category of data that may be stored in the organization's data repository 130. In step 202, the regular expression generator 110 may retrieve and display the requested data via a user interface that supports generating a regular expression based on selected input data. Various embodiments and examples of such a user interface are described in detail below.

[0049] In step 203, the user may select one or more input character sequences from the data displayed in the user interface provided by the regular expression generator 110. In some embodiments, the data may be displayed in a tabular form within the user interface, including labeled columns with specific data types and / or data categories. In this case, the selection of input data in step 203 may correspond to the user selecting a data cell, or selecting (e.g., highlighting) a single text fragment within a data cell. However, in other embodiments, the regular expression generator 110 may support retrieval and display of semi-structured and unstructured data via a user interface, and the user may select input data for regular expression generation by selecting a character sequence from the semi-structured or unstructured data. As described in the example below, the user selecting an input character sequence from the displayed tabular data is only an example use case. In other examples, the user (e.g., a software developer or advanced user may attempt to write a regular expression for a Linux command line tool such as grep, sed, or awk) may enter examples from scratch instead of extracting them from a spreadsheet.

[0050] In step 204, the regular expression generator 110 may generate one or more regular expressions based on the input data selected by the user in step 203. In step 205, the regular expression generator 110 may update the user interface, for example, to display the generated regular expressions and / or highlight matching / unmatching data within the displayed data. In step 206 (which may be optional in some embodiments), the user interface may support functionality that allows the user to modify the underlying data based on the generated regular expressions. For example, the user interface may support features that allow the user to filter, modify, delete, or extract specific data fields from the tabular data based on whether these fields match the regular expressions. Filtering or modifying the data may include modifying the underlying data stored in the repository 130, and in some cases, the extracted data may be stored in the repository 130 as new columns and / or new tables.

[0051] Although these steps illustrate a general and high-level overview of exemplary user interactions with the user interface of the regular expression generator 110, various additional features and functionality may be supported in other embodiments. For example, in some embodiments, a regular expression code (or category code) may be associated with a minimum number of occurrences of the code. Additionally or alternatively, a regular expression code may be associated with a maximum number of occurrences of the code. As an example, a set of regular expression codes may include the code L<0, 1> to indicate that a particular portion of an LCS includes a letter at least zero times and at most once.

[0052] In addition, in some embodiments, the input data may include three or more character sequences. In such an embodiment, a technique may be used to determine the order in which the LCS algorithm is executed for three or more character sequences, so that the resulting regular expression may be generated in a high-performance manner to avoid the exponential increase in the running time caused by three or more input character sequences. In contrast, the regular expression generator 110 may execute the LCS algorithm for two character sequences at a time, and the order for selecting the pair of character sequences may be determined based on the graph. For example, a fully connected graph may indicate that the first execution of the LCS algorithm (e.g., LCS1) should be executed for sequence 1 and sequence 3, and then the second execution of the LCS algorithm (e.g., LCS2) should be executed for LCS1 and sequence 2, and so on. The graph may be a fully connected graph, with nodes representing character sequences, and edges connecting nodes to represent the length of the LCS shared by the connected nodes. Each node in the graph may be connected to each other node in the graph, and the order for selecting the character sequence may be determined by performing a depth-first traversal of the minimum spanning tree for the graph.

[0053] In other embodiments, input data may be provided in a variety of different ways via a user interface. For example, the input data may indicate a first user selection of one or more characters within a second user selection of a character set. For example, a user may highlight a character in a previously highlighted character set. Therefore, the second user selection may provide a context for the first user selection, which may enable the input data to be provided to the regular expression generator 110 in a more targeted manner. In some embodiments, the regular expression generator 110 may generate and display a regular expression in near real time in response to each user selection. For example, when a user highlights a first character range, the regular expression generator 110 may display a regular expression representing the first character range. Then, when a user highlights a second character range within the first character range, the regular expression generator 110 may update the displayed regular expression.

[0054] In addition, in some embodiments, the regular expression generator 110 can generate a regular expression based on an input including positive and negative examples. As described above, a positive example can refer to a character sequence to be included by the regular expression, while a negative example can refer to a character sequence that will not be included by the regular expression. In this case, the regular expression generator 110 can identify the shortest subsequence of one or more characters that distinguish the positive example from the negative example at a specific position. The shortest subsequence can then be hard-coded in the regular expression generated by the regular expression generator 110. In various examples, the shortest subsequence can be included in a prefix / suffix portion or an intermediate span within a negative example.

[0055] Other examples for automatically generating regular expressions according to certain embodiments are described below. These examples may correspond to Figure 2 Various specific possible implementations of the general technology in the present invention can be implemented by software (e.g., code, instructions, programs, etc.) executed by one or more processing units (e.g., processors, cores) of the corresponding system, hardware, or a combination thereof. The software can be stored on a non-volatile storage medium (e.g., on a storage device). The other examples described below are intended to be illustrative and non-limiting. Although these examples describe various processing steps that occur in a specific sequence or order, this is not intended to be limiting. In some alternative embodiments, these steps can be performed in a different order, or some steps can also be performed in parallel.

[0056] In some examples, the user input received via the user interface (e.g., step 203) may include one or more "positive examples" that the regular expression output is to match, and zero or more "negative examples" that the regular expression output does not match. Optionally, one or more positive examples may be highlighted to select characters of a specific range (or subsequence). In some cases, in step 204, the positive examples received via the user interface may be converted into spans of regular expression codes (e.g., character category codes such as Unicode category codes). For each positive example, a series of spans may be generated. In some embodiments, a graph may be created in which each vertex corresponds to one of the sequences of spans, and the edge weight is equal to the length of the output of the LCS algorithm executed on the two span sequences corresponding to the endpoints of the edge. A minimum spanning tree may be determined for the graph. For example, in some embodiments, the Prim algorithm may be used to obtain a minimum spanning tree. A depth-first traversal may be performed on the minimum spanning tree to determine the traversal order, after which the LCS algorithm may be performed on the first two elements of the traversal. Then, by executing the LCS algorithm again on the output of the previous LCS iteration and the next current traversal element, each additional element of the traversal can be merged into the current LCS output in order, one by one. Then, the final output of the LCS algorithm (which can be a span sequence) can be converted to a regular expression. In some embodiments, the conversion can be a one-to-one conversion, and some optional modifications described herein may not correspond to a one-to-one conversion. Finally, in step 203, the resulting regular expression can be tested for all positive and negative examples received via the user interface. If any test fails, the above process can be repeated using all failed positive examples and any negative examples.

[0057] II. Regular expression generation using the longest common subsequence algorithm on the regular expression code

[0058] As noted above, certain aspects described herein relate to generating regular expressions based on computing the longest common subsequence (LCS) shared by a set of different regular expression codes corresponding to input data.

[0059] Figure 3 1 is a flowchart illustrating a process of generating a regular expression using an LCS algorithm on a regular expression code set according to one or more embodiments described herein. In step 301, the regular expression generator 110 may receive one or more character sequences as input data. As described above, in some examples, the input data may correspond to positive example data selected from tabular data displayed in a user interface, but it should be understood that in some embodiments, the user interface is optional, and in various examples, the input data may correspond to any character sequence received from any other communication channel (e.g., non-user interface).

[0060] In step 302, each character sequence received in step 301 can be converted into a corresponding regular expression code. In various embodiments, the regular expression code can be a Loogle code, a Unicode category code, or any other character class code representing a regular expression character class. For example, the input character sequence "May 3" can be converted into the Loogle code "LLLZN". In some embodiments, the regular expression code can be a category code based on the UNICODE category code shown in https: / / www.regular-expressions.info / unicode.html#category. For example, code L can represent letters, code N can represent numbers, code Z can represent spaces, code S can represent symbols, code P can represent punctuation marks, and so on. For example, code L can correspond to Unicode\p{L}, and code N can correspond to Unicode\p{N}.

[0061] In step 303, the longest common subsequence can be determined based on the regular expression code set generated in step 302. In some embodiments, the LCS algorithm can be executed using two regular expression code sets as input. Various different features of the execution of the LCS algorithm (e.g., processing direction, anchoring, pushing space, merging low cardinality spans, aligning common tokens, etc.) can be used in different embodiments. In step 304, a regular expression can be generated based on the output of the LCS algorithm. In some cases, step 304 can include capturing the output of the LCS algorithm in the regular expression code and converting the regular expression code into a regular expression. In step 305, the regular expression can be simplified and output, for example, by displaying the regular expression to the user via a user interface.

[0062] FIG. 4 is an example diagram for generating a regular expression using the longest common subsequence (LCS) algorithm on a regular expression code set based on two character sequence examples. Figure 3. As shown in FIG. 4 , the regular expression in this example is generated based on two input strings: “iPhone 5” and “iPhone X”. Each sequence in this example can be converted into a corresponding set of regular expression codes. Thus, iPhone 5 can be converted into “LLLLLLZN” and iPhone X can be converted into “LLLLLLZL”. As shown in FIG. 4 , these category codes are then provided as input to the LCS algorithm, which determines that both IRECs (or category codes) include six Ls and one Z. Category codes excluded from LCS can be expressed as optional and / or alternative. Thus, a regular expression containing two character sequences can be expressed as: \pL{6}\pZ\pN? \pL? . In this example, the regular expression includes Unicode category codes (e.g., \pL represents letters, \pZ represents spaces, and \pN represents numbers). The curly braces containing the number 6 indicate six instances of letters, and the question mark indicates that the number / letter at the end is optional. Finally, the regular expression generator may perform a simplification process during which the regular expression is simplified by reinserting the generic text fragment "iphone" back into the final regular expression, replacing the broader "\pL{6}\" portion of the regular expression.

[0063] As shown in this example, the input string received by the regular expression generator 110 can be converted into a "regular expression code" representing a generalized category of a regular expression (also referred to as a "category code"), and the LCS algorithm can be run on these regular expression codes. In some embodiments, Unicode category codes can be used for regular expression codes. For example, an input text string can be converted into a code representing a regex Unicode generalized category (e.g., \pL represents letters, \pP represents punctuation, etc.). Figure 3 This method, as shown in Figure 4, may be referred to as an indirect method. However, in other embodiments, a direct method may be used, in which the LCS algorithm operates directly on the character sequence received as input.

[0064] In some embodiments, the indirect method can provide additional technical advantages because it does not require a large amount of training data and can generate valid regular expressions with relatively few input examples. This is because the indirect method uses heuristics to reduce uncertainty in regular expression generation and eliminate potential false positives and false negatives. For example, when generating regular expressions based on the input strings "May 3" and "Apr 4", the direct method may need at least one example per month to generate a valid regular expression that matches the date pattern. Relying only on these two examples, the direct method may generate a regex of "[AM][ap][yr]

[13] 1?". ​​In contrast, based on the Unicode broad category, the indirect method can generate a more efficient regular expression "\pL{3}\d{1,2}". In addition, as described above, one of the technical advantages described herein includes efficiently generating regular expressions using very little input data (potentially even from a single example). For example, with respect to generating a regular expression from a single example "am", a heuristic can determine whether to generate "am" or "\pL\pL" for the regular expression. Both are arguably correct, but so programmed heuristics may implement user preferences and / or criteria to determine how to generate the best regular expression (e.g., should it also match "pm").

[0065] Additionally, the indirect method can also simplify the generated regular expression "\pL{3}\d{1,2}" to "[A-Za-z]{3}\d{1,2}" to make it more human readable. This may be beneficial in some embodiments, such as when outputting to non-complex regular expression users who may not be familiar with Unicode expressions for regular expressions.

[0066] In addition, in some embodiments, when executing the LCS algorithm, instead of treating each character independently, sequential and equal regular expression codes can be converted into span data structures (which can also be referred to as spans). In some cases, a span can include a representation of a single regular expression code (e.g., a Unicode generalized category code), as well as a repeat count range (e.g., a minimum number and / or a maximum number). The conversion from regular expression codes to spans can facilitate some of the various additional features described below, such as identifying alternatives (e.g., disjunctions), and can also facilitate merging adjacent optional spans to further simplify the generated regular expression.

[0067] As described above, the LCS algorithm can be configured to store and preserve the underlying text fragments within the input character sequence, which may be inserted back into the final regular expression, such as the string "iphone" in Figure 4. By tracking the text fragment that originally caused the category code assigned to the span, such an embodiment can allow literal text (e.g., am and pm) to be included directly in the generated regular expression, which can reduce false positives and make the regular expression output more human readable.

[0068] III. Generate regular expressions using the longest common subsequence algorithm on regular expression code combinations

[0069] Additional aspects described herein relate to generating regular expressions based on input data including three or more strings (e.g., three or more independent character sequences). When three or more strings are identified as input data, regular expression generator 110 can use performance optimization features, wherein the sequence executed for the LCS algorithm determines the best order. As described below, the performance optimization features for more than two strings can include building a graph, which has vertices corresponding to each string, and edge lengths / weights of the size of the LCS output based on each string and every other string. Then, these edge weights can be used to derive a minimum spanning tree, and a depth-first traversal can be performed to determine the order of the input strings. Finally, a series of LCS algorithms can be completed using a determined input string order.

[0070] Figure 5 is a flowchart showing the process of generating a regular expression using the longest common subsequence (LCS) algorithm on a larger set of regular expression codes (e.g., three or more character sequences). Therefore, steps 502-505 in this example may correspond to the steps described above in Figure 3 303 discussed in . However, since the present example involves generating a regular expression based on three or more input character sequences, the LCS algorithm may be executed multiple times. For example, to avoid an exponential increase in the running time for three or more input strings, the LCS algorithm may be executed multiple times, with each execution being performed on only two input strings. For example, the regular expression generator 110 may perform an initial execution of the LCS algorithm on two strings (e.g., two input character sequences or two converted regular expression codes), then may perform a second execution of the LCS algorithm on the output of the first LCS algorithm and a third string, then may perform a third execution of the LCS algorithm on the output of the second LCS algorithm and a fourth string, and so on.

[0071] In order to improve and / or optimize the performance of such embodiments, it may be desirable to determine the best order for input strings (e.g., input character sequences or regular expression codes) to execute the LCS algorithm sequence. For example, obtaining a good order for the input strings may affect the readability of the generated regular expressions, such as by minimizing the number of optional spans. In order to maintain the simplicity of the generated regex, the additional string of LCS to the current regex should preferably already be somewhat similar to the current regex (the intermediate result of LCS on the string already seen).

[0072] Therefore, in step 501, multiple (e.g., 3 or more) input character sequences are converted into regular expression codes. In step 502, the order in which the regular expression codes are processed using the LCS algorithm is determined. Figure 7 The order determination in step 502 is discussed in more detail. In step 503, either the first two regular expression codes of the determined order are selected (for the first iteration of step 503), or the next regular expression code of the determined order is selected (for subsequent iterations of step 503). In step 504, the LCS algorithm is executed on two input strings corresponding to the regular expression code format. For the first iteration of step 504, the LCS algorithm is executed on the first two regular expression codes of the determined order, and for subsequent iterations of step 504, the LCS algorithm is executed on the next regular expression code of the determined order and the output of the previous LCS algorithm (which may also be a regular expression code of the same format). In step 505, the regular expression generator 110 determines whether there are additional regular expression codes in the determined order that have not been provided as input to the LCS algorithm. If so, the process returns to step 503 for another execution of the LCS algorithm. If not, in step 506, a regular expression is generated based on the output of the last execution of the LCS algorithm.

[0073] Figure 6 is an example diagram of generating a regular expression based on five input character sequence examples. In this example, each input character sequence is converted into a regular expression code, and then the LCS algorithm is repeatedly executed based on the determined order of the regular expression code. Therefore, Figure 6 Shows the application of the above Figure 5 An example of the process discussed in , in which the determined order of the five regular expression codes is code #1 to code #5, and each code is input to the LCS algorithm in the determined order to generate a regular expression output. The final regular expression output (Reg Ex #4) corresponds to the final regular expression generated based on all five input character sequences.

[0074] Figure 7is a flow chart illustrating a process for determining the execution order of the longest common subsequence (LCS) algorithm on a larger set of regular expression codes (e.g., three or more). Thus, as shown in this example, steps 701-704 may correspond to the order determination in step 502 discussed above. In step 701, the LCS algorithm may be run on each unique regular expression code pair corresponding to the input data, and the resulting output LCS may be stored for each execution. Thus, for k input data, this may represent all (k(k-1)) / 2 possible string pairings to be run on the LCS algorithm, or in some embodiments, k(k-1). For example, if k=3 input character sequences are received, the LCS algorithm may be run three times in step 701; if k=4 input character sequences are received, the LCS algorithm may be run six times in step 701; if k=5 input character sequences are received, the LCS algorithm may be run 10 times in step 701, and so on. In step 702, a fully connected graph may be constructed from k nodes representing the strings, where the edge weight of (k(k-1)) / 2 edges is the length of the original LCS output between two nodes. In step 703, a minimum spanning tree may be derived from the fully connected graph in step 702. In step 704, a depth-first traversal may be performed on the minimum spanning tree. The output of the traversal may correspond to the order in which the regular expression code will be input into the LCS algorithm execution sequence.

[0075] Brief reference Fig. 8A and 8B , Figure 5 An example of a fully connected graph generated based on a received k=5 input character sequence is shown in Figure 8B The minimum spanning tree representation for a fully connected graph is shown in .

[0076] In some embodiments, Figure 5-8B The methods described in may provide additional technical advantages with respect to performance. For example, some conventional implementations of the LCS algorithm may exhibit O(n 2 ), where n is the length of the string. Extending this implementation to k strings instead of just 2 would likely result in exponential runtime performance, O(n k ), because the LCS algorithm may be required to search the k-dimensional space. Such traditional implementations of the LCS algorithm may not perform well or be insufficient for real-time online user experience.

[0077] As described above, the LCS algorithm may be executed (k(k-1)) / 2 times, where sometimes the copies are exactly the same as those that have been seen before, because the LCS algorithm may be executed when the original input examples from the user have been converted into regex class codes. Therefore, memoization can be achieved in some cases, where a cache can be used to map previously seen LCS problems to previously worked LCS solutions.

[0078] IV. Regular Expression Generation Based on Positive and Negative Pattern Matching Examples

[0079] Additional aspects described herein relate to generating regular expressions based on input data corresponding to both positive examples and negative examples. As described above, positive examples may refer to input data character sequences that are designated as example strings that should be matched with the regular expression that the regular expression generator will generate. On the contrary, negative examples may refer to input data character sequences that are designated as example strings that should not be matched with the regular expression that the regular expression generator will generate. As described below, in some embodiments, regular expression generator 110 may be configured to identify the position that distinguishes positive examples from negative examples and the shortest subsequence of characters at that position. Then, the shortest subsequence may be hard-coded into the generated regular expression so that the positive example will match the regular expression, and the negative example will be excluded (e.g., will not match) by the regular expression.

[0080] Fig. 9 9 is a flow chart illustrating a process of generating a regular expression based on positive and negative character sequence examples. In step 901, the regular expression generator 110 may receive one or more input data character sequences corresponding to positive examples. In step 902, the regular expression generator 110 may generate a regular expression based on the received positive examples. Thus, steps 901-902 may include the steps discussed above. Figure 3 or Figure 5 to generate a regular expression based on an input data character sequence.

[0081] In step 903, the regular expression generator 110 may receive an additional input data character sequence corresponding to a negative example. Therefore, the negative example is specifically specified so that it does not match the regular expression generated in step 902. In some embodiments, the negative example received in step 903 may be tested first against the regular expression generated in step 902, and if it is determined that the negative example does not match the regular expression, no further action is taken. However, in this example, it may be assumed that at least one of the negative examples received in step 903 matches the regular expression generated in step 902. Therefore, in step 904, a disambiguation position may be determined within the regular expression generated in step 902. In some embodiments, the disambiguation position may be selected as a prefix position (e.g., at the beginning of the regular expression) or a suffix position (e.g., at the end of the regular expression). For example, the regular expression generator 110 may determine a first number of characters required at the prefix to distinguish a positive example from a negative example, and a second number of characters required at the suffix to distinguish a positive example from a negative example. The regular expression generator 110 can then select a suffix or prefix based on the shortest number of replacement characters required. In some cases, using a prefix as a disambiguation position may be preferred (e.g., emphasized) for readability purposes. In other examples, the disambiguation position may be a mid-span position that does not correspond to a prefix or suffix of the regular expression.

[0082] In step 905, the regular expression generator 110 may determine a replacement sequence for the custom character class that, when inserted into the regular expression at the determined position, can distinguish between positive examples and negative examples. In some embodiments, the regular expression generator 110 may retrieve a text fragment from each of the positive and negative examples corresponding to the disambiguation position (or replacement position) in step 905, and then use the text fragment to determine a discriminator that will be used as a replacement sequence to distinguish between positive examples and negative examples. In addition, the discriminator replacement sequence determined in step 905 may include multiple different replacement sequences for the custom character class, which may be replaced at the same position or different positions within the regular expression.

[0083] As described above, in some cases, the determination of the replacement sequence in step 905 can be performed in conjunction with the determination of the disambiguation position (or replacement position) in step 904. For example, the regular expression generator 110 can determine one or more replacement sequences that can distinguish positive and negative examples at the first possible replacement position. The regular expression generator 110 can also determine one or more other replacement sequences that can distinguish positive and negative examples at the second different possible replacement position. In this example, when selecting between different possible replacement positions and corresponding replacement sequences, the regular expression generator 110 can apply a heuristic formula to perform the selection based on one or more of the character size of the replacement position and the number and / or size of the corresponding replacement sequence. Finally, in step 906, the regular expression can be modified by inserting one or more determined replacement sequences into the determined position to replace the previous part of the regular expression. In some cases, after modifying the regular expression in step 906, the positive example and / or the negative example can be tested against the modified regular expression to confirm that the positive example matches and the negative example does not match the regular expression.

[0084] Fig. 10A and 10B is an example user interface screen showing the generation of regular expressions based on positive and negative character sequence examples. Fig. 10A and 10B The example shown in may correspond to performing the above-discussed Fig. 9 The user interface that is displayed during the process. Fig. 10A 4B , the user provides three positive examples of a data input character sequence 1001, and the regular expression generator 110 generates a regular expression 1002 that matches each positive example. Then, in FIG. 4B , the user provides a negative example 1004, and the regular expression generator 110 generates a modified regular expression 1005 that is based on the two current sets of positive examples 1003 and negative examples 1004.

[0085] As described above, in some embodiments, when both positive examples and negative examples are received, the regular expression generator 110 may identify a discriminator or the shortest subsequence of one or more characters that distinguishes the positive example from the negative example. The selected discriminator may be the shortest sequence (e.g., expressed in terms of a class code), and may be both positive and negative, such that the positive example will match and the negative example will not match. In some cases, the discriminator may correspond to a replacement subsequence, which may then be hard-coded into the regular expression in step 905. As an example, in "[AL][az]+", [AL] is a positive discriminator, assuming that it is applied to a street suffix, it will match (or allow) "Alley", "Avenue" and "Lane", but not (or allow) any other words. As another example, in "[BC][o][az]+", [BC][o] is a positive discriminator consisting of a sequence of two character classes, which matches "Boulevard" and "Court". As yet another example, in "[^A][az]+", [^A] can be a negative discriminator, which disallows "Alley" and "Avenue". In some cases, the algorithm correctly discriminates by generating a negative backtracking. For example, (?<!Av)[A-Za-z]+ will exclude "Avenue" but allow "Alley".

[0086] As another example, if the user provides positive examples "202-456-7800" and "313-678-8900" and negative examples "404-765-9876" and "515-987-6570", then in some embodiments, the regular expression generator 110 may generate the regular expression "\d\d\d-\d\d\d-\d\d00". That is, based on determining that a phone number ending with 00 distinguishes between positive examples and negative examples (e.g., assuming the goal is to match a regular expression for a business phone number), a replacement character subsequence may be identified for the suffix of the regular expression. This is an example of a negative example through a suffix (or more specifically, an example of accommodating a negative example by using a positive suffix), but various other embodiments may support replacement of prefixes, suffixes, or mid-span positions. In the example of mid-span position replacement, the character offset in the span may be tracked and may be split at the mid-span point.

[0087] To decide whether to use a prefix or a suffix, in some embodiments, a heuristic is used, where a Select the smallest score among all combinations of prefixes / suffixes:

[0088]

[0089] in,

[0090] k a = Number of characters to consider for disambiguating affixes (prefixes or suffixes)

[0091] |F p | = number of unique text fragments in the positive example required to disambiguate the affix

[0092] |F n | = number of unique text fragments in the negation example required to disambiguate the affix

[0093] |E p | = number of (complete) positive examples provided by the user

[0094] |E n | = number of (complete) negative examples provided by the user

[0095] In the example above, the heuristic is designed to favor shorter disambiguating text fragments over longer ones (e.g., thus multiplying k by a ). The heuristic is also designed to favor prefixes over suffixes (e.g., so the penalty for suffixes is 0.1), to improve readability. Finally, the heuristic is designed to favor disambiguating (e.g., replacing) longer prefixes or suffixes over disambiguating by using a larger number of string fragments (e.g., so the square of the number of string fragments to replace).

[0096] As described above, some embodiments may also support negative mid-span examples as well as negative backtracking examples and negative lookahead examples.

[0097] Once the prefix / suffix and k (the number of characters to disambiguate) are determined, the regular expression generator 110 can still determine how to represent the disambiguation in the generated regular expression. The generated regular expression can allow affixes (e.g., prefixes or suffixes) that look like positive examples, or exclude affixes that look like negative examples.

[0098]

[0099] If usePermissive is greater than zero, then content that looks like a positive example is allowed to pass by generating a regular expression that allows characters to be taken from the positive example one by one (for each character position). In other cases, the regular expression generator 110 can take an approach of not allowing content that looks like a negative example by generating a regular expression that does not allow characters to be taken from the negative example one by one (for each character position).

[0100] As another example, the regular expressions generated for the positive example 8am and the negative example 9pm may be \d[^p]m. This uses the caret syntax. In some cases, the regular expression generator 110 may be configured to favor shorter regular expressions, which are not only more readable to the user, but also more likely to be correct. The reasoning is that frequently occurring characters are more likely to occur again in the future, so emphasis should be placed on frequently occurring characters. If the unique character |F p | is less (fewer unique characters because the characters that do occur occur more frequently), then the bonus for this in the heuristic is to put it in the denominator.

[0101] Referring again to the usePermissive example heuristic above, if there is only one positive example from a user, it is not a big deal to determine a unique positive affix. Therefore, in this heuristic, low |E p |Punish (i.e., reward high |E in this heuristic) by placing it in the numerator p |).

[0102] Additionally, in some embodiments, negative examples may be based on backtracking and / or lookahead. For example, a user may provide a positive example of "323-1234" and a negative example of "202-754-9876", which would involve using regex backtracking syntax (?<!) to exclude phone numbers with area codes.

[0103] In some cases, negative examples may also be based on optional spans. For example, a user may provide positive examples of "ab" and "a2b" and a negative example of "a3b". In this case, the example implementation may fail because it may try to distinguish based only on the required span, and the "2" number is in the optional span. In this example, failure may refer to a situation where the generated regular expression matches all positive examples (correct) and also matches one or more negative examples (wrong). In this case, the user may be warned of the failure and may be provided with the option of manually fixing the generated regular expression and / or removing some negative examples via the user interface.

[0104] V. User Interface for Regular Expression Generation

[0105] Additional aspects described herein include several different features and functions related to regular expression generation within a graphical user interface. As described below, some of these features may include various options for user selection and highlighting of positive and negative examples, color coding for positive and negative examples, and multiple overlapping / nested highlights within a data cell.

[0106] Fig.11is a flow diagram illustrating a process for generating a regular expression based on user data selections received within a user interface. Fig.11 The example process in can correspond to any of the previously discussed examples of generating a regular expression based on an input data character sequence. However, Fig.11 A process is described with respect to a user interface that can be generated and displayed on a client device 120. In step 1101, in response to a request from a user via a user interface, the regular expression generator 110 can retrieve data (e.g., from a data repository 130) and present / display the data in a tabular form within a graphical user interface. Although tabular data is used in this example, it should be understood that tabular data need not be used or displayed in other examples. For example, in some cases, a user can directly type in raw data (rather than selecting data from a user interface). In addition, when data is presented via a user interface, the data need not be in a tabular form, but can be unstructured data (e.g., a document) or semi-structured (e.g., a spreadsheet of unformatted / unstructured data items such as tweets or posts). In various examples, the tabular data can correspond to transaction data, customer data, performance data, forecast data, and / or any other category of data that can be stored in a data repository 130 of an enterprise or other organization. In step 1102, a user's selection of input data can be received via a user interface. For example, the selected input data can correspond to an entire data unit selected by a user, or a subsequence of characters within a data unit. In step 1103, the regular expression generator 110 may generate a regular expression based on the input data (e.g., a data unit or a portion thereof) received in step 1102. In step 1104, the user interface may be updated in response to the generation of the regular expression. In some cases, the user interface may be simply updated to display the generated regular expression to the user, while in other cases, the user interface may be updated in various other ways discussed below. As shown in this example, the user may select a plurality of different input data character sequences via the user interface, and in response to each new input data received, the regular expression generator 110 may generate an updated regular expression containing the first and second (positive) examples of the character sequence. Then, when the user highlights the third character sequence (e.g., outside the two character sequences, or within the first or second character sequence), the regular expression generator 110 may update the regular expression again, and so on. In some embodiments, the regular expression generator 110 may execute the algorithm in real time (or near real time) so that a completely new regular expression is generated in response to each new keystroke or each new highlighted portion performed by the user.

[0107] Therefore, if Fig.11As shown, in response to the user's selection of a character sequence via a user interface, the regular expression generator 110 can generate and display a regular expression. For example, when the user highlights a first character sequence, the regular expression generator can generate and display a regular expression representing the first character sequence. When the user highlights a second character sequence, the regular expression generator can generate an updated regular expression containing the first and second character sequences. Then, when the user highlights a third character sequence (e.g., within the first or second sequence), the regular expression generator can update the regular expression again, and so on.

[0108] Fig.12 1 is another flow chart showing the process of generating a regular expression and extracting data based on a capture group via a user data selection received in a user interface. In step 1201, as discussed above in step 1101, the regular expression generator 110 can retrieve data (e.g., from a data repository 130) and present / display the data in a table form in a graphical user interface. In step 1202, the regular expression generator 110 can receive a user-highlighted selection of a text fragment in a specific data cell. In step 1203, the regular expression generator 110 can generate a regular expression based on a positive example of the selected data cell, and in step 1204, a regular expression capture group can be created based on the highlighted text fragment in the cell. In step 1205, the regular expression generator 110 can determine one or more additional cells in the displayed table data that match the generated regular expression, and in step 1206, the corresponding text fragments in the additional cells that match the generated regular expression can be extracted.

[0109] Thus, in addition to providing positive examples, the user may also select (e.g., via mouse text highlighting) a text fragment within any selected positive example. In response, the regular expression generator 110 may create a regular expression capture group to extract the text fragment from the example and corresponding fragments from all other matches in the text to which the regular expression is applied. Extracting text fragments from matched data units may also include deletions and modifications, and in some cases may be used to create new data columns out of existing columns of semi-structured or unstructured text.

[0110] Using the example of a user selecting a positive data example, and if the user highlights the year, the regular expression generator 110 can generate the regular expression (?:[AZ]{3}\s+\d\d,\s+|\d\d / \d\d)(\d\d\d\d). As shown in this example, the regular expression generator 110 has placed parentheses around the year, and has also converted the old parentheses around the month and day (for alternatives) to "non-capturing" groups by using the ?: regular expression syntax. In some embodiments, it may be necessary for the extraction / capture group to fall on a span boundary, and in such an embodiment, the regular expression generator 110 can take the highlighted character range as input and expand it to include the nearest anchor span boundary. However, in other examples, the user interface can support mid-span extraction / capture.

[0111] In some embodiments, the user interface can support input data from a user including a selection of a first character sequence within a second character sequence. For example, a user can highlight one or more characters within a larger previously highlighted character sequence, and the second user selection can provide context for the larger first user selection. Such embodiments can enable input data to be provided to the regular expression generator 110 in a more targeted manner.

[0112] In addition, in some examples, an operation can be initiated in response to a user selecting (e.g., highlighting text) in a user interface and a dialog box can be opened. In some cases, the dialog box can be a non-model dialog box, such as a floating toolbox window that does not prevent the user from interacting with the main screen. Depending on the main operation being performed by the user, the dialog box can also change in appearance and / or function. Therefore, in this case, the user does not need to search for other menu items after highlighting the selected text in order to initiate modification, extraction, etc. of the captured group text fragment. In addition, in some embodiments, a user interface provided for generating a regular expression may include three highlighting modes: nested automatic, nested manual, and single layer. In some cases, the default operation mode may be to identify the entire cell as a highlighted area, and the user may also highlight one or more additional subsequences within the highlighted cell. In other modes, the user may be allowed to manually specify two highlights within a data cell displayed in a table of data. In other modes, the user may be allowed to manually specify external highlights without internal highlights. These other modes may be more suitable for "semi-structured" data, for example, a data column consisting of tweets or other long strings (e.g., a browser "user agent" string). "Semi-structured" data refers to data that can be displayed in a table within a user interface, but the columns within the table consist of unstructured text.

[0113] In some such embodiments, internal and external selections (e.g., highlights) made by a user via a user interface may be distinguished by color coding. For example, an external highlight of an affirmative example may be displayed in a first text / background color combination, while an internal highlight of the affirmative example may be displayed in a different, contrasting text / background color combination.

[0114] As described above, the user can specify the selection of a capture group via selecting a character subsequence. The GUI can be used to facilitate user selection via highlighting (or other indication). Fig.13 An example is shown, in which an example user interface screen with a display of tabular data is shown. In this example, Fig.13 Depicted is highlighting within a column value, e.g., caused by a user dragging a mouse over one or more desired elements of the column value. Note that the "cell" in which the user highlighting is performed may exhibit a color change indicating selection of the column value. This color change may be interpreted as automatic highlighting in response to the user highlighting.

[0115] Fig.14 and 15 are example user interface screens showing the generation of regular expressions and capture groups based on selecting data from a tabular display. In these examples, Fig.14 and Fig.15 An additional user interface window is shown that automatically displays a user highlight 1401 within a tabular data display that is detected. The window includes a field 1402 for displaying a positive example, a field for displaying a negative example, and a field for displaying a dynamically (and almost instantly) generated regular expression in response to selecting a positive example from the tabular data display. In these examples, the user highlight within the column value 1401 can be equivalent to the user highlight within the automatic highlight. Thus, a user highlighting an area code results in not only the user-highlighted area code 1401, but also the remainder of the phone number being populated in the positive example field 1402.

[0116] However, it should be understood that user highlighting is not limited to performance within automatic highlighting. For example, user highlighting may alternatively be performed within other user highlights. As another example, user highlighting may alternatively be performed without any inner highlighting (e.g., further highlighting within highlighted text). These alternative examples are particularly suitable for semi-structured data, such as data columns that include "Tweets" or other long strings (e.g., browser "user agent" strings).

[0117] Additionally, when a corresponding regular expression is generated, other column values ​​that match the regular expression may be identified based on additional automatic highlighting 1402. Fig.14 and15 In the example shown, additional automatic highlighting indicates elements of these other column values ​​that match the capture groups of the generated regular expression. The additional automatic highlighting may be performed using a different color than that used for user highlighting.

[0118] like Fig.15 As shown, additional user highlighting is shown to indicate that the user has selected other examples. Additional user highlighting can be performed in a similar manner as described above. Thus, Fig.15 The user interface in shows the filling of other examples in the field 1502 for displaying positive examples. This can occur in response to detecting additional user highlights. In addition, the generated regular expression 1503 can be dynamically and almost instantly updated so that it matches all positive examples 1502. In response to generating an updated regular expression, the automatic highlighting of other column values ​​1504 that match the updated regular expression can also be updated. In some implementations, dynamic color coding can also be used. For example, matches can be color-coded using a first color (e.g., blue), while positive examples can be color-coded using a second color (e.g., green), and negative examples can be color-coded using a third color (e.g., red).

[0119] Fig.16A and 16B is an example user interface screen showing the generation of a regular expression based on selecting positive examples and negative examples from a table display. Figures 16A-16B , individual examples from the positive examples field 1602 can be removed from the positive examples field 1603 and / or moved to the negative examples field 1603. Within the user interface, this can be performed, for example, by the user clicking (e.g., right clicking) on ​​one of the examples to select it. The selection can cause the user interface to display a menu 1602 including a delete option and a change option. Thereafter, clicking on an option can result in the execution of the corresponding function.

[0120] exist Fig.16A and 16BIn the example shown, the result of the user selecting the change option is to move the selected example to the negative example field 1603, so that the regular expression 1601 is updated to the regular expression 1604, which can be generated dynamically and nearly instantly (e.g., between 30ms and 9000ms in some embodiments). In response to generating the updated regular expression 1604, the automatic highlighting of other column values ​​that match the updated regular expression can also be updated within the tabular data display. In addition, automatic highlighting can be performed on some or all negative examples, including any column values ​​corresponding to negative examples, which can be highlighted using a color different from any of the colors used above, or otherwise distinguished within the user interface using other visual techniques.

[0121] In some embodiments, specifying a negative example via a user interface does not require first specifying the example as a positive example and then converting it to a negative example, such as Fig.16A and 16B As shown. In contrast, negative examples can be specified in a variety of ways. For example, a user can select (e.g., right-click) a column value (e.g., one of the other column values ​​that is automatically highlighted to indicate that it matches the generated regular expression) via a user interface, which can cause a menu to be displayed that includes an option to designate the selected column value as a negative example (e.g., "Create a new counterexample").

[0122] Therefore, use Fig.16A and 16B In the example shown in , in response to generating an updated regular expression 1604, the automatic highlighting of other column values ​​matching the updated regular expression can also be updated. In these examples, the updated regular expression specifies telephone numbers ending with "9".

[0123] Briefly return to Fig.14 and 15 , when the user clicks or otherwise selects the "Extract" button, an operation may be initiated to extract the highlighted text fragment within all cells that match the current regular expression 1403 or 1503. Although Fig.14 and Fig.15Not shown, but in some embodiments, the user interface may provide other selectable buttons in addition to or in place of the "Extract" button. For example, a "Replace" button may be presented as an option to replace the user-highlighted element with an element specified by the user. Additionally or alternatively, one or more "Delete" buttons may be presented as options to actually replace the user-highlighted element with nothing. For example, one or both of a "Delete Segment" operation and / or a "Delete Row" operation may be implemented, which will delete the user-highlighted text segment or row, respectively. Additional operations that may be implemented in various embodiments may include: a "Keep Row" operation, a "Split" operation (e.g., highlighting commas and then extracting the comma-separated components into separate multiple new columns), and a "Fuzzy" operation (e.g., replacing highlighted text / capture groups with a "#" sequence or other symbol).

[0124] In this example, in response to selecting the "Extract" button, an extract operation can be added to a list of transformation scripts to be performed by downstream operations. In some embodiments, the list of transformation scripts can be displayed in a portion of the user interface for the user to review / modify. Alternatively, the extract operation can be performed in situ to generate a new column including the contents of the regular expression capture group (e.g., elements corresponding to the user-highlighted portion of the affirmative example). Fig.14 and 15 In the example shown, a new column and / or new table of area codes may be generated in response to selection of the "Extract" button.

[0125] Fig.17 is another example user interface screen illustrating the generation of regular expressions and capture groups based on selecting data from a tabular display according to one or more embodiments described herein.

[0126] VI. Regular Expression Generation Using the Longest Common Subsequence Algorithm on Span

[0127] Additional aspects described herein relate to regular expression generation based on the LCS algorithm from one or more data input character sequences, but wherein the regular expression generator 110 can also process characters that appear only in some examples. In order to process characters that appear only in some input examples, spans can be defined, wherein the minimum and maximum occurrences of the regular expression code are tracked. For example, for the character sequence input of "9pm" and "9pm", there is an optional space between the number and the "pm" text. In this case, when a certain span (e.g., a single space between "9" and "pm") may not exist in all given input examples, the minimum number of occurrences can be set to zero. These minimum and maximum numbers can then be mapped to regular expression multiplicity grammars. The longest common subsequence (LCS) algorithm can run on the span of characters obtained from the input example, including "optional" spans (e.g., minimum length is zero) that do not appear in each input example. As described below, continuous spans can be merged during the execution of the LCS algorithm. In this case, when the extra optional spans carried no longer appear continuously, the LCS algorithm can also run recursively on these optional spans. That is, although the operation of the LCS algorithm is inherently recursive, in these cases, the entire LCS algorithm can be run recursively (e.g., recursively running the recursive LCS algorithm). Among other technical advantages, this can allow shorter, cleaner, and more readable regular expression generation. For example, (am|am) (i.e., with optional space before am) can be generated without recursively running the LCS algorithm, while recursively running the LCS algorithm can result in the regular expression being generated as (?am), which is shorter and cleaner.

[0128] Fig.181 is a flowchart illustrating a process of generating a regular expression including an optional span using the longest common subsequence (LCS) algorithm according to one or more embodiments described herein. In step 1801, the regular expression generator 110 may receive one or more character sequences corresponding to a positive regular expression example as input data. In step 1802, the regular expression generator 110 may convert the character sequence into a regular expression code. Therefore, steps 1801 and 1802 may be similar or identical to the previous corresponding examples discussed above. Then, in step 1802, the regular expression code may also be converted into a span data structure (or span). As described above, each span may include a data structure storing a character class code (e.g., a regex code) and a repetition count range (e.g., a minimum count and / or a maximum count). In step 1804, the regular expression generator 110 may execute the LCS algorithm, providing the span set as input to the algorithm. The output of the LCS algorithm in this example may include an output set of spans, including at least one span having a minimum repetition count range equal to zero, which corresponds to an optional span within the output of the LCS algorithm. Finally, in step 1805 , the regular expression generator 110 may generate a regular expression based on the output of the LCS algorithm (including the optional span).

[0129] Fig.19 1802 is an example diagram showing the generation of a regular expression using the Longest Common Subsequence (LCS) algorithm, wherein the generated regular expression includes an optional span. In this example, the two input data character sequences are "8am" and "9pm". As described above, the input data character sequences are first converted into regular expression codes (step 1802) and then converted into spans (step 1803). The spans can be provided as input to the LCS algorithm (step 1804), and the LCS output includes the optional span Z. <0,1> , indicating that an optional single space can be a text sequence of numbers and two letters. That is, the upper notation in this example can include two numbers, namely the minimum repetition count range (e.g., 0) and the maximum repetition count range (e.g., 1) applied to the previous code (e.g., Z=space). Finally, a regular expression can be generated based on the output span of the LCS algorithm, and the optional span can be converted to the corresponding regular expression code "pZ * ”.

[0130] In some embodiments, the regular expression generator 110 can provide additional technical advantages in terms of performance and readability by reproducing and using optional spaces during execution of the LCS algorithm. For example, when generating regular expressions, in some cases it is desirable to be able to handle characters that are common in all given examples and characters that appear only in some examples.

[0131] In some embodiments, for each span data structure, a minimum number of occurrences of a category code and a maximum number of occurrences of a category code may be tracked. In the event that a span does not exist at all in one or more given examples, the minimum number is set to zero. As another example, to generate a regular expression to handle spelled out months of the year, the minimum and maximum numbers may then be mapped to a regular expression multiplicity syntax including curly braces (e.g., [A-Za-z]{3,9}).

[0132] In some embodiments, the regular expression generator 110 can track the minimum and maximum number of occurrences for each span, but can also handle additional implementation details. For example, as a result of processing the combination of optional spans and running LCS on character spans, the regular expression generator 110 can be configured to detect and merge continuous spans throughout the execution of the LCS algorithm. In addition, any additional optional spans carried may sometimes appear continuously, and the LCS algorithm may also need to run recursively on these spans. For example, in some cases, the regular expression generator 110 modifies and / or extends the LCS algorithm to tend to (or weight) less conversion between optional and required sequence elements (e.g., spans). For example, grouping optional spans together can minimize the number of grouping parentheses that must be used in the regular expression, thereby improving the human readability of the generated regular expression. In some cases, if the resulting lengths are equal even after considering the optional spans, the regular expression generator 110 can show a preference for alternatives with fewer conversions between the optional and required spans. For example, in some cases, the standard LCS algorithm may be implemented to prefer longer sequences at its decision points. However, at decision points where the options are of equal length, a configuration preference may be programmed into the regular expression generator 110. For example, one such configuration preference may be to prefer shorter sequences (once the optional span is taken into account). Thus, the custom LCS within the configuration may be optimized for both longer sequences (of the required span) and shorter sequences (of the total required span and the optional span).

[0133] In some embodiments, if the generated regular expression starts with a required span (which can also be used as a psychological anchor for human readers), rather than starting the regular expression with an optional span, the generated regular expression may be more readable. Therefore, in some cases, if the resulting options have an equal number of transitions, the option with the earlier non-optional span can be selected. In addition, in some embodiments, the LCS algorithm executed by the regular expression generator 110 can be configured to push all spaces (including optional spans corresponding to spaces) to the right within the regular expression. By pushing all spaces to the right, the possibility of merging space spans together may be increased, which can simplify the resulting regular expression and improve readability. Therefore, during the execution of the LCS algorithm, when it is determined that two substring sets have the same LCS, a set that helps to improve readability can be selected instead of arbitrarily selecting one of the two sets. In addition, in some embodiments, the LCS algorithm can be configured to favor a larger number of required spans and / or fewer optional spans in order to improve readability.

[0134] As described above, in some cases, negative examples may also be based on optional spans. For example, a user may provide positive examples of "ab" and "a2b" and a negative example of "a3b". In this case, the example implementation may fail because it may attempt to discriminate based only on the required span, and the "2" digit is in the optional span. In this case, the user may be warned of the failure and may be provided with options via the user interface to manually fix the generated regular expression and / or remove some negative examples.

[0135] In some embodiments, there may be an isSuccess returned as part of the JSON returned from the REST service. In some embodiments, when isSuccess=false, the generated regex may turn a different color (eg, red).

[0136] VII. Regular Expression Generation Using the Combined Longest Common Subsequence Algorithm

[0137] Other aspects described herein relate to combinatorial searches, in which the LCS algorithm performed by the regular expression generator 110 can be run multiple times to generate a "correct" regular expression (e.g., a regular expression that properly matches all given positive examples and properly excludes all given negative examples), and / or to generate multiple correct regular expressions from which the most desirable or optimal regular expression can be selected. For example, during a combinatorial search, the complete LCS algorithm and regular expression generation process can be run multiple times, including different combinations / permutations of text processing directions, different anchors, and other different features of the LCS algorithm.

[0138] Fig. 20 2 is a flowchart illustrating a process of generating a regular expression based on a combination of executions of the longest common subsequence (LCS) algorithm. In step 2001, the regular expression generator 110 may receive an input data character sequence corresponding to a positive example. In step 2002, the regular expression generator 110 may iterate various different combinations of execution techniques of the LCS algorithm. As shown in this example, during each iteration of step 2002, the regular expression generator 110 may select different combinations of the following LCS algorithm execution parameters (or characteristics): anchoring (i.e., no anchoring, anchoring to the beginning of the line, anchoring to the end of the line), processing direction (i.e., right-to-left order, left-to-right order), pushing spaces (i.e., pushing or not pushing spaces), and folding spans (i.e., folding or not folding spans). In step 2003, the LCS algorithm is run on the input data character sequence (or, if the input character sequence is first converted, it is run on the regular expression code), wherein the LCS algorithm is configured based on the parameters / characteristics selected in step 2002. In step 2004, the output of the LCS algorithm may be stored by the regular expression generator 110, including data such as whether the algorithm successfully identified the LCS and the length of the corresponding regular expression. In step 2005, the process may be iterated until the LCS algorithm has been run with all possible combinations of parameters / features of the combined search. Finally, in step 2006, a particular output from one of the LCSs is selected as the best output (e.g., based on success and regular expression length), and a regular expression may be generated based on the selected LCS algorithm output.

[0139] In various embodiments, the above references may be performed for various combinations of parameters / characteristics. Fig. 20 For example, in some embodiments, the LCS algorithm can anchor the regular expression to the beginning of the text using the caret symbol ^, and / or anchor the regular expression to the end of the text using the dollar sign $. In some cases, this anchoring may result in a shorter regular expression being generated. Anchors may be particularly useful when a user wants to find a specific pattern at the beginning and / or end of a string. For example, a user may want a product name at the beginning. To avoid confusing the LCS algorithm with a different number of words describing a product name, the regex can be anchored to the beginning of the string using a caret symbol, as shown in the following figure.

[0140] Furthermore, in some embodiments, the LCS algorithm may be performed with input data in either forward or reverse order (or similarly, the LCS algorithm may be configured to receive input data in a usual order and then reverse that order before executing the algorithm). Thus, in some embodiments, a combinatorial search of the LCS algorithm that may be performed on an input character sequence or code pair may be:

[0141] 1. Normal (right to left) order, not anchored to the beginning or end

[0142] 2. In normal (right to left) order, use the caret ^ to anchor to the beginning of the line

[0143] 3. In normal (right to left) order, use the dollar dollar to anchor to the end of the line

[0144] 4. Reverse (left to right) order, not anchored to the beginning or end

[0145] 5. Reverse the order (from left to right) and use the caret ^ to anchor to the beginning of the line

[0146] 6. Reverse the order (from left to right) and use the dollar dollar to anchor to the end of the line

[0147] In this example, among the six executions of LCS, the shortest resulting regular expression may be selected (step 2006).

[0148] In some embodiments, the combined search of the LCS algorithm can also iterate over greedy quantifiers "?" and non-greedy quantifiers "??". For example, by default, if there is an optional span, a single question mark is emitted, such as [AZ]+(?:[AZ]\.)? [AZ]+ represents the first and last name, with an optional middle initial. If a satisfactory regular expression cannot be found when using greedy quantifiers, the combined search can try to replace all question mark quantifiers with double question mark quantifiers (e.g., [AZ]+(?:[AZ]\.)??[AZ]+). The double question mark corresponds to a non-greedy quantifier, which can instruct the downstream regular expression matcher to enter backtracking mode in order to find a match.

[0149] In addition, in some embodiments, the combinatorial search of the LCS algorithm can also iterate whether spaces on the right are preferred. For example, as described above, a strategy of pushing spaces to the right can be used in some embodiments, for example, when the LCS algorithm faces an arbitrary choice of otherwise equal options, it is hoped that space spans can be merged together to produce fewer overall spans. This feature adds another option to the combinatorial search, that is, either push the spaces to the right or perform according to the traditional LCS method, that is, make a decision arbitrarily.

[0150] In addition, in some embodiments, the combined search of the LCS algorithm can also traverse the common words in all examples by running LCS on the original string. In such an embodiment, the LCS algorithm can be configured to identify and align common words. As used herein, "common words" can refer to words that appear in each positive example. Once a common word is identified, its span type can be converted from letters to words, and then it can be naturally aligned by subsequent operation of the LCS algorithm.

[0151] Thus, in the example below, the combined search may iterate over several parameters / features to achieve 96 executions of the full LCS algorithm. The various parameters / features to be iterated over in this example are:

[0152] Anchor (3) (value = ^, $, or neither)

[0153] Push Space (2) (Value = Yes or No)

[0154] Merge low cardinality spans into wildcard (2) (value = yes or no)

[0155] Greedy quantifier? (2) (value = yes or no)

[0156] Align LCS algorithm (2) on common tokens (value = yes or no)

[0157] Use "\w" to represent alphanumerics, or treat letters "\pL" and numbers "\pN" as separate spans (2) (value = yes or no)

[0158] As described above, in this example, the complete LCS algorithm will be executed 96 times (eg, 3*2*2*2*2*2*2=96).

[0159] However, in other embodiments, the regular expression generator 110 can provide a performance enhancement by which only the first three features in the above list (anchoring, pushing spaces, and merging low-cardinality spans into wildcards) can participate in the combined search. This may result in much fewer times (e.g., 3*2*2=12 times) of the full LCS algorithm to be executed. In such an embodiment, although the last three features in the above list (greedy quantifiers, aligning the LCS algorithm to common tokens, and using "\w" to represent alphanumerics, or treating letters "\pL" and numbers "\pN" as separate spans) do not participate in the combined search, these features can be tested separately and sequentially at the end. A technical advantage can be achieved in such an embodiment because dividing the search space in this way can still find a satisfactory regular expression, but with a performance acceleration of approximately 8 times.

[0160] To illustrate this, the following combined search example can provide performance advantages over the previous examples. In this example, the combined search can be performed based on the following parameters / features to be traversed:

[0161] ·Anchoring (3): BEGINNING_OF_LINE_MODE, END_OF_LINE_MODE, NO_EOL_MODE

[0162] Order / Direction (2): Right to left (normal) LCS and left to right (reverse) LCS

[0163] Push (2): whether to try to push spaces to the right within the LCS algorithm

[0164] Compress to wildcards (2): Should we try to compress long sequences that occur only sometimes down to the wildcard character .*?

[0165] The combination in this example may result in running the full algorithm 3*2*2*2=24 times. Regular expression generator 110 may then take the best of the 24 results of the LCS algorithm, where "best" may mean (a) the LCS algorithm succeeded, and (b) the shortest regular expression was generated. Regular expression generator 110 may then perform the following three additional tasks:

[0166] 1. Try to compress sequences of letters and numbers that are not interrupted by spaces, punctuation, or symbols into a new span type I, called alphanumeric, corresponding to the generated regular expression \w. This may be useful for hexadecimal numbers found in IPv6 addresses in clickstream logs (see Novelty 64, April 2019).

[0167] 2. Try using non-greedy quantifier ? ? instead of greedy quantifier ?

[0168] 3. Try to align the text

[0169] VIII. Hardware Overview

[0170] Fig.21 A simplified diagram of a distributed system 2100 for implementing an embodiment is depicted. In the illustrated embodiment, the distributed system 2100 includes one or more client computing devices 2102, 2104, 2106, and 2108 coupled to a server 2112 via one or more communication networks 2110. The client computing devices 2102, 2104, 2106, and 2108 may be configured to execute one or more applications.

[0171] In various embodiments, the server 2112 may be adapted to run one or more services or software applications that enable automatic generation of regular expressions, as described in the present disclosure. For example, in some embodiments, the server 2112 may receive user input data sent from a client device, where the user input data is received by the client device through a user interface displayed at the client device. The server 2112 may then convert the user input data into a regular expression that is transmitted to the client device for display through the user interface.

[0172] In some embodiments, the server 2112 may also provide other services or software applications that may include non-virtualized and virtualized environments. In some embodiments, these services may be provided as web-based services or cloud services, such as under a software as a service (SaaS) model, to users of client computing devices 2102, 2104, 2106, and / or 2108. Users operating client computing devices 2102, 2104, 2106, and / or 2108 may in turn utilize one or more client applications to interact with the server 2112 to utilize the services provided by these components.

[0173] exist Fig.21 In the configuration shown, the server 2112 may include one or more components 2118, 2120, and 2122 that implement the functions performed by the server 2112. These components may include software components, hardware components, or a combination thereof that can be executed by one or more processors. It should be understood that a variety of different system configurations are possible, which may differ from the distributed system 2100. Therefore, Fig.21 The illustrated embodiment is one example of a distributed system for implementing an embodiment system and is not intended to be limiting.

[0174] The user may use client computing devices 2102, 2104, 2106, and / or 2108 to execute one or more applications that may generate regular expressions according to the teachings of the present disclosure. The client device may provide an interface that enables a user of the client device to interact with the client device. The client device may also output information to the user via the interface. Fig.21 Only four client computing devices are depicted, but any number of client computing devices may be supported.

[0175] Client devices may include various types of computing systems, such as portable handheld devices, general-purpose computers such as personal computers and laptops, workstation computers, wearable devices, gaming systems, thin clients, various messaging devices, sensors or other sensing devices, etc. These computing devices may run various types and versions of software applications and operating systems (e.g., Microsoft apple or UNIX-like operating systems, such as Google Chrome TM LINUX or LINUX-like operating systems for OS), including various mobile operating systems (e.g., Microsoft Windows Windows Android TM , Portable handheld devices may include cellular phones, smart phones (e.g., ), tablets (e.g. ), personal digital assistants (PDAs), etc. Wearable devices can include Google Head mounted displays and other devices. Gaming systems can include various handheld gaming devices, Internet-enabled gaming devices (e.g., with or without Gesture Input Devices Microsoft Game consoles, Sony System, by Various game systems provided by the Internet, etc.) etc. The client device can execute various different applications, such as various Internet-related applications, communication applications (e.g., email applications, short message service (SMS) applications), and can use various communication protocols.

[0176] Network 2110 may be any type of network familiar to those skilled in the art that may support data communications using any of a variety of available protocols, including but not limited to TCP / IP (Transmission Control Protocol / Internet Protocol), SNA (Systems Network Architecture), IPX (Internetwork Packet Exchange), For example only, the network 2110 may be a local area network (LAN), an Ethernet-based network, a token ring, a wide area network (WAN), the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a public switched telephone network (PSTN), an infrared network, a wireless network (e.g., any one of the Institute of Electrical and Electronics Engineers (IEEE) 1002.11 protocol suite, and / or any other network operating under a wireless protocol).

[0177] The server 2112 may be composed of one or more general-purpose computers, dedicated server computers (including, for example, PC (personal computer) servers, The server 2112 may include one or more virtual machines running virtual operating systems, or other computing architectures involving virtualization, such as one or more flexible logical storage device pools that can be virtualized to maintain a virtual storage device of the server. In various embodiments, the server 2112 may be suitable for running one or more services or software applications that provide the functionality described in the foregoing disclosure.

[0178] The computing system in the server 2112 can run one or more operating systems, including any of the operating systems discussed above, as well as any commercially available server operating system. The server 2112 can also run any of a variety of additional server applications and / or middle-tier applications, including HTTP (Hypertext Transfer Protocol) servers, FTP (File Transfer Protocol) servers, CGI (Common Gateway Interface) servers, Servers, database servers, etc. Exemplary database servers include but are not limited to (International Business Machines) and other commercially available database servers.

[0179] In some implementations, server 2112 may include one or more applications to analyze and consolidate data feeds and / or event updates received from users of client computing devices 2102, 2104, 2106, and 2108. As examples, data feeds and / or event updates may include, but are not limited to: feed, Real-time updates and continuous data streams updated or received from one or more third-party information sources, which may include real-time events related to sensor data applications, financial tickers, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, automobile traffic monitoring, etc. Server 2112 may also include one or more applications for displaying data feeds and / or real-time events via one or more display devices of client computing devices 2102, 2104, 2106, and 2108.

[0180] The distributed system 2100 may also include one or more data repositories 2114, 2116. In some embodiments, these data repositories may be used to store data and other information. For example, one or more of the data repositories 2114, 2116 may be used to store information, such as new data columns that match regular expressions generated by the system. The data repositories 2114, 2116 may reside in a variety of locations. For example, the data repository used by the server 2112 may be local to the server 2112 or may be remote from the server 2112 and communicate with the server 2112 via a network-based or dedicated connection. The data repositories 2114, 2116 may be of different types. In some embodiments, the data repository used by the server 2112 may be a database, such as a relational database, such as Oracle Database, or a database server. And other suppliers of databases. One or more of these databases can be adapted to store, update and retrieve data to and from the database in response to SQL format commands.

[0181] In some embodiments, one or more of the data stores 2114, 2116 may also be used by applications to store application data. The data stores used by the applications may be of different types, such as a key-value store, an object store, or a general store backed by a file system.

[0182] In certain embodiments, the functionality described in this disclosure may be provided as a service via a cloud environment. Fig. 22 is a simplified block diagram of a cloud-based system environment in which various services may be provided as cloud services, according to certain examples. Fig. 22 In the example shown, cloud infrastructure system 2202 may provide one or more cloud services that may be requested by users using one or more client computing devices 2204, 2206, and 2208. Cloud infrastructure system 2202 may include one or more computers and / or servers, which may include those described above with respect to server 2112. The computers in cloud infrastructure system 2202 may be organized as general-purpose computers, dedicated server computers, server farms, server clusters, or any other suitable arrangement and / or combination.

[0183] The network 2210 can facilitate data communication and exchange between the clients 2204, 2206, and 2208 and the cloud infrastructure system 2202. The network 2210 can include one or more networks. The networks can be of the same type or different types. The network 2210 can support one or more communication protocols, including wired and / or wireless protocols, to facilitate communication.

[0184] Fig. 22 The example depicted in FIG. 2 is only one example of a cloud infrastructure system and is not intended to be limiting. It should be understood that in some other examples, the cloud infrastructure system 2202 may have more Fig. 22 More or fewer components may be included than those depicted, two or more components may be combined, or there may be a different configuration or arrangement of components. For example, although Fig. 22 Three client computing devices are depicted, but any number of client computing devices may be supported in alternative examples.

[0185] The term cloud service is generally used to refer to services provided to users on demand and via a communication network such as the Internet by a service provider's system (e.g., cloud infrastructure system 2202). Typically, in a public cloud environment, the servers and systems that make up the cloud service provider's system are different from the customer's own enterprise servers and systems. The cloud service provider's system is managed by the cloud service provider. Therefore, customers can take advantage of the cloud services provided by the cloud service provider without having to purchase separate licenses, support, or hardware and software resources for those services. For example, the cloud service provider's system can host applications, and users can order and use the applications on demand via the Internet without having to purchase infrastructure resources for executing the applications. Cloud services are designed to provide easy, scalable access to applications, resources, and services. Several providers offer cloud services. For example, several cloud services are provided by Oracle in Redwood Shores, California. Provides, for example, middleware services, database services, Java cloud services, etc.

[0186] In some embodiments, the cloud infrastructure system 2202 may provide one or more cloud services using different models such as under a software as a service (SaaS) model, a platform as a service (PaaS) model, an infrastructure as a service (IaaS) model, and other models including hybrid service models. The cloud infrastructure system 2202 may include a set of applications, middleware, databases, and other resources that can provide various cloud services.

[0187] The SaaS model enables applications or software to be delivered to customers as a service over a communications network (such as the Internet) without the customer having to purchase hardware or software for the underlying application. For example, the SaaS model can be used to provide customers with access to on-demand applications hosted by the cloud infrastructure system 2202. Oracle Examples of SaaS services provided include, but are not limited to, various services for human resource / capital management, customer relationship management (CRM), enterprise resource planning (ERP), supply chain management (SCM), enterprise performance management (EPM), analytical services, social applications, etc.

[0188] The IaaS model is usually used to provide infrastructure resources (such as servers, storage, hardware, and network resources) to customers in the form of cloud services to provide elastic computing and storage capabilities. Provides various IaaS services.

[0189] The PaaS model is typically used to provide platform and environment resources as a service, enabling customers to develop, run, and manage applications and services without having to purchase, build, or maintain such resources. Examples of PaaS services provided include, but are not limited to, Oracle Java Cloud Service (JCS), Oracle Database Cloud Service (DBCS), data management cloud services, various application development solution services, etc.

[0190] Cloud services are typically provided on an on-demand self-service, subscription-based, elastically scalable, reliable, highly available, and secure basis. For example, a customer may order one or more services provided by the cloud infrastructure system 2202 via a subscription order. The cloud infrastructure system 2202 then performs processing to provide the services requested in the customer's subscription order. The cloud infrastructure system 2202 may be configured to provide one or more cloud services.

[0191] The cloud infrastructure system 2202 can provide cloud services via different deployment models. In the public cloud model, the cloud infrastructure system 2202 can be owned by a third-party cloud service provider, and the cloud services are provided to any general public customer, where the customer can be an individual or an enterprise. Under the private cloud model, the cloud infrastructure system 2202 can run within an organization (e.g., within an enterprise organization) and provide services to customers within the organization. For example, customers can be various departments of an enterprise, such as the human resources department, the payroll department, etc., or even individuals within the enterprise. In the community cloud model, the cloud infrastructure system 2202 and the services provided can be shared by several organizations in a related community. Various other models can also be used, such as a hybrid of the above models.

[0192] Client computing devices 2204, 2206, and 2208 may be of different types (e.g., Fig.21 2102, 2104, 2106, and 2108) and is capable of operating one or more client applications. A user can use a client device to interact with the cloud infrastructure system 2202, such as requesting services provided by the cloud infrastructure system 2202.

[0193] In some embodiments, the processing performed by cloud infrastructure system 2202 for providing management-related services may involve big data analysis. Such analysis may involve using, analyzing, and manipulating large data sets to detect and visualize various trends, behaviors, relationships, etc. in the data. The analysis may be performed by one or more processors, possibly processing data in parallel, performing simulations using the data, etc. For example, big data analysis may be performed by cloud infrastructure system 2202 to determine regular expressions in an automated manner. The data used for the analysis may include structured data (e.g., data stored in a database or structured according to a structured model) and / or unstructured data (e.g., data blobs (binary large objects)).

[0194] like Fig. 22 As shown in the example of , cloud infrastructure system 2202 may include infrastructure resources 2230 for facilitating the provision of various cloud services provided by cloud infrastructure system 2202. Infrastructure resources 2230 may include, for example, processing resources, storage or memory resources, networking resources, and the like.

[0195] In some embodiments, in order to facilitate the efficient provision of these resources to support the various cloud services provided by the cloud infrastructure system 2202 for different customers, these resources can be bundled into resource sets or resource modules (also referred to as "pods"). Each resource module or pod may include a pre-integrated and optimized combination of one or more types of resources. In some embodiments, different pods can be pre-provisioned for different types of cloud services. For example, a first set of pods can be provisioned for a database service, a second set of pods can be provided for a Java service, which may include a different combination of resources than the pods in the first set of pods, and so on. For some services, the resources allocated to provide the service can be shared between services.

[0196] The cloud infrastructure system 2202 itself may internally use services 2232 that are shared by different components of the cloud infrastructure system 2202 and facilitate the provision of services by the cloud infrastructure system 2202. These internal shared services may include, but are not limited to, security and identity services, integration services, enterprise repository services, enterprise manager services, virus scanning and whitelisting services, high availability, backup and recovery services, services for enabling cloud support, email services, notification services, file transfer services, and the like.

[0197] The cloud infrastructure system 2202 may include multiple subsystems. These subsystems may be implemented in software or hardware or a combination thereof. Fig. 22As shown, the subsystem may include a user interface subsystem 2212 that enables users or customers of cloud infrastructure system 2202 to interact with cloud infrastructure system 2202. User interface subsystem 2212 may include a variety of different interfaces, such as a web interface 2214, an online store interface 2216 in which cloud services provided by cloud infrastructure system 2202 are advertised and can be purchased by consumers, and other interfaces 2218. For example, a customer may use a client device to request (service request 2234) one or more services provided by cloud infrastructure system 2202 using one or more of interfaces 2214, 2216, and 2218. For example, a customer may visit an online store, browse cloud services provided by cloud infrastructure system 2202, and place a subscription order for one or more services provided by cloud infrastructure system 2202 to which the customer wishes to subscribe. The service request may include information identifying the customer and one or more services to which the customer wishes to subscribe. For example, a customer may subscribe to services related to automatically generating regular expressions provided by cloud infrastructure system 2202.

[0198] In some embodiments, for example Fig. 22 In the example depicted in , cloud infrastructure system 2202 may include an order management subsystem (OMS) 2220 configured to process new orders. As part of this processing, OMS 2220 may be configured to: create an account for the customer (if one does not already exist); receive billing and / or account information from the customer, which will be used to charge the customer for providing the requested service to the customer; verify the customer information; after verification, book the order for the customer; and coordinate various workflows to prepare the order for supply.

[0199] Once properly verified, the OMS 2220 can then call the order provisioning subsystem (OPS) 2224, which is configured to provision resources for the order, including processing, memory, and networking resources. Provisioning can include allocating resources to the order and configuring resources to assist in the services requested by the customer order. The manner in which resources are provisioned for the order and the type of resources provisioned can depend on the type of cloud service that the customer has ordered. For example, according to a workflow, the OPS 2224 can be configured to determine the specific cloud service requested and identify multiple pods that may have been pre-configured for the specific cloud service. The number of pods allocated for the order can depend on the size / amount / level / scope of the requested service. For example, the number of pods to be allocated can be determined based on the number of users to be supported by the service, the duration of the requested service, etc. The allocated pods can then be customized for the specific requesting customer to provide the requested service.

[0200] The cloud infrastructure system 2202 may send a response or notification 2244 to the requesting client to indicate when the requested service is ready for use. In some cases, information (e.g., a link) may be sent to the client that enables the client to begin using and taking advantage of the benefits of the requested service. In some embodiments, for a client requesting an automatic generation of regular expression related services, the response may include instructions that, when executed, cause a user interface to be displayed.

[0201] The cloud infrastructure system 2202 can provide services to multiple customers. For each customer, the cloud infrastructure system 2202 is responsible for managing information related to one or more subscription orders received from the customer, maintaining customer data related to the order, and providing the requested services to the customer. The cloud infrastructure system 2202 can also collect usage statistics about the customer's use of the subscription services. For example, statistics can be collected about the amount of storage used, the amount of data transferred, the number of users, and the amount of system uptime and system downtime. This usage information can be used to bill the customer. For example, billing can be based on a monthly cycle.

[0202] The cloud infrastructure system 2202 may provide services to multiple customers in parallel. The cloud infrastructure system 2202 may store information for these customers, which may include proprietary information. In certain embodiments, the cloud infrastructure system 2202 includes an identity management subsystem (IMS) 2228 that is configured to manage customer information and provide separation of managed information so that information associated with one customer cannot be accessed by another customer. The IMS 2228 may be configured to provide various security-related services, such as identity services; information access management, authentication and authorization services; services for managing customer identities and roles and related capabilities, etc.

[0203] Fig.23 An example of a computer system 2300 is shown. In some embodiments, the computer system 2300 can be used to implement any of the above systems. Fig.23 As shown, computer system 2300 includes various subsystems, including a processing subsystem 2304 that communicates with multiple other subsystems via a bus subsystem 2302. These other subsystems may include a processing acceleration unit 2306, an I / O subsystem 2308, a storage subsystem 2318, and a communication subsystem 2324. Storage subsystem 2318 may include non-transitory computer-readable storage media, including storage media 2322 and system memory 2310.

[0204] The bus subsystem 2302 provides a mechanism for the various components and subsystems of the computer system 2300 to communicate with each other as expected. Although the bus subsystem 2302 is schematically shown as a single bus, alternative examples of the bus subsystem can utilize multiple buses. The bus subsystem 2302 can be any of several types of bus structures, including a memory bus or memory controller, a peripheral device bus, a local bus using any of a variety of bus architectures, etc. For example, such an architecture can include an industry standard architecture (ISA) bus, a microchannel architecture (MCA) bus, an enhanced ISA (EISA) bus, a video electronics standard association (VESA) local bus, and a peripheral component interconnect (PCI) bus, which can be implemented as a mezzanine bus manufactured in accordance with the IEEE P1386.1 standard, etc.

[0205] The processing subsystem 2304 controls the operation of the computer system 2300 and may include one or more processors, application specific integrated circuits (ASICs), or field programmable gate arrays (FPGAs). The processor may include a single-core or multi-core processor. The processing resources of the computer system 2300 may be organized into one or more processing units 2332, 2334, etc. A processing unit may include one or more processors, one or more cores from the same or different processors, a combination of cores and processors, or other combinations of cores and processors. In some embodiments, the processing subsystem 2304 may include one or more dedicated coprocessors, such as a graphics processor, a digital signal processor (DSP), etc. In some embodiments, some or all of the processing units of the processing subsystem 2304 may be implemented using custom circuits (e.g., application specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs)).

[0206] In some embodiments, the processing unit in the processing subsystem 2304 can execute instructions stored in the system memory 2310 or the computer-readable storage medium 2322. In various examples, the processing unit can execute various programs or code instructions, and can maintain multiple concurrently executed programs or processes. At any given time, some or all of the program code to be executed may reside on the system memory 2310 and / or the computer-readable storage medium 2322, including possibly on one or more storage devices. Through appropriate programming, the processing subsystem 2304 can provide the various functions described above. In the case where the computer system 2300 is executing one or more virtual machines, one or more processing units can be assigned to each virtual machine.

[0207] In some embodiments, the processing acceleration unit 2306 may optionally be used to perform custom processing or to offload some of the processing performed by the processing subsystem 2304 , thereby speeding up the overall processing performed by the computer system 2300 .

[0208] The I / O subsystem 2308 may include devices and mechanisms for inputting information to the computer system 2300 and / or for outputting information from or via the computer system 2300. In general, use of the term input device is intended to include all possible types of devices and mechanisms for inputting information to the computer system 2300. User interface input devices may include, for example, keyboards, pointing devices such as a mouse or trackball, touch pads or touch screens integrated into a display, scroll wheels, click wheels, dials, buttons, switches, keypads, audio input devices with voice command recognition systems, microphones, and other types of input devices. User interface input devices may also include motion sensing and / or gesture recognition devices, such as Microsoft Windows, that enable a user to control and interact with input devices. Motion Sensor, Microsoft 360 game controllers, devices that provide an interface for receiving input using gestures and verbal commands. User interface input devices may also include eye gesture recognition devices such as Google Blink detectors, which detect eye activity from the user (e.g., a "blink" when taking a picture and / or making a menu selection) and convert eye gestures to input devices (e.g., Google ) input. In addition, the user interface input device may include a user to enable the user to communicate with the voice recognition system (e.g., Navigator) interactive voice recognition sensing device.

[0209] Other examples of user interface input devices include, but are not limited to, three-dimensional (3D) mice, joysticks or pointing sticks, game pads and graphics tablets, and audio / visual devices such as speakers, digital cameras, digital video cameras, portable media players, webcams, image scanners, fingerprint scanners, barcode readers 3D scanners, 3D printers, laser rangefinders, and eye gaze tracking devices. In addition, user interface input devices may include, for example, medical imaging input devices such as computed tomography, magnetic resonance imaging, position emission tomography, and medical ultrasound imaging devices. User interface input devices may also include, for example, audio input devices such as MIDI keyboards, digital musical instruments, etc.

[0210] In general, the use of the term output device is intended to include all possible types of devices and mechanisms for outputting information from the computer system 2300 to a user or other computers. User interface output devices may include display subsystems, indicator lights, or non-visual displays such as audio output devices, etc. The display subsystem may be a cathode ray tube (CRT), a flat panel device (e.g., a flat panel device using a liquid crystal display (LCD) or plasma display), a projection device, a touch screen, etc. For example, user interface output devices may include, but are not limited to, various display devices that visually convey text, graphics, and audio / video information, such as monitors, printers, speakers, headphones, car navigation systems, plotters, voice output devices, and modems.

[0211] The storage subsystem 2318 provides a repository or data storage for storing information and data used by the computer system 2300. The storage subsystem 2318 provides a tangible, non-transitory computer-readable storage medium for storing basic programming and data constructs that provide the functionality of certain examples. The storage subsystem 2318 can store software (e.g., programs, code modules, instructions) that provide the above-described functionality when executed by the processing subsystem 2304. The software can be executed by one or more processing units of the processing subsystem 2304. The storage subsystem 2318 can also provide a repository for storing data used in accordance with the teachings of the present disclosure.

[0212] The storage subsystem 2318 may include one or more non-volatile storage devices, including volatile and non-volatile storage devices. Fig.23 As shown, the storage subsystem 2318 includes a system memory 2310 and a computer-readable storage medium 2322. The system memory 2310 may include multiple memories, including a volatile main random access memory (RAM) for storing instructions and data during program execution and a non-volatile read-only memory (ROM) or flash memory in which fixed instructions are stored. In some implementations, a basic input / output system (BIOS) may typically be stored in ROM, which contains basic routines such as helping to transfer information between elements within the computer system 2300 during startup. RAM typically contains data and / or program modules currently operated and executed by the processing subsystem 2304. In some implementations, the system memory 2310 may include multiple different types of memory, such as static random access memory (SRAM), dynamic random access memory (DRAM), etc.

[0213] By way of example and not limitation, Fig.23As shown, the system memory 2310 can load the executing application 2312, which can include various applications such as a web browser, a middle-tier application, a relational database management system (RDBMS), etc., program data 2314 and an operating system 2316. As an example, the operating system 2316 can include various versions of Microsoft Apple and / or Linux operating system, various commercially available or UNIX-like operating systems (including but not limited to various GNU / Linux operating systems, Google OS, etc.) and / or such as iOS, Phone, OS, OS, OS operating system, etc. Mobile operating system.

[0214] Computer-readable storage media 2322 may store programming and data structures that provide some example functionality. Computer-readable media 2322 may provide storage for computer-readable instructions, data structures, program modules, and other data for computer system 2300. Software (programs, code modules, instructions) that provide the above functionality when executed by processing subsystem 2304 may be stored in storage subsystem 2318. By way of example, computer-readable storage media 2322 may include non-volatile memory, such as a hard drive, a magnetic disk drive, an optical disk drive (e.g., a CD ROM, a DVD, Disk) or other optical media. Computer readable storage media 2322 may include, but is not limited to, Drive, flash memory card, universal serial bus (USB) flash drive, secure digital (SD) card, DVD disk, digital video tape, etc. The computer readable storage medium 2322 may also include a solid-state drive (SSD) based on non-volatile memory (such as a flash memory-based SSD, an enterprise flash drive, a solid-state ROM, etc.), a volatile memory-based SSD (such as a solid-state RAM, a dynamic RAM, a static RAM, a DRAM-based SSD, a magnetoresistive RAM (MRAM) SSD), and a hybrid SSD using a combination of DRAM and a flash memory-based SSD.

[0215] In certain embodiments, storage subsystem 2318 may also include a computer-readable storage medium reader 2320, which may also be connected to computer-readable storage medium 2322. Reader 2320 may receive data from a storage device such as a disk, flash drive, etc. and be configured to read data from the storage device.

[0216] In some embodiments, computer system 2300 can support virtualization technology, including but not limited to virtualization of processing and memory resources. For example, computer system 2300 can provide support for executing one or more virtual machines. In some embodiments, computer system 2300 can execute programs such as hypervisors that are convenient for configuring and managing virtual machines. Each virtual machine can be allocated memory, computing (e.g., processor, core), I / O and networking resources. Each virtual machine is usually independent of other virtual machines. Virtual machines usually run their own operating system, which can be the same or different from the operating system executed by other virtual machines executed by computer system 2300. Therefore, computer system 2300 can run multiple operating systems simultaneously.

[0217] The communication subsystem 2324 provides an interface to other computer systems and networks. The communication subsystem 2324 serves as an interface for receiving data from the computer system 2300 and sending data to other systems. For example, the communication subsystem 2324 can enable the computer system 2300 to establish a communication channel to one or more client devices via the Internet for receiving information from the client devices and sending information to the client devices.

[0218] The communication subsystem 2324 can support wired and / or wireless communication protocols. In some embodiments, the communication subsystem 2324 may include a radio frequency (RF) transceiver component for accessing a wireless voice and / or data network (e.g., using cellular phone technology, advanced data network technology, such as 3G, 4G or EDGE (Enhanced Data Rates for Global Evolution)), WiFi (IEEE 802.XX series standards, or other mobile communication technologies, or any combination thereof), a global positioning system (GPS) receiver component, and / or other components. In some embodiments, in addition to or in lieu of a wireless interface, the communication subsystem 2324 may provide a wired network connection (e.g., Ethernet).

[0219] The communication subsystem 2324 can receive and send data in various forms. In some embodiments, the communication subsystem 2324 can receive input communications in the form of structured and / or unstructured data feeds 2326, event streams 2328, event updates 2330, etc., among other forms. For example, the communication subsystem 2324 can be configured to receive (or send) data feeds 2326 in real time from users of social media networks and / or other communication services, such as feed, Updates, web feeds such as Rich Site Summary (RSS) feeds, and / or real-time updates from one or more third-party information sources.

[0220] In some embodiments, the communication subsystem 2324 can be configured to receive data in the form of a continuous data stream, which can include an event stream 2328 of real-time events and / or event updates 2330, which can be continuous or unbounded in nature, without an explicit end. Examples of applications that generate continuous data can include, for example, sensor data applications, financial tickers, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, automobile traffic monitoring, etc.

[0221] The communication subsystem 2324 may also be configured to transmit data from the computer system 2300 to other computer systems or networks. The data may be transmitted in a variety of different forms (e.g., structured and / or unstructured data feeds 2326, event streams 2328, event updates 2330, etc.) to one or more databases that may be in computer communication with one or more stream data source computers coupled to the computer system 2300.

[0222] Computer system 2300 may be one of various types, including a handheld portable device (e.g., Cellular phone, computing tablets, PDAs), wearable devices (e.g., Google head mounted display), personal computer, workstation, mainframe, kiosk, server rack or any other data processing system. Due to the ever-changing nature of computers and networks, Fig.23 The description of the computer system 2300 depicted in FIG. 2 is for specific examples only. Fig.23 Many other configurations of more or fewer components of the systems shown are also possible. Based on the disclosure and teachings provided herein, a person of ordinary skill in the art will appreciate other ways and / or methods to implement the various examples.

[0223] Although specific examples have been described, various modifications, changes, alternative structures and equivalents are possible. The examples are not limited to operating in certain specific data processing environments, but can be freely operated in multiple data processing environments. In addition, although certain examples have been described using a series of specific transactions and steps, it will be apparent to those skilled in the art that this is not intended to be limiting. Although some flowcharts describe operations as sequential processes, many operations can be performed in parallel or concurrently. In addition, the order of operations can be rearranged. The process may have other steps not included in the figure. The various features and aspects of the above examples can be used alone or in combination.

[0224] In addition, although certain examples have been described using a particular combination of hardware and software, it should be appreciated that other combinations of hardware and software are possible. Certain examples may be implemented only in hardware, or only in software, or using a combination thereof. The various processes described herein may be implemented in any combination on the same processor or on different processors.

[0225] Where a device, system, component or module is described as being configured to perform certain operations or functions, such configuration may be achieved, for example, by designing electronic circuits to perform the operations, by programming programmable electronic circuits (such as microprocessors) to perform the operations, for example by executing computer instructions or code, or a processor or core programmed to execute code or instructions stored on a non-volatile memory medium, or any combination thereof. Processes may communicate using a variety of techniques, including but not limited to conventional techniques for inter-process communication, and different pairs of processes may use different techniques, or the same pair of processes may use different techniques at different times.

[0226] Specific details are given in this disclosure to provide a thorough understanding of the examples. However, the examples can be practiced without these specific details. For example, well-known circuits, processes, algorithms, structures and techniques are shown without unnecessary details to avoid confusing the examples. This specification only provides examples and is not intended to limit the scope, applicability or configuration of other examples. On the contrary, the above description of the examples will provide those skilled in the art with an enabling description for implementing various examples. Various changes may occur in the functions and arrangements of the elements.

[0227] The description and drawings are therefore to be regarded as illustrative rather than restrictive. However, it will be apparent that additions, subtractions, deletions and other modifications and changes may be made thereto without departing from the broader spirit and scope set forth in the claims. Therefore, although specific examples have been described, these examples are not intended to be limiting. Various modifications and equivalents are within the scope of the appended claims.

[0228] In the foregoing description, various aspects of the present disclosure are described with reference to specific examples thereof, but those skilled in the art will recognize that the present disclosure is not limited thereto. The various features and aspects disclosed above may be used alone or in combination. In addition, without departing from the broader spirit and scope of the present specification, examples may be used in any number of environments and applications in addition to those described herein. Therefore, the description and drawings should be regarded as illustrative, rather than restrictive.

[0229] In the foregoing description, for the purpose of illustration, the methods are described in a particular order. It should be understood that in alternative examples, the methods may be performed in an order different from the order described. It should also be understood that the above methods may be performed by hardware components, or may be implemented in a machine executable instruction sequence, which may be used to enable a machine, such as a general or special processor or logic circuit, to perform the methods using instruction programming. These machine executable instructions may be stored on one or more machine readable media, such as a CD-ROM or other type of optical disk, floppy disk, ROM, RAM, EPROM, EEPROM, magnetic card or optical card, flash memory or other types of machine readable media suitable for storing electronic instructions. Alternatively, the methods may be performed by a combination of hardware and software.

[0230] Where a component is described as being configured to perform certain operations, such configuration may be accomplished, for example, by designing electronic circuits or other hardware to perform the operations, by programming programmable electronic circuits (e.g., a microprocessor or other suitable electronic circuits) to perform the operations, or any combination thereof.

[0231] While illustrative examples of the present application are described herein in detail, it should be understood that the concepts of the present invention may be otherwise variously embodied and used, except as limited by the prior art, and the appended claims are intended to be interpreted to include such variations.

[0232] Where a component is described as being “configured to” perform certain operations, such configuration may be accomplished, for example, by designing electronic circuits or other hardware to perform the operations, by programming programmable electronic circuits (e.g., a microprocessor or other suitable electronic circuits) to perform the operations, or any combination thereof.

Claims

1. A method for generating a regular expression, comprising: Retrieving data from one or more data sources by a regular expression generator including one or more processors; The regular expression generator displays the retrieved data in a table form in a user interface, wherein the displayed table data includes a plurality of data cells; receiving, by the regular expression generator via the user interface, a first selection of data from within the displayed tabular data; generating, by the regular expression generator, a first regular expression based on the first selection of data from within the displayed tabular data; as well as Displaying the first regular expression in the user interface by the regular expression generator; After displaying the first regular expression, receiving, by the regular expression generator via the user interface, a second selection of data within the displayed table data; generating, by the regular expression generator, a second regular expression in response to the second selection of data, wherein the second regular expression matches both the first selection of data within the displayed tabular data and the second selection of data; and The second regular expression is displayed in the user interface by the regular expression generator. 2 . The method of claim 1 , wherein receiving the first selection of data comprises receiving a selection of a first data cell via the user interface, and wherein generating the first regular expression comprises confirming that the first regular expression matches the data within the first data cell.

3. The method of claim 1 , wherein receiving the first selection of data comprises receiving, via the user interface, a selection of a subsequence of characters within a first data cell, and wherein one or more additional characters within the first data cell are not selected within the first selection of data.

4. The method of claim 3, wherein generating the first regular expression comprises confirming that the first regular expression matches the entire first data cell, and wherein the method further comprises: determining one or more additional data cells within the displayed tabular data that match the first regular expression; determining, within each additional data cell that matches the first regular expression, a character subsequence that corresponds to the selected character subsequence within the first data cell; as well as An indication of a selection of each determined subsequence of characters within the additional data cell is displayed via the user interface.

5. The method of claim 4, further comprising: receiving, via the user interface, a selection of a batch modification operation within the displayed tabular data, wherein the selected batch modification operation corresponds to one of an extract operation, a replace operation, or a delete operation; as well as The selected batch modification operation is performed on the character subsequences in the first data cell and on the corresponding character subsequences in the additional data cells that match the first regular expression.

6. The method of claim 1 , wherein receiving the first selection of data comprises receiving, via the user interface, a selection of a first data cell or a subsequence of characters within the first data cell, and wherein the method further comprises: identifying a first subset of data cells within the displayed tabular data that match the first regular expression; as well as A first graphical indicator associated with each data cell in a first subset of data cells matching the first regular expression is displayed within the user interface.

7. The method of claim 6, further comprising: identifying a second subset of data cells within the displayed tabular data that do not match the first regular expression; as well as A second different graphical indicator is displayed within the user interface in association with each data cell in a second subset of the data cells that do not match the first regular expression.

8. The method of claim 1, further comprising: receiving, by the regular expression generator via the user interface, a third selection of data within the displayed tabular data; determining, based on the first selection and the third selection of data, that the first selection of data corresponds to a positive example to be matched with the generated regular expression and that the third selection of data corresponds to a negative example not to be matched with the generated regular expression; generating, by the regular expression generator, a third regular expression in response to the third selection of data, wherein the third regular expression matches the first selection of data but does not match the third selection of data within the displayed tabular data; as well as The third regular expression is displayed in the user interface by the regular expression generator.

9. A system for generating a regular expression, the system comprising: a processing unit comprising one or more processors; as well as a memory storing instructions that, when executed by the processing unit, cause the system to: Retrieving data from one or more data sources; Displaying the retrieved data in a table format in a user interface, wherein the displayed table data includes a plurality of data cells; receiving, via the user interface, a first selection of data from within the displayed tabular data; generating a first regular expression based on said first selection of data from within the displayed tabular data; Displaying the first regular expression in the user interface; After displaying the first regular expression, receiving, via the user interface, a second selection of data within the displayed tabular data; generating a second regular expression in response to the second selection of data, wherein the second regular expression matches both the first selection of data within the displayed tabular data and the second selection of data; and The second regular expression is displayed in the user interface.

10. The system of claim 9, wherein receiving the first selection of data comprises receiving a selection of a first data cell via the user interface, and wherein generating the first regular expression comprises confirming that the first regular expression matches the data within the first data cell.

11. The system of claim 10, wherein receiving the first selection of data comprises receiving, via the user interface, a selection of a subsequence of characters within a first data cell, and wherein one or more additional characters within the first data cell are not selected within the first selection of data.

12. The system of claim 11, wherein generating the first regular expression comprises confirming that the first regular expression matches the entire first data cell, and wherein the memory stores further instructions that, when executed by the processing unit, cause the system to: determining one or more additional data cells within the displayed tabular data that match the first regular expression; determining, within each additional data cell that matches the first regular expression, a character subsequence that corresponds to the selected character subsequence within the first data cell; and An indication of a selection of each determined subsequence of characters within the additional data cell is displayed via the user interface.

13. The system of claim 12, wherein the memory stores other instructions that, when executed by the processing unit, cause the system to: receiving, via the user interface, a selection of a batch modification operation within the displayed tabular data, wherein the selected batch modification operation corresponds to one of an extract operation, a replace operation, or a delete operation; and The selected batch modification operation is performed on the character subsequences in the first data cell and on the corresponding character subsequences in the additional data cells that match the first regular expression.

14. The system of claim 9, wherein receiving the first selection of data comprises receiving, via the user interface, a selection of a first data cell or a subsequence of characters within the first data cell, and wherein the memory stores further instructions that, when executed by the processing unit, cause the system to: identifying a first subset of data cells within the displayed tabular data that match the first regular expression; and A first graphical indicator associated with each data cell in a first subset of data cells matching the first regular expression is displayed within the user interface.

15. The system of claim 14, the memory storing other instructions that, when executed by the processing unit, cause the system to: identifying a second subset of data cells within the displayed tabular data that do not match the first regular expression; and A second different graphical indicator is displayed within the user interface in association with each data cell in a second subset of the data cells that do not match the first regular expression.

16. The system of claim 9, wherein the memory stores other instructions that, when executed by the processing unit, cause the system to: receiving, via the user interface, a third selection of data within the displayed tabular data; determining, based on the first selection of data and the third selection of data, that the first selection of data corresponds to a positive example to be matched with the generated regular expression and that the third selection of data corresponds to a negative example not to be matched with the generated regular expression; generating a third regular expression in response to the third selection of data, wherein the third regular expression matches the first selection of data but does not match the third selection of data within the displayed tabular data; and The third regular expression is displayed in the user interface.

17. A non-transitory computer-readable medium for generating a regular expression, the computer-readable medium comprising computer-executable instructions that, when executed on a computer system, cause the computer system to: Retrieving data from one or more data sources; Displaying the retrieved data in a table format in a user interface, wherein the displayed table data includes a plurality of data cells; receiving, via the user interface, a first selection of data from within the displayed tabular data; generating a first regular expression based on said first selection of data from within the displayed tabular data; Displaying the first regular expression in the user interface; After displaying the first regular expression, receiving, via the user interface, a second selection of data within the displayed tabular data; generating a second regular expression in response to the second selection of data, wherein the second regular expression matches both the first selection of data within the displayed tabular data and the second selection of data; and The second regular expression is displayed in the user interface.

18. A computer program product comprising instructions which, when executed by a computer, cause the computer to perform the method according to any one of claims 1 to 8.

Images