Automatic conversion of programs written in procedural programming languages to dataflow graphs and related systems and methods

Abstract

This paper describes tools for enhancing the functionality of data processing systems by converting programs written in procedural languages ​​into computer-executable components that can be used as data flow graphs. Data flow graphs generated from programs written in procedural programming languages ​​can support various forms of parallelism, such as pipelined parallelism and / or component parallelism. In some embodiments, parallelism can be achieved by parsing the program based on syntax and identifying control flow relationships between data operations performed by the program. Specifically, the types of data flow graph components can be identified based on syntax to correspond to specific data operations (or groups of data operations) of the program. Data flow graphs can be generated to include the identified components, which can be connected together by flows in order according to the control flow relationships between the identified data operations.

Description

Background Technology

[0001] Procedural programming is a paradigm in which a program is written as a series of explicit instructions to be executed sequentially by a processor. Subsequences of instructions can be grouped and called as needed by other parts of the program, thus extending the range of instructions that the rest of the program can execute. Depending on the specific programming language used, these subsequences are sometimes called subroutines, functions, or subroutines.

[0002] In contrast, dataflow programming emphasizes the flow of data through a series of operations performed by a program called a dataflow graph, which defines the operations and how data flows between them. A dataflow graph is typically configured to obtain data records from one or more sources, perform operations on those records sequentially, and produce some type of output. Generally, the parts of a dataflow graph that perform operations on the data records are called components, and the connections between components are called flows.

[0003] The difference between a program written in a procedural language and a data flow graph is that multiple operations can be executed concurrently in a data flow graph because operations can be performed on data records arriving at earlier components ("upstream components") in the data flow graph while other data records have already propagated to later components ("downstream" components). This phenomenon is a type of parallelism, sometimes referred to as pipelined parallelism. In this way, a data flow graph can be imagined as a series of workers on an assembly line, where components represent workers, while a program written in a procedural language is more like a single worker completing a series of tasks, where the next task only begins after the previous one is finished. Summary of the Invention

[0004] According to some aspects, a non-transitory computer-readable medium is provided, comprising instructions for converting a program into a data flow graph when executed by at least one processor, wherein the program includes data operations and a control flow indicating the execution sequence of the data operations, the method comprising analyzing the program using the at least one processor and a syntax stored by the at least one computer-readable medium, the analysis comprising: identifying a plurality of sequential steps of the program based on the syntax, wherein each of the plurality of steps includes one or more data operations of the program; generating a directory storing control flow relationships between the plurality of identified steps of the program; and for each of the identified plurality of steps of the program, based on the step... The data manipulation and, based on the syntax, identifies one of a plurality of data flow graph component types corresponding to the step, thereby identifying a plurality of data flow graph components each corresponding to a corresponding step in a plurality of steps of the program; and, based on each corresponding step in a plurality of steps of the program and based on the syntax, configures the plurality of data flow graph components, thereby generating a plurality of configured data flow graph components; and generates a data flow graph using the at least one processor, the data flow graph including: the plurality of configured data flow graph components, one or more streams representing the flow of data records between the one or more data flow graph components, wherein the one or more data flow graph components are connected via the one or more streams in the order of the generated directory.

[0005] According to some aspects, a computer-implemented method is provided for automatically converting a program into a data flow graph, wherein the program includes data operations and a control flow indicating the execution sequence of the data operations. The method includes: analyzing the program using at least one processor and a syntax stored by the at least one computer-readable medium, the analysis including: identifying a plurality of sequential steps of the program based on the syntax, wherein each of the plurality of steps includes one or more data operations of the program; generating a directory storing control flow relationships between the plurality of identified steps of the program; and for each of the identified plurality of steps of the program, based on the data operations of that step and based on... The syntax identifies one of a plurality of data flow graph component types corresponding to the step, thereby identifying a plurality of data flow graph components each corresponding to a corresponding step in a plurality of steps of the program; and configures the plurality of data flow graph components based on each corresponding step in the plurality of steps of the program and based on the syntax, thereby generating a plurality of configured data flow graph components; and generates a data flow graph using at least one processor, the data flow graph including: the plurality of configured data flow graph components, one or more streams representing the flow of data records between the one or more data flow graph components, wherein the one or more data flow graph components are connected via the one or more streams in the order of the generated directory.

[0006] According to some aspects, a system is provided for automatically converting a program into a data flow graph, wherein the program includes data operations and a control flow indicating the execution sequence of these data operations. The system includes: means for analyzing the program using at least one processor and a syntax stored by the at least one computer-readable medium, the analysis including: identifying a plurality of sequential steps of the program based on the syntax, wherein each of the plurality of steps includes one or more data operations of the program; generating a directory storing control flow relationships between the plurality of identified steps of the program; and for each of the identified plurality of steps of the program, based on the data operations of that step and... In this syntax, one of a plurality of data flow graph component types is identified as corresponding to the step, thereby identifying a plurality of data flow graph components each corresponding to a corresponding step in a plurality of steps of the program; and based on each corresponding step in the plurality of steps of the program and based on this syntax, the plurality of data flow graph components are configured to generate a plurality of configured data flow graph components; and an apparatus for generating a data flow graph, the data flow graph including: the plurality of configured data flow graph components; and one or more streams representing the flow of data records between the one or more data flow graph components, wherein the one or more data flow graph components are connected via the one or more streams in the order of the generated directory.

[0007] Embodiments of the foregoing apparatus and methods may be implemented using any suitable combination of aspects, features, and actions further described in detail above or below. These and other aspects, embodiments, and features of this teaching can be more fully understood from the following description and in conjunction with the accompanying drawings. Attached Figure Description

[0008] Various aspects and embodiments will be described with reference to the following accompanying drawings. It should be understood that the drawings are not necessarily drawn to scale. In the drawings, every identical or nearly identical component shown in the various figures is represented by the same numbers. For clarity, not every component may be labeled in every figure.

[0009] Figure 1 An illustrative data flow diagram is depicted according to some embodiments;

[0010] Figure 2 A flowchart is depicted illustrating a method for automatically converting a program written in a procedural language into a data flow graph according to some embodiments;

[0011] Figure 3 This is a flowchart of a method for automatically converting a program written in a procedural language into a data flow diagram according to some embodiments;

[0012] Figure 4A These are illustrative SAS programs based on some embodiments;

[0013] Figure 4B Depicting according to some embodiments, according to Figure 4A An illustrative data flow diagram generated by the procedure shown;

[0014] Figure 4C Depicting according to some embodiments Figure 4B An illustrative data flow diagram, in which the data flow diagram components are annotated with the corresponding steps of the SAS program;

[0015] Figure 4D Demonstrates according to some embodiments Figure 4A The first step of the program shown is converted to Figure 4B Details of the first component of the data flow diagram shown;

[0016] Figure 4E This demonstrates the use of syntax to parse according to some embodiments. Figure 4A The steps shown are an example of identifying the step and the control flow relationship between the step and other steps, selecting the corresponding data flow graph component, and configuring the selected data flow graph component.

[0017] Figure 4F Demonstrates according to some embodiments Figure 4A The second step of the procedure shown is converted to Figure 4B Details of the second component of the data flow diagram shown; and

[0018] Figure 5 Examples of computing system environments on which various aspects of the present invention can be implemented are shown. Detailed Implementation

[0019] The inventors have recognized and understood that the functionality of a data processing system can be enhanced by converting a program written in a procedural language into a computer-executable tool that can be used as a data flow graph. Specifically, the computer-executable tool can identify the types of data flow graph components corresponding to different parts of the program, allowing the behavior of each part to be reproduced through the appropriate portions of the data flow graph. In some embodiments, the graph generated from a program written in a procedural programming language can support various forms of parallelism, such as pipelined parallelism and / or component parallelism, which programs written in procedural languages ​​may lack. In some embodiments, parallelism can be achieved by parsing the program based on syntax and identifying the control flow relationships between data operations performed by the program. Furthermore, syntax can be used to identify the types of data flow graph components corresponding to specific data operations (or groups of data operations) of the program. The data flow graph can be generated to include the identified components, which can be connected together by flows in order according to the control flow relationships between the identified data operations.

[0020] Compared to procedural programming, using dataflow programming to process data can offer several advantages. In particular, as discussed above, dataflow graphs naturally provide parallel processing through pipelined parallelism, making dataflow programming a natural choice for processing datasets where specific operations are repeated across many segments of data, such as the data records of the dataset. Another type of parallelism provided by dataflow graphs is component parallelism, which arises when different branches of the dataflow graph operate simultaneously. For example, data records from a single data source can be output along different flows, and different operations can be executed concurrently along different branches.

[0021] While data flow diagrams offer some advantages, some organizations (e.g., enterprises) may still rely on programs written in procedural languages ​​for data processing. In some cases, organizations can partially utilize data flow diagrams while still leveraging programs written in procedural languages. This approach typically involves executing the data flow diagram instead of the previously executed program in the procedural language, while incorporating components of the code for the procedural language program—which executes externally—into the data flow diagram. In this way, the data flow diagram acts as a kind of "wrapper," and although the data flow diagram is ultimately a program executed by the system, a significant portion of the program's functionality can still be executed externally by the procedural language.

[0022] One drawback of this approach is that the language of the data flow diagram may not understand what is happening within a component executing external code written in a procedural language. Another useful feature of data flow diagrams is the ability to trace data progression through a single or multiple data flow diagrams to understand where data comes from and goes in an organization's operations. However, the component executing external code may be like a "black box" to the data flow diagram software in terms of data progression, as the software may not be able to trace this progression because the operations performed by that component are unknown to the software. Furthermore, this approach may hinder pipeline parallelism, as downstream components may need to execute after the external code has finished.

[0023] As discussed above, the inventors have recognized and understand that the functionality of data processing systems can be enhanced by converting programs written in procedural languages ​​into computer-executable tools that can be executed as components of a data flow graph. These techniques can be used to generate native data flow graph code within the resulting entire data flow graph, enabling the tracing of data progression through the data flow graph. The inherent efficiency advantages of data flow graphs (such as the component parallelism and / or pipeline parallelism discussed above) are naturally provided in the resulting data flow graph. The inventors further recognize and understand the techniques for implementing such tools based on syntax, as discussed further below.

[0024] According to some embodiments, a system for converting a program written in a procedural language into a data flow graph can execute a parser to parse the text of the program. In some embodiments, this parsing can identify the operations performed by the program on data, and can further identify the type of such operations. Based on the type of data operations performed by the program, specific types of data flow graph components capable of performing the same operations can be identified by accessing data structures configured to associate specific operations with specific data flow graph components. For example, if the language within the program is recognized by the parser as performing a merge between data, then the data flow graph component that combines streams of data records can be identified by a data structure as corresponding to a merge operation. In some embodiments, such a parser can also determine the execution order of the data operations performed by the program. When a data flow graph is generated based on the program, streams representing data operations between components can be generated, and components can be connected together via these streams in an order based on the determined execution order.

[0025] The following is a more detailed description of various concepts related to automatically converting programs written in procedural languages ​​into data flow diagrams, various embodiments of automatically converting programs written in procedural languages ​​into data flow diagrams, and various techniques for automatically converting programs written in procedural languages ​​into data flow diagrams. It should be understood that the various aspects described herein may be implemented in any of a variety of ways. This document provides examples of specific implementations for illustrative purposes only. Furthermore, the various aspects described in the following embodiments can be used alone or in any combination, and are not limited to the combinations explicitly described herein.

[0026] Figure 1 An illustrative data flow diagram is depicted according to some embodiments. Figure 1 The diagram 100 illustrates some of the features described above regarding data flow diagrams. A data flow diagram is a visual representation of the flow of data records through components and flows. As mentioned herein, a component is the part of a data flow diagram that performs operations on data records, and the connections between components are referred to herein as flows.

[0027] Typically, data flow diagrams are created by arranging components and flows in a graphical user interface. Figure 1 This example illustrates this view of a data flow diagram, where rectangles represent components and the lines between these rectangles each represent a flow. Data flow diagrams are a convenient way to write programs that manipulate data records because the data flow can be visualized when viewing the diagram. Once the data flow is ready to be executed, it can be compiled and then executed by a suitable computing system.

[0028] exist Figure 1 In the example, three data inputs 102, 104, and 106 are accessed, manipulated, and combined in a manner described below to produce two data outputs 122 and 124. Data inputs can be provided from various sources, including files, collections of files, or database tables stored on computer-readable media. Figure 1 In the example, data inputs 102, 104, and 106 are depicted in a manner typically used to depict a single data file stored on a computer-readable medium.

[0029] exist Figure 1 In the example, input data 102 is read and manipulated by filtering component 108, which outputs data records that satisfy a specific logical expression. For example, if input data 102 includes a data field specifying a country code, filter 108 can be configured to output only those data records where the value of that data field is "US". The data records output from filter 108 are input to sorting component 110, which sorts the data records in an order specified by a selected sorting "key" that identifies one or more data fields on which the records are to be sorted. The sorted data records are output to combining component 120.

[0030] Additionally, input data 104 is read and manipulated by transformation component 112, which manipulates the input data records according to transformation codes stored within (or otherwise accessed by) transformation component 112. For example, the transformation may add or remove data fields for each data record, and / or may manipulate the values ​​of data fields when generating output data. It should be noted that in Figure 1 In the example, filtering component 108 and transformation component 112 exhibit component parallelism. That is, while data records from input data 104 are being processed by transformation 112, data records from input data 102 can be processed by filtering 108. In some cases, component parallelism can be utilized by using different processors to execute filtering component 108 and transformation component 112, and in other cases, it can be utilized by using different computing systems to execute the filtering component and transformation component.

[0031] exist Figure 1In the example, the combining component 116 receives data records output from transform 112 and data records output from filtering component 114 (which operates as discussed above with respect to filtering 108). The combining component can be configured to produce a single set of data records from the two sets of data records by matching the records of each set according to the key of a data field that identifies each set of input data records. Transformation code stored within (or otherwise accessed by) the combining component can further specify how the output data records of the combining component are constructed based on the data field values ​​contained in the matching records of the two sets of input data records.

[0032] The copy component 118 copies the data records output from the combination component 116 to produce two identical sets of data records. One set of data records is stored as output data 124, while the other set is combined with the data records output from the sorting component 110, and the result is stored as output data 122.

[0033] Typically, components in a data flow diagram are configured to define their functionality to some extent. For example, in Figure 1 In the examples of filtering and transformation components, these components are configured by specifying transformation codes for how to select or transform records, respectively. In some cases, transformation components may also define an output record format different from the input data record format. Sorting components can be configured by specifying a sort key, and in some cases by specifying how to perform the sorting (e.g., ascending versus descending values). In each of these cases, a configuration method is applied to describe how the component should operate on the data records. The data describing this configuration is collectively referred to herein as "configuration data." Therefore, transformation codes, key values, record formats, etc., are each types of configuration data. In at least some cases, Figure 1 The copy component in the example can work without defining configuration data because the component is designed to simply copy the data stream and may not require configuration in other respects.

[0034] Figure 2 A flowchart illustrating a method for automatically converting a program written in a procedural language into a data flow graph according to some embodiments is provided. As discussed above, according to some aspects of this application, the functionality of a data processing system can be enhanced by a computer-executable tool that converts a program written in a procedural language into a component that can be executed as a data flow graph. Method 200 illustrates how such a tool can be operated to perform this conversion by converting program 201 into data flow graph 208.

[0035] exist Figure 2In the example, a program 201 written in a procedural language is provided as input to a conversion engine 205, which includes one or more computing devices that execute the tools discussed above. Program 201 can be written in any procedural language, as the techniques described herein are not limited to any particular language. Examples of suitable languages ​​include Statistical Analysis System (SAS), BASIC, Fortran, Java, Mathematica, MATLAB, Pascal, Perl, Python, R, Unix shell, and Visual Basic.

[0036] During execution by the transformation engine, the tool can utilize predefined syntax 206. Syntax 206 provides various definitions upon which the tool, executed by transformation engine 205, relies to allow the tool to parse program 201 and generate data flow graph 208. These definitions provided by syntax 206 can instruct the tool on how to interpret words and symbols within program 201, how to identify data operations (or groups of data operations) within the program as having functionality corresponding to specific types of data flow graph components, how to generate data flow graph components, how to identify data operations (or groups of data operations) within the program as being associated with configuration data of the identified data flow graph component types, how to configure data flow graph components corresponding to data operations within the program, and / or how to determine the control flow of data operations within the program.

[0037] As used in this article, "control flow" refers to the order in which various data operations are performed, consistent with a program. For example, a program can perform operations C, D, and E, and can be configured to perform operation C before either operation D or operation E. This dependency on the relative order of operations C, D, and E is an example of control flow.

[0038] exist Figure 2 In the example, the data generated by parsing the program using this tool can be stored in directory 207. This directory can include any suitable data structure to store, for example, the control flow relationships determined by data operations within the program. As further described below, this directory can be accessed by the transformation engine during subsequent parsing operations.

[0039] As mentioned in this article, a program's "data operation" refers to a part of the program that, when executed, modifies the program's execution environment. For example, a data operation may include modifying the values ​​of one or more variables or other data existing in the execution environment. It should be understood that the modification of a value itself can include one or more data operations. For instance, when a command in a procedural program sets the value of a variable to be equal to the value of a computation result, each part of the computation can represent a separate data operation, and the final result storing the computational value in the variable also represents a separate data operation. Therefore, in at least some cases, a single command of a program can represent multiple data operations because the execution environment changes various times during command execution.

[0040] According to some embodiments, the transformation engine 205 can generate one or more components and flows of the data flow graph 208 corresponding to a series of data operations in program 201. In some cases, the components generating the data flow graph 208 may include identifying a type of component from a plurality of available data flow graph components. In some cases, the components generating the data flow graph 208 may also include generating configuration data for the identified components, which may include defining parameters or other configurable aspects of the components based on data operations within the program. For example, Figure 2 An instance of the illustrative data flow graph transformation code 209 shown can be generated by a tool executed by the transformation engine and can be associated with the identified components.

[0041] According to some embodiments, a tool executed by conversion engine 205 can generate flows between identified components in data flow graph 208 based on the determined control flow relationships of data operations within the program. For example, if the tool identifies a first component and a second component for specific data operations (or multiple sets of data operations), the tool can also identify that the data operation(s) associated with the first component(s) occurs earlier in the control flow than the data operation(s) associated with the second component(s). As a result, the tool can generate one or more flows connecting the first and second components, such that the second component executes downstream of the first component in the data flow graph (i.e., in the component sequence, data flows out of the first component earlier than data flows into the second component).

[0042] The net result of the process executed by the transformation engine 205 based on syntax 206 is the generation of a data flow graph 208, which is a data flow graph representation of program 201. This relationship between the program and the equivalent data flow graph is... Figure 2 And as shown by the zigzag arrows in the subsequent attached diagrams.

[0043] Figure 3This is a flowchart of a method for automatically converting a program written in a procedural language into a data flow diagram according to some embodiments. Method 300 is an illustrative process for automatically converting a program 301 written in a procedural language into a data flow diagram. For example, method 300 can be achieved by... Figure 2 The tool shown in the diagram and executed by the conversion engine 205 discussed above is used to perform the conversion.

[0044] Method 300 begins with action 302, whereby program 301 is divided into steps based on syntax 303. Figure 2 Similar to the example of grammar 206, the tool executing method 300 can rely on grammar 303 to specifically instruct how to interpret words and symbols within the program, and how to identify data operations (or groups of data operations) within the program as having functionality corresponding to a specific type of data flow graph component. Specifically, in action 302, the tool can use grammar 303 to parse program 301 to identify one or more data operations. Each group of one or more data operations can be referred to as a "step," such that the program's data operations are divided into a series of steps, where each step contains one or more data operations performed by the program.

[0045] In action 304, the tool executing method 300 may store or otherwise record information identifying the steps in directory 305. The directory generated in action 304 may identify the steps of the program and the control flow relationships of the steps, and may further identify one or more data operations of program 301 corresponding to each step.

[0046] In action 306, the tool executing method 300 selects a data flow graph component type corresponding to the identified (and possibly already stored in a directory in action 304) program step. The selected data flow graph component type can be based on the following operation: using syntax 303 to parse the data operations of the identified step, and based on the parsing, determining which type of data flow graph component corresponds to the data operations of that step. As discussed above, syntax 303 can be configured to associate a specific type of data flow graph component with a specific language within the program, such that data flow graph components that perform the same function as steps in which that language appears can be appropriately generated.

[0047] In some embodiments, the tool may use a pre-configured library of data flow graph component types, thereby enabling the identification of data flow graph components to include selecting one of the component types from said library (e.g., as per the information provided). Figure 1 (Components of the "sorting" or "filtering" type discussed). In some embodiments, instances of data flow graph components of the selected component type with functions corresponding to program steps can be generated.

[0048] In action 308, the tool executing method 300 can configure the data flow graph components selected in action 306. Action 308 may accordingly include generating configuration data for one or more components selected in action 306. The tool can access syntax to determine how to configure a given data flow graph component based on the data operations present in the corresponding program steps. As discussed above, the data describing the configuration of a data flow graph component is referred to herein as "configuration data". Therefore, action 308 may include generating values ​​for the component's operational parameters and / or generating transformation code for the component. It should be understood that some of the components identified in action 306 may not require configuration data, and therefore, configuration data may not be generated for all identified components.

[0049] In action 310, the tool performing method 300 can generate a data flow graph that includes (where applicable) identified components configured by action 308 (e.g., including generated configuration data) and has flows connecting these components in an order according to the identified program step control flow relationships. In some embodiments, action 310 may include recording the identified and configured components and the generated flows connecting these components together to a suitable computer-readable storage medium. For example, the data flow graph may be stored as a data file including data describing or otherwise identifying the data flow graph components, flows, and configuration data of these components, and action 310 may include recording the data file. In some embodiments, such a data file may be arranged to be compiled by another program prior to execution. In some embodiments, the generated data flow graph may be produced as an executable data file, in which case action 310 may include the step of compiling the data flow graph according to the identified data flow graph components, flows, and configuration data of these components. Action 310 may include generating flows to connect the data flow graph components in an order according to the control flow relationships identified by the directory.

[0050] According to some embodiments, program 301 may include one or more procedures (also referred to as routines, functions, subroutines, or macros), and the tools executing method 300 can identify these procedures when parsing the program using syntax parsing. For example, the tools can identify one or more procedure calls and / or can identify procedure definitions. In some embodiments, the tools executing method 300 can generate a reusable set of data flow graph components (sometimes referred to as "subgraphs") to represent the data operations of the identified procedures. In some cases, the subgraph can be stored in a data file accessible to the generated data flow graph. This allows the generated data flow graph to reuse the same set of components in a manner similar to how program 301 makes repeated procedure calls and thus reuses the same piece of code. The subgraph can represent an entire identified program step, or it can represent a portion of an identified program step.

[0051] When the tool executing method 300 generates a subgraph to represent the process of program 301, the process can use one or more values ​​as inputs. In this case, the generated subgraph can be parameterized such that the behavior of the data flow graph components in the subgraph differs based on the values ​​of the subgraph parameters. For example, branches of components can be enabled or disabled in the subgraph based on the component values.

[0052] According to some embodiments, program 301 may include one or more global variables, or otherwise operate based on one or more global variables, which are values ​​defined in the execution environment and can modify the way data operations in the program are performed. In this case, the tool executing method 300 may generate one or more data flow graph parameters, which are parameters set at the data flow graph level, and whose values ​​can be accessed by one or more components of the data flow graph to specify the behavior of that component. As an illustrative example, program 301 may be written to initialize global variables with country code values ​​and perform subsequent data processing based on said values. The generated data flow graph can be defined using graph parameters to define the country codes, and components of the data flow graph may utilize the parameter definition in transformation code or otherwise utilize the parameter definition.

[0053] According to some embodiments, the tool executing method 300 can identify comments within program 301. Comments are parts of the program that are not executed and are typically identified at the beginning or end of the comment section using delimiters. In some cases, the tool executing method 300 can identify such comments within the program and copy those comments to relevant locations within the generated data flow graph. For example, the tool can identify comments as related to specific data operations within the program based on the proximity of the comment text to the data operation. Such comments can be placed in the transformation code of the component corresponding to those data operations, so that any information within those comments can be preserved in the generated data flow graph.

[0054] According to some embodiments, the tool executing method 300 can recognize and preserve whitespace within program 301 when generating a data flow graph. As mentioned herein, "whitespace" refers to any character or series of characters representing horizontal or vertical space in the layout of a program, the most common example being the "space" character. In some cases, the tool executing method 300 can recognize such whitespace in the data operations of the program and can preserve the presence of whitespace when generating transformation code (or other configuration data) for data operations within components of the data flow graph.

[0055] Figures 4A to 4FIllustrative examples are provided for converting programs written in the procedural language SAS into data flow diagrams according to some embodiments. Figure 4A An illustrative SAS program is depicted, which performs a series of operations on two input data files and produces an output data file. As discussed above, computer-executable tools can be used to transform the demonstrated program into components that can be executed as data flow diagrams.

[0056] According to some embodiments, when executing and providing program 400 as input, the tool can identify four steps of the program labeled 401a, 401b, 401c, and 401d. (See above regarding...) Figure 3 As discussed in action 302, the tool can access the syntax to identify data operations or groups of data operations corresponding to the program steps. In illustrative procedure 400, these steps are configured to perform the following actions: Step 401a reads a data file named “FILE_A”, parses its contents into three data fields, and creates an additional data field (ORDER_FULL_NAME) based on the contents of these three data fields. Step 401b reads a data file named “FILE_B”, parses its contents into three data fields, and creates an additional data field (ORDER_FULL_NAME) based on the contents of these three data fields. Step 401c merges the results of steps 401a and 401b. Step 401d filters out any data records that do not have a specific value (“US”) in the first two characters of the ORDER_FULL_NAME data field.

[0057] According to some embodiments, the executed tool can identify the contents of these portions of program 400 as corresponding to operations that can be performed by components of a data flow graph. Specifically, the tool can identify the contents of code portions labeled 401a and 401b as each performing a transformation on the read data file. Further, the tool can identify the contents of code portion labeled 401c as performing a combination operation, and the contents of code portion labeled 401d as performing a filtering operation. Furthermore, the tool can identify the control flow relationship of these steps by recognizing that step 401c utilizes the results of steps 401a and 401b (“MERGEDATASET_A DATASET_B”) and step 401d utilizes the results of step 401c (“SET DATASET_C”). As a result, it can be determined that... Figure 4B The components and flows are shown in the diagram. The tool can also generate appropriate configuration data for components 406, 408, 410, and 412. As a final step, the tool can then generate, as shown in the diagram. Figure 4B The data flow diagram shown includes the generated configuration data.

[0058] Figure 4C Will Figure 4B Shown along with the annotations associated with components 406, 408, 410, and 412, these annotations illustrate... Figure 4A The corresponding steps of program 400 are shown. The procedural language code corresponding to the components of data flow diagram 450 is shown as connected by gray zigzag arrows. Figure 4C The components described in the data flow diagram 450. Therefore, it should be understood that the commented portions of program 400 are not stored verbatim or otherwise executed by the corresponding components in data flow diagram 450. Rather, the components of data flow diagram 450 perform functions equivalent to the associated portions of the program.

[0059] Figure 4D Demonstrates according to some embodiments Figure 4A The first step of the program shown is converted to Figures 4B to 4C The data flow diagram shown includes details of the first component. To further illustrate the process of configuring a component by having a tool generate configuration data for that component,... Figure 4D It shows Figure 4A Step 401a shows how the resulting data flow graph portion can be generated.

[0060] exist Figure 4D In the example, as a result of the data operation "INFILE FILE_A" (which instructs the program to read FILE_A) in part 401a, stream 405 is generated to connect the identified transformation component 406 to the input file FILE_A. Furthermore, transformation component 406 is configured to read a series of bytes from the input file as specified by the program's data operation "INPUT ORDER_NAME$10EXT$4CHANGE_FLAG$1," which instructs that ten bytes be read as the value of the data field ORDER_NAME, the next four bytes as the value of the data field EXT, and the next byte as the value of the data field CHANGE_FLAG. Assuming additional bytes are present, component 406 can be configured to continue reading bytes from the input FILE_A; in this way, each 15-byte block in the input file becomes part of a continuous data record.

[0061] Additionally, component 406 is configured with transformation code 451 to generate additional output data fields by concatenating the values ​​of the three fields read from the input file with underscore strings, as specified in the data operation defined in program section 401a for ORDER_FULL_NAME. Component 406 can be configured to generate data records on stream 407 according to the presented output record format 452. The tool for generating the data flow graph can generate record format 452 using syntax interpreter section 401a to identify the type and length of the data fields produced by this step of the program. In this case, the lengths of the ORDER_NAME, EXT, and CHANGE_FLAG fields can be inferred from how they are read from the input file, and the length of the ORDER_FULL_NAME field can be inferred as the result of concatenating the other three fields with two underscore strings, totaling 15 characters.

[0062] Figures 4E to 4F Demonstrates according to some embodiments Figure 4A The second step of the procedure shown is converted to Figures 4B to 4C Details of the second component of the data flow diagram shown. Figure 4E It reproduced Figure 4A The content of step 401c is described, and three different parsing operations for this step are described using syntax 480.

[0063] exist Figure 4E In the first instance, the program uses syntax 480 to parse the shown data operation 481 to identify these data operations as steps, namely step 401c mentioned above. The tool can use syntax 480 to identify data operation 481 as corresponding to the step based on the highlighted language, which: a) identifies data operations that begin with "DATA.." and end with "...RUN;" as defining a step; and b) identifies that the step depends on "DATASET_A" and "DATASET_B", and this thus implies a specific control flow relationship between the step and the rest of the program.

[0064] exist Figure 4E In the second instance, the tool can use syntax 480 to parse the illustrated data operation 482 of the procedure (which may now be identified as corresponding to step 401c) to determine the type of data flow graph component corresponding to the function of that step in the procedure. Figure 4E In the example, the syntax can be used to identify the highlighted keyword "MERGE" as corresponding to a data flow graph component of type "JOIN".

[0065] exist Figure 4EIn the third instance, the tool can use the data operation 483 shown in the syntax 480 parser (which may now be identified as corresponding to step 401c) to determine how to configure instances of data flow graph components of the type selected in the second parsing instance above. Figure 4E In the example, the highlighted language "BYORDER NAME;" can be identified using syntax as an indication that an instance of the "JOIN" data flow graph component should be configured to use a field named "ORDER NAME" as the key field for the JOIN operation.

[0066] therefore, Figure 4E Syntax 480 provides an example of how it can generate these three different types of information about a part of a program by parsing that part. While parsing has been described above in terms of three separate parsing instances, it should be understood that tools can utilize Syntax 480 to parse parts of a program in any number of ways, each parsing any suitable portion of the program in any order, and the above is provided only as an illustrative process.

[0067] Based on the analysis of step 401c above, Figure 4F This demonstrates how to generate the resulting data flow diagram portion. Figure 4F In the example, streams 407 and 409 provide data records with the shown input record format from two upstream transformation components 406 and 408. During step 401c of the parsing procedure, the tool can identify which combining components are suitable as the result of the "MERGE" command in the program, and can further generate configuration data for the combining components, specifying the ORDER_NAME combining key 462 as a field for matching the incoming records to be combined. As a result of the combining, the only records that can be output from component 410 along stream 411 are those records that have the same ORDER_NAME value in both sets of incoming data records from the transformation components.

[0068] Figure 5 An example of a suitable computing system environment 500 on which the techniques described herein can be implemented is shown. The computing system environment 500 is merely one example of a suitable computing environment and is not intended to impose any limitation on the scope of use or functionality of the techniques described herein. Nor should the computing environment 500 be construed as having any dependencies or requirements relating to any one or combination of the components shown in the exemplary operating environment 500.

[0069] The techniques described herein operate in conjunction with many other general-purpose or special-purpose computing system environments or configurations. Examples of well-known computing systems, environments, and / or configurations that may be suitable for use with the techniques described herein include, but are not limited to: personal computers, server computers, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the systems or devices described above, etc.

[0070] A computing environment can execute computer-executable instructions, such as program modules. Typically, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. The techniques described in this paper can also be practiced in distributed computing environments, where tasks are performed by remote processing devices linked via a communication network. In a distributed computing environment, program modules can reside in local and remote computer storage media, including memory storage devices.

[0071] Reference Figure 5 An exemplary system for implementing the techniques described herein includes a general-purpose computing device in the form of a computer 510. Components of computer 510 may include, but are not limited to, a processing unit 520, system memory 530, and a system bus 521 that couples various system components, including the system memory, to the processing unit 520. System bus 521 may be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, and a local bus using any architecture of various bus architectures. By way of example and not limitation, such architectures include the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MCA) bus, the Enhanced ISA (EISA) bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus (also known as a mezzanine bus).

[0072] Computer 510 typically includes a variety of computer-readable media. Computer-readable media can be any available medium accessible to computer 510, and includes volatile and non-volatile media, removable and non-removable media. By way of example, and not limitation, computer-readable media can include computer storage media and communication media. Computer storage media includes volatile and non-volatile media, removable and non-removable media implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, Digital Universal Disc (DVD) or other optical disc storage devices, magnetic tape cassettes, magnetic tape, disk storage devices or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to computer 510. Communication media typically implement computer-readable instructions, data structures, program modules, or other data in the form of modulated data signals such as carrier waves or other transmission mechanisms, and includes any information delivery medium. The term "modulated data signal" refers to a signal in which one or more of its characteristics are set or altered in such a way as to encode information in the signal. By way of example and not limitation, communication media include wired media (such as wired networks or direct wired connections) and wireless media (such as acoustic, RF, infrared, and other wireless media). Any combination of the above should also be included within the scope of computer-readable media.

[0073] System memory 530 includes computer storage media in the form of volatile and / or non-volatile memory, such as read-only memory (ROM) 531 and random access memory (RAM) 532. A basic input / output system 533 (BIOS), containing basic routines such as those that facilitate the transfer of information between components within computer 510 during startup, is typically stored in ROM 531. RAM 532 typically contains data and / or program modules that can be immediately accessed and / or are currently in operation by processing unit 520. This is by way of example and not limitation. Figure 5 The operating system 534, application program 535, other program modules 536, and program data 537 are shown.

[0074] Computer 510 may also include other removable / non-removable, volatile / non-volatile computer storage media. This is by way of example only. Figure 5A hard disk drive 541 for reading or writing non-removable non-volatile magnetic media, a flash drive 551 for reading or writing removable non-volatile memory 552 (such as flash memory), and an optical disc drive 555 for reading or writing removable non-volatile optical disc 556 (such as CD ROM or other optical media) are illustrated. Other removable / non-removable, volatile / non-volatile computer storage media that can be used in the exemplary operating environment include, but are not limited to, magnetic tape cassettes, flash memory cards, digital universal disks, digital videotapes, solid-state RAM, solid-state ROM, etc. Hard disk drive 541 is typically connected to system bus 521 via a non-removable memory interface such as interface 540, and disk drive 551 and optical disc drive 555 are typically connected to system bus 521 via a removable memory interface such as interface 550.

[0075] The above discussion and Figure 5 The drive and its associated computer storage media shown provide storage for computer-readable instructions, data structures, program modules, and other data for computer 510. For example, in Figure 5 In this diagram, hard disk drive 541 is shown storing operating system 544, application programs 545, other program modules 546, and program data 547. Note that these components may be the same as or different from operating system 534, application programs 535, other program modules 536, and program data 537. Different designations are given to operating system 544, application programs 545, other program modules 546, and program data 547 to indicate that they are at least different copies. Users can input commands and information into computer 510 through input devices such as keyboard 562 and pointing device 561 (commonly referred to as mouse, trackball, or touchpad). Other input devices (not shown) may include microphones, joysticks, game controllers, disc satellite dishes, scanners, etc. These and other input devices are typically connected to processing unit 520 via user input interface 560 coupled to the system bus, but may be connected via other interfaces and bus structures such as parallel ports, game ports, or Universal Serial Bus (USB). Monitor 591 or other types of display devices are also connected to system bus 521 via interfaces such as video interface 590. In addition to the monitor, the computer may also include other peripheral output devices, such as speakers 597 and printers 596, which can be connected via the peripheral output interface 595.

[0076] Computer 510 can operate in a networked environment using a logical connection to one or more remote computers (such as remote computer 580). Remote computer 580 can be a personal computer, server, router, network PC, peer-to-peer device, or other common network node, and typically includes many or all of the elements described above relative to computer 510, but in... Figure 5 Only the memory storage device 581 is shown in the image. Figure 5 The logical connections described include Local Area Networks (LANs) 571 and Wide Area Networks (WANs) 573, but may also include other networks. This networking environment is common in offices, enterprise-wide computer networks, intranets, and the Internet.

[0077] When used in a LAN networking environment, computer 510 connects to LAN 571 via a network interface or adapter 570. When used in a WAN networking environment, computer 510 typically includes a modem 572 or other device for establishing communication over a WAN 573, such as the Internet. Modem 572 may be built-in or external and may be connected to system bus 521 via user input interface 560 or other suitable mechanism. In a networking environment, program modules or portions thereof described with respect to computer 510 may be stored in a remote memory storage device. By way of example and not limitation, Figure 5 The diagram shows a remote application 585 residing on a memory device 581. It should be understood that the network connection shown is exemplary, and other means of establishing communication links between computers can be used.

[0078] Having described several aspects of at least one embodiment of the present invention, it should be understood that various changes, modifications, and improvements will readily occur to those skilled in the art.

[0079] Such changes, modifications, and improvements are intended to be part of this disclosure and are intended to fall within the spirit and scope of the invention. Furthermore, while advantages of the invention have been indicated, it should be understood that not every embodiment of the technology described herein will include every described advantage. Some embodiments may not implement any features described herein as advantageous, and in some cases, one or more of the described features may be implemented to achieve further embodiments. Therefore, the preceding description and figures are by way of example only.

[0080] The embodiments of the technology described herein can be implemented in any of a variety of ways. For example, these embodiments can be implemented using hardware, software, or a combination thereof. When implemented in software, the software code can be executed on any suitable processor or set of processors, whether the processor or set of processors is located in a single computer or distributed across multiple computers. Such processors can be implemented as integrated circuits (wherein an integrated circuit component has one or more processors), including commercially available integrated circuit components known in the art, such as CPU chips, GPU chips, microprocessors, microcontrollers, or coprocessors. Alternatively, the processor can be implemented as a custom circuit system (such as an ASIC) or a semi-custom circuit system resulting from configuring a programmable logic device. As yet another alternative, the processor can be part of a larger circuit or a semiconductor device, whether commercial, semi-custom, or custom. As a specific example, some commercial microprocessors have multiple cores, such that one or a subset of these cores can constitute a processor. However, the processor can be implemented using a circuit system in any suitable format.

[0081] Furthermore, it should be understood that a computer can be implemented in any of a variety of forms, such as a rack-mount computer, desktop computer, laptop computer, or tablet computer. Additionally, a computer can be embedded in a device that is not typically considered a computer but has suitable processing capabilities, including a personal digital assistant (PDA), smartphone, or any other suitable portable or stationary electronic device.

[0082] Furthermore, a computer may have one or more input and output devices. These devices can be used, in particular, to present a user interface. Examples of output devices that can be used to provide a user interface include printers or displays for visual presentation of output, and speakers or other sound-generating devices for auditory presentation of output. Examples of input devices that can be used for a user interface include keyboards and pointing devices (such as mice, touchpads, and digitizers). As another example, a computer may receive input information via speech recognition or in other audible formats.

[0083] Such computers can be interconnected by one or more networks of any suitable form, including local area networks (LANs) or wide area networks (WANs), such as corporate networks or the Internet. These networks can be based on any suitable technology and can operate according to any suitable protocol, and can include wireless networks, wired networks, or fiber optic networks.

[0084] Furthermore, the various methods or processes outlined in this paper can be encoded as software executable on one or more processors employing any of a variety of operating systems or platforms. Additionally, such software can be written using any of a variety of suitable programming languages ​​and / or programming or scripting tools, and can also be compiled into executable machine language code or intermediate code that executes on a framework or virtual machine.

[0085] In this regard, the invention can be implemented as a computer-readable storage medium (or multiple computer-readable media) encoded with one or more programs (e.g., computer memory, one or more floppy disks, compact optical discs (CDs), optical discs, digital video discs (DVDs), magnetic tape, flash memory, circuit configurations in field-programmable gate arrays or other semiconductor devices, or other tangible computer storage media), which, when executed on one or more computers or other processors, performs methods for implementing the various embodiments of the invention discussed above. As will be clear from the foregoing examples, a computer-readable storage medium can retain information for a sufficient time to provide computer-executable instructions in a non-transitory form. Such a computer-readable storage medium can be transportable such that one or more programs stored thereon can be loaded onto one or more different computers or other processors to implement the various aspects of the invention discussed above. As used herein, the term "computer-readable storage medium" covers only non-transitory computer-readable media that can be regarded as articles of manufacture (i.e., articles of manufacture) or machines. Alternatively or additionally, the invention can be implemented as a computer-readable medium other than a computer-readable storage medium, such as a propagating signal.

[0086] The terms "program" or "software" are used in a general sense as used herein to refer to any type of computer code or computer-executable instruction set that can be used to program a computer or other processor to implement the various aspects of the invention discussed above. Furthermore, it should be understood that, according to one aspect of this embodiment, one or more computer programs that perform the methods of the invention when executed do not necessarily reside on a single computer or processor, but can be distributed in a modular manner across multiple different computers or processors to implement the various aspects of the invention.

[0087] Computer-executable instructions can take many forms, such as program modules, that are executed by one or more computers or other devices. Typically, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. The functionality of program modules can typically be combined or distributed as desired in the various embodiments.

[0088] Furthermore, data structures can be stored in any suitable form on a computer-readable medium. For simplicity, a data structure can be shown as having fields related by their position within the data structure. This relationship can also be achieved by assigning positions in a computer-readable medium that convey the relationship between fields to the storage of those fields. However, any suitable mechanism can be used to establish relationships between the information in the fields of a data structure, including by using pointers, labels, or other mechanisms that establish relationships between data elements.

[0089] Various aspects of the present invention can be used alone, in combination, or in various arrangements not precisely discussed in the embodiments described above, and therefore are not limited in their application to the details and arrangements of the components set forth in the foregoing description or shown in the drawings. For example, an aspect described in one embodiment can be combined in any way with aspects described in other embodiments.

[0090] Furthermore, the invention can be implemented using a method already provided as an example. The actions performed as part of this method can be ordered in any suitable manner. Therefore, embodiments can be constructed in which the actions are performed in a different order than that shown, which may include performing several actions simultaneously, even if these actions are shown as sequential actions in the illustrative embodiments.

[0091] Furthermore, some actions are described as being performed by a “user.” It should be understood that a “user” does not necessarily have to be a single individual, and in some embodiments, actions attributable to a “user” may be performed by a team of individuals and / or a combination of individuals and computer-aided tools or other entities.

[0092] The use of ordinal terms such as “first,” “second,” and “third” to modify a claim element does not imply any priority, precedence, or order of precedence of one claim element over another, or the chronological order of the actions of the method. Rather, it serves only as a label to distinguish one claim element with a certain name from another element with the same name (but using ordinal terms), thereby differentiating these claim elements.

[0093] Furthermore, the wording and terminology used herein are for descriptive purposes and should not be considered restrictive. The terms “including,” “comprising,” “having,” “containing,” “involving,” and their variations, as used herein, are intended to cover the items listed thereafter and their equivalents, as well as other items.

Claims

1. A non-transitory computer-readable medium comprising instructions, when executed by at least one processor, for converting a program written in a procedural programming language into a data flow graph, wherein, The program includes data manipulation and a control flow that instructs the execution sequence of these data manipulations, the method comprising: The program is analyzed using at least one processor and a syntax stored by at least one non-transitory computer-readable medium, wherein the syntax indicates: how to process words and symbols within the program written in the procedural programming language, how to identify data operations within the program written in the procedural programming language as having functions corresponding to specific types of data flow graph components, and how to configure data flow graph components corresponding to data operations within the program written in the procedural programming language. The analysis includes: Based on the syntax indicating how to identify data operations within the program, multiple sequential steps of the program are identified, wherein each of the multiple sequential steps includes one or more data operations of the program; Generate a directory that stores the control flow relationships between the multiple identified steps of the program; For each of the multiple steps identified in the program, based on the data operations of that step and based on the syntax indicating how to identify one of the multiple data flow graph component types as corresponding to that step, multiple data flow graph components corresponding to the respective steps in the multiple steps of the program are identified; and Based on the corresponding steps in the multiple steps of the program and based on the syntax indicating how to configure the multiple data flow graph components, the multiple data flow graph components are configured, thereby generating multiple configured data flow graph components; and A data flow graph is generated using at least one processor, the data flow graph comprising: These multiple configured data flow graph components; and One or more flows represent the flow of data records between one or more of the configured data flow graph components. The one or more data flow graph components are connected via the one or more streams in the order of the generated directories.

2. The non-transitory computer-readable medium as described in claim 1, wherein, This program is a statistical analysis system program, namely a SAS program.

3. The non-transitory computer-readable medium as described in claim 1, wherein, The analysis of the program further includes identifying macros defined in the program, and the configuration of the plurality of data flow graph components further includes generating at least one parameter set based on the macro definition.

4. The non-transitory computer-readable medium as claimed in claim 1, wherein, The program includes annotations, and configuring the plurality of data flow graph components includes configuring at least some of the plurality of data flow graph components to include these annotations.

5. The non-transitory computer-readable medium as claimed in claim 1, wherein, The generated data flow graph includes a first set of data flow graph components and a second set of data flow graph components configured to execute in parallel with each other.

6. The non-transitory computer-readable medium as described in claim 5, wherein: The program includes a first set of data operations for one or more of the plurality of data operations, and a second set of data operations for one or more of the plurality of data operations. The first set of data operations and the second set of data operations are independent of each other. The first set of data flow graph components performs the first set of data operations, and The second set of data flow graph components performs the second set of data operations.

7. The non-transitory computer-readable medium as claimed in claim 1, wherein, The program includes whitespace between the symbols of the program, and wherein the plurality of data flow graph components are configured to retain whitespace between the portions of the generated instructions corresponding to these symbols.

8. The non-transitory computer-readable medium as claimed in claim 1, wherein, The analysis of the program further includes identifying subroutines defined in the program, and the method further includes generating at least one subgraph based on the subroutine definition.

9. The non-transitory computer-readable medium as claimed in claim 8, wherein, The subroutine is defined as having one or more input values ​​that change the executable behavior of the subroutine, and wherein at least one subgraph is generated having at least one subgraph parameter corresponding to one or more input values ​​of the subroutine.

10. The non-transitory computer-readable medium of claim 1, wherein, Each of these multiple sequential steps includes multiple data operations of the program.

11. The non-transitory computer-readable medium of claim 1, wherein, This directory identifies the one or more data operations corresponding to each step of the program.

12. A computer-implemented method for automatically converting a program written in a procedural programming language into a data flow graph, wherein, The program includes data manipulation and a control flow that instructs the execution sequence of these data manipulations, the method comprising: The program is analyzed using at least one processor and a syntax stored by at least one computer-readable medium, wherein the syntax instructs: how to process words and symbols within the program written in the procedural programming language, how to identify data operations within the program written in the procedural programming language as having functions corresponding to specific types of data flow graph components, and how to configure data flow graph components corresponding to data operations within the program written in the procedural programming language. The analysis includes: Based on the syntax that indicates how to identify data operations within the program, multiple sequential steps of the program are identified, wherein each of the multiple sequential steps includes one or more data operations of the program; Generate a directory that stores the control flow relationships between the multiple identified steps of the program; For each of the multiple steps identified in the program, based on the data operations of that step and based on the syntax indicating how to identify one of the multiple data flow graph component types as corresponding to that step, multiple data flow graph components corresponding to the respective steps in the multiple steps of the program are identified; and Based on the corresponding steps in the multiple steps of the program and based on the syntax indicating how to configure the multiple data flow graph components, the multiple data flow graph components are configured, thereby generating multiple configured data flow graph components; and A data flow graph is generated using at least one processor, the data flow graph comprising: These multiple configured data flow graph components; and One or more flows represent the flow of data records between one or more of the configured data flow graph components. The one or more data flow graph components are connected via the one or more streams in the order of the generated directories.

13. The method of claim 12, wherein, This program is a statistical analysis system program, namely a SAS program.

14. The method of claim 12, wherein, The analysis of the program further includes identifying macros defined in the program, and the configuration of the plurality of data flow graph components further includes generating at least one parameter set based on the macro definition.

15. The method of claim 12, wherein, The program includes annotations, and configuring the plurality of data flow graph components includes configuring at least some of the plurality of data flow graph components to include these annotations.

16. The method of claim 12, wherein, The generated data flow graph includes a first set of data flow graph components and a second set of data flow graph components configured to execute in parallel with each other.

17. The method of claim 16, wherein: The program includes a first set of data operations for one or more of the plurality of data operations, and a second set of data operations for one or more of the plurality of data operations. The first set of data operations and the second set of data operations are independent of each other. The first set of data flow graph components performs the first set of data operations, and The second set of data flow graph components performs the second set of data operations.

18. The method of claim 12, wherein, The program includes whitespace between the symbols of the program, and wherein the plurality of data flow graph components are configured to retain whitespace between the portions of the generated instructions corresponding to these symbols.

19. The method of claim 12, wherein, The analysis of the program further includes identifying subroutines defined in the program, and the method further includes generating at least one subgraph based on the subroutine definition.

20. The method of claim 19, wherein, The subroutine is defined as having one or more input values ​​that change the executable behavior of the subroutine, and wherein at least one subgraph is generated having at least one subgraph parameter corresponding to one or more input values ​​of the subroutine.

21. The method of claim 12, wherein, Each of these multiple sequential steps includes multiple data operations of the program.

22. The method of claim 12, wherein, This directory identifies the one or more data operations corresponding to each step of the program.

23. A system for automatically converting programs written in a procedural programming language into data flow graphs, wherein, The program includes data manipulation and a control flow that instructs the execution sequence of these data manipulations. The system includes: A means for analyzing a program using at least one processor and a syntax stored by at least one computer-readable medium, wherein the syntax instructs: how to process words and symbols within a program written in a procedural programming language, how to identify data operations within a program written in the procedural programming language as having functions corresponding to specific types of data flow graph components, and how to configure data flow graph components corresponding to data operations within a program written in the procedural programming language, the means being configured to perform the method as described in any one of claims 12 to 21; and A means for generating a data flow graph, the data flow graph comprising: These multiple configured data flow graph components; and One or more flows represent the flow of data records between one or more of the configured data flow graph components. The one or more data flow graph components are connected via the one or more streams in the order of the generated directories.

24. The system of claim 23, wherein, This directory identifies the one or more data operations corresponding to each step of the program.

25. A computer program product comprising a computer program, wherein, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 12 to 22.

Images