An intelligent data processing system that generates metadata from iterative data analysis

Abstract

The method includes obtaining (601) a first data model (501) from a data exploration phase performed in a first environment (511), the first data model including first metadata (531). The method also includes obtaining (603) a second data model (502) from a data exploration phase performed in a second environment (512) different from the first environment, the second data model including second metadata (532). The method further includes generating (605) a third data model (503) using the first metadata and the second metadata, the third data model including one or more software artifacts (545). Each of the one or more software artifacts is configured as one or more files configured to execute at least one artificial intelligence (AI) / machine learning (ML) application (320).

Description

[Technical field]

[0001] The present disclosure is directed generally to data processing systems, and more particularly to intelligent data processing systems that generate metadata from iterative data analysis. [Background technology]

[0002] Many data science tools, such as Pandas, are designed to solve challenges and requirements during the exploratory "data exploration" phase of a project. The purpose of such tools is to provide a good interactive experience and fast turnaround of experiments on the data. While such tools are appropriate for the data exploration phase of a project, the "production" phase of a project typically has very different requirements. For example, scale and performance are often critical to building software applications. When it comes time to build and move a software application into production, the code written for initial data exploration using data science tools may not be suitable for production for multiple reasons. One reason is that the code may not have been designed with scalability in mind. Another reason is that the syntax and artifacts associated with software applications may be very different from the syntax and artifacts of data exploration. Summary of the Invention

[0003] The present disclosure relates to an intelligent data processing system that generates metadata from iterative data analysis.

[0004] In a first embodiment, the method includes obtaining a first data model from a data exploration phase performed in a first environment, the first data model including first metadata. The method also includes obtaining a second data model from a data exploration phase performed in a second environment different from the first environment, the second data model including second metadata. The method further includes generating a third data model using the first metadata and the second metadata, the third data model including one or more software artifacts, each of the one or more software artifacts configured as one or more files configured to execute at least one artificial intelligence (AI) / machine learning (ML) application.

[0005] In a second embodiment, the apparatus includes at least one processing device configured to obtain a first data model from a data exploration phase performed in a first environment, the first data model including first metadata. The at least one processing device is also configured to obtain a second data model from a data exploration phase performed in a second environment different from the first environment, the second data model including second metadata. The at least one processing device is further configured to generate a third data model using the first metadata and the second metadata, the third data model including one or more software artifacts, each of the one or more software artifacts configured as one or more files configured to execute at least one AI / ML application.

[0006] In a third embodiment, a non-transitory computer readable medium includes computer readable program code that, when executed, causes one or more processors to obtain a first data model from a data exploration phase performed in a first environment, the first data model including a first metadata. The non-transitory computer readable medium also includes computer readable program code that, when executed, causes one or more processors to obtain a second data model from a data exploration phase performed in a second environment different from the first environment, the second data model including a second metadata. The non-transitory computer readable medium further includes computer readable program code that, when executed, causes the one or more processors to generate a third data model using the first metadata and the second metadata, the third data model including one or more software artifacts, each of the one or more software artifacts configured as one or more files configured to execute at least one AI / ML application.

[0007] Other technical features will be readily apparent to those skilled in the art from the following figures, descriptions and claims. [Brief description of the drawings]

[0008] For a more complete understanding of the present disclosure, reference is now made to the following descriptions taken in conjunction with the accompanying drawings, in which:

[0009] [Figure 1] 1 illustrates an exemplary system supporting intelligent data processing with metadata generation from iterative data analysis in accordance with the present disclosure.

[0010] [Diagram 2] 1 illustrates an exemplary apparatus supporting intelligent data processing with metadata generation from iterative data analysis in accordance with the present disclosure.

[0011] [Diagram 3]1 illustrates an example architecture of an intelligent data processing system having a multi-interface front end and back end in accordance with the present disclosure.

[0012] [Figure 4] 1 illustrates an example process for generating execution engine specific code for execution by a selected backend according to the present disclosure.

[0013] [Diagram 5] 1 illustrates an exemplary workflow for intelligent data processing with metadata generation from iterative data analysis according to the present disclosure.

[0014] [Figure 6] 1 illustrates an exemplary method for intelligent data processing with metadata generation from iterative data analysis according to the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] 1 through 6 described below and the various embodiments used to illustrate the principles of the present invention in this patent document are for illustrative purposes only and should not be construed as limiting the scope of the invention. Those skilled in the art will understand that the principles of the present invention may be implemented in any type of suitably arranged device or system.

[0016] As mentioned above, many data science tools, such as Pandas, are designed to solve challenges and requirements during the exploratory "data exploration" phase of a project. The purpose of such tools is to provide a good interactive experience and fast turnaround of experiments on the data. While such tools are appropriate for the data exploration phase of a project, the "production" phase of a project typically has very different requirements. For example, scale and performance are often critical to building software applications. When it comes time to build and move a software application into production, the code written for initial data exploration using data science tools may not be suitable for production for multiple reasons. One reason is that the code may not have been designed with scalability in mind. Another reason is that the syntax and artifacts associated with software applications may be very different from the syntax and artifacts of data exploration.

[0017] Building an application that leverages artificial intelligence (AI) or machine learning (ML) models typically requires designing multiple components that are used together, such as a data exploration component, a data integration component, a feature store, an ML pipeline, and ML models. However, existing AI / ML tools are often characterized by each component working in a different language, and such tools typically have different (and often incompatible) interfaces between the different components. There is usually no common thread linking all the components together.

[0018] The present disclosure provides an intelligent data processing system capable of generating metadata from iterative data analysis. As described in more detail below, the disclosed intelligent data processing system includes a multi-interface front end and back end. For example, the data processing system may support at least one front end (interface), which is used by one or more users to identify data and transformations (code) to be applied to the data. Depending on the implementation, the data processing system may support the use of a single front end or multiple front ends. The data processing system may capture the transformations and store information identifying the transformations, such as in a database. The data processing system may also use information and context associated with the data to perform data processing operations. For example, the data processing system may select a particular execution engine from multiple execution engines (representing multiple back ends) depending on the context, and code implementing the transformations to be performed may be generated or obtained for that particular execution engine. The particular execution engine may execute the code to perform the data processing operations on the data. The particular context may change based on many factors, examples of which are provided below. The particular execution engine selected for use may also change, such as when different users or applications are associated with different contexts, or when the context associated with the same user or application changes over time. Based on the stored information, different code for performing data transformations can be generated or obtained, and the different codes can be executed by different execution engines. This allows the same set of transformations to be requested one or more times later and executed using one or more execution engines. Furthermore, the disclosed embodiments allow building data models on top of existing data models without requiring users to redo previous work. Thus, the disclosed embodiments alleviate some of the drawbacks of reusing existing data transformations.

[0019] Collaborative workflows allow intelligent data processing systems to auto-generate application files (or artifacts) used to run production-level enterprise-level AI / ML applications or other AI / ML applications. These artifacts can be auto-generated from data exploration work performed in any tool of a user's choosing. The resulting artifacts are configured to run at scale. Additionally, the data exploration work performed by the user may be in a different environment than the environment in which the application runs. Furthermore, there is no association with a specific user. That is, the auto-generation of production-level software application artifacts can be performed by a user different from the user who originally created the data model. Workflows can also compose data models from one schema to another. The ability to take data models from multiple different environments and create data models in yet another environment (accounting for connectors, data sources, and systems related to other data models) is useful.

[0020] FIG. 1 illustrates an exemplary system 100 supporting intelligent data processing with metadata generation from iterative data analysis according to the present disclosure. For example, the illustrated system 100 may be used to support one or more of the metadata generation techniques described below. As illustrated in FIG. 1, the system 100 includes user devices 102a-102d, one or more networks 104, one or more application servers 106, and one or more database servers 108 associated with one or more databases 110. Each of the user devices 102a-102d communicates over the network 104, such as via a wired or wireless connection. Each of the user devices 102a-102d represents any suitable device or system used by at least one user to provide or receive information, such as a desktop computer, a laptop computer, a smartphone, and a tablet computer. However, any other or additional types of user devices may be used with the system 100.

[0021] Network 104 facilitates communications between various components of system 100. For example, network 104 may communicate Internet Protocol (IP) packets, Frame Relay frames, Asynchronous Transfer Mode (ATM) cells, or other suitable information between network addresses. Network 104 may include one or more local area networks (LANs), metropolitan area networks (MANs), wide area networks (WANs), all or a portion of a global network such as the Internet, or any other communications system in one or more locations.

[0022] The application server 106 is coupled to the network 104 and is coupled to or otherwise in communication with a database server 108. The application server 106 supports techniques for intelligent data processing involving metadata generation from iterative data analysis, as described below. For example, the application server 106 may execute one or more applications 112 that perform the metadata generation using data from a database 110. It should be noted that the database server 108 may also be used within the application server 106 to store information, in which case the application server 106 may itself store the information used to perform the metadata generation.

[0023] The database server 108 operates to store and facilitate retrieval of various information used, generated, or collected by the application server 106 and the user devices 102a-102d in a database 110. For example, the database server 108 may store various information related to intelligent data processing involving metadata generation from iterative data analysis.

[0024] Although FIG. 1 illustrates an example of a system 100 supporting intelligent data processing with metadata generation from iterative data analysis that may be subject to various modifications of the present disclosure and details, various modifications may be made to FIG. 1. For example, the system 100 may include any number of user devices 102a-102d, a network 104, an application server 106, a database server 108, and a database 110. Moreover, these components may be located in any suitable location and may be distributed over a wide area. Moreover, although FIG. 1 illustrates an example operating environment in which intelligent data processing with metadata generation from iterative data analysis may be used, this functionality may be used in any other suitable system.

[0025] FIG. 2 illustrates an example apparatus 200 supporting intelligent data processing with metadata generation from iterative data analysis according to the present disclosure. One or more instances of the apparatus 200 may be used, for example, to at least partially implement the functionality of the application server 106 of FIG. 1. However, the functionality of the application server 106 may be implemented in any other suitable manner. In some embodiments, the apparatus 200 illustrated in FIG. 2 may form at least a portion of the user devices 102a-102d, the application server 106, or the database server 108 of FIG. 1. However, each of these components may be implemented in any other suitable manner.

[0026] As shown in FIG. 2, device 200 illustrates a computing device or system including at least one processing unit 202, at least one storage device 204, at least one communication unit 206, and at least one input / output (I / O) unit 208. Processing unit 202 can execute instructions loadable into memory 210. Processing unit 202 can include any suitable number and type of processors or other processing devices in any suitable arrangement. Examples of types of processing units 202 include one or more microprocessors, microcontrollers, reduced instruction set computers (RISC), complex instruction set computers (CISC), graphics processing units (GPUs), data processing units (DPUs), virtual processing units, associative processing units (APUs), tensor processing units (TPUs), vision processing units (VPUs), neuromorphic chips, AI chips, quantum processing units (QPUs), cerebral wafer scale engines (WSEs), digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or discrete circuits.

[0027] Memory 210 and persistent storage 212 are examples of storage 204 and represent any structure capable of storing and facilitating retrieval of information (such as data, program code, and / or other suitable information, either temporary or permanent). Memory 210 may represent random access memory or any other suitable volatile or non-volatile storage. Persistent storage 212 may include one or more components or devices that support long-term storage of data, such as read-only memory, a hard drive, a flash memory, or an optical disk.

[0028] The communications unit 206 supports communication with other systems or devices. For example, the communications unit 206 may include a network interface card or a wireless transceiver that facilitates communication over a wired or wireless network, such as the network 104. The communications unit 206 may support communication over any suitable physical or wireless communication link.

[0029] I / O unit 208 allows for the input and output of data. For example, I / O unit 208 may provide a connection for user input via a keyboard, mouse, keypad, touch screen, or other suitable input device. I / O unit 208 may also send output to a display, printer, or other suitable output device. It should be noted, however, that if device 200 does not require local I / O, such as when device 200 represents a remotely accessible server or other device, I / O unit 208 may be omitted.

[0030] Figure 2 illustrates an example of an apparatus 200 supporting intelligent data processing with metadata generation from iterative data analysis that may be subject to various modifications of the present disclosure and details, although various modifications may be made to Figure 2. For example, computing and communication devices and systems come in a wide variety of configurations, and Figure 2 is not intended to limit this disclosure to any particular computing or communication device or system.

[0031] Figure 3 illustrates an example of an architecture 300 of an intelligent data processing system having a multi-interface front end and back end in accordance with the present disclosure. For ease of explanation, the architecture 300 illustrated in Figure 3 is described as being implemented on or supported by the application server 106 in the system 100 illustrated in Figure 1, where the application server 106 may be implemented using one or more instances of the device 200 illustrated in Figure 2. However, the architecture 300 illustrated in Figure 3 may be used in any other suitable device and in any other suitable system.

[0032] 3, architecture 300 is generally divided into functionality associated with an authoring environment 302 and functionality associated with an execution environment 304. Authoring environment 302 generally includes functionality that allows data and data transformations (code) to be defined, and information related to data transformations may be stored in database 110 or other suitable storage location. Execution environment 304 generally includes functionality that allows code to be translated (if necessary) and executed to perform data transformations on demand.

[0033] In this exemplary embodiment, the authoring environment 302 may be used by one or more users 306 to define at least one set of data transformations to be applied to at least one input data set 308. For example, the authoring environment 302 may include one or more interfaces 310 representing one or more available front ends. Each interface 310 may enable at least one user 306 to load a data set 308 and define operations to be performed on the data set 308. It should be noted that, depending on the implementation, a single interface 310 or multiple interfaces 310 may be provided in the architecture 300. Each interface 310 includes any suitable logic configured to receive information defining data and transformations to be applied to the data, such as an application programming interface (API). Examples of specific types of interfaces 310 that may be used herein include a Pandas API and a Structured Query Language (SQL) API.

[0034] The tracking module 312 is used in conjunction with the interface 310 to track each sequence of transformations that have been applied to each data set 308 by one or more users 306 using the interface 310. In other words, the tracking module 312 can identify how each user uses the interface 310 to modify the data set 308, and the tracking module 312 can store this information in the database 110. The tracking module 312 can generate information identifying the sequence of transformations and store information associated with each sequence of transformations in the database 110 in any suitable manner. For example, in some cases, the tracking module 312 can generate a directed acyclic graph (DAG) that identifies each sequence of transformations applied to the data set 308.

[0035] The context module 314 is also used within the authoring environment 302 to identify information about each context in which the user 306 is performing a sequence of transformations, which can be stored in the database 110 or provided to the execution environment 304. Examples of the types of information identified or generated by the context module 314 are described below. In this example, the tracking module 312 and the context module 314 are shown as collectively storing data 316 associated with the sequence of transformations and optionally the context as part of a "save_transformations" action. In some cases, the stored data 316 for each sequence of transformations may be associated with a user-defined identifier (ID) or other type of identification. Also, in some cases, the context module 314 can provide context information to the tracking module 312 for storage in the database 110, the tracking module 312 can provide sequence of transformations information to the context module 314 for storage in the database 110, or the context module 314 can provide context information directly to the execution environment 304.

[0036] It should be noted here that the user 306 is free to use any of the supported interfaces 310 when defining the sequence of transformations to be applied to the data set 308. For example, in some embodiments, the authoring environment 302 can include one or more predefined or pre-installed interfaces 310, and the user 306 can be limited to using these particular interfaces 310. In other embodiments, one or more interfaces 310 can be installed in the authoring environment 302 as needed or desired (with or without one or more predefined or pre-installed interfaces 310 being used in the authoring environment 302). In general, the authoring environment 302 can include any suitable number and type of interfaces 310, regardless of how or when the interfaces 310 are made available to the user 306. This approach thus allows different users 306 to define the sequence of transformations and other information to be performed on the input data set 308 using different interfaces 310 as needed or desired.

[0037] Once a sequence of one or more transformations has been defined using at least one interface 310 in the authoring environment 302, each sequence of transformations may then be applied within the execution environment 304. In this example, the execution environment 304 may be used to execute the sequence of transformations to process an input data set 308 defined by a user 306 (during an initial execution of each defined sequence of transformations). One or more users 318 and / or one or more applications 320 may also request that the same sequence of transformations be executed on input data 322 (during subsequent executions of the transformation sequences). The one or more users 318 represent users who wish to apply previously defined data transformations to the input data 322, and the one or more applications 320 represent logic (executed by one or more computing devices or other devices) that requests the application of previously defined data transformations to the input data 322. It should be noted that the one or more users 318 may or may not be the same as the one or more users 306. In this example, the application of the previously defined data transformations may be requested using a "run_transformations" request 324. In some cases, a previously defined data transformation may be requested by including in the request 324 a user-defined identifier or other type of identification associated with the previously defined data transformation.

[0038] The execution module 326 generally operates to control the execution of transformation sequences for a user 306 and its input data set 308, as well as the execution of transformation sequences for a user 318 / application 320 and its input data 322. For each transformation sequence that is executed, the execution module 326 may receive information identifying a current context 328 associated with the data being processed. Each current context 328 is identified using the context module 314. For example, the context module 314 may be used to identify information regarding the context for each user 306, as described above, and the context module 314 may be used to identify information regarding the context in which the user 318 or application 320 is requesting the execution of a transformation sequence for each request 324.

[0039] The context module 314 may be used to identify any suitable characteristics related to the context in which the data transformation is occurring. For example, the context module 314 may determine values ​​of different context dimensions related to each sequence of data transformation. In some embodiments, examples of different context dimensions that may be used by the context module 314 may include any single one or any combination of the following context dimensions: An interactivity context dimension may represent an amount or level of user or application interaction during the processing of data in a transformation sequence. A data size context dimension may approximate a size or amount of data processed during a transformation sequence. A data type context dimension may represent a type of data processed during a transformation sequence, such as real-time data, batch data, streaming data, etc. A data shape context dimension may represent whether the data processed during a transformation sequence is structured or unstructured data, and in the case of unstructured data, may represent the particular type of unstructured data (e.g., image data, video data, audio data, etc.) being processed. A security profile context dimension may represent a level of security required when processing data during a transformation sequence. A resource availability context dimension may represent an amount or level of processing resources, memory resources, or other resources required to process data during a transformation sequence. The personal identifiable information (PII) context dimension may represent whether the data processed during the transformation sequence contains personally identifiable information and, if so, how that data is processed. The retention policy context dimension may represent the period for which the data or its results processed during the transformation sequence must be retained. The calculation comprehensiveness context dimension may represent the overall amount or level of data included in the calculation during the transformation sequence.Depending on the implementation, the context may be defined along each of these one or more context dimensions using discrete values ​​or continuous values ​​within a particular range of values.

[0040] For each sequence of transformations to be performed (either for user 306, user 318, or application 320), execution module 326 may receive information identifying a current context 328 associated with the data being processed. Execution module 326 may also obtain information regarding the particular sequence of transformations to be performed from database 110, such as by searching for information defining the sequence of transformations associated with an identifier included in request 324 or the sequence of transformations defined by user 306. In some cases, for example, database 110 may be queried using a "retrieve_transformations" request 330, which may be used to obtain information related to the sequence of transformations to be performed (such as a directed acyclic graph).

[0041] For each sequence of transformations to be performed, the execution module 326 can use at least a portion of the obtained information to select one of a plurality of execution engines 332 to be used to execute the sequence of transformations. The execution engines 332 represent different backends that may use different tools or techniques to execute the sequence of transformations requested. Here, any suitable execution engines 332 may be used to execute the data transformations, such as backends that support different types of machine learning or artificial intelligence (ML / AI) algorithms or other data processing algorithms. In some cases, the execution module 326 can select the particular execution engine 332 to be used for each sequence of transformations to be performed based on the current context 328 of the data to be processed. As a particular example, the execution module 326 can use an in-memory data structure or other mechanism that maps different combinations of values ​​of the context dimensions to different combinations of execution engines 332. Thus, the execution module 326 can obtain the values ​​of the current context 328 and select the execution engine 332 that maps to those values.

[0042] For each transformation sequence to be executed, the execution module 326 may further generate execution engine specific code 334 to be executed by the selected execution engine 332. For example, the execution module 326 may traverse a directed acyclic graph associated with the transformation sequence to be executed and modify the directed acyclic graph in an appropriate manner to enable code 334 to be generated for the selected execution engine 332. Examples of the types of modifications that may be performed on the directed acyclic graph may include removing one or more nodes from the directed acyclic graph, replacing one or more nodes of the directed acyclic graph with one or more other nodes, and / or shuffling the position or order of one or more nodes of the directed acyclic graph. Each node of the directed acyclic graph may generally represent a data operation that is performed as part of the transformation sequence. The particular manner in which the nodes of the directed acyclic graph are modified may vary based on many factors, such as the particular execution engine 332 on which the code 334 is executed, and in some cases the directed acyclic graph may not require modification. The modified or unmodified directed acyclic graph may be used by a compiler to generate code 334 for a selected execution engine 332. The compiler may also be used to generate execution engine specific deliverables based on more general deliverables received in response to the request 330. Regardless of how the code 334 is generated, the code 334 may be executed by the selected execution engine 332 to perform the desired transformation sequence.

[0043] It should be noted that although the generation of the code 334 for the execution engines 332 is described above as being performed during run-time (e.g., after a user 306, 318 or application 320 requests execution of a sequence of transformations), other approaches can be used to generate the code 334. For example, the code 334 for each execution engine 332 can be generated after data 316 associated with each sequence of transformations is obtained and stored in the database 110. In some cases, the code 334 for each execution engine 332 can also be stored in the database 110. Once a particular execution engine 332 is identified (such as in response to a particular request 324), the pre-generated code 334 for the appropriate sequence of transformations is obtained and provided to that particular execution engine 332 (without further compilation) for execution. In general, this disclosure is not limited to a particular order of receiving requests and generating code.

[0044] As seen herein, this approach allows the authoring environment 302 to be used to define any desired sequence of data transformations, where information defining the data transformation sequence can be stored in the database 110 or elsewhere for later use. This can be accomplished using any suitable interface 310 within the authoring environment 302. This approach also allows the execution environment 304 to be used to execute transformation sequences on demand based (at least in part) on information retrieved from the database 110 or elsewhere. Notably, the execution engine 332 for each transformation sequence to be executed can be dynamically selected based on the associated context 328.

[0045] In this manner, the architecture 300 can provide various benefits or advantages depending on the implementation. The following are non-limiting examples of various types of benefits or advantages that can be obtained using the architecture 300. For example, the architecture 300 can enable code to be developed once and then reused across different use cases (different contexts). For example, a user 306 can write code to perform a data transformation sequence using a data set 308 in the authoring environment 302, and that transformation sequence is associated with one context and used to generate code for execution by one execution engine 332. That code can later be translated by the execution module 326 for execution by any number of other execution engines 332, which may be associated with different contexts. The particular execution engine 332 selected for the user's 306 original use and the particular execution engine 332 selected for use in a subsequent request 324 can change based on the particular context 328 associated with those operations. This allows the context, and therefore the execution engine 332 used to perform the same transformation sequence, to change based on the changing needs of the user 306, 318, or application 320 (among other things).

[0046] As another example, the architecture 300 can be used to provide flexibility in selecting between a system-driven backend and a user-driven backend. For example, in some embodiments, the execution module 326 can allow the user 306, 318 and / or application 320 to select the execution engine 332 that will be used to process its data. Among other things, this allows the user 306, 318 or application 320 to select a particular execution engine 332 for use with particular data, which is useful when a particular execution engine 332 is known to provide good results in performing a particular type of data processing task. However, the execution module 326 can also dynamically select the execution engine 332 to be used, such as when the user 306, 318 or application 320 does not specifically identify the execution engine 332 or when the user 306, 318 or application 320 specifically requests dynamic selection of the execution engine 332.

[0047] As yet another example, the architecture 300 can be used to accelerate time-to-value for executing data processing tasks. For example, data-intensive workloads often have changing back-end needs over time, typically involving time-consuming changes to the execution engine used for the data-intensive workload. The architecture 300 can be used to reduce the complexity of authoring data transformation sequences, and the same data transformation sequences can be leveraged to execute code 334 on any number of execution engines 332 (including execution engines 332 selected by the user 306, 318 / application 320 or dynamically selected by the architecture 300). In some cases, the user 306 can define the transformation using a single interface 310, and the particular back-end used to execute the transformation can vary depending on the nature of the data processing task being performed (which can change dynamically). This can significantly reduce the time required to execute the data processing task.

[0048] As yet another example, the architecture 300 can be used to make server-aware and client-aware decisions when selecting which execution engine 332 to use. That is, a customer (user 306, 318 and / or application 320) can implement logic using devices acting as clients or servers depending on their particular installation. As a particular example of this, some installations support declarative programming while other installations support imperative programming. Thus, different execution engines 332 can be customized for different types of customer installations. In some embodiments, the architecture 300 uses self-declarative programming that can leverage the benefits of both declarative and imperative programming.

[0049] It should be noted that the architecture 300 can still achieve a high level of performance when executing code using a dynamically selected execution engine 332. Among other reasons, this is because the architecture 300 can provide resource efficiency, time efficiency, and reliability to the users 306, 318 and / or applications 320 when it selects (and switches as needed between) different execution engines 332 that best suit the needs of the users 306, 318 and / or applications 320. For example, time efficiency can be obtained by providing a good trade-off between interactive and non-interactive contexts. In some cases, for example, this allows for faster output when reliability is low and slower output when reliability is high. Furthermore, the architecture 300 can be used to perform data transformations on a wide range of data types, including real-time and streaming data, even when minimal resources are available to process the data. Furthermore, security-related aspects of the context of the data processing task can be identified and respected within the architecture 300.

[0050] The results generated through execution of the code 334 by the execution engine 332 selected for each transformation sequence may be used in any suitable manner. For example, the results generated by each execution engine 332 may be provided to the user 306, 318 or application 320 that initiated the execution of the transformation sequence via a suitable interface 310 or other mechanism. Because both the data processing tasks and the data processed by the data processing tasks can vary widely, the results generated by the execution engines 332 may be used for any suitable purpose.

[0051] Although Figure 3 illustrates an example of an intelligent data processing system architecture 300 with multiple interface front ends and back ends, various modifications may be made to Figure 3. For example, functions and components may be added, omitted, combined, further subdivided, duplicated, or arranged in any other suitable configuration within architecture 300 according to particular needs. As a particular example, architecture 300 may include any suitable number of interfaces 310 and any suitable number of execution engines 332.

[0052] FIGURE 4 illustrates an example process 400 for generating execution engine specific code for execution by a selected backend according to the present disclosure. The process 400 illustrated in FIGURE 4 may represent, for example, a process for collecting user input via an interface 310, where the user input defines a data transformation sequence to be executed. The process 400 illustrated in FIGURE 4 also illustrates how the tracking module 312 generates information regarding the data transformation sequence, and how the execution module 326 uses this information to generate execution engine specific code 334 suitable for execution by a selected execution engine 332.

[0053] 4, one or more data sources 402 represent one or more interfaces (such as one or more interfaces 310) that may be used to obtain data from one or more users 306. In some embodiments, each data source 402 may be associated with a different programming language and a different data storage mechanism. Depending on the implementation, there may be a single data source 402 or multiple data sources 402 provided for use herein.

[0054] Each user 306 can use one or more tools 404 to write code that defines data transformation sequences in one or more specific programming languages. In some embodiments, the tools 404 accessed and used by each user 306 can vary depending on which data source 402 is used by the user 306, and therefore different data sources 402 can be associated with different tools 404. In some cases, for example, a tool 404 can represent a Jupyter Lab tool that uses the Pandas programming language. Of course, different tools 404 can be provided to support different programming languages, as needed.

[0055] Each tool 404 here can be used to define a sequence of operations to be performed, which represents a data transformation sequence defined by the user 306. Each operation in the sequence can be represented by a cell 406, which can identify at least one operation defined by the user 306. In this example, the first cell 406 represents a request to load one or more source data files. The second cell 406 represents at least one request to perform one or more data integration operations, which can include any suitable operations to prepare source data for processing. The third cell 406 represents at least one request related to feature engineering, which can include any suitable operations required to identify features of the source data to be further processed. The fourth cell 406 represents at least one request related to feeding the processed data (e.g., identified features) to a machine learning pipeline or other data processing architecture. The fifth cell 406 represents a request to update a package that encapsulates the operation sequence. Of course, the operations represented by the cells 406 will vary based on the particular sequence of data transformations defined by the user 306. In some cases, the cell 406 may be digitally signed (such as using a user's digital key) to associate a particular user 306 with a particular cell 406 and / or to prevent unauthorized modification of the cell 406.

[0056] Each user-defined operation sequence generated by the user 306 using the tool 404 is converted into system-generated code 408, which (when executed) can be used to execute the user-defined operation sequence. In some cases, the system-generated code 408 can be defined using a directed acyclic graph, where the directed acyclic graph includes multiple nodes 410. Each node 410 of the directed acyclic graph represents one or more operations to be executed, and each node 410 can correspond to one of the cells 406 generated using the tool 404. For example, each node 410 can include one or more data specifications ("data specs") that identify the particular operation to be executed on the corresponding cell 406. The information defined within the node 410 can identify the operation that corresponds to each line of code generated by the user 306 using the tool 404. Thus, the nodes 410 of the directed acyclic graph can represent the logic required to implement the user-defined operation sequence. However, the directed acyclic graph can be general in that it is not tied to a particular execution engine 332. As can be seen, the nodes 410 form a directed acyclic graph because the nodes 410 are ordered in a particular sequence defined by the arrows to / from / between the nodes 410. In some cases, a directed acyclic graph is at least one of the items that the tracking module 312 may generate and store in the database 110 when the user 306 uses the interface 310 to define a transformation sequence.

[0057] After the user 306 issues a terminal transformation (meaning the final transformation defined by the user 306), the directed acyclic graph may be stored in the database 110. Also, a particular execution engine 332 may be selected as described above based on the current context associated with the user 306. To execute the transformation sequence defined by the directed acyclic graph on the selected execution engine 332, the directed acyclic graph may be translated into code 412 suitable for execution by the selected execution engine 332. Because the directed acyclic graph generically defines the operations to be performed, the directed acyclic graph may be easily translated into code 412 specific to the execution engine 332 on which the code 412 is to be executed. The code 412 is then compiled or otherwise prepared and sent to the selected execution engine 332 for execution. If the same transformation sequence is requested again (e.g., via request 324), but a different execution engine 332 is selected, the directed acyclic graph is again retrieved and used to generate code 412 for execution by the selected execution engine 332. It should be noted that after the code 412 is generated for a particular execution engine 332, the code 412 may be stored (e.g., in database 110) so that the code 412 can be executed again without delay if later requested.

[0058] It should be noted that the particular operations performed within the code 412 will typically vary depending (among other things) on the execution engine 332 used to execute the code 412. Because the directed acyclic graph contains a data specification or other information that generally defines the operations to be performed as part of a user-defined sequence of operations, knowledge of a particular execution engine 332 can be used to generate logic that enables the execution module 326 to translate the data specification into execution-engine-specific code. By defining appropriate logic for all available execution engines 332, the execution module 326 can translate the directed acyclic graph into the appropriate code 412 each time execution of a user-defined sequence of operations is requested.

[0059] Although Figure 4 illustrates one example process 400 for generating execution engine specific code for execution by a selected backend, various modifications can be made to Figure 4. For example, the user defined code can include any suitable number of operations, which can be transformed into any suitable operations to be executed by the selected execution engine 332. Also, the particular code 412 illustrated in Figure 4 is for a particular execution engine 332, and any other suitable code 412 can be generated depending on the execution engine 332.

[0060] 5 illustrates an exemplary workflow 500 for intelligent data processing with metadata generation from iterative data analysis according to the present disclosure. In some embodiments, the workflow 500 may be implemented using a data modeling system or an application development system (such as an intelligent data processing system having architecture 300), which may be executed using one or more devices (such as application server 106, one or more of user devices 102a-102d of FIG. 1, or one or more devices 200 of FIG. 2). However, the workflow 500 illustrated in FIG. 5 may be implemented using any other suitable device and any other suitable system.

[0061] As shown in FIG. 5, workflow 500 allows various users (e.g., users 306 and 318) to collaborate iteratively on one or more data models from a single interface (e.g., interface 310), thereby reducing time to production. One or more contexts from the data exploration phase can be saved as software deliverables (files, code, etc.) and used for application development. Thus, some users 306, 318 can leverage the iterative explorations performed by other users 306, 318 to generate deliverables simply by function calls. In some embodiments, at least some of the deliverables can be human readable, machine executable, or both, enabling continuously integrated and deployable application development. This is an advantageous advantage over conventional systems that do not generate human readable or machine executable files or code.

[0062] 5, a first user 306, 318 ("Person A") can construct a data model 501 ("Data Model 1") for a first application 320 in a first environment 511 ("Env1"). For example, the first user 306, 318 can construct the data model 501 using an interface 310 in an authoring environment 302. A second user 306, 318 ("Person B") can construct a data model 502 ("Data Model 2") for a first application 320 in a first environment 512 ("Env2"). The environments 511, 512 can represent (or be represented by) the authoring environment 302, the execution environment 304, or a combination thereof. Using workflow 500, the system can automatically create graphs 521, 522 (e.g., directed acyclic graphs) for each of data models 501, 502 and store the graphs 521, 522 (e.g., in database 110). As used herein, "data model" refers to an abstract model that organizes elements of data and relationships between elements of data.

[0063] Each data model 501, 502 includes a corresponding group of metadata 531, 532. The metadata 531, 532 describes the actual data, their types and relationships, and the sources from which the actual data is generated, generated by the corresponding data model 501, 502. In some embodiments, the metadata 531, 532 may additionally or alternatively include information defining data transformations required to create one or more features or feature sets used in a machine learning model, such as those used by one of the backends described above. Further details regarding the use of metadata to create or identify features or feature sets are described in U.S. Patent Application Serial No. 17 / 699,025, entitled "METADATA-DRIVEN FEATURE STORE FOR MACHINE LEARNING SYSTEMS," which is incorporated herein by reference in its entirety.

[0064] After the data models 501, 502 and metadata 531, 532 are developed, a third user 306, 318 ("Person C") can operate in a third environment 513 ("Env3") and utilize the data models 501, 502 and metadata 531, 532 to develop a new data model 503 ("Data Model 3"). The data model 503 includes a group of metadata 533 that is generated using the metadata 531 associated with the data model 501 and the metadata 532 associated with the data model 502. As a particular example, the third user 306, 318 can request that the system retrieve either or both of the graphs 521, 522 and the corresponding metadata 531, 532 and load the graphs 521, 522 and the metadata 531, 532 into the separate authoring environment 302 upon the request of the third user 306, 318. In some embodiments, the system can propagate actual data (such as features or feature sets of machine learning models) from the metadata 531, 532 for reuse in the data model 503. The system can also build on the graphs 521, 522 and metadata 531, 532 by taking some or all of each of the graphs 521, 522 and metadata 531, 532 and combining them to generate the graphs 523 and metadata 533 that are part of or associated with the data model 503. Thus, while the data model 503 is being generated, the source of the graphs 523 and metadata 533 may be the existing graphs 521, 522 and metadata 531, 532 rather than new data. The existence of the graphs 521, 522 and the type of system allows having the same API available across multiple run times and multiple environments. In this way, an application that uses one of the data models 501-503 in one of the environments 511-513 can use the same data models 501-503 in a different environment 511-513. Therefore, the data models 501-503 themselves are compatible across different environments 511-513.

[0065] While the third user 306, 318 is developing the data model 503, the third user 306, 318 can provide instructions to the system to perform one or more operations 541-543. Each operation 541-543 includes one or more definitions for one or more data transformations, which can be stored in the database 110 in some cases. The one or more data transformations can be used by the system to generate a set of new artifacts 545 that can be part of or associated with the data model 503. Each artifact 545 represents a file or code configured to be executed by an application. The data model 503 can be executed in the same or a different execution environment 304 as the execution environment 304 of the other two data models 501, 502.

[0066] To prepare the data models 501, 502 for generation of the data model 503, the system can generate an intermediate representation 550 of each of the data models 501, 502. Each intermediate representation 550 is a data structure that contains information for maintaining the sequence of transformations from each data model 501, 502 that are used to create the deliverables 545 of the data model 503. Each intermediate representation 550 also includes the context in which these transformations were input. In some embodiments, the system can use the intermediate representation 550 to perform one or more operations 541-543. The system can also perform one or more runtime optimizations (such as vertex fusion, vertex expansion, sub-expression elimination, etc.) on these transformations by using the intermediate representation 550. The system can take the intermediate representations 550 and the associated context and convert them into one or more deliverables 545 for the data model 503.

[0067] In some embodiments, the system can support version control of the data models 501-503 and their generated artifacts 545 to support continuous iterative application development in each environment 511-513 by one or more users 306, 318. With each iteration that generates a new version, the version identifier can be updated to track progress. Thus, each environment 501-503 can be used for continuous iterative application development. Any suitable technique can be used to support version control in the system. Furthermore, the system can improve the generated artifacts 545 and thus support collaborative application development on top of the artifacts 545. For example, one user 306, 318 can write a transformation in terms of a user interface and a second user 306, 318 can write the same transformation in terms of a code interface and both users can collaborate iteratively. Thus, multiple users 306, 318 operating in different contexts can collaborate. Most traditional data science tools (such as Pandas) support collaboration primarily at the interface level. In contrast, the workflow 500 can enable collaboration at both the interface level and the generated deliverable level, thereby improving overall collaboration between multiple users 306, 318.

[0068] Also, in some embodiments, the interface 310 that a user 306, 318 uses to develop one or more of the data models 501-503 may be different from the interface 310 that a user uses to develop another one or more of the data models 501-503. That is, different data models 501-503 may be developed using different interfaces 310. In some cases, one interface 310 may include APIs that are not present in another interface 310 and that are not supported in a particular execution environment 304. The workflow 500 can handle such cases. For example, when a deliverable 545 is generated for a graph 523 in an environment 513, assume that a node in the graph 523 describes an operation that is not supported in the environment 513. In this case, the workflow 500 allows the system to take the output of all parent nodes, move the output to another environment that supports the operation, execute the operation in the other environment, and return the results to the environment 513 to continue the process.

[0069] In summary, the workflow 500 enables the system to automatically generate a deliverable 545 that can be used to run a production-grade enterprise-level AI / ML application or other AI / ML application. The deliverable 545 can be automatically generated from data exploration activities performed by a user 306, 318 using any suitable interface 310. The resulting deliverable 545 is configured to be scalable. Also, the data exploration activities performed by the user may be in a different environment than the environment in which the application runs. Furthermore, because there is no association with a particular user 306, 318, the automatic generation of the production-grade software application deliverable 545 can be performed by a different user 306, 318 than the user 306, 318 that originally created the data models 501-503.

[0070] Although Figure 5 illustrates an example of a workflow 500 for intelligent data processing with metadata generation from iterative data analysis that may be subject to various modifications of this disclosure and details, various modifications may be made to Figure 5. For example, functions and components may be added, omitted, combined, further subdivided, duplicated, or arranged in any other suitable configuration within workflow 500 according to particular needs. As a particular example, workflow 500 may include or be associated with any suitable number of data models, environments, users, and operations.

[0071] It should be noted that the functionality shown in or described in connection with Figures 3-5 may be implemented in an electronic device, such as a computing device, in any suitable manner. For example, in some embodiments, at least some of the functionality shown in or described in connection with Figures 3-5 may be implemented or supported using one or more software applications or other software instructions executed by one or more processing units of the application server 106, the device 200, or other devices. In other embodiments, at least some of the functionality shown in or described in connection with Figures 3-5 may be implemented or supported using dedicated hardware components. In general, the functionality shown in or described in connection with Figures 3-5 may be performed using any suitable hardware or any suitable combination of hardware and software / firmware instructions.

[0072] Figure 6 illustrates an exemplary method 600 of intelligent data processing with metadata generation from iterative data analysis according to the present disclosure. For ease of explanation, the method 600 illustrated in Figure 6 is described as being implemented using one or more devices 200 illustrated in Figure 2 and including the use of the application server 106 illustrated in Figure 1 supporting the architecture 300 illustrated in Figure 3 and the workflow 500 illustrated in Figure 5. However, the method 600 illustrated in Figure 6 may be used with any other suitable electronic device and any other suitable architecture or workflow.

[0073] As shown in Fig. 6, a first data model is obtained in step 601 from a data exploration phase performed in a first environment. The first data model includes first metadata. This may for example include the server 106 obtaining the data model 501 from a data exploration phase performed in environment 511, the data model 501 including metadata 531. A second data model is obtained in step 603 from a data exploration phase performed in a second environment different from the first environment. The second data model includes second metadata. This may for example include the server 106 obtaining the data model 502 from a data exploration phase performed in environment 512, the data model 502 including metadata 532.

[0074] The method includes, at step 605, generating a third data model using the first metadata and the second metadata, the third data model including one or more software deliverables. Each of the software deliverables is configured as one or more files configured to execute at least one enterprise-level AI / ML application or other AI / ML application. Third metadata associated with the third data model is also generated using the first metadata and the second metadata. This may include, for example, server 106 using metadata 531 and metadata 532 to generate data model 503. Data model 503 may include one or more deliverables 545 and may also include metadata 533.

[0075] Figure 6 illustrates an example method 600 of intelligent data processing with metadata generation from iterative data analysis that may be subject to various modifications of the present disclosure and details, although various modifications may be made to Figure 6. For example, although shown as a series of steps, various steps in Figure 6 may overlap, occur in parallel, occur in a different order, or occur any number of times.

[0076] In some embodiments, various functions described in this patent document are implemented or supported by a computer program formed from computer readable program code and embodied in a computer readable medium. The phrase "computer readable program code" includes any type of computer code, including source code, object code, and executable code. The phrase "computer readable medium" includes any type of medium accessible by a computer, such as read only memory (ROM), random access memory (RAM), hard disk drive (HDD), compact disk (CD), digital video disk (DVD), or any other type of memory. "Non-transitory" computer readable medium excludes wired, wireless, optical, or other communication links that transmit transient electrical or other signals. Non-transitory computer readable medium includes media that can store data permanently, and media that can store data and later be overwritten, such as rewritable optical disks or erasable storage devices.

[0077] It is advantageous to establish definitions of certain words and phrases used throughout this patent document. The terms "application" and "program" refer to one or more computer programs, software components, sets of instructions, procedures, functions, objects, classes, instances, associated data, or portions thereof adapted for implementation in suitable computer code (including source code, object code, or executable code). The term "communicate" and its derivatives include both direct and indirect communication. The terms "include" and "comprise" and its derivatives mean inclusion without limitation. The term "or" is inclusive and / or. The term "related" and its derivatives can mean include, be contained within, interconnect, contain, be contained within, connect, connect or connect, be able to communicate, cooperate, connect, juxtapose, be adjacent, couple or connect, have a property of, or have a relationship, and the like. The phrase "at least one of," when used in conjunction with a list of items, means that different combinations of one or more of the listed items can be used, and only one item in the list may be required. For example, "at least one of A, B, C" includes any combination of A, B, C, A and B, A and C, B and C, and A and B and C.

[0078] The description in this application should not be construed as implying that a particular element, step, or function is an essential or critical element that must be included in the scope of the claims. The scope of the patent is defined solely by the allowed claims. Moreover, no claim invokes 35 U.S.C. §112(f) with respect to any of the appended claims or claim elements, except where the precise words "means" or "step" are expressly used in a particular claim, followed by a participial phrase identifying the function. The use of terms such as, but not limited to, "mechanism," "module," "apparatus," "unit," "component," "element," "member," "device," "machine," "system," "processor," or "controller" in the claims is understood and intended to refer to structures known to those skilled in the relevant art, as further modified or enhanced by the features of the claims themselves, and is not intended to invoke 35 U.S.C. §112(f).

[0079] While this disclosure has described certain embodiments and generally associated methods, modifications and permutations of these embodiments and methods will be apparent to those skilled in the art. Accordingly, the above description of exemplary embodiments does not define or limit the disclosure. Other modifications, substitutions, and alterations are possible without departing from the spirit and scope of the disclosure, as defined by the following claims.

Claims

1. obtaining a first data model from a data exploration phase performed in a first environment, the first data model including first metadata; obtaining a second data model from a data exploration phase performed in a second environment different from the first environment, the second data model including second metadata; generating a third data model using the first metadata and the second metadata, the third data model including one or more software artifacts; Including, wherein each of the one or more software deliverables is configured as one or more files configured for execution of at least one artificial intelligence (AI) / machine learning (ML) application.

2. generating the third data model includes using the first metadata and the second metadata to generate third metadata associated with the third data model; 2. The method of claim 1, wherein each of the first, second, and third metadata includes information defining data transformations to create one or more features or feature sets for use in a machine learning model.

3. The step of generating the third data model comprises: performing one or more operations on at least one of the first data model and the second data model, the one or more operations defining one or more data transformations; generating the one or more software deliverables using the one or more data transformations; The method of claim 1 , comprising:

4. The method of claim 3 , wherein the one or more operations are performed using an intermediate representation that maintains a sequence of the one or more data transformations, the intermediate representation including a context associated with the one or more data transformations.

5. 4. The method of claim 3, wherein generating the third data model further comprises combining at least a portion of a first graph associated with the first data model and at least a portion of a second graph associated with the second data model to a third graph associated with the third data model.

6. The method of claim 1 , wherein generating the third data model comprises iteratively generating multiple versions of the third data model based on input from multiple users.

7. The method of claim 1 , wherein the one or more files are human readable and machine executable.

8. An apparatus comprising at least one processing unit configured to carry out the method according to any one of claims 1 to 7.

9. A non-transitory computer readable medium comprising computer readable program code which, when executed, causes one or more processors to perform the method according to any one of claims 1 to 7.

Images