Pipelining data processing optimizations for machine learning workloads

Abstract

Techniques are provided for improving a data pipeline. An original file of serialized objects is received and an original pipeline including a plurality of transforms is identified based on the original file. A first computational cost of a first transform of the plurality of transforms is determined. The first transform is modified using a predefined optimization and a second cost of the modified first transform is determined. Upon determining that the second cost is lower than the first cost, the first transform in the original pipeline is replaced with the optimized first transform.

Description

BACKGROUND

[0001] The present disclosure relates to machine learning optimization, and more specifically to improved techniques for optimizing a machine learning pipeline.

[0002] Machine learning (ML) systems often rely on large amounts of data to operate effectively and accurately. For example, large amounts of data are often required to adequately train a system. Similarly, during use, models are often expected to process large amounts of data to provide desired inferences, predictions, and classifications. Further, during a retraining or refinement process, previous data (which can include training data and deployed data) is often used to reconfigure a model, which can require evaluating large amounts of records.

[0003] In existing systems, data is often passed through a preprocessing system in order to prepare it for use as input to an actual model. The preprocessing system can involve one or more evaluations and transformations of the raw input data. This preprocessing can result in significant computational costs, including computation time (e.g., cycles on a processor) and memory requirements. In fact, in many systems, preprocessing input data involves more computational cost than actually evaluating the data with a ML model. For these and other reasons, it is important that the preprocessing system operate efficiently in order to reduce the expense and latency in the system.

[0004] For example, in deployments that rely on real-time inference (e.g., where input data is received as a stream), the input data is often high-throughput and needs to be evaluated quickly. If the preprocessing system is not efficient, it acts as a significant bottleneck in the process. Similarly, in batch inference systems, large amounts of records are often evaluated at a single time. If the preprocessing system is inefficient, then each batch therefore requires additional and significant resources to ingest. Optimizations in this preprocessing system are critical to the overall performance of the model. However, existing systems rely on significant manual effort to identify and implement optimizations in the system. This again introduces additional cost and delay. Further, these manual approaches are often inherently subjective and do not provide intelligent and systematic techniques to refine the system. SUMMARY

[0005] According to one embodiment of the present disclosure, a method is provided. The method includes receiving an original file of serialized objects, identifying an original pipeline including a plurality of transformations based on the original file, determining a first computational cost of a first transformation of the plurality of transformations, modifying the first transformation using a predefined optimization, determining a second cost of the modified first transformation, and replacing the first transformation with the optimized first transformation in the original pipeline upon determining that the second cost is lower than the first cost. Advantageously, such a method is able to implement significant data-driven improvements when processing a pipeline.

[0006] According to some embodiments of the present disclosure, any combination of the above embodiments can further include the technique where identifying the original pipeline comprises generating a data processing graph, where each respective vertex in the data processing graph represents a respective transform in the plurality of transforms, and where each respective edge in the data processing graph specifies a data flow in the original pipeline. Such embodiments are advantageous at least because they enable granular evaluation and understanding of the pipeline in order to dynamically analyze each element and identify potential improvements.

[0007] According to some embodiments of the present disclosure, any combination of the above embodiments can further include the technique where generating the data processing graph comprises: instantiating the original pipeline based on the original file; executing the original pipeline on the sample data; identifying the plurality of transforms based on the execution; and identifying the sequence of the plurality of transforms based on the execution. Advantageously, such embodiments allow the system to identify the specific transforms and data flows involved in the pipeline while allowing the system to monitor the original pipeline in use. This reduces complexity and speeds up improvements.

[0008] According to some embodiments of the present disclosure, any combination of the above embodiments can further include the technique where generating the data processing graph comprises evaluating the original file to identify the plurality of transforms and data flows in the original pipeline without instantiating the original pipeline. Such embodiments can be advantageous because they allow the system to begin evaluation of the pipeline by directly looking at the file without requiring resource usage to instantiate the pipeline or process actual data.

[0009] According to some embodiments of the present disclosure, any combination of the above embodiments can further include the technique where determining the first computational cost of the first transform comprises: executing the first transform on the sample data; and evaluating the execution to determine a latency of executing the first transform. Advantageously, this allows the system to dynamically identify costs associated with processing data in a realistic manner, which better reflects what is expected at runtime.

[0010] According to some embodiments of the present disclosure, any combination of the above embodiments can further include the technique where the predefined optimization comprises parallelization added in the first transform. Such embodiments enable significant improvements to the non-improved transform because parallelism is often useful in reducing processing latency.

[0011] According to some embodiments of the present disclosure, any combination of the above embodiments can further include the technique where the predefined optimization comprises a one-line optimization for the first transform. Such embodiments enable significant improvements to the non-improved transform because these one-line optimizations are often much faster and utilize less resources when the data allows it.

[0012] According to some embodiments of the present disclosure, any combination of the above embodiments can further include generating an optimized file of the serialized objects based on the original pipeline and the optimized first transformation. Advantageously, such embodiments enable the system to return the improved pipeline to the requesting entity in a useful format that is easily transmitted and analyzed.

[0013] According to different embodiments of the present disclosure, any combination of the above embodiments can be implemented by one or more computer-readable storage media. The computer-readable storage media collectively contain computer program code that, when executed by operation of one or more computer processors, performs operations. In embodiments, the performed operations can correspond to any combination of the above methods and embodiments.

[0014] According to yet different embodiments of the present disclosure, any combination of the above embodiments can be implemented by a system. The system includes one or more computer processors and one or more memories collectively containing a program that, when executed by the one or more computer processors, performs operations. In embodiments, the performed operations can correspond to any combination of the above methods and embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 An environment for machine learning is depicted according to one embodiment disclosed herein, including a pipeline optimizer configured to automatically identify and replace transformation modifications to improve functionality of a system.

[0016] Figure 2 A workflow for pipeline optimization is shown according to one embodiment disclosed herein.

[0017] Figure 3 A processing pipeline is depicted according to one embodiment disclosed herein, including a sequence of transformation operations and a set of alternative optimization transformations.

[0018] Figure 4 Pipeline optimization utilizing parallelism in data frames is shown according to one embodiment disclosed herein.

[0019] Figure 5 is a flowchart showing a method for automated pipeline optimization using alternative transformations according to one embodiment disclosed herein.

[0020] Figure 6 is a flowchart showing a method for automatically evaluating and implementing pipeline optimization according to one embodiment disclosed herein.

[0021] Figure 7 is a block diagram showing a pipeline optimizer configured to automatically evaluate an optimization transformation pipeline according to one embodiment disclosed herein. DETAILED DESCRIPTION

[0022] Embodiments of the present disclosure provide techniques for intelligently and automatically optimizing a data preprocessing system by reducing the computational cost required by the data preprocessing system. In one embodiment, the system receives a data transformation pipeline as its input and returns a modified / refined pipeline that reduces the computational cost of processing. In some embodiments, rather than receiving the pipeline itself, the system receives a data file that specifies a set of serialized objects. That is, the input can be a file generated by serializing the pipeline structure. For example, in one such embodiment, the input is a pickle file. In embodiments, a client that designed its preprocessing pipeline can serialize it and transmit or otherwise provide it to the optimization system, which evaluates it for potential optimizations.

[0023] In embodiments, the system can first identify the relationships between the steps of the pipeline. This can include, for example, generating a data preprocessing graph for the pipeline, where each vertex or node in the graph corresponds to a step or transformation in the pipeline, and each edge specifies the data flow between operations as indicated in the pipeline. In one embodiment, the system does so by instantiating the pipeline using the received file (e.g., deserializing the file to construct the pipeline) and using the pipeline to process a sample set of data. By observing this execution, the system can identify the transformation functions that are called and the sequence of these transformations. This can be achieved by observing how the data changes in each step, by adding reflection hooks to each transformation to emit information about the processing, etc. In another embodiment, the system analyzes the serialized file itself to construct the graph without actually instantiating or running the pipeline.

[0024] In some embodiments, the system can further determine the computational cost of one or more of the transformations in the pipeline. These costs can include, for example, computational resource usage (such as memory usage, storage requirements, processor usage, network usage, etc.), latency costs, etc. In one embodiment, the system determines these costs by instantiating the pipeline and adding hooks to each transformation operation. When the pipeline is subsequently used during runtime (to process training or runtime data), the system can determine the cost of each operation based on the actual data used with the ML system. In another embodiment, the system performs the transformations on sample data in order to estimate or determine the cost of each sample. This sample data can be provided by the client (so that it better aligns with the expected real data).

[0025] ​In embodiments, the system can similarly determine the cost of potential optimizations or modifications to the transformation operations. In one embodiment, the optimization system can maintain a library or set of predefined modified / alternative transformations. For each transformation in the original pipeline, the system can identify a set of zero or more alternatives / modifications from this library. The system can then evaluate each such alternative to determine its cost. In at least one embodiment, each transformation is associated with a name or other identifier, a set of inputs, and a set of outputs. Based on this information, the system can identify alternative / modified transformations that can replace operations in the existing pipeline.

[0026] For example, a transformation can be designed or modified to exploit parallelism, which can significantly reduce the cost when the input data includes a large number of rows or columns. Similarly, a line of optimization can improve the cost of a single record or smaller data sets. In embodiments, the system can maintain any number of alternative transformation functions for a given transformation in the original pipeline. By executing each alternative using the same sample data (or real data during runtime), the system can identify the most efficient operation relative to the actual data and pipeline being evaluated.

[0027] Once each modification has been evaluated, in one embodiment, the system can apply the selected optimization to the original pipeline, such as by replacing the original function signature with the selected (optimized) version. In one embodiment, the system can then begin processing received data for the pipeline / client using this optimized pipeline. In another embodiment, the system can serialize this new pipeline and return it to the client (or another system that will execute the pre-processing pipeline and / or ML model). This modified pipeline can operate more efficiently and effectively than the original pipeline, thereby reducing computational expense and improving the functionality of the system.

[0028] Figure 1 An environment 100 for machine learning is depicted in accordance with one embodiment disclosed herein, including a pipeline optimizer 115 configured to automatically identify and replace transformation modifications to improve the functionality of the system. In the illustrated workflow, input data 105 is received and processed during an initial pre-processing step labeled data preparation 110. This data preparation 110 can include, for example, restructuring and / or reformatting the input data 105, aggregating or partitioning the input data 105, etc. In at least one embodiment, the data preparation 110 includes using a transformation pipeline, in which the input data 105 is sequentially processed with a variety of transformations and operations. In embodiments, this pipeline can be linear or non-linear. That is, in at least one embodiment, the pipeline can include loops, recursive portions, etc. In many real-world use cases, the data preparation 110 is computationally complex and is often the source of a large portion of the computational cost required to evaluate the input data 105.

[0029] In the illustrated embodiment, the pipeline optimizer 115 can be used to refine the data preparation 110 so that it operates more efficiently, with reduced latency, and / or using fewer computational resources. In some embodiments, the pipeline optimizer 115 iteratively analyzes each transformation in the pipeline in order to identify alternatives, and evaluates each such alternative to identify the optimal replacement. As shown, after passing through the data preparation 110, model building 120 is performed. Model building 120 generally includes building, training, and refining ML models. These models can include supervised models (e.g., trained with labeled training data), unsupervised models (which are prepared without labeled data), and the like. Generally, model building 120 can include the creation of any number and variety of models.

[0030] After the model has been built, in some embodiments, the model is evaluated or validated (e.g., using test data). In some embodiments, this can result in the collection and preparation of new input data 105 in order to refine the system. Once the model is acceptable, the process proceeds to model deployment 125, in which the model is used during run time. In embodiments, each element in the workflow can of course be repeated at any time. For example, after model deployment 125, the model can be continuously or periodically refined using new data. Similarly, these models can sometimes be completely rebuilt (e.g., completely retrained). Each of these rebuilds requires input data 105 to be processed by the data preparation 110 pipeline. Similarly, during use, when new data is received and evaluated by the deployed model, it must first be passed through the data preparation 110 stage. Thus, data preparation 110 is performed not only during training, but throughout the lifetime of the model. This increases the importance of an optimized and efficient transformation pipeline.

[0031] Figure 2 A workflow 200 for pipeline optimization is shown in accordance with one embodiment disclosed herein. In the illustrated embodiment, an original pipeline 205 is received. In embodiments, the original pipeline 205 corresponds to a set or sequence of data transformations that are applied to input data in order to make it ready for use in a ML system. For example, the original pipeline 205 can include a sequence of operations that transform raw input data into prepared data that is suitable for training a model and / or inputting into a model at runtime. In at least one embodiment, the original pipeline 205 is received by the pipeline optimizer 115 in the form of a serialized data file. For example, in one such embodiment, the input is a pickle file.

[0032] ​As shown, the pipeline optimizer 115 relies on a set of predefined optimized transforms 210 to evaluate the original pipeline 205 to generate a refined pipeline 215. In embodiments, the optimized transforms 210 generally include transform operators, functions, methods, techniques, modules, or other components. Each optimized transform 210 is configured to receive one or more elements of input data and perform one or more operations or transforms. In embodiments, some or all of the optimized transforms 210 are also configured to produce some output (e.g., output modified / transformed input).

[0033] In one embodiment, each optimized transform 210 is a refined or modified version of one or more original transforms. For example, assume that an original transform function (which can be used in the original pipeline 205) involves reformatting input data. In one embodiment, the optimized transforms 210 can include a set of zero or more modified or refined versions of the reformatting transform. That is, a user can have previously defined possible alternatives or modifications to this transform, which can make it more efficient on some datasets. In such embodiments, the optimized transforms 210 can be configured to receive the same input and produce the same output as the original transform, but can do so differently (e.g., exploit parallelism, outsource some transforms to other more qualified components, etc.).

[0034] In one embodiment, each optimized transform 210 is associated with a corresponding signature that can be used to identify it. In some embodiments, this signature includes an indication of the input(s) and / or output(s) of the operation. In some embodiments, the signature further includes a name of the optimized transform 210. In one embodiment, for each transform operation in the original pipeline 205, the pipeline optimizer 115 searches the optimized transforms 210 to determine if any alternatives exist. This can include identifying optimized transforms 210 that have the same or overlapping signatures as the original transform (e.g., have the same or similar name / description and the same input / output).

[0035] The pipeline optimizer 115 can then iteratively evaluate each such alternative to determine which should be applied. In one embodiment, the pipeline optimizer 115 does so by processing test data using the alternative transform and the original transform. In embodiments, the best transform operator can be different depending on the nature and structure of the input data. For example, if the input data frame can be partitioned across multiple parallel pipelines, then parallelism can reduce the computational cost of some input data. Similarly, many standard transform functions are designed to operate on input matrices (e.g., having dimensions of M columns and N rows). If the actual input data exists in a single dimension (e.g., a single column with multiple rows, or a single row with multiple columns), then a row optimization can be applied to reduce complexity.

[0036] In embodiments, the pipeline optimizer 115 can select the optimal function based on any number of criteria, including latency of operations when operating on data, computing resources required, etc. Once the optimal function is identified (which can be an optimized transform 210 or the original transform), the pipeline optimizer 115 selects that operation for the pipeline and proceeds to the next step in the pipeline. Once the process is complete, the pipeline optimizer 115 outputs a modified pipeline 215, which can have some or all of its functions replaced with custom / modified operations, as shown. In some embodiments, the modified pipeline 215 is deployed and used by the system. In at least one embodiment, the modified pipeline 215 is serialized and returned to the client that originally provided the original pipeline 205 for analysis.

[0037] Figure 3 A processing pipeline 300 including a sequence of transform operations and a set of alternative optimized transforms is depicted in accordance with one embodiment disclosed herein. In the illustrated embodiment, the original pipeline 205 includes a sequence of transform operations 305A-N. Although depicted as a sequential workflow, in some embodiments, the original pipeline 205 can include iterations, loops, cycles, and other complexities. Generally, each transform operation 305A-N is a defined transform applied to input data. Arrows in the original pipeline 205 indicate the flow of data through the pipeline. For example, data is processed by transform operation 305A. After that processing, the data is passed to transform operation 305B, which performs a corresponding operation. Subsequently, the data is provided to transform operation 305C.

[0038] In the illustrated embodiment, the pipeline optimizer 115 has identified a set of optimized transforms 310A-N for each transform operation 305A-N. That is, the pipeline optimizer 115 has determined that transform 305A can be replaced with any of the optimized transforms 310A, as they are each configured to receive the same input, perform the same operation, and return the same output. Similarly, transform operation 305B can be replaced with optimized transform 310B, and so on. In embodiments, each transform operation 305 can have zero or more alternative optimized transforms 310. That is, some transform operations 310 can not have any equivalent optimized version, while other transform operations 310 have a single alternative or multiple alternatives.

[0039] In one embodiment, for a given transform operation 305, a set of potential alternative optimized transforms 310 are identified based on their signatures. For example, assume that transform operation 305A is associated with two inputs and one output, and has a name / description of "DBLookup." In some embodiments, the pipeline optimizer 115 can identify an optimized transform 310A that utilizes the same inputs and outputs. In at least one embodiment, the pipeline optimizer 115 identifies an optimized transform 310 that has the same or similar name (e.g., "DBLookup_vl. l"). In some embodiments, the optimized transforms 310 can have the same name as the underlying transform they were designed to replace, but are associated with other metadata indicating the type of optimization, the user who created it, the time and / or date it was created, version information, etc. Once a set of potential alternatives are identified for a given transform operation 305, they can be iteratively evaluated (along with the original operation) in order to select one or more to be substituted into the original pipeline 205.

[0040] Figure 4 A pipeline optimization 400 that utilizes parallelism in data frames is shown in accordance with one embodiment disclosed herein. In the illustrated embodiment, one or more portions of the original pipeline have been modified to utilize parallelism that can improve computational efficiency in some data sets. Specifically, as shown, an input data frame 405 undergoes partitioning 410, and each portion of the partitioned data frame 405 undergoes separate (parallel) transforms in operations 415A and 415B. After operation 415B, the separate streams are recombined in a gather 420 to produce a single data frame. The frame can then proceed to the next step in the pipeline.

[0041] In some embodiments, upon determining that parallelism improves a given transform operation, the pipeline optimizer 115 can insert a partition 410 and a gather 420 before and after the given operation, respectively. In at least one embodiment, if the pipeline optimizer 115 determines that two or more adjacent operations can both be improved using the same parallelism, the pipeline optimizer 115 can place the partition 410 and gather 420 near that set of sequential operations. That is, the pipeline optimizer 115 can instead create a pipeline that partitions data, processes that data using a sequence of transforms, and gathers the data at a point where the next operation does not utilize parallelism, rather than repeatedly partitioning and gathering data frames at each operation.

[0042] Figure 5is a flowchart illustrating a method 500 for automated pipeline optimization using alternative transformations according to one embodiment disclosed herein. In the illustrated embodiment, the method 500 begins at block 505, where the pipeline optimizer 115 receives a data preparation pipeline. In some embodiments, the pipeline is received from a client. For example, a client can provide a pipeline and request that it be evaluated / optimized. In at least one embodiment, the pipeline optimizer 115 is one component in a system that provides ML services. For example, a client can define the structure and configuration of a preparation pipeline as well as the ML model itself, and transmit these specifications to an entity that hosts the model and provides computational resources.

[0043] In some embodiments, the pipeline is received as a serialized data file. In general, the received pipeline specifies a set of one or more transformations, operations, or functions to be applied, and defines the data flow between the transformations. At block 510, the pipeline optimizer 115 generates a processing graph for the received pipeline. In one embodiment, each vertex or node in the processing graph corresponds to one of the transformations or operations, and each edge in the graph corresponds to how data flows through the pipeline. In one embodiment, the pipeline optimizer 115 generates the graph by evaluating the data file itself to identify the transformations and data flow, without actually instantiating or constructing the pipeline. In at least one embodiment, the pipeline optimizer 115 constructs the graph by constructing / instantiating the pipeline based on the received specifications.

[0044] In some embodiments, the pipeline optimizer 115 then adds hooks or other structures to each transformation in order to track the data as it moves through the pipeline. The pipeline optimizer 115 can then run a set of sample data through the instantiated pipeline, and use this reflective mechanism to identify the transformations / operations performed at each node as well as the sequence of transformations applied.

[0045] At block 515, the pipeline optimizer 115 selects one of the transformation operations from the original pipeline. In one embodiment, selecting the transformation operation is performed by selecting one of the nodes / vertices in the generated graph and identifying the underlying function in the pipeline. The method 500 then continues to block 520, where the pipeline optimizer 115 determines whether there is one or more alternatives for the selected transformation operation. In one embodiment, this includes searching a set of predefined transformations (e.g., optimized transformations 210) to identify an alternative function that utilizes the same input data, returns the same output data, and performs the same operation / transformation as the selected function. In some embodiments, as discussed above, the pipeline optimizer 115 uses the signature of the selected operation to identify matching or related modified transformations.

[0046] If no alternative or modified transformation is available for the selected node, the method 500 continues to block 550, discussed in more detail below. Returning to block 520, if there is at least one modified, optimized, or alternative transformation operation for the selected function, the method 500 continues to block 525. At block 525, the pipeline optimizer 115 selects one of the identified alternative operations. The method 500 then proceeds to block 530, where the pipeline optimizer 115 quantifies the cost of the selected alternative.

[0047] In one embodiment, the pipeline optimizer 115 does so by running the selected alternative on sample data. This sample data can be created or selected by the pipeline optimizer 115, or can be provided by the client. For example, in one such embodiment, the client can provide a sample of data that reflects expected run-time data, which allows the pipeline optimizer 115 to determine the cost of the selected alternative based on actual data. In some embodiments, the pipeline optimizer 115 instantiates and uses the original pipeline during run-time (e.g., to begin processing actual data in deployment to train or use a model). In such embodiments, the pipeline optimizer 115 can also use the identified alternative to process data during run-time in order to quantify the cost of each using actual data.

[0048] In embodiments, the costs collected can include any number and kind of metrics, including latency, cost, memory usage, processor usage, storage usage, etc. In some embodiments, the client can select and / or weight the costs to be considered. Once the cost of a transformation is determined, the method 500 proceeds to block 535, where the pipeline optimizer 115 determines whether there is at least one alternative that has not been evaluated. If so, the method 500 returns to block 525. If all possible alternative transformation functions have been identified for cost, the method 500 continues to block 540, where the pipeline optimizer 115 similarly determines the cost of the original transformation function in the pipeline. In embodiments, this can include evaluating sample data using the original pipeline. In some embodiments, as discussed above, this can include instantiating and using the pipeline during run-time, and monitoring the cost of operations as it processes real data in the system.

[0049] At block 545, the pipeline optimizer 115 selects the best transform among the evaluated transforms for use in the pipeline. That is, in one embodiment, the pipeline optimizer 115 identifies the function with the lowest cost (which can be the original operation, or one of the identified alternatives). In some embodiments, this determination is made based in part on client-defined weights for each cost metric. If the original transform has the lowest cost, the pipeline optimizer 115 refrains from modifying that portion of the pipeline, and leaves the original element in place. However, if one of the alternatives is more optimal, the pipeline optimizer 115 can modify the pipeline by replacing the original transform with the identified optimal transform. For example, the pipeline optimizer 115 can modify the function call or other element of the pipeline to route data through the optimized version rather than the original operation.

[0050] The method 500 then proceeds to block 550, where the pipeline optimizer 115 determines whether there are at least one additional transform operation in the original pipeline that has not yet been evaluated. If so, the method 500 returns to block 515. If every step in the original pipeline has been evaluated, the method 500 continues to block 555, where the pipeline optimizer 115 returns the modified pipeline. In some embodiments, this includes generating a serialized data file for the modified pipeline (including the selected replacement / alternative elements) and returning it to the client. In one embodiment, the system can instead instantiate the modified pipeline and begin using it in the deployed environment to begin processing runtime data for the client.

[0051] Figure 6 is a flowchart illustrating a method 600 for automatically evaluating and implementing pipeline optimizations according to one embodiment disclosed herein. The method 600 begins at block 605, where the pipeline optimizer 115 receives an original file of serialized objects. At block 610, the pipeline optimizer 115 identifies an original pipeline including a plurality of transforms based on the original file. The method 600 then continues to block 615, where the pipeline optimizer 115 determines a first computational cost of a first transform in the plurality of transforms. Further, at block 620, the pipeline optimizer 115 modifies the first transform using a predefined optimization. At block 625, the pipeline optimizer 115 determines a second cost of the modified first transform. The method 600 then proceeds to block 630, where upon determining that the second cost is lower than the first cost, the pipeline optimizer 115 replaces the first transform with the optimized first transform in the original pipeline.

[0052] Figure 7is a block diagram illustrating a pipeline optimizer 115 according to one embodiment disclosed herein, the pipeline optimizer 115 configured to automatically evaluate optimization transform pipelines. Although depicted as a physical device, in embodiments, the pipeline optimizer 115 can be implemented using virtual devices, and / or across multiple devices (e.g., in a cloud environment). As shown, the pipeline optimizer 115 includes a processor 710, a memory 715, a storage 720, a network interface 725, and one or more I / O interfaces 730. In the illustrated embodiment, the processor 710 retrieves and executes programmed instructions stored in the memory 715, as well as stores and retrieves application data residing in the storage 720. The processor 710 generally represents a single CPU and / or GPU, multiple CPUs and / or GPUs, a single CPU and / or GPU with multiple processing cores, etc. The memory 715 is generally included to represent random access memory. The storage 720 can be any combination of disk drives, flash-based storage devices, etc., and can include fixed and / or removable storage devices, such as fixed disk drives, removable memory cards, caches, optical storage, network attached storage (NAS), or storage area networks (SAN).

[0053] In some embodiments, input and output devices, such as a keyboard, monitor, etc., are connected via the I / O interface 730. Further, via the network interface 725, the pipeline optimizer 115 can be communicatively coupled with one or more other devices and components (e.g., via a network 780, which can include the Internet, local network(s), etc.). As shown, the processor 710, the memory 715, the storage 720, the network interface(s) 725, and the I / O interface(s) 730 are communicatively coupled by one or more buses 775.

[0054] In the illustrated embodiment, the storage 720 includes a set of predefined modified transforms 755. In some embodiments, these modified transforms 755 are user-defined functions, transforms, and operations designed to perform one or more operations in different ways (e.g., with parallelism). Each modified transform 755 can be performed differently based on the underlying data being processed. In at least one embodiment, each modified transform 755 includes an indication of modifications that have been or should be applied to the original function to implement the modification. In some embodiments, each modified transform 755 further indicates the original function, allowing the pipeline optimizer 115 to easily identify the relevant / appropriate alternative for each step in the original pipeline.

[0055] In the illustrated embodiment, the memory 715 includes an optimizer application 735. While described as software residing in memory 715, in embodiments the functionality of the optimizer application 735 can be implemented using hardware, software, or a combination of hardware and software. As shown, the optimizer application 735 includes a pipeline component 740, a modification component 745, and a cost component 750. Although depicted as discrete components for conceptual clarity, in embodiments the operations of the pipeline component 740, the modification component 745, and the cost component 750 can be combined or distributed across any number of components and devices.

[0056] In one embodiment, the pipeline component 740 is configured to receive a pipeline (or data file including serialized objects) for processing and transforming data in preparation for use with one or more ML models. The pipeline component 740 can then construct a graph based on the file / pipeline, where each node in the graph corresponds to a transformation step and each edge indicates a flow of data between steps. In some embodiments, once the best modifications are identified, the pipeline component 740 modifies the original pipeline as needed (e.g., by replacing original transformations with modified transformations, by changing pointers or data flows, etc.). The pipeline component 740 can then return the optimized pipeline.

[0057] In the illustrated embodiment, the modification component 745 evaluates each step in the original pipeline to identify potential modified transformations 755. In some embodiments, as discussed above, the modification component 745 does so by identifying modified transformations 755 that have the same or similar signatures as the original steps. In at least one embodiment, the modified transformations 755 can specifically identify the original functions that they are intended to replace. In embodiments, the cost component 750 is used to collect the cost of each original element and the corresponding alternatives, as discussed above. For example, the cost component 750 can run test data through the transformations, and / or monitor the transformers when they are processing real data during runtime. This allows the cost component 750 to identify the lowest cost transformation operation for each element of the pipeline.

[0058] The description of the various embodiments of the present application have been presented for purposes of illustration but is not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein was chosen to best explain the principles of the embodiments, the practical application or technical improvement over technologies found in the marketplace, or to enable others skilled in the art to understand the embodiments disclosed herein.

[0059] In the foregoing and / or below, reference is made to examples of the present disclosure. These examples are not meant to limit the scope of the disclosure, as those skilled in the art will understand that the features and aspects of the examples can be combined, modified, or otherwise altered to achieve and practice the intended embodiments. Furthermore, although the examples disclosed herein can have advantages over other possible solutions or over the prior art, whether or not a specific advantage is mentioned here is not limiting of the scope of the disclosure. Thus, the aspects, features, embodiments, and advantages discussed herein are merely illustrative and are not considered elements or limitations of the appended claims, unless the claim explicitly states otherwise. Likewise, reference to "the invention" shall not be construed as being a reference to any one of the particular examples disclosed herein and shall instead be understood to mean any or all of the aspects, features, embodiments, and advantages discussed herein, unless the claim explicitly states otherwise.

[0060] Various aspects of the present application can be implemented as follows: an embodiment of all hardware, an embodiment of all software (including firmware, resident software, micro-code, etc.), or an embodiment of a combination of hardware and software aspects, which can all generally be referred to herein as a "circuit," "module" or "system."

[0061] The present application can be a system, a method, and / or a computer program product. The computer program product can include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present application.

[0062] The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium can be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non- exhaustive list of more specific examples of the computer readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.

[0063] Computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions to a computer readable storage medium within the respective computing / processing device for storage and / or execution.

[0064] Computer readable program instructions for carrying out operations of the present application can be assembly instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like, and conventional procedural programming languages such as the "C" programming language or similar programming languages. The computer readable program instructions can execute entirely on the user's computing device, partly on the user's computing device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device or entirely on the remote computing device or server. In the latter scenario, the remote computing device can be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computing device, for example, through the Internet using an Internet Service Provider. In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present application.

[0065] The computer readable program instructions can also be loaded onto a computing / processing device, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computing / processing device, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computing / processing device, other programmable apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0066] These computer readable program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions can also be stored in a computer readable storage medium that can include a non-transitory computer readable storage medium that can be a computer- readable storage medium having no data, programs, program modules, e.g., instructions for operation, or digital information thereon. The

[0067] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0068] The flow diagrams and block diagrams in the drawings are representative of the architectures, functions, and operations of possible implementations of systems, methods and computer program products according to various embodiments of the present application. To this end, each block in the flowcharts or block diagrams can represent a module, segment, or portion of instructions, which includes one or more executable instructions for implementing the specified logical functions (‘instructions’). In some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks can sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations of blocks in the block diagrams and / or flowchart illustrations, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and

[0069] Embodiments of the present application can be provided to end users through a cloud computing infrastructure. Cloud computing generally refers to the provision of scalable computing resources as a service over a network. More formally, cloud computing can be defined as a computing capability that provides an abstraction between the computing resource and its underlying utility (e.g., the ability to provision storage, processors, networks, applications, and complete virtualized systems) delivered as a service to customers. The concept of cloud computing can be described as a broad network of resource pools where the computing resources are delivered to a customer as a service over the Internet. Pool of resources can include applications (e.g., web-based e-mail), storage, processing power, and / or networks. Cloud computing is composed of five essential characteristics, three service models, and four deployment models.

[0070] Generally, cloud computing resources are provided to users on a pay-per-use basis, where users are invoiced only for the computing resources actually used (e.g., amount of storage space consumed, or number of virtualized systems instantiated). Users can access any of their resources over the Internet from anywhere they have access to the Internet. In the context of the present application, users can access applications (e.g., optimizer application 735) or related data available in the cloud. For example, optimizer application 735 can execute on a computing system in the cloud and analyze and refine data pipelines. In such a case, optimizer application 735 can receive and optimize such pipelines and store the refined pipelines and predefined modifications at a storage location in the cloud. Doing so allows users to access this information from any computing system attached to a network (e.g., the Internet) connected to the cloud.

[0071] While the foregoing is directed to embodiments of the present application, other and further embodiments of the application can be devised without departing from the basic scope thereof, and the scope of the application is determined by the claims that follow.

Claims

1. A method for pipeline optimization, comprising: receiving an original file of a serialized object; identifying an original pipeline comprising a plurality of transforms based on the original file; determining a first computational cost of a first transform of the plurality of transforms; modifying the first transform using a predefined optimization; determining a second computational cost of the modified first transform; and replacing the first transform with the modified first transform in the original pipeline upon determining that the second computational cost is lower than the first computational cost.

2. The method of claim 1, wherein identifying the original pipeline comprises generating a data processing graph, wherein each respective vertex in the data processing graph represents a respective transform of the plurality of transforms, and wherein each respective edge in the data processing graph specifies a data flow in the original pipeline. generating the data processing graph comprises:

3. The method of claim 2, wherein, instantiating the original pipeline based on the original file; executing the original pipeline on sample data; identifying the plurality of transforms based on the execution; and identifying a sequence of the plurality of transforms based on the execution. generating the data processing graph comprises evaluating the original file to identify the plurality of transforms and the data flows in the original pipeline without instantiating the original pipeline.

4. The method of claim 2, wherein, determining the first computational cost of the first transform comprises:

5. The method of claim 1, wherein, executing the first transform on sample data; and evaluating the execution to determine a latency of executing the first transform. the predefined optimization comprises increased parallelization in the first transform.

6. The method of claim 1, wherein, the predefined optimization comprises a one-line optimization for the first transform.

7. The method of claim 1, wherein, 8. The method of claim 1, the method further comprising: generating an optimized file of a serialized object based on the original pipeline and the modified first transform.

9. A computer program product containing computer program code that, when executed by operation of one or more computer processors, performs operations comprising: receiving an original file of a serialized object; identifying an original pipeline comprising a plurality of transforms based on the original file; determining a first computational cost of a first transform of the plurality of transforms; modifying the first transform using a predefined optimization; determining a second computational cost of the modified first transform; and replacing the first transform with the modified first transform in the original pipeline upon determining that the second computational cost is lower than the first computational cost.

10. The computer program product of claim 9, wherein identifying the original pipeline comprises generating a data processing graph, wherein each respective vertex in the data processing graph represents a respective transform of the plurality of transforms, and wherein each respective edge in the data processing graph specifies a data flow in the original pipeline. generating the data processing graph comprises: instantiating the original pipeline based on the original file; 11. The computer program product of claim 10, wherein, executing the original pipeline on sample data; identifying the plurality of transforms based on the execution; and identifying a sequence of the plurality of transforms based on the execution. ​ ​ 12. The computer program product of claim 10, wherein, Generating the data processing graph includes evaluating the original file to identify the plurality of transformations and the data flow in the original pipeline without instantiating the original pipeline.

13. The computer program product of claim 9, wherein, Determining the first computational cost of the first transformation includes: executing the first transformation on sample data; and evaluating the execution to determine a latency of executing the first transformation.

14. The computer program product of claim 9, wherein, The predefined optimization includes (i) increased parallelization in the first transformation, or (ii) one-line optimization of the first transformation.

15. The computer program product of claim 9, the operations further comprising: generating an optimized file of serialized objects based on the original pipeline and the modified first transformation.

16. A system for pipeline optimization, comprising: one or more computer processors; and one or more memories collectively containing one or more programs which, when executed by the one or more computer processors, perform operations comprising: receiving an original file of serialized objects; identifying an original pipeline comprising a plurality of transformations based on the original file; determining a first computational cost of a first transformation in the plurality of transformations; modifying the first transformation using a predefined optimization; determining a second computational cost of the modified first transformation; and replacing the first transformation with the modified first transformation in the original pipeline upon determining that the second computational cost is lower than the first computational cost.

17. The system of claim 16, wherein identifying the original pipeline includes generating a data processing graph, wherein each respective vertex in the data processing graph represents a respective transformation in the plurality of transformations, and wherein each respective edge in the data processing graph specifies a data flow in the original pipeline.

18. The system of claim 17, wherein, Generating the data processing graph includes: instantiating the original pipeline based on the original file; executing the original pipeline on sample data; identifying the plurality of transformations based on the execution; and identifying a sequence of the plurality of transformations based on the execution.

19. The system of claim 17, wherein, Generating the data processing graph includes evaluating the original file to identify the plurality of transformations and the data flow in the original pipeline without instantiating the original pipeline.

20. The system of claim 16, wherein, Determining the first computational cost of the first transformation includes: executing the first transformation on sample data; and evaluating the execution to determine a latency of executing the first transformation.

Images