Non-coding machine learning pipeline

Abstract

A method (500) for implementing a non-coding machine learning pipeline includes receiving, by a GUI (200, 400), a user selection of a mode button (226) displayed in the GUI, wherein the mode button, when selected, causes the GUI to display a first set of user-selectable buttons (402) corresponding to respective machine learning routines (308), and the mode button, when not selected, causes the GUI to display a second set of user-selectable buttons (208) corresponding to respective machine learning sub-routines (310). In response to receiving the user selection of the mode button, the method further includes displaying the first set of user-selectable buttons, receiving a user selection of one or more of the first set of user-selectable buttons, displaying a graphical representation of a machine learning model (414) defined by one or more machine learning routines corresponding to the user selection of the one or more of the first set of user-selectable buttons, and generating a file representing the machine learning model.

Description

TECHNICAL FIELD

[0001] This specification relates to cloud computing. BACKGROUND

[0002] Cloud computing systems can provide access to a variety of computing resources. For example, a cloud computing system can store data accessible by a plurality of different devices for a client device, allow a plurality of different client devices to access a single application executing on the cloud, and provide access to other computer resources. SUMMARY

[0003] This specification describes a cloud computing non-coding machine learning pipeline for building, executing, and publishing machine learning models.

[0004] Generally, one innovative aspect of the subject matter described in this specification can be implemented in a computer-implemented method that includes providing a graphical user interface (GUI) for generating a machine learning model; receiving, by the GUI, a user selection of a mode button displayed in the GUI, where the mode button, when selected, causes the GUI to display a first set of user-selectable buttons corresponding to respective machine learning routines, and, when not selected, causes the GUI to display a second set of user-selectable buttons corresponding to respective machine learning sub-routines, where a machine learning routine includes a respective plurality of connected machine learning sub-routines; in response to receiving the user selection of the mode button, displaying the first set of user-selectable buttons in the GUI; receiving, by the GUI, a user selection of one or more of the first set of user-selectable buttons; displaying, in the GUI, a graphical representation of a machine learning model defined by one or more machine learning routines corresponding to the one or more of the first set of user-selectable buttons selected by the user, and generating a file representing the machine learning model.

[0005] Other implementations of these aspects include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the methods. A system of one or more computer systems and / or one or more computers can be configured to perform particular operations or actions by virtue of having software, firmware, hardware, or a combination thereof installed on the system that in operation

[0006] The foregoing and other embodiments can each optionally include one or more of the following features, alone or in combination. In some embodiments, the machine learning subroutines include a data formatting algorithm, a data splitting algorithm, a feature selection algorithm, a machine learning training algorithm, a machine learning evaluation algorithm, or a statistical algorithm.

[0007] In some embodiments, each of the first set of user-selectable buttons represents a package file corresponding to a machine learning routine, and includes a respective plurality of connected machine learning subroutines.

[0008] In some embodiments, the method further includes receiving, through the GUI, a user selection of a source code button displayed in the GUI; in response to the user selection of the source code button, displaying, in the GUI, source code representing the machine learning model, wherein the generated file includes the source code; receiving, through the GUI, one or more user adjustments to the displayed source code; and generating an updated file representing an updated machine learning model defined by (i) one or more machine learning routines corresponding to one or more of the first set of user-selectable buttons selected by the user, and (ii) the adjustments to the displayed source code.

[0009] In some embodiments, the adjustments to the displayed source code include adjustments to machine learning model parameters or functions included in the source code.

[0010] In some embodiments, the method further includes receiving, through the GUI, a request from a user to download the generated file representing the machine learning model; providing, through the GUI, access to the generated file representing the machine learning model.

[0011] In some embodiments, the request to download the generated file representing the machine learning model includes a request to reformat the generated file according to an indicated type, and wherein the method further includes: generating a reformatted file representing the machine learning model of the indicated type; providing, through the GUI, access to the reformatted file representing the machine learning model.

[0012] In some embodiments, further including storing the generated file representing the machine learning model at a location associated with the user.

[0013] In some embodiments, the method further includes receiving, through the GUI, a selection of a data set; receiving, through the GUI, a request to execute the generated file representing the machine learning model using the data set; executing the generated file representing the machine learning model using the data set to generate a corresponding machine learning model output; displaying the generated machine learning model output in the GUI.

[0014] In some embodiments, receiving a selection of a data set includes receiving a selection of a sample data set or receiving a selection of a data set imported by the user.

[0015] In some embodiments, receiving, through the GUI, a request to execute the generated file representing the machine learning model using the data set includes receiving, through the GUI, a user selection of a run button, wherein the run button is enabled for selection when a machine learning model input data set is selected.

[0016] In some embodiments, the method further includes receiving a request to publish the generated file representing the machine learning model; in response to receiving the request, publishing the generated file as a new asset in the GUI.

[0017] In some embodiments, the provided GUI for generating a machine learning model includes a drag-and-drop interface, wherein the GUI includes: a searchable list of a first set of user-selectable buttons or a second set of user-selectable buttons; an edit area; wherein a user can select, drag, and drop a user-selectable button from the first set of user-selectable buttons or the second set of user-selectable buttons into the edit area to define a machine learning model.

[0018] In some embodiments, the machine learning subroutines or the types of machine learning routines that are used frequently are listed higher than the machine learning subroutines or the types of machine learning routines that are used less frequently.

[0019] In some embodiments, the edit area includes a template graphic to assist a user in selecting, dragging, and dropping the user-selectable buttons into the edit area to define a machine learning model.

[0020] In some embodiments, the provided GUI for generating a machine learning model includes a plurality of user-selectable tabs, including a machine learning model creation tab, a machine learning model publishing tab, and a machine learning model versioning tab.

[0021] The subject matter described in this specification can be implemented in particular implementations so as to realize one or more of the following advantages.

[0022] Systems implementing the non-coding machine learning pipelines described in this specification provide users who are unable or unwilling to code with the possibility to quickly and easily generate, execute, and publish machine learning models. Furthermore, through the presently described simple mode functionality, users of the non-coding machine learning pipelines do not need to understand basic or complex machine learning concepts. Thus, the presently described non-coding machine learning pipelines are accessible to a large number of users and improve the interaction between the user and the pipeline, as even inexperienced users can quickly and with minimal effort generate and run machine learning solutions.

[0023] Furthermore, systems implementing the non-coding machine learning pipelines described in this specification provide users with enhanced flexibility, as the front end of the pipeline provides the possibility to directly modify the source code underlying the generated machine learning models. Users can use the provided simple mode to more quickly generate machine learning models, but also have increased control over the machine learning models. Furthermore, machine learning models or building blocks of machine learning models established by different users can be easily reused. Thus, the efficiency of generating personalized models can be improved. Furthermore, the computational efficiency of running machine learning models can be improved, as users with experience in machine learning concepts have the possibility to simplify the provided machine learning routines.

[0024] Systems implementing the non-coding machine learning pipelines described in this specification provide users with enhanced execution transparency and data security, as machine learning models generated by the non-coding machine learning pipelines can be executed on resources provided for users in the system or can be downloaded and securely run offline.

[0025] Furthermore, systems implementing the non-coding machine learning pipelines described in this specification provide type safety checks at design time, as users cannot connect building blocks when the output of a block does not match the input of another block. Thus, errors can be identified or prevented during the construction process and not only at runtime.

[0026] The details of one or more implementations of the subject matter described in this specification are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages of the subject matter will become apparent from the description, the drawings, and the claims. BRIEF DESCRIPTION OF DRAWINGS

[0027] Figure 1 is a block diagram of an example system implementing a non-coding machine learning pipeline.

[0028] Figure 2 is a schematic diagram of an example graphical user interface for generating, executing, and publishing machine learning models using a non-coding machine learning pipeline.

[0029] Figure 3is a schematic diagram of a wrapped file representing an example machine learning routine.

[0030] Figure 4 is a schematic diagram of an example graphical user interface using simple mode functionality in a non-coding machine learning pipeline to generate, execute, and publish machine learning models.

[0031] Figure 5 is a flowchart of an example process to generate a machine learning model using a non-coding machine learning pipeline.

[0032] Figures 6A and 6B are flowcharts of an example process to execute a machine learning model that has been generated using a non-coding machine learning pipeline.

[0033] The same reference numbers and designations in the various drawings indicate the same elements. DETAILED DESCRIPTION

[0034] SUMMARY

[0035] Machine learning is the study of algorithms and statistical models that computer systems use to effectively perform a specific task without using explicit instructions, but rather by relying on patterns and inference. Machine learning algorithms build models of training data in order to make predictions or decisions on unseen data without being explicitly programmed to perform a specific task.

[0036] This specification describes a cloud computing service for building machine learning models. The service includes a GUI, e.g., a web-based GUI, in which a user can drag and drop graphical representations of machine learning subroutines into an editing area. The user can link the dropped-in machine learning subroutines to form a graph representing a machine learning model.

[0037] To help users with little to no understanding of the concepts of machine learning, the GUI provides a simple mode presentation in which machine learning subroutines are packaged to form a "black box" representation of complete machine learning routines that only need to specify data inputs and target outputs. The user can then drag and drop graphical representations of these machine learning routines into an editing area to form a graph representing a machine learning model.

[0038] The cloud service can provide access to a machine learning engine in which the generated machine learning model can be trained and / or used for inference. Alternatively, the user can download a file representing the generated machine learning model and train and / or perform inference offline, e.g., on an open source stack.

[0039] Example Operating Environment

[0040] Figure 1is a block diagram of an example system 100 that implements a non-code machine learning pipeline. A computer network 102, such as a local area network (LAN), a wide area network (WAN), the Internet, or a combination thereof, connects a non-code machine learning pipeline front end 104 to various components of a cloud computing environment 106.

[0041] A user can access the non-code machine learning pipeline front end 104 via a user device (e.g., user device 118). The user device 118 can include a computing device, such as a desktop computer, a laptop computer, a tablet computer, a smartphone, or any device capable of displaying a graphical user interface (GUI) of the non-code machine learning pipeline front end provided by the non-code machine learning pipeline back end 112.

[0042] The GUI provided by the non-code machine learning pipeline back end 112 can be a web-based GUI and provides an interactive visual workspace to the user to build, execute, and publish machine learning models. The GUI allows the user to drag-and-drop machine learning subroutines into an editing area where the machine learning subroutines can be connected to form a machine learning model - the user does not need programming skills to generate a machine learning model. The formed machine learning model can be trained on a dataset by the cloud computing environment 106, or can be downloaded and trained offline by a third party system. Similarly, once trained, the cloud computing environment 106 can perform inference on a dataset using the machine learning model, or a third party system can download the trained machine learning model and perform inference using the trained machine learning model. Example processes of generating a machine learning model using the non-code machine learning pipeline are described below with reference to Figure 2 and 4 Example GUIs provided by the non-code machine learning pipeline back end are shown and described. Example processes of generating a machine learning model using the non-code machine learning pipeline are described below with reference to Figure 5 , 6A and 6B.

[0043] The various components of the cloud computing environment 106 include a cloud storage component 108, a relational or non-relational database 110, e.g., a NewSQL database, a non-code machine learning pipeline back end 112, a cloud machine learning engine 114, and an inventory back end 116.

[0044] The cloud storage component 108, the relational or non-relational database 110, e.g., a NewSQL database, the non-code machine learning pipeline back end 112 are components that perform operations related to generating or editing a machine learning model based on input received from a user through the non-code machine learning pipeline front end. The non-code machine learning pipeline back end provides the GUI described above and processes input received from the user via the GUI. Example operations performed by the non-code machine learning pipeline back end include those described below with reference to Figure 5 , 6Asome or all of those operations described in connection with FIGS. 6A and 6B. Cloud storage components 108 and databases 110 store information related to machine learning models created by users through the non-code machine learning pipeline front end. For example, cloud storage components 108 and databases 110 can store intermediate states, revision histories, execution logs, or input / output of machine learning models.

[0045] Cloud machine learning engine 114 performs operations related to executing machine learning models generated by the non-code machine learning pipeline front end and back end. For example, cloud machine learning engine 114 can execute machine learning routines or subroutines on specified datasets in a machine learning model to train the machine learning model and / or perform inference. Outputs (results) generated by cloud machine learning engine 114 can be provided directly to non-code machine learning pipeline front end 104 or non-code machine learning pipeline back end 112.

[0046] Inventory back end 116 performs operations related to publishing machine learning models generated by the non-code machine learning pipeline front end and back end. For example, inventory back end can include storage (with security checks) that is exposed to users of the non-code ML pipeline.

[0047] Figure 2 is a schematic diagram of an example GUI 200 for generating, executing, and publishing machine learning models using a non-code machine learning pipeline.

[0048] Example GUI 200 includes a search box 202 in which a user can search for a list of machine learning subroutines. Example machine learning subroutines can include algorithms or functions such as: various data formatting algorithms; various data splitting algorithms; various feature selection algorithms; various machine learning training algorithms, various machine learning evaluation algorithms, or various statistical algorithms. Other example machine learning subroutines include datasets or machine learning commands such as inputs, outputs, or evaluations. In some implementations, GUI 200 can also include an area 204 in which search options and filters can be set by a user. For example, a user can select a type of machine learning task as a search option.

[0049] In some implementations, a default list of machine learning subroutines, e.g., a list of frequently used machine learning subroutines, can be displayed in box 206 in the GUI before a user enters text into search box 202, where the frequently used machine learning subroutines listed are higher than the infrequently used machine learning subroutines. In addition, after a user begins entering text into search box 202, the list of machine learning subroutines displayed in box 206 can be updated based on the entered text.

[0050] Each of the machine learning subroutines in the list of machine learning subroutines in block 206 is displayed as a user-selectable button 208, e.g., user-selectable buttons 208, 208a-c. With the pointer tool function activated by selecting the pointer tool button 216, the user can select a button corresponding to a particular machine learning subroutine and drag-and-drop the button corresponding to the particular machine learning subroutine into the edit area 210. In the example GUI 200, the user has selected, dragged-and-dropped buttons corresponding to the machine learning subroutines "input <data>", "machine learning algorithm III", "data splitter IV", "train algorithm I", "evaluate", and "output <model>, output <visual>".

[0051] The edit area 210 is used to define a machine learning model based on the selected machine learning subroutines. Once the user has placed buttons corresponding to machine learning subroutines into the edit area 210, the user can further connect the machine learning subroutines to form a graph representing the machine learning model. Connections between machine learning subroutines (e.g., connection 214) can be created by activating a connector function by selecting the connector button 212. In some implementations, the connector function can not allow connections between incompatible machine learning subroutines, e.g., can not allow the user to connect an input subroutine directly to an output subroutine, and can automatically guide connections between connected machine learning subroutines, e.g., can connect a data splitting routine to a training algorithm even if the user first selects the training algorithm and then selects the data splitting routine. These aspects of the connector function can increase the likelihood that the generated machine learning model will run properly and can execute without errors.

[0052] In some implementations, the edit area 210 can display a template graph that is used to assist the user when selecting, dragging-and-dropping the user-selectable buttons related to machine learning subroutines into the edit area to define a machine learning model. For example, when the GUI is first presented to the user, a template graph showing the connected building blocks of an example machine learning model can be shown in the edit area with 50% opacity. In some implementations, the template graph presented in the edit area can be updated based on the search terms or search options entered by the user in the search box 202 or area 204. For example, if the user enters "image classification" into the area 204, the template graph displayed in the edit area 210 can show an example graph of a plurality of connected machine learning subroutines that together represent a convolutional neural network.

[0053] As the user operates in the edit area, i.e., drags, drops, and connects machine learning subroutines in the edit area, the system providing the GUI calls a backend with a prototype file representing the graph being constructed. In some implementations, the system can store a complete or partial history of the operations performed in the edit area.

[0054] The example GUI 200 also includes a plurality of user-selectable tabs, such as a machine learning model creation or design tab 218a, a machine learning model publishing tab 218b, an events tab 218c, and a machine learning model version tab 218d. If the user selects the design tab 218a, the GUI presents options and features for creating a machine learning model, as shown in the example GUI 200.

[0055] If the user selects the publishing tab 218b, the GUI presents, alternatively or additionally, options and / or features related to publishing the created machine learning model. For example, the GUI presentation can include a user-selectable publish button that, when selected, allows the system to publish the graph as an asset to other users of the system for editing, execution, and saving. Such published assets can be accessible to users under the design tab 218a, for example, presented in a list in the boxes 208a-c.

[0056] If the user selects the events tab 218c, the GUI presentation can present a page listing an activity log for a particular project, where the user can view timestamps and each revision of the machine learning model, for example, corresponding to one or more steps the user performed the model or stored files in the cloud. The events tab 218c can also serve as a summary of the charges incurred by the cloud service user, as the timestamps and revisions can be matched to corresponding charges that appear on the user’s bill.

[0057] If the user selects the version tab 218d, the GUI presentation can include a history of edits made in the edit area, for example, presented as a list of selectable snapshots. The user can restore a snapshot by selecting the snapshot, which can automatically copy the snapshot into a new project or pipeline, for example, a second GUI presentation. In some implementations, the user can also view a summary of the execution history when operating the GUI under the design tab, for example, in the area 222 of the GUI 200.

[0058] The example GUI 200 also includes a run button 220. When the run button is enabled, the user can select the run button to execute the machine learning model generated in the edit area 210. When the run button is selected, the machine learning model is immediately executed in the cloud, for example, by the cloud machine learning engine. The results from selecting the run button 220 and executing the machine learning model can be displayed directly in the edit area or can be seen by hovering or selecting the corresponding output module in the edit area. However, in some implementations, the run button 220 can be disabled, for example, if the machine learning model generated in the edit area is missing routines or cannot be executed. In these implementations, the GUI can also present a warning flag in, for example, the edit area 210 or next to the run button 220.

[0059] The example GUI 200 also includes a save button 224 and a download button 230. A user can select the save button 224 to save the machine learning model generated in the edit area locally, for example, to the cloud. Alternatively or additionally, a user can select the download button 230 to download a file representing the machine learning model generated in the edit area 210. The download button 230 can be selected before the run button 220 is selected, i.e., before the machine learning model is trained and / or evaluated, or can be selected after the run button 220 is selected, e.g., to download the trained machine learning model. The following references the use of the GUI to generate a machine learning model. Figure 6A and 6B The execution and saving of a machine learning model generated using the GUI for generating a machine learning model is described in more detail.

[0060] The example GUI also includes a view source code button 228. A user can select the view source code button to view the source code corresponding to the machine learning model presented in the edit area 210. For example, in response to selecting the view source code button, the GUI can split the edit area into two portions, with the machine learning model presented in one portion and the corresponding source code presented in the other portion. Alternatively, the source code can be presented in another portion of the GUI.

[0061] Displaying the source code representing the machine learning model enables a user to adjust the machine learning model. For example, a user with some experience in machine learning can be able to identify adjustments that will simplify the machine learning model, e.g., reduce the training time required by the machine learning model and / or reduce the inference time, without compromising the accuracy of the solution. Such adjustments can include adjusting the number of neural network layers used by a particular machine learning subroutine or routine, or specifying a different activation function. As another example, a user with some experience in machine learning can wish to compare multiple machine learning models and their effectiveness in solving a particular task in a particular data set. Thus, manually adjusting the source code behind the machine learning model provides increased personalization and enables a user to generate machine learning models that are increasingly tailored to a particular task.

[0062] In some implementations, hovering or pointing to a portion of the machine learning model can cause the GUI to highlight or otherwise indicate the relevant portion of the displayed source code. This can help the user and improve the accuracy of adjustments made to the source code.

[0063] The example GUI 200 also includes a simple mode button 226. A user can select the simple mode button 226 to change the presentation of the GUI 200. For example, selecting the simple mode button 226 can change the display list 208a-c of machine learning subroutines. Instead of displaying only user-selectable buttons corresponding to machine learning subroutines, user-selectable buttons corresponding to one or more machine learning routines can be displayed. In this context, a machine learning routine is a process that takes input data, processes the data according to a sequence of one or more machine learning subroutines, and outputs an evaluation result. A machine learning routine can represent a machine learning algorithm building block, or an entire machine learning algorithm. In either case, a machine learning routine appears as a black box to the user - the user does not need to construct the machine learning routine itself according to multiple machine learning subroutines. Thus, the user does not need to understand machine learning concepts or how to transform or split data. By selecting the simple mode button 226, the presented GUI can be operated by more users - those with machine learning experience and those without.

[0064] For convenience, the example machine learning routines described in this specification represent entire machine learning algorithms. However, as described above, the machine learning routines represented by the user-selectable buttons displayed in simple mode can represent building blocks of machine learning algorithms. In these cases, a user can have the flexibility to use one machine learning routine in simple mode and another routine in advanced or "normal" mode.

[0065] Each user-selectable button corresponding to a machine learning routine can represent a wrapper file that includes multiple connected machine learning subroutines that define the machine learning routine. For example, the wrapper file representing a machine learning routine can include one or more of (i) a data transformation algorithm, (ii) a machine learning training algorithm, or (iii) a machine learning evaluation algorithm. Wrapping of multiple machine learning subroutines can be performed by a standard wrapper.

[0066] Figure 3 is a schematic diagram of an example wrapper file. For convenience, Figure 3 shows an example machine learning model defined by a user in the edit area 210 of the GUI 200. Figure 2 of the GUI 200.

[0067] In the standard mode 302, the example machine learning model includes a particular data splitting routine (Data Split Module IV) 310, 310a, a training algorithm (Training Algorithm I) 310, 310b, a machine learning algorithm (Machine Learning Algorithm III) 310, 310c, and an evaluation routine 310, 310d. This configuration of subroutines 310, 310a-d together represents the corresponding machine learning routine 308, as indicated by the dashed line.

[0068] The configuration of machine learning subroutines 310 shown in standard mode 302 can be wrapped using subroutine wrapper (wrapper file) 304, as indicated by the dashed line representing machine learning routine 308. This wrapping results in a simple mode "black box" presentation 306. When operating the GUI in standard mode 302, simple mode presentation 306 corresponds to the same machine learning model defined by the user. That is, the system generates the same file / source code representing the machine learning model regardless of whether the user is operating the GUI in standard mode or simple mode.

[0069] Figure 4 is a schematic diagram of an example graphical user interface 400 for generating, executing, and publishing machine learning models using simple mode functionality. The example GUI 400 builds on the example GUI 200 of Figure 2 . Specifically, Figure 4 differences in the GUI presentation after the user selects the simple mode button 226 are shown.

[0070] For example, after the simple mode button 226 is selected, the presented list of user-selectable buttons representing multiple machine learning subroutines in box 206 Figure 2 ) is replaced with a simple mode list. The simple mode list includes at least one user-selectable button 402, such as user-selectable buttons 402a-c, representing a machine learning routine, i.e., a wrapper file containing one or more machine learning subroutines. In some implementations, the simple mode list can include user-selectable buttons corresponding to both machine learning subroutines and machine learning routines. In other implementations, the simple mode list can include only user-selectable buttons corresponding to machine learning routines. As described above with reference to Figure 2 the user can drag and drop a button corresponding to a machine learning routine into the edit area 210 to generate a machine learning model.

[0071] Hardware Programming: Edit Experience

[0072] Figure 5 is a flowchart of an example method 500 for a process for generating a machine learning model using a non-coding machine learning pipeline. For convenience, the process 500 will be described as performed by a system of one or more computers located in one or more locations. For example, a system 100, such as that described above with reference to Figure 1 , appropriately programmed according to the present specification, can perform the process 500.

[0073] The system provides a graphical user interface (GUI) for generating a machine learning model (step 502). For example, the system can provide the GUI 200 described above with reference to Figure 2The GUI 200 is described. In some implementations, the GUI may be a web-based GUI, and providing the GUI may include providing the GUI via a web browser.

[0074] The system receives user selections of mode buttons displayed in the GUI via the provided GUI (step 504). A mode button is a button that, when selected, causes the GUI to display a first set of user-selectable buttons corresponding to the corresponding machine learning routine. When the mode button is not selected, the GUI displays a second set of user-selectable buttons corresponding to the corresponding machine learning subroutine, where the machine learning routine includes multiple connected machine learning subroutines. For example, the mode button could correspond to the above reference... Figures 2-4 The simplified mode button is described.

[0075] In some implementations, a machine learning routine includes multiple machine learning subroutines and corresponds to a corresponding wrapper file. Each wrapper file may include one or more machine learning subroutines, such as (i) a data transformation algorithm, (ii) a machine learning training algorithm, or (iii) a machine learning evaluation algorithm.

[0076] In response to receiving the user's selection of the simplified mode button, the system displays the first set of user-selectable buttons in the GUI (step 506). Reference Figure 4 A sample GUI rendering is shown after the select mode button.

[0077] The system receives user selections from one or more of the first set of user-selectable buttons via a GUI (step 508). The system then displays a graphical representation of a machine learning model in the GUI, defined by one or more machine learning routines corresponding to one or more of the user-selected buttons from the first set (step 510). Figure 4 The example graphical representation is illustrated in the editing area of ​​the example GUI 400. The system generates a file representing the machine learning model (step 512).

[0078] In some implementations, the system may also receive a user's selection of a source code button displayed in the GUI. In response to receiving a user's selection of the source code button, the system may display source code representing a machine learning model in the GUI, defined by the received user selection of one or more user-selectable buttons (e.g., source code included in the file generated in step 510). In some implementations, when the user hovers over the selected button corresponding to the appropriate type of machine learning algorithm, the corresponding portion of the displayed source code is highlighted.

[0079] The system can then receive one or more adjustments to the displayed source code via the GUI, such as adjustments to machine learning model parameters or functions included in the source code. The system can then generate an updated file representing the machine learning model, defined by (i) the user's selections of one or more user-selectable buttons, and (ii) the adjustments to the displayed source code.

[0080] In some implementations, the system may also store the generated file representing the machine learning model at a location associated with the user. In some implementations, the system may receive requests to publish the generated file representing the machine learning model via a GUI. In these implementations, the system may publish the generated file as a new resource within the GUI.

[0081] Note that the optional step of receiving the user's selection of the source code button displayed in the GUI and displaying the source code representing the machine learning model in the GUI can be used as a... Figure 5 The machine learning model is defined by the user's selection of one or more user-selectable buttons received from the user. For example, this optional step could be performed as part of a process that generates a machine learning model using the GUI only in standard mode.

[0082] Hardware Programming: Execution Experience

[0083] Figure 6A This is a flowchart of a first example process 600 for executing a machine learning model that has already been generated using a non-coded machine learning pipeline. For convenience, process 600 will be described as being executed by a system of one or more computers located in one or more locations. For example, a system appropriately programmed according to this specification... Figure 1 System 100 can execute process 600.

[0084] After the system generates a file representing the machine learning model, for example, as shown in the reference above. Figure 5 As described in step 510, the system can also receive a request from the user to download a generated file representing the machine learning model via a GUI for generating the machine learning model (step 602). In some implementations, for example, where the user wants to train the machine learning model offline, the generated file can represent an untrained machine learning model. In other implementations, for example, where the user uploads a training dataset to the system via a GUI or selects a sample dataset provided by the system for training, the user can request the system to train the machine learning model to generate a file representing the trained machine learning model, and the user can further request to download the generated file representing the trained machine learning model, thereby enabling offline inference.

[0085] In some implementations, the received request to download the generated file may include a request to reformat the generated file according to an indicated type (e.g., a third-party streaming format). In these implementations, the system may generate a reformatted file representing a machine learning model of the indicated type and provide access to the reformatted file representing the machine learning model via a GUI.

[0086] The system provides access to the generated files representing the machine learning model via a GUI (step 604). The user can then provide the generated files to other system components or modules, or to third parties for execution.

[0087] Example process 600 decouples the GUI that the user uses to generate the machine learning model from the execution process. This provides the user with enhanced execution transparency, as well as enhanced security, since the user needs not to share their data.

[0088] Note that example procedure 600 can also be used in the system with... Figure 5 The different processes described herein are executed after the system generates a file representing a machine learning model. For example, example process 600 can be executed after the system generates a file representing a machine learning model, which has already been generated by the operation. Figure 2 The example GUI 200 is created by users (i.e., users who operate the GUI in standard mode).

[0089] Figure 6B This is a flowchart of a second example process 650 for executing a machine learning model that has already been generated using a non-coded machine learning pipeline. For convenience, process 650 will be described as being executed by a system of one or more computers located in one or more locations. For example, a system appropriately programmed according to this specification... Figure 1 System 100 can execute process 650.

[0090] After the system generates a file representing the machine learning model, for example, as shown in the reference above. Figure 5 As described in step 510, the system receives the selection of the dataset via the GUI (step 652). For example, the user can select a sample dataset from one of several options displayed in the GUI, or select a dataset imported by the user.

[0091] The system receives a request via the GUI to execute a generated file representing a machine learning model using a dataset (step 654). Receiving the request to execute a generated file representing a machine learning model using a dataset may include receiving a user's selection of a run button. In some implementations, the system may enable the run button displayed in the GUI when a machine learning model input dataset is selected, i.e., after step 652.

[0092] The system executes the generated file representing the generation of the machine learning model using the dataset to generate a corresponding machine learning model output (step 656). The generated machine learning model output can include one or more trained machine learning models or solutions to machine learning tasks produced by trained machine learning models. The system provides the generated machine learning model output for display in the GUI (step 658).

[0093] The embodiments described in this specification and all of the functional operations described in this specification can be implemented in digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Embodiments can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a computer readable medium for execution by, or to control the operation of, data processing apparatus. The computer readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter effecting a machine-readable propagated signal, or a combination of one or more of them. The term “data processing apparatus” encompasses all apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers. The apparatus can include, in addition to hardware, code that creates an execution environment for the computer program in question, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them. The propagated signal is an artificially generated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to suitable receiver apparatus.

[0094] A computer program (also known as a program, software, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub programs, or portions of code). A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and are interconnected by a communication network.

[0095] The processes and logic flows described herein can be executed by one or more programmable processors, which execute one or more computer programs to perform functions by manipulating input data and generating outputs. The processes and logic flows can also be executed by special-purpose logic circuits such as FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits), and the apparatus can also be implemented as special-purpose logic circuits such as FPGAs or ASICs.

[0096] Processors suitable for executing computer programs include, for example, general-purpose microprocessors, special-purpose microprocessors, and any one or more processors of any kind of digital computer. Generally, a processor receives instructions and data from read-only memory or random access memory, or both.

[0097] The essential components of a computer are: a processor for executing instructions, and one or more memory devices for storing instructions and data. Generally, a computer will also include one or more mass storage devices for storing data, or the computer may be operatively coupled to receive data from or transfer data to or both from such mass storage devices, such as magnetic disks, magneto-optical disks, or optical disks. However, a computer does not necessarily have such devices. Furthermore, a computer may be embedded in another device, such as a tablet computer, mobile phone, personal digital assistant (PDA), mobile audio player, GPS receiver, etc. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and storage devices, including, for example, semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices), magnetic disks (e.g., internal hard disks or removable disks), magneto-optical disks, and CD-ROM and DVD-ROM disks. The processor and memory may be supplemented by or incorporated into a dedicated logic circuit system.

[0098] To provide interaction with a user, embodiments can be implemented on a computer having: a display device for displaying information to the user, such as a CRT (cathode ray tube) or LCD (liquid crystal display) monitor; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including sound input, voice input, or tactile input.

[0099] Embodiments can be implemented in a computing system that includes a back end component, e.g., as a data server, or that includes a middleware component, e.g., an application server, or that includes a front end component, e.g., a client computer having a graphical user interface or a Web browser through which a user can interact with an implementation, or any combination of one or more such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication network. Examples of communication networks include a local area network ("LAN") and a wide area network ("WAN"), e.g., the Internet.

[0100] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

[0101] While this specification contains many specifics, these should not be construed as limitations on the scope of the disclosure or of what can be claimed, but rather as descriptions of features specific to particular embodiments. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented separately or in any suitable subcombination. Moreover, although features can be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination and the claimed combination can be directed to a subcombination or variation of a subcombination.

[0102] Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring such order, nor that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing can be advantageous. Moreover, the separation of various system components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated in a single software product or packaged into multiple software products.

[0103] In each instance where a file is mentioned, different file types or formats can be expected, e.g., YAML, HTML, XML, JSON, plain text, or other types of files. Moreover, where tables are mentioned, different data structures can be expected (e.g., spreadsheets, relational databases, or structured files).

[0104] Thus, particular embodiments have been described. Other embodiments are within the scope of the following claims. For example, acts recited in the claims can occur in a different order and still achieve the desired results.

Claims

1. A computer-implemented method (500) for implementing a non-coding machine learning pipeline, characterized in that, comprising: providing a graphical user interface (GUI) (200, 400) for generating a machine learning model (414), wherein the GUI is arranged to display, in accordance with a user selection, a first set of user-selectable buttons (402) corresponding to a respective machine learning routine (308) or a second set of user-selectable buttons (208) corresponding to a respective machine learning sub-routine (310), wherein each user-selectable button (402) of the first set of user-selectable buttons (402) represents a wrapper file (304) corresponding to a machine learning routine (306) and generated by a sub-routine wrapper wrapping a respective plurality of connected machine learning sub-routines (310), wherein the machine learning routine (308) comprises the respective plurality of connected machine learning sub-routines (310); displaying, by the GUI (200, 400), the second set of user-selectable buttons (208) corresponding to the respective machine learning sub-routine (310); receiving, by the GUI (200, 400), a user selection of a mode button (226) displayed in the GUI (200, 400), wherein the mode button (226) when selected causes the GUI (200, 400) to display the first set of user-selectable buttons (402) corresponding to the respective machine learning routine (308) and when not selected causes the GUI (200, 400) to display the second set of user-selectable buttons (208) corresponding to the respective machine learning sub-routine (310); in response to receiving the user selection of the mode button (226), displaying the first set of user-selectable buttons (402) in the GUI (200, 400) in place of the second set of user-selectable buttons (208); receiving, by the GUI (200, 400), a user selection of one or more of the first set of user-selectable buttons (402) by a user; displaying, in the GUI (200, 400), a graphical representation of a machine learning model (414) defined by one or more machine learning routines (308) corresponding to the user selection of one or more of the first set of user-selectable buttons (402); and generating a file representing the machine learning model (414).

2. The method (500) of claim 1, wherein, The machine learning subroutines (310) include a data formatting algorithm, a data splitting algorithm, a feature selection algorithm, a machine learning training algorithm, a machine learning evaluation algorithm, or a statistical algorithm and / or the GUI (200, 400) provided for generating a machine learning model (414) includes a plurality of user selectable tabs (218a-c) including a machine learning model creation tab (218a), a machine learning model publishing tab (218b), and a machine learning model versioning tab (218d).

3. The method (500) according to claim 1 or 2, characterized by Also included are: receiving, through the GUI (200, 400), a user selection of a source code button (228) displayed in the GUI (200, 400); in response to receiving the user selection of the source code button (228), displaying, in the GUI (200, 400), source code representing the machine learning model (414), wherein the generated file includes the source code; receiving, through the GUI (200, 400), one or more user adjustments to the displayed source code; and generating an updated file representing an updated machine learning model (414) defined by (i) the one or more machine learning routines (308) corresponding to one or more of the user selected first set of user selectable buttons (402), and (ii) the adjustments to the displayed source code.

4. The method (500) according to claim 3, characterized by The adjustments to the displayed source code include adjustments to machine learning model parameters or functions included in the source code.

5. The method (500) according to any one of claims 1-4, characterized by, Also included are: receiving, through the GUI (200, 400), a request from a user to download the generated file representing the machine learning model (414); and providing, through the GUI (200, 400), access to the generated file representing the machine learning model (414).

6. The method (500) according to claim 5, characterized by The request to download (230) the generated file representing the machine learning model (414) includes a request to reformat the generated file according to an indicated type, and wherein the method further includes: generating a reformatted file representing the machine learning model (414) of the indicated type; and providing, through the GUI (200, 400), access to the reformatted file representing the machine learning model (414).

7. The method (500) according to any of claims 1-6, characterized by, Also included is storing the generated file representing the machine learning model (414) at a location associated with the user (108).

8. The method (500) according to any one of claims 1-7, characterized by, Also included are: receiving, through the GUI (200, 400), a selection of a data set; receiving, through the GUI (200, 400), a request to execute the generated file representing the machine learning model (414) using the data set; executing the generated file representing the machine learning model (414) using the data set to generate a corresponding machine learning model output; and providing the generated machine learning model (414) output for display in the GUI (200, 400).

9. The method (500) of claim 8, characterized by Receiving a selection of a dataset includes receiving a selection of a sample dataset or receiving a selection of a dataset imported by the user.

10. The method (500) according to claim 8 or 9, characterized by, Receiving, through the GUI (200, 400), a request to execute the generated file representing the machine learning model (414) using the dataset includes receiving, through the GUI (200, 400), a user selection of a run button (220), wherein the run button (220) is enabled for selection when a machine learning model input dataset is selected.

11. The method (500) according to any one of claims 1-10, characterized by, Also included are: Receiving a request to publish the generated file representing the machine learning model (414); And In response to receiving the request, publishing the generated file as a new asset in the GUI (200, 400), wherein the publishing includes making the new asset available for user access in the GUI (200, 400).

12. The method (500) according to any one of claims 1-11, characterized by, The provided GUI (200, 400) for generating a machine learning model (414) includes a drag-and-drop interface, wherein the GUI (200, 400) includes: A searchable list of the first set of user-selectable buttons or the second set of user-selectable buttons (402, 208); and An edit area (210); Wherein a user can select a user-selectable button from the first set of user-selectable buttons (402-c) or the second set of user-selectable buttons (208) and drag-and-drop it into the edit area (210) to define a machine learning model (414).

13. The method (500) of claim 12, wherein, The frequently used machine learning subroutines (310) or types of machine learning routines (308) are listed higher than the less frequently used machine learning subroutines (310) or types of machine learning routines (308); or, The frequently used machine learning subroutines (310) or types of machine learning routines (308) are listed higher than the less frequently used machine learning subroutines (310) or types of machine learning routines (308), wherein the edit area (210) includes a template graph arranged to assist a user in selecting the user-selectable buttons (402, 208) and dragging-and-dropping them into the edit area (210) to define a machine learning model (414). One or more computers and one or more storage devices storing instructions operable to cause the one or more computers to perform operations of the method of any of claims 1-13 when executed by the one or more computers.

14. A system (100) implementing a non-coding machine learning pipeline, characterized in that, The instructions, when executed by one or more computers, cause the one or more computers to perform operations of the method of any of claims 1-13.

15. A computer storage medium encoded with instructions, the computer storage medium comprising: ​

Images