Distributed computing system with integrated data as a service feedback loop engine

Abstract

Various embodiments, methods, and systems for implementing a distributed computing system feedback loop engine are provided. Initially, a training data set report is accessed. The training data set report identifies a synthetic data asset having a value for a data change parameter. The synthetic data asset is associated with a framework set. Based on the training data set report, the synthetic data asset having a synthetic data asset change is updated. The framework set is updated using the updated synthetic data asset.

Description

Background Art

[0001] Users rely on different types of technical systems to complete tasks. Technical systems can be improved based on machine learning, which uses statistical techniques to enable computers to use data to gradually improve the performance of specific tasks without being explicitly programmed. For example, machine learning can be used for data security, personal safety, fraud detection, healthcare, natural language processing, online search and recommendation, financial transactions, and smart cars. For each of these fields or industries, the machine learning model is trained using a training data set, which is an example data set used to create a framework for matching learning tasks and machine learning applications. For example, a facial recognition system can be trained to compare the unique features of a person's face with a known set of features of the face to correctly identify a person. With the growing use of machine learning in different fields and the importance of correctly training machine learning models, improvements to the computational operations of machine learning training systems will provide more efficient performance of machine learning tasks and applications, and will improve user navigation of the graphical user interface of the machine learning training system. Summary of the invention

[0002] Embodiments of the present invention relate to methods, systems, and computer storage media for providing a distributed computing system that supports integrated data as a service. As background, a distributed computing system can operate based on a service-oriented architecture, where services are provided using different service models. At a high level, a service model can provide an abstraction of the underlying operations associated with providing a corresponding service. Examples of service models include infrastructure as a service, platform as a service, software as a service, and function as a service. Using any of these models, customers can develop, operate, and manage various aspects of a service without having to maintain or develop operational features that are abstracted using a service-oriented architecture.

[0003] Turning to machine learning and training datasets, machine learning uses statistical techniques to enable computers to leverage data to gradually improve performance on specific tasks without being explicitly programmed. Training datasets are an integral part of the field of machine learning. High-quality datasets can help improve machine learning algorithms and the computational operations associated with machine learning hardware and software. Creating high-quality training datasets can require a significant amount of work. For example, labeling the data used for training datasets can be particularly tedious, which often leads to inaccuracies in the labeling process.

[0004] Conventional methods for finding training datasets are clearly insufficient when it comes to making training datasets democratized or generally available for use across several different domains. Moreover, theoretical solutions for developing machine learning training datasets have not even been fully defined or described because the infrastructure for implementing such solutions is either unavailable or too expensive to pursue alternatives to current techniques for developing training datasets. Overall, comprehensive functionality around developing machine learning training datasets is limited in conventional machine learning training services.

[0005] Embodiments described in the present disclosure relate to techniques for improving access to machine learning training data sets using a distributed computing system that provides comprehensive data as a service ("SDaaS"). SDaaS may refer to a distributed (cloud) computing system service that is implemented using a service-oriented architecture to provide machine learning training services while abstracting the underlying operations that are managed via the SDaaS service. For example, SDaaS provides a machine learning training system that allows customers to configure, generate, access, manage, and process comprehensive data training data sets for machine learning. In particular, SDaaS operates without the complexity typically associated with manual development of training data sets. SDaaS may be delivered in a variety of ways based on an SDaaS engine, manager, module, or component, including an asset assembly engine, a scene assembly engine, a frameset assembly engine, a frameset package generator, a frameset package repository, a feedback loop engine, and a crowdsourcing engine. The observable effect of implementing SDaaS as a service on a distributed computing system is the massive generation and availability of synthetic data assets that support the generation of training data sets based on intrinsic-parameter variation and extrinsic-parameter variation, where the intrinsic-parameter variation and extrinsic-parameter variation provide programmable machine learning data representations of assets and scenarios. Using the components of SDaaS provides additional specific functionality, as described in more detail below.

[0006] Therefore, an example embodiment of the present invention provides a distributed computing system asset assembly engine. The asset assembly engine is configured to receive a first source asset from a first distributed integrated data as a service (SDaaS) upload interface. The engine is also configured to receive a second source asset from a second distributed SDaaS upload interface. The engine is also configured to ingest the first source asset and the second source asset. Ingesting the source asset includes automatically calculating the value of the asset change parameter for the source asset. The asset change parameter is programmable for machine learning. The engine is also configured to generate a first integrated data asset, the first integrated data asset including a first set of values ​​for the asset change parameter. The engine is also configured to generate a second integrated data asset, the second integrated data asset including a second set of values ​​for the asset change parameter. The engine is also configured to store the first integrated data asset and the second integrated data asset in a comprehensive data asset repository.

[0007] Therefore, an example embodiment of the present invention provides a distributed computing system scenario assembly engine. The scenario assembly engine is configured to receive a selection of a first integrated data asset and a selection of a second integrated data asset from a distributed integrated data as a service (SDaaS) integrated development environment (IDE). The integrated data asset is associated with an asset change parameter and a scenario change parameter. The asset change parameter and the scenario change parameter are programmable for machine learning. The engine is also configured to receive a value for generating a comprehensive data scenario. The value corresponds to an asset change parameter or a scenario change parameter. The engine is also configured to generate a comprehensive data scenario using the first integrated data asset and the second integrated data asset based on the value.

[0008] Therefore, an example embodiment of the present invention provides a distributed computing system frameset assembly engine. The frameset assembly engine is configured to access a comprehensive data scene. The engine is also configured to determine a first set of values ​​for a scene change parameter. The first set of values ​​is automatically determined for generating a comprehensive data scene frameset. The engine is also configured to generate a comprehensive data scene frameset based on the first set of values. The comprehensive data scene frameset includes at least a first frame in the frameset, the first frame including a comprehensive data scene updated based on the value for the scene change parameter. The engine is also configured to store the comprehensive data scene frameset.

[0009] Therefore, an example embodiment of the present invention provides a distributed computing system frameset package generator. The frameset package generator is configured to access a frameset package generator profile. The frameset package generator profile is associated with a first image generating device. The frameset package generator profile includes known device variability parameters associated with the first image generating device. The engine is further configured to generate a frameset package based on the frameset package generator profile. The frameset package generator profile includes values ​​for the known device variability parameters. The engine is further configured to store the frameset package.

[0010] Therefore, an exemplary embodiment of the present invention provides a distributed computing system frameset package repository. The frameset package repository is configured to receive a query for a frameset package from a frameset package query interface. The frameset query interface includes a plurality of frameset package categories. The engine is further configured to identify a query result frameset package based on a frameset package profile. The engine is further configured to transmit the query result frameset package.

[0011] Therefore, an example embodiment of the present invention provides a distributed computing system feedback loop engine. The feedback loop engine is configured to access a training data set report. The training data set report identifies a synthetic data asset having a value for an asset change parameter. The synthetic data asset is associated with a framework set. The engine is further configured to update the synthetic data asset with the synthetic data asset change based on the training data set report. The engine is further configured to update the framework set using the updated synthetic data asset.

[0012] Therefore, an example embodiment of the present invention provides a distributed computing system crowdsourcing engine. The crowdsourcing engine is configured to receive source assets from a distributed synthetic data as a service (SDaaS) crowdsourcing interface. The engine is also configured to receive crowdsourcing tags for source assets via the distributed SDaaS crowdsourcing interface. The engine is also configured to ingest source assets based in part on the crowdsourcing tags. Ingesting source assets includes automatically calculating values ​​for asset change parameters for the source. The asset change parameters are programmable for machine learning. The engine is also configured to generate crowdsourced synthetic data assets, which include values ​​for asset change parameters.

[0013] Thus, the embodiments described herein improve computing functions and operations for generating training data sets based on implementations of providing integrated data as a service using a distributed computing system. For example, computing operations required for manual development (e.g., labeling and tagging) and refinement (e.g., searching) of training data sets are eliminated based on SDaaS operations that automatically develop training data sets using integrated data assets and automatically refine training data sets based on training data set reports that indicate additional integrated data assets or scenarios that will improve machine learning models in machine learning training services. In this regard, SDaaS solves specific problems of manually developing machine learning training data sets and improves existing processes for training machine learning models in distributed computing systems.

[0014] This Summary is provided to introduce some concepts in a simplified form that are further described in the Detailed Description below. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The present technology is described in detail below with reference to the accompanying drawings, in which:

[0016] Figure 1A and Figure 1B is a block diagram of an example distributed computing for providing integrated data as a service according to an embodiment of the present invention;

[0017] Figure 2A and Figure 2B is a flow chart illustrating an example implementation of a distributed computing system integrated data as a service according to an embodiment of the present invention;

[0018] Figure 3 is a schematic diagram illustrating an example distributed computing system integrated data-as-a-service interface according to an embodiment of the present invention;

[0019] Figure 4 is a schematic diagram illustrating an example distributed computing system integrated data-as-a-service workflow according to an embodiment of the present invention;

[0020] Figure 5 is a schematic diagram illustrating an example distributed computing system integrated data-as-a-service interface according to an embodiment of the present invention;

[0021] Figure 6 is a flow chart illustrating an example distributed computing system integrated data-as-a-service operation according to an embodiment of the present invention;

[0022] Figure 7 is a flow chart illustrating an example distributed computing system integrated data-as-a-service operation according to an embodiment of the present invention;

[0023] Figure 8 is a flow chart illustrating an example distributed computing system integrated data-as-a-service operation according to an embodiment of the present invention;

[0024] Fig. 9 is a flow chart illustrating an example distributed computing system integrated data-as-a-service operation according to an embodiment of the present invention;

[0025] Fig.10 is a flow chart illustrating an example distributed computing system integrated data-as-a-service operation according to an embodiment of the present invention;

[0026] Fig.11 is a flow chart illustrating an example distributed computing system integrated data-as-a-service operation according to an embodiment of the present invention;

[0027] Fig.12 is a flow chart illustrating an example distributed computing system integrated data-as-a-service operation according to an embodiment of the present invention;

[0028] Fig.13is a block diagram of an example distributed computing environment suitable for implementing embodiments of the present invention; and

[0029] Fig.14 is a block diagram of an example computing environment suitable for implementing embodiments of the present invention. DETAILED DESCRIPTION

[0030] Distributed computing systems can be used to provide different types of service-oriented models. As background, a service model can provide an abstraction of the underlying operations associated with providing a corresponding service. Examples of service models include infrastructure as a service, platform as a service, software as a service, and function as a service. With any of these models, customers can develop, operate, and manage various aspects of a service without having to maintain or develop the operational features that are abstracted using a service-oriented architecture.

[0031] Machine learning uses statistical techniques to enable computers to use data to gradually improve the performance of specific tasks without being explicitly programmed. For example, machine learning can be used for data security, personal safety, fraud detection, healthcare, natural language processing, online search and recommendation, financial transactions, and smart cars. For each of these fields or industries, the machine learning model is trained using a training data set, which is an example data set used to create a framework for matching learning tasks and machine learning applications. Training data sets are an integral part of the field of machine learning. High-quality data sets can help improve machine learning algorithms and computing operations associated with machine learning hardware and software. Machine learning platforms operate based on training data sets that support supervised and semi-supervised machine learning algorithms; however, high-quality training data sets are often difficult to produce and expensive because it takes a lot of time to label data. Machine learning models rely on high-quality labeled training data sets for supervised learning so that the model can provide reliable results in predicting, classifying, and analyzing different types of phenomena. Without the right type of training data set, it may be impossible to develop a reliable machine learning model. Training data sets include labeled, tagged, and annotated entries to effectively train machine learning algorithms.

[0032] Conventional approaches to finding training datasets fall significantly short when it comes to making them democratized or generally available for use across several different domains. Currently, such limited solutions include outsourcing labeling functionality, repurposing existing training data and labels, collecting your own training data and labels from free sources, relying on third-party models that have been pre-trained on labeled data, and leveraging crowdsourced labeling services. Most of these solutions are either time-consuming, expensive, unsuitable for sensitive projects, or clearly not robust enough to handle large-scale machine learning projects. Moreover, theoretical solutions for developing machine learning training datasets have not even been fully defined or described because the infrastructure for implementing such solutions is not available or is too expensive to implement alternatives to current techniques for developing training datasets. Overall, comprehensive functionality around developing machine learning training datasets is limited in conventional machine learning training services.

[0033] The embodiments described herein provide simple and effective methods and systems for implementing a distributed computing system that provides comprehensive data as a service ("SDaaS"). SDaaS may refer to a distributed (cloud) computing system service that is implemented using a service-oriented architecture to provide machine learning training services while abstracting the underlying operations that are managed via the SDaaS service. For example, SDaaS provides a machine learning training system that allows customers to configure, generate, access, manage, and process comprehensive data training data sets for machine learning. In particular, SDaaS operates without the complexity typically associated with manual development of training data sets. SDaaS can be delivered in a variety of ways based on an SDaaS engine, manager, module, or component, including an asset assembly engine, a scene assembly engine, a frameset assembly engine, a frameset package generator, a frameset package repository, a feedback loop engine, and a crowdsourcing engine. The observable effect of implementing SDaaS on a distributed computing system is the massive generation and availability of synthetic data assets that support the generation of training data sets based on intrinsic and extrinsic parameter changes that provide programmable machine learning data representations of assets and scenarios. Using the components of SDaaS provides additional specific functionality, as described in more detail below.

[0034] It is contemplated herein that a source asset may include several different parameters that may be computationally determined based on known techniques in the art. By way of example, a source asset may refer to a three-dimensional representation of geometric data. A source asset may be represented as a mesh consisting of triangles, where the smoother the triangles and the more details on the surface of the model, the larger the size of the source. In this regard, a source asset may be represented across a range from a high polygon model with a large amount of detail to a low polygon model with less detail. The process of representing a source asset as varying levels of detail may be referred to as decimation. Low polygon models may be used in different types of processes, but these processes may be computationally expensive for high models. Thus, an automatic decimation process may be implemented to store source assets at different levels of detail. Other types of programmable parameters may be determined and associated with source assets stored as composite assets.

[0035] Embodiments of the present invention operate on a two-tier programmable parameter system, where a machine learning training service can train a model automatically or based on manual intervention based on accessing and determining first-tier parameters (e.g., asset parameters) and / or second-tier parameters (e.g., scenario or framework set parameters) required to improve training data sets and expand model training. The machine learning training service can support deep learning and deep learning networks, and can support other types of machine learning algorithms and networks. The machine learning training service can also implement generative adversarial networks as a type of unsupervised machine learning. SDaaS can utilize these basic hierarchical parameters in different ways. For example, how much to charge for a framework set; how to develop different types of framework sets for specific devices when knowing the device parameters and being able to manipulate these parameters when developing training data sets.

[0036] Example operating environment and schematic instructions

[0037] refer to Figure 1A and Figure 1B , the components of the distributed computing system 100 can operate together to provide the functional form of SDaaS described herein. The distributed computing system 100 supports processing of comprehensive data assets to generate and process training data sets for machine learning. At a high level, distributed computing supports a distributed framework for generating training data sets in large quantities. In particular, the distributed computing architecture built on features including file compression, large-scale GPU-enabled hardware, unstructured storage devices, and distributed backbone networks inherently supports the ability to provide SDaaS functionality in a distributed manner so that multiple users (e.g., artists or data managers) can access operations on comprehensive data assets simultaneously.

[0038] Figure 1AThe distributed computing system also includes several components that support SDaaS functionality, including an asset assembly engine 110, a scene assembly engine 112, a frameset assembly engine 114, a frameset package generator 116, a frameset package repository 118, a feedback loop engine 120, a crowdsourcing engine 122, a machine learning training service 124, and an SDaaS repository 126. Figure 1B Illustrated are assets 126A and framesets 126B stored in an SDaaS repository 126 and integrated with a machine learning training service for automated access to assets, scenes, and framesets, as described in more detail below.

[0039] The asset assembly engine 110 may be configured to receive a first source asset from a first distributed synthetic data as a service (SDaaS) upload interface and may receive a second source asset from a second distributed SDaaS upload interface. The first source asset and the second source asset may be ingested, wherein ingesting the source assets includes automatically calculating values ​​of asset change parameters for the source assets. For example, Figure 2A The method includes ingesting source assets 210 into an asset repository (i.e., assets 220). The asset change parameters are programmable for machine learning. The asset assembly engine may generate a first synthetic data asset, the first synthetic data asset including a first set of values ​​for the asset change parameters, and may generate a second synthetic data asset, the second synthetic data asset including a second set of values ​​for the asset change parameters. The first synthetic data asset and the second synthetic data asset are stored in a synthetic data asset repository.

[0040] The distributed SDaaS upload interface (e.g., interface 128A or 128B) is associated with the SDaaS integrated development environment (IDE). The SDaaS IDE supports an additional value that identifies an asset change parameter for a source asset. This value is associated with generating a training data set based on internal parameter changes and external parameter changes, where the internal parameter changes and external parameter changes provide programmable machine learning data representations of assets and scenarios. Ingesting source assets is based on machine learning comprehensive data standards, which include file formats and data set training architectures. File formats can refer to hard standards, while data set training architectures can refer to soft standards, such as automatic or manual human intervention.

[0041] refer to Figure 2A and Figure 2B Ingesting a source asset (eg, source asset 202) further includes automatically calculating a value of a scene change parameter for the source asset, wherein the scene change parameter is programmable for machine learning. A comprehensive data asset profile may be generated, wherein the comprehensive data asset profile includes the value of the asset change parameter. Figure 2A and Figure 2B Additional artifacts are also illustrated, such as bounding box 208 , thumbnail 210 , 3D visualization 212 , and optimized assets 214 .

[0042] The scenario assembly engine 112 may be configured to receive a selection of a first synthetic data asset and a selection of a second synthetic data asset from a distributed synthetic data as a service (SDaaS) integrated development environment (IDE). Figure 4 , the assets and parameters 410 at the first layer can be used to generate scenarios and parameters 420 at the second layer, and can also be used to define a framework set 430. The synthetic data asset is associated with the asset change parameter and the scenario change parameter. The asset change parameter and the scenario change parameter are programmable for machine learning. The scenario assembly engine can receive a value for generating a synthetic data scenario, wherein the value corresponds to the asset change parameter or the scenario change parameter. Based on the value, the synthetic data scenario is generated using the first synthetic data asset and the second synthetic data asset.

[0043] A scenario assembly engine client (e.g., client device 130B) may be configured to receive a query for a synthetic data asset, wherein the query is received via the SDaaS IDE, and generate a query result synthetic data asset; and cause display of a synthetic data scenario generated based on the query result synthetic data. The synthetic data scenario may be generated based on values ​​for scenario generation received from at least two scenario assembly engine clients. The synthetic data scenario is associated with a scenario preview and metadata.

[0044] The frameset assembly engine 114 may be configured to access the synthetic data scenario and determine a first set of values ​​for the scene change parameter, wherein the first set of values ​​is automatically determined for generating the synthetic data scenario frameset. The frameset assembly engine may also generate the synthetic data scenario frameset based on the first set of values, wherein the synthetic data scenario frameset includes at least a first frame in the frameset, the first frame including the synthetic data scenario updated based on the value for the scene change parameter; and store the synthetic data scenario frameset. A second set of values ​​for the scene change parameter is manually selected for generating the synthetic data scenario frameset. The second set of values ​​is manually selected using a synthetic data as a service (SDaaS) integrated development environment (IDE), which supports machine learning synthetic data standards including file formats and data set training architectures. Generating the synthetic data scenario frameset includes iteratively generating frames for the synthetic data scenario frameset based on updating the synthetic data scenario according to the first set of values.

[0045] The frameset package generator 116 may be configured to access a frameset package generator profile, wherein the frameset package generator profile is associated with a first image generating device, wherein the frameset package generator profile includes a known device variability parameter associated with the first image generating device. The frameset package is based on the frameset package generator profile, wherein the frameset package generator profile includes values ​​for the known device variability parameter; and storing the frameset package. The frameset package includes categories based on at least two synthetic data scenarios. Generating the frameset package is based on an expected machine learning algorithm to be trained with the frameset package, wherein the expected machine learning algorithm is identified in the frameset package generator profile. The frameset package includes assigning a value quantizer to the frameset package. The frameset package is generated based on a synthetic data scenario including synthetic data assets.

[0046] The frameset bundle storage 118 may be configured to: receive a query for a frameset bundle from a frameset bundle query interface, wherein the frameset bundle query interface includes a plurality of frameset bundle categories; identify a query result frameset bundle based on a frameset bundle profile; and transmit the query result frameset bundle. At least a portion of the query triggers an automatically suggested frameset bundle, wherein the automatically suggested frameset bundle is associated with a synthetic data scene of the frameset, the synthetic data scene having synthetic data assets. The frameset bundle is associated with an image generation device, wherein the image generation device includes known device variability parameters, the known device variability parameters being programmable for machine learning. The query result frameset bundle is transmitted to an internal machine learning model training service (e.g., the machine learning training service 124) or an external machine learning model training service operating on a distributed computing system.

[0047] The feedback loop engine 120 can be configured to access a training data set report, wherein the training data set report identifies a synthetic data asset having a value for an asset change parameter, wherein the synthetic data asset is associated with a framework set. Based on the training data set report, the synthetic data asset is updated with the synthetic data asset change; and the framework set is updated using the updated synthetic data asset. The value is manually or automatically identified in the training data set report for updating the framework set. A value quantifier (e.g., a cost of making an update) is assigned to the update framework set. The training data set report is associated with an internal machine learning model training service or an external machine learning model training service operating on a distributed system.

[0048] The crowdsourcing engine 122 can be configured to: receive source assets from a distributed synthetic data as a service (SDaaS) crowdsourcing interface; receive crowdsourcing tags for source assets via the distributed SDaaS crowdsourcing interface; ingest the source assets based in part on the crowdsourcing tags, wherein ingesting the source assets includes automatically calculating values ​​for asset change parameters for the source assets, wherein the asset change parameters are programmable for machine learning; and generate a crowdsourced synthetic data asset, the crowdsourced synthetic data asset including values ​​for the asset change parameters. A value quantizer is used for the crowdsourced synthetic data asset. A crowdsourced synthetic data asset profile includes the asset change parameters. Reference Figure 5 , the crowdsourcing interface 500 can support uploading and tagging source assets for ingestion.

[0049] Example Flowchart

[0050] refer to Figure 6-Figure 12 , a flow chart is provided that illustrates a method for implementing a distributed computing system integrated data as a service. The method can be performed using the distributed computing system described herein. In various embodiments, one or more computer storage media have computer executable instructions embodied thereon, which, when executed by one or more processors, can cause the one or more processors to perform the method in the distributed computing system 100.

[0051] Figure 6 6 is a flowchart illustrating a process 600 for implementing a distributed computing system asset assembly engine according to an embodiment. Initially, at box 610, a first source asset is received from a first distributed integrated data as a service (SDaaS) upload interface. At box 620, a second source asset is received from a second distributed SDaaS upload interface. At box 630, the first source asset and the second source asset are ingested. Ingesting the source assets includes automatically calculating values ​​for asset change parameters for the source assets, wherein the asset change parameters are programmable for machine learning. At box 640, a first integrated data asset is generated, the first integrated data asset including a first set of values ​​for the asset change parameters. At box 650, a second integrated data asset is generated, the second integrated data asset including a second set of values ​​for the asset change parameters. At box 660, the first integrated data asset and the second integrated data asset are stored in an integrated data asset repository.

[0052] Figure 77 is a flow chart illustrating a process 700 for implementing a distributed computing system scenario assembly engine according to an embodiment. Initially, at block 710, a selection of a first synthetic data asset and a selection of a second synthetic data asset are received from a distributed synthetic data as a service (SDaaS) integrated development environment (IDE). The synthetic data assets are associated with asset change parameters and scenario change parameters, and the asset change parameters and scenario change parameters are programmable for machine learning. At block 720, a value for generating a synthetic data scenario is received. The value corresponds to an asset change parameter or a scenario change parameter. At block 730, based on the value, a synthetic data scenario is generated using the first synthetic data asset and the second synthetic data asset.

[0053] Figure 8 8 is a flow chart illustrating a process 800 for implementing a distributed computing system frameset assembly engine according to an embodiment. Initially, at block 810, a synthetic data scene is accessed. At block 820, a first set of values ​​for a scene change parameter is determined. The first set of values ​​is automatically determined for generating a synthetic data scene frameset. At block 830, a synthetic data scene frameset is generated based on the first set of values. The synthetic data scene frameset includes at least a first frame in the frameset, the first frame including a synthetic data scene updated based on the value for the scene change parameter. At block 840, the synthetic data scene frameset is stored.

[0054] Fig. 9 9 is a flow chart illustrating a process 900 for implementing a distributed computing frameset package generator according to an embodiment. At block 910, a frameset package generator profile is accessed. The frameset package generator profile is associated with a first image generating device. The frameset package generator profile includes known device variability parameters associated with the first image generating device. At block 920, a frameset package is generated based on the frameset package generator profile. The frameset package generator profile includes values ​​for the known device variability parameters. At block 930, the frameset package is stored.

[0055] Fig.10 1 is a flow chart illustrating a process 1000 for implementing a distributed computing system frameset package repository according to an embodiment. At block 1010, a query for a frameset package is received from a frameset package query interface. The frameset query interface includes a plurality of frameset package categories. At block 1020, a query result frameset package is identified based on a frameset package profile. At block 1030, the query result frameset package is transmitted.

[0056] Fig.111 is a flow chart illustrating a process 1100 for implementing a distributed computing system feedback loop engine according to an embodiment. At block 1110, a training data set report is accessed. The training data set report identifies synthetic data assets having values ​​for asset change parameters. The synthetic data assets are associated with a framework set. At block 1120, the synthetic data assets are updated with synthetic data asset changes based on the training data set report. At block 1130, the framework set is updated using the updated synthetic data assets.

[0057] Fig.12 1 is a flow diagram illustrating a process 1200 for implementing a distributed computing system crowdsourcing engine according to an embodiment. At block 1210, a source asset is received from a distributed synthetic data as a service (SDaaS) crowdsourcing interface. At block 1220, crowdsourcing tags for the source asset are received via the distributed SDaaS crowdsourcing interface. At block 1230, based in part on the crowdsourcing tags, the source asset is ingested. Ingesting the source asset includes automatically calculating a value for an asset change parameter for the source asset. The asset change parameter can be programmable for machine learning. At block 1240, a crowdsourced synthetic data asset including a value for the asset change parameter is generated.

[0058] Advantageously, the embodiments described herein improve computing functions and operations for generating training data sets based on an implementation of providing integrated data as a service using a distributed computing system. In particular, the improvements to computing functions and operations are associated with a distributed infrastructure for mass production of training data sets based on SDaaS operations. For example, computing operations required to manually develop (e.g., label and tag) and refine (e.g., search) training data sets are eliminated based on SDaaS operations that automatically develop training data sets using integrated data assets and automatically refine training data sets based on training data set reports that indicate additional integrated data assets or scenarios that will improve machine learning models in machine learning training services.

[0059] In addition, an internal machine learning training service operating in the same distributed computing system is used to improve the storage and retrieval of training data sets, thereby reducing computational overhead. SDaaS operations are implemented based on an unconventional arrangement of engines and a set of defined unconventional rules for an ordered combination of steps for the SDaaS system. In this regard, SDaaS solves specific problems of manually developing machine learning training data sets and improves existing processes for training machine learning models in distributed computing systems. Overall, these improvements also result in less CPU computation, smaller memory requirements, and increased flexibility in generating and utilizing machine learning training data sets.

[0060] Sample distributed computing environment

[0061] Reference now Fig.13 , Fig.13 An example distributed computing environment 1300 is illustrated in which implementations of the present disclosure may be employed. In particular, Fig.13 The high-level architecture of the distributed computing system integrated data as a service in the cloud computing platform 1310 is shown, wherein the system supports seamless modification of software components. It should be understood that this arrangement and other arrangements described herein are only set forth by example. For example, as mentioned above, many elements in the elements described herein can be implemented as discrete components or distributed components or combined with other components, and are implemented in any suitable combination and position. In addition to or in place of those arrangements and elements shown, other arrangements and elements (e.g., groups of machines, interfaces, functions, commands and functions, etc.) can also be used.

[0062] The data center can support a distributed computing environment 1300, which includes a cloud computing platform 1310, a rack 1320, and nodes 1330 (e.g., computing devices, processing units, or blades) in the rack 1320. The system can be implemented using a cloud computing platform 1310 that runs cloud services across different data centers and geographic regions. The cloud computing platform 1310 can implement a fabric controller 1340 component for provisioning and managing resource allocation, deployment, upgrades, and management of cloud services. Typically, the cloud computing platform 1310 is used to store data or run service applications in a distributed manner. The cloud computing infrastructure 1310 in the data center can be configured to host and support the operation of endpoints for specific service applications. The cloud computing infrastructure 1310 can be a public cloud, a private cloud, or a dedicated cloud.

[0063] Node 1330 may be provided with a host 1350 (e.g., an operating system or runtime environment) that runs a defined software stack on node 1330. Node 1330 may also be configured to perform specialized functionality (e.g., a computing node or a storage node) within cloud computing platform 1310. Node 1330 is allocated to run one or more parts of a tenant's service application. A tenant may refer to a customer that utilizes the resources of cloud computing platform 1310. The service application components of cloud computing platform 1310 that support a specific tenant may be referred to as tenant infrastructure or leasing. The terms service application, application, or service are used interchangeably herein and generally refer to any software or software portion that runs on top of a data center, or an access storage device and computing device location within a data center.

[0064] When node 1330 supports more than one separate service application, node 1330 can be divided into virtual machines (e.g., virtual machine 1352 and virtual machine 1354). Physical machines can also run separate service applications concurrently. Virtual machines or physical machines can be configured as individualized computing environments supported by resources 1360 (e.g., hardware resources and software resources) in cloud computing platform 1310. It should be expected that resources can be configured for specific service applications. In addition, each service application can be divided into functional parts so that each functional part can run on a separate virtual machine. In cloud computing platform 1310, multiple servers can be used to run service applications and perform data storage operations in a cluster. In particular, servers can perform data operations independently, but are disclosed as a single device called a cluster. Each server in a cluster can be implemented as a node.

[0065] The client device 1380 may be linked to a service application in the cloud computing platform 1310. The client device 1380 may be any type of computing device, for example, it may correspond to a reference Fig.13 13. The computing device 1300 described herein. The client device 1380 may be configured to issue commands to the cloud computing platform 1310. In various embodiments, the client device 1380 may communicate with the service application via a virtual Internet Protocol (IP) and a load balancer or other means of directing communication requests to a designated endpoint in the cloud computing platform 1310. The components of the cloud computing platform 1310 may communicate with each other via a network (not shown), which may include, but is not limited to, one or more local area networks (LANs) and / or wide area networks (WANs).

[0066] Sample computing environment

[0067] After briefly describing an overview of embodiments of the present invention, an exemplary operating environment in which embodiments of the present invention may be implemented is described below to provide a general context for various aspects of the present invention. Fig.14 , an exemplary operating environment for implementing embodiments of the present invention is shown and generally designated as computing device 1400. Computing device 1400 is only one example of a suitable computing environment and is not intended to suggest any limitation as to the functionality or scope of use of the present invention. Neither should computing device 1400 be interpreted as having any dependency or requirement relating to any one or combination of components illustrated.

[0068] The present invention may be described in the general context of computer code or machine-usable instructions (including computer-executable instructions such as program modules) executed by a computer or other machine (such as a personal data assistant or other handheld device). Generally, program modules, including routines, programs, objects, components, data structures, etc., refer to code that performs specific tasks or implements specific abstract data types. The present invention may be practiced in a variety of system configurations, including handheld devices, consumer electronics, general-purpose computers, more special-purpose computing devices, etc. The present invention may also be practiced in distributed computing environments, where tasks are performed by remote processing devices linked through a communications network.

[0069] refer to Fig.14 , computing device 1400 includes bus 1410 that directly or indirectly couples the following devices: memory 1412, one or more processors 1414, one or more presentation components 1416, input / output ports 1418, input / output components 1420, and an illustrative power supply 1422. Bus 1410 represents what may be one or more buses (such as an address bus, a data bus, or a combination thereof). For conceptual clarity, the lines are shown. Fig.14 , and other arrangements of the components described and / or component functionality are also contemplated. For example, a presentation component such as a display device may be considered an I / O component. In addition, a processor has memory. We recognize that this is the nature of the art and reiterate that Fig.14 The schematic diagrams are merely illustrative of exemplary computing devices that may be used in conjunction with one or more embodiments of the present invention. No distinction is made between categories such as "workstation," "server," "laptop," "handheld device," etc., as all of these are referred to in the same context. Fig.14 within the scope of and with reference to a “computing device”.

[0070] The computing device 1400 typically includes a variety of computer-readable media. Computer-readable media can be any available media that can be accessed by the computing device 1400, and includes volatile and non-volatile media, removable and non-removable media. By way of example and not limitation, computer-readable media can include computer storage media and communication media.

[0071] Computer storage media includes volatile and nonvolatile, and removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by the computer 1400. Computer storage media does not include the signals themselves.

[0072] Communication media typically embodies computer-readable instructions, data structures, program modules, or other data in the form of a modulated data signal (such as a carrier wave or other transmission mechanism), and includes any information delivery medium. The term "modulated data signal" refers to a signal that has one or more of its characteristics set or changed in a manner that encodes information in the signal. By way of example and not limitation, communication media include wired media such as a wired network or a direct wired connection, and wireless media such as acoustic, RF, infrared, and other wireless media. Combinations of any of the above should also be included within the scope of computer-readable media.

[0073] Memory 1412 includes computer storage media in the form of volatile and / or non-volatile memory. Memory may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard drives, optical drives, and the like. Computing device 1400 includes one or more processors that read data from various entities such as memory 1412 or I / O components 1420. Presentation component(s) 1416 present data indications to a user or other device. Exemplary presentation components include display devices, speakers, printing components, vibration components, and the like.

[0074] I / O ports 1418 allow computing device 1400 to be logically coupled to other devices including I / O components 1420, some of which may be built-in. Illustrative components include a microphone, joystick, game pad, satellite dish, scanner, printer, wireless devices, and the like.

[0075] With reference to the distributed computing system integrated data as a service, the distributed computing system integrated data as a service component refers to the integrated components used to provide the integrated data as a service. The integrated components refer to the hardware architecture and software framework that support the functionality within the system. The hardware architecture refers to the physical components and their interrelationships, and the software framework refers to the software that provides the functionality that can be implemented using the hardware embodied on the device.

[0076] End-to-end software-based systems can operate within system components to operate computer hardware to provide system functionality. At a low level, a hardware processor executes instructions selected from a machine language (also referred to as machine code or native machine) instruction set for a given processor. The processor recognizes native machine instructions and performs corresponding low-level functions related to, for example, logic, control, and memory operations. Low-level software written in machine code can provide more complex functionality to high-level software. As used herein, computer executable instructions include any software, including low-level software written in machine code, higher-level software such as application software, and any combination thereof. In this regard, system components can manage resources and provide services for system functionality. Utilizing embodiments of the present invention, any other variations and combinations thereof can be expected.

[0077] By way of example, the distributed computing system integrated data as a service may include an API library including specifications for routines, data structures, object classes, and variables that may support interaction between a hardware architecture of a device and a software framework of the distributed computing system integrated data as a service. These APIs include configuration specifications for the distributed computing system integrated data as a service so that different components therein may communicate with each other in the distributed computing system integrated data as a service, as described herein.

[0078] Various components utilized herein have been identified, and it should be understood that within the scope of the present disclosure, any number of components and arrangements can be used to achieve desired functionality. For example, for conceptual clarity, the components in the embodiment depicted in the figure are shown with lines. Other arrangements of these and other components can also be implemented. For example, although some components are depicted as single components, many elements in the elements described herein can be implemented as discrete components or distributed components or combined with other components, and implemented in any suitable combination and position. Some elements can be omitted completely. In addition, as described below, various functions described herein as being performed by one or more entities can be implemented by hardware, firmware and / or software. For example, various functions can be implemented by processors executing instructions stored in memory. In this way, in addition to or in place of the arrangements and elements shown, other arrangements and elements (for example, groups of machines, interfaces, functions, commands and functions, etc.) can also be used.

[0079] The embodiments described in the following paragraphs may be combined with one or more of the specifically described alternatives. In particular, the claimed embodiments may alternatively include references to more than one other embodiment. The claimed embodiments may specify further limitations to the claimed subject matter.

[0080] The subject matter of the embodiments of the present invention is described in detail herein to satisfy statutory requirements. However, the description itself is not intended to limit the scope of this patent. On the contrary, the inventors have anticipated that the claimed subject matter may also be embodied in other ways in conjunction with other current or future technologies to include steps or combinations of steps different from those described in this document. In addition, although the terms "step" and / or "box" may be used herein to refer to different elements of the method employed, these terms should not be interpreted as implying any particular order among or between the various steps disclosed herein unless and except when the order of the various steps is explicitly described.

[0081] For purposes of this disclosure, the word "including" has the same broad meaning as the word "comprising," and the word "access" includes "receiving," "referencing," or "retrieving." Additionally, the word "transmit" has the same broad meaning as the word "receiving" or "transmitting" supported by a software or hardware-based bus, receiver, or transmitter using the communication media described herein. Additionally, unless otherwise indicated to the contrary, words such as "a" and "an" include the plural and singular. Thus, for example, where there are one or more features, the constraint of "feature" is satisfied. Likewise, the term "or" includes conjunctions, disjunctive conjunctions, and both (thus, a or b includes a or b and a and b).

[0082] For the purpose of the above detailed discussion, embodiments of the present invention are described with reference to a distributed computing environment. However, the distributed computing environment depicted herein is merely exemplary. Components may be configured to perform novel aspects of the embodiments, wherein the term "configured for" may refer to "programmed to" use code to perform a particular task or implement a particular abstract data type. Furthermore, although embodiments of the present invention may generally refer to the distributed computing system integrated data as a service and schematic diagrams described herein, it should be understood that the described techniques may be extended to other implementation contexts.

[0083] Embodiments of the present invention have been described with respect to particular embodiments, which are intended in all respects to be illustrative rather than restrictive. Alternative embodiments will become apparent to those of ordinary skill in the art to which the present invention pertains without departing from the scope of the present invention.

[0084] From the foregoing it will be seen that the present invention is well adapted to attain all the intentions and objects set forth above, together with other obvious advantages and advantages inherent in its construction.

[0085] It will be understood that certain features and subcombinations are of utility and may be employed without reference to other features or subcombinations. This is contemplated by and is within the scope of the claims.

Claims

1. A system for implementing a distributed computing system feedback loop engine, the system comprising: one or more computer processors; as well as a computer memory storing computer usable instructions which, when used by the one or more computer processors, cause the one or more computer processors to perform: Feedback loop engine, configured to: accessing a training data set report, wherein the training data set report identifies synthetic data assets having values ​​for an asset variation parameter, wherein the synthetic data assets are associated with a framework set, wherein the training data set report is associated with a comprehensive Data as a Service (SDaaS) integrated development environment (IDE), the comprehensive Data as a Service (SDaaS) integrated development environment (IDE) supporting both SDaaS distributed computing service operations and SDaaS machine learning training service operations, wherein the SDaaS distributed computing service operation includes storing and retrieving source assets at varying levels of detail for performing the SDaaS machine learning training service operation on the source assets; Based on the training data set report, updating the integrated data asset with integrated data asset changes; as well as The frameset is updated using the updated composite data asset. 2 . The system of claim 1 , wherein the training data set report identifies a synthetic data scenario associated with the synthetic data asset, the synthetic data scenario having a value for a scenario variation parameter.

3. The system of claim 1 , wherein the value is associated with generating a training data set based on intrinsic parameter changes and extrinsic parameter changes, wherein the intrinsic parameter changes and extrinsic parameter changes provide a programmable machine learning data representation of assets and scenarios. 4 . The system of claim 1 , wherein the values ​​are manually identified in the training data set report for use in updating the framework set. 5 . The system of claim 1 , wherein the value is automatically identified in the training data set report for use in updating the framework set.

6. The system of claim 1, wherein updating the frameset is assigned a value quantizer.

7. A system according to claim 1, wherein the training data set report is associated with an internal machine learning model training service, or an external machine learning model training service, operating on the distributed system.

8. One or more computer storage media storing thereon instructions for implementing a distributed computing system feedback loop engine, the instructions, when executed by one or more processors of a computing device, causing the computing device to perform actions comprising: accessing a training data set report, wherein the training data set report identifies a synthetic data asset having an asset variation parameter, wherein the synthetic data asset is associated with a framework set, wherein the training data set report is associated with a comprehensive Data as a Service (SDaaS) integrated development environment (IDE), the comprehensive Data as a Service (SDaaS) integrated development environment (IDE) supporting both SDaaS distributed computing service operations and SDaaS machine learning training service operations, wherein the SDaaS distributed computing service operation includes storing and retrieving source assets at varying levels of detail for performing the SDaaS machine learning training service operation on the source assets; Based on the training data set report, updating the integrated data asset with integrated data asset changes; as well as The frameset is updated using the updated composite data asset.

9. The one or more computer storage media of claim 8, wherein the training data set report identifies a synthetic data scenario associated with the synthetic data asset, the synthetic data scenario having a value for a scenario variation parameter.

10. The one or more computer storage media of claim 8, wherein the values ​​are manually identified in the training data set report for use in generating the frameset.

11. The one or more computer storage media of claim 8, wherein the value is automatically identified in the training data set report for use in updating the framework set.

12. One or more computer storage media according to claim 8, wherein the integrated data asset includes asset variation parameters associated with intrinsic parameter variability and extrinsic parameter variability, wherein the intrinsic parameter variability and extrinsic parameter variability are programmable machine learning data representations based on the asset variation parameters.

13. The one or more computer storage media of claim 8, wherein updating the frameset is assigned a value quantizer.

14. One or more computer storage media according to claim 8, wherein the training data set report is associated with an internal machine learning model training service, or an external machine learning model training service, operating on the distributed system.

15. A computer-implemented method for implementing a distributed computing system feedback loop engine, the method comprising: accessing a training data set report, wherein the training data set report includes a synthetic data asset having values ​​for an asset variation parameter, wherein the training data set report is associated with a comprehensive Data as a Service (SDaaS) integrated development environment (IDE), the comprehensive Data as a Service (SDaaS) integrated development environment (IDE) supporting both SDaaS distributed computing service operations and SDaaS machine learning training service operations, wherein the SDaaS distributed computing service operation includes storing and retrieving source assets at varying levels of detail for performing the SDaaS machine learning training service operation on the source assets; updating the synthetic data asset using a value of an asset change parameter based on the training data set report; as well as The updated composite data asset is transmitted.

16. The method of claim 15, wherein the training data set report identifies a synthetic data scenario associated with the synthetic data asset, the synthetic data scenario having a value for a scenario variation parameter.

17. The method of claim 15, wherein the value is associated with generating a training data set based on intrinsic parameter changes and extrinsic parameter changes, wherein the intrinsic parameter changes and the extrinsic parameter changes provide a programmable machine learning data representation of assets and scenarios.

18. The method of claim 15, wherein the values ​​are manually identified in the training data set report for use in updating the framework set.

19. The method of claim 15, wherein updating the set of frames is assigned a value quantizer.

20. The method of claim 15, wherein the training data set report is associated with an internal machine learning model training service, or an external machine learning model training service, operating on the distributed system.

Images