Automated machine learning adaptation file delivery

Abstract

In one example, a computing system comprising a memory that stores instructions and one or more processors, executes, with the one or more processors, the instructions to: receive a plurality of adaptation files, wherein each adaptation file from the plurality of adaptation files is compatible with a version from a plurality of versions of one or more machine learning models; store, in a repository, for each adaptation file, a respective association between the adaptation file and a particular version of the one or more machine learning models that is compatible with the adaptation file; receive, from a computing device, an indication of a version of a machine learning model installed at the computing device; identify, based on the respective associations stored in the repository, a particular adaptation file that is associated with the particular version; and send, to the computing device, the particular adaptation file.

Description

Docket No.: 1333-900WO01 AUTOMATED MACHINE LEARNING ADAPTATION FILE DELIVERY BACKGROUND

[0001] Computing devices may execute a variety of applications that utilize machine learning models, such as to process input and generate output. Many machine learning applications utilize machine learning models provided by machine learning model providers. These machine learning models may be generalized models and thus may not provide the desired level of performance with respect to machine learning tasks particular to an application. SUMMARY

[0002] In general, various aspects of the techniques described in this disclosure are directed to automated machine learning adaptation file delivery. An adaptation file may include weights that may be applied to a machine learning model of a computing device to fine tune the machine learning module for one or more particular machine learning tasks. The adaptation file may only be compatible with a particular version of the machine learning model and, accordingly, should not be applied to other versions of the machine learning model. For example, the weights of the adaptation file may be nonsensical or incoherent when applied to incompatible versions of the machine learning model.

[0003] In accordance with the techniques disclosed herein, a computing system may automatically deliver a compatible adaptation file based on the version of the machine learning model on a computing device. The computing system may cause the computing device to fine tune the computing device’s machine learning model, such as for the operation of a particular application, by sending the compatible adaptation file to the computing device. For example, the computing system may cause the computing device to apply the adaptation file to the machine learning model to output answers, summaries, tips, and other responses that correspond to formatting, content, or both of various example responses in a fine tuning data set used to generate the adaptation file.

[0004] Different models, brands, or configurations of computing devices may be capable of executing different versions of a machine learning model, with at least some computing devices being limited (e.g., software / hardware limited) to executing particular versions (e.g., older versions) of the machine learning model. Rather than requiring developers to track and deliverDocket No.: 1333-900WO01 adaptation files across a multitude of computing devices and machine learning model versions, the disclosed techniques determine and automatically deliver a compatible adaptation file to the computing devices.

[0005] In one example, various aspects of the techniques are directed to a method comprising: receiving, by a computing system, a plurality of adaptation files, wherein each adaptation file from the plurality of adaptation files includes a plurality of weights compatible with a version from a plurality of versions of one or more machine learning models; storing, by the computing system and in a repository, for each adaptation file from the plurality of adaptation files, a respective association between the adaptation file and a particular version from the plurality of versions of the one or more machine learning models that is compatible with the adaptation file; receiving, by the computing system and from a computing device, an indication of a version of a machine learning model installed at the computing device; identifying, by the computing system and based on the respective associations stored in the repository, a particular adaptation file from the plurality of adaptation files that is associated with the particular version that corresponds to the version of the machine learning model installed at the computing device; and sending, by the computing system and to the computing device, the particular adaptation file.

[0006] In another example, various aspects of the techniques are directed to a computing system comprising: a memory that stores instructions, and one or more processors that execute the instructions to: receive a plurality of adaptation files, wherein each adaptation file from the plurality of adaptation files includes a plurality of weights compatible with a version from a plurality of versions of one or more machine learning models; store, in a repository, for each adaptation file from the plurality of adaptation files, a respective association between the adaptation file and a particular version from the plurality of versions of the one or more machine learning models that is compatible with the adaptation file; receive, from a computing device, an indication of a version of a machine learning model installed at the computing device; identify, based on the respective associations stored in the repository, a particular adaptation file from the plurality of adaptation files that is associated with the particular version that corresponds to the version of the machine learning model installed at the computing device; and send, to the computing device, the particular adaptation file.

[0007] In another example, various aspects of the techniques are directed to non-transitory computer-readable storage media comprising instructions, that when executed by one or moreDocket No.: 1333-900WO01 processors, cause the one or more processors to: receive a plurality of adaptation files, wherein each adaptation file from the plurality of adaptation files includes a plurality of weights compatible with a version from a plurality of versions of one or more machine learning models; store, in a repository, for each adaptation file from the plurality of adaptation files, a respective association between the adaptation file and a particular version from the plurality of versions of the one or more machine learning models that is compatible with the adaptation file; receive, from a computing device, an indication of a version of a machine learning model installed at the computing device; identify, based on the respective associations stored in the repository, a particular adaptation file from the plurality of adaptation files that is associated with the particular version that corresponds to the version of the machine learning model installed at the computing device; and send, to the computing device, the particular adaptation file.

[0008] The details of one or more examples are set forth in the accompanying drawings and the description below. Other features, objects, and advantages will be apparent from the description and drawings, and from the claims. BRIEF DESCRIPTION OF DRAWINGS

[0009] FIG.1 is a conceptual diagram illustrating an example environment for automated machine learning adaptation file delivery, in accordance with one or more aspects of the present disclosure.

[0010] FIG.2 is a block diagram illustrating an example environment for automated machine learning adaptation file delivery, in accordance with one or more aspects of the present disclosure.

[0011] FIG.3 is an example of compatibility information, in accordance with one or more aspects of the present disclosure.

[0012] FIG.4 is a flowchart illustrating an example process for automated machine learning adaptation file delivery, in accordance with one or more aspects of the present disclosure. DETAILED DESCRIPTION

[0013] FIG.1 is a conceptual diagram illustrating an example environment for automated machine learning adaptation file delivery, in accordance with one or more aspects of the present disclosure. As can be seen from the example of FIG.1, environment 100 may include one orDocket No.: 1333-900WO01 more computing devices 120A–120N (collectively, “computing devices 120”) that may communicate with computing system 110 over network 102. In some examples, computing devices 120 and computing system 110 may be peer devices that operate in a client / server fashion. For instance, computing devices 120 may be clients that are used to access services, such as application store services (e.g., application search services, download services) of an application store 113 provided by computing system 110.

[0014] As such, computing device 120 may enable users to interact with an application store 113 provided by computing system 110. Computing device 120 may be an example of a smartphone, mobile phone, a tablet computer, a laptop computer, a desktop computer, a wearable device, a gaming system, a media player, an e-book reader, camera device, or a wearable computing device (e.g., a computerized watch, computerized eyewear, etc.), or other computing device. FIG.1 illustrates a particular example of computing device 120, and many other examples of computing device 120 may be used in other instances and may include a subset of the components included in example computing device 120 or may include additional components not shown in FIG. 1.

[0015] Computing device 120 may include an operating system that provides an execution environment for store client 122, one or more applications 126A–126N (collectively, “applications 126”), or both. Computing device 120 may execute store client 122 to download applications 126 by downloading corresponding application packages 116A–116N (collectively, “application packages 116”), such as from a repository 114 of computing system 110. Examples of applications 126 include social networking applications, utility applications, productivity applications, entertainment applications, creativity applications, communication applications, shopping applications, games, and other software applications. Each application package 116 may include a respective one of applications 126. As such, to install application 126A, computing device 120 may download application package 116A that includes application 126A and install application 126A using application package 116A, such as by extracting application 126A from application package 116A.

[0016] Store client 122 may interact with store module 112 of computing system 110 and may perform other functions associated with application packages 116 or applications 126, including downloading application packages 116, installing applications 126, updating applications 126, and deleting applications 126, or various subsets thereof. As shown in FIG. 1 for instance, storeDocket No.: 1333-900WO01 client 122 may present a representation of application store 113 provided by store module 112, such as by presenting one or more user interface elements that constitute the representation of application store 113 or a portion (e.g., page, screen) thereof, through one or more user interface devices 128 of computing device 120. Application store 113 may include indications of application packages 116 stored by repository 114, that are available for download and / or installation, to computing device 120 through application store 113. Application store 113 may present application packages 116, such as within a page, screen, or other user interface element of application store 113.

[0017] Computing device 120 may receive user input (e.g., requests) for store client 122, applications 126, or both as well as present output of store client 122, applications 126, or both through user interface device 128 of computing device 120. User interface device 128 of computing device 120 may be hardware that functions as an input and / or output device for computing device 120. For example, user interface device 128 may include a display component (e.g., liquid crystal display (LCD), organic light-emitting diode (OLED) display), which may be a screen at which information is displayed by user interface device 128 and a presence-sensitive input device that may detect an object at and / or near the display component. The presence- sensitive input device may, for example, detect a user’s touch or other input. User interface device 128 may provide tactile, audio, and video output. User interface device 128, in some examples, includes one or more of a presence-sensitive display, speaker, liquid crystal display (LCD), organic light-emitting diode (OLED) display, haptic motors, linear actuating devices, or any other type of device for receiving input or generating output to a human or machine.

[0018] Computing system 110 may be any suitable computing system, such as one or more desktop computers, laptop computers, mainframes, servers, cloud computing systems, virtual machines, etc. capable of sending and receiving information via network 102. In some examples, computing system 110 may represent a cloud computing system that provides one or more services via network 102. That is, in some examples, computing system 110 may be a distributed computing system. One or more computing devices, such as computing devices 120, may access the services provided by the cloud by communicating with computing system 110. FIG.1 illustrates only one particular example of computing system 110, and many other examples of computing system 110 may be used in other instances and may include a subset ofDocket No.: 1333-900WO01 the components included in example computing system 110 or may include additional components not shown in FIG.1.

[0019] Computing system 110 may include store module 112 that computing system 110 may invoke to provide application store 113. For example, store module 112 may provide application store 113 by providing one or more application store services, such as to store client 122 of computing devices 120. Store module 112 may publish application packages 116 including respective applications 126 developed by application developers such that computing devices 120 may download applications 126 for installation. In some examples, store module 112 may process payment information and / or authorizations from computing devices 120, such as to allow purchase prior to download and installation of paid applications 126 or paid features of applications 126.

[0020] Store module 112 may publish application listings for application packages 116 that may include metadata about corresponding applications 126. As shown in FIG. 1 for example, application store 113 may present application packages 116 in a list along with application metadata for each corresponding application 126, such as in the form of an application name (e.g., “APPLICATION 1”) and application description (e.g., “APPLICATION 1 DESCRIPTION”). Other examples of application metadata for applications 126 include technical information about features, functionality, or other characteristics of application 126, promotional content advertising, promoting, marketing, selling, or otherwise enticing users to download and / or install application 126, or other information about application 126.

[0021] Store module 112 may receive application packages 116 from various sources. For example, store module 112 may receive applications packages 116 uploaded by application developers through one or more development systems 104. As will be described further below, application developers may use development system 104 to develop (e.g., code) applications 126. Store module 112 may receive application package 116A and including application 126A from a first application developer and may receive application package 116N including application 126N from an nth application developer. Store module 112 may receive a variety of developer provided metadata about application 126, such as the application developer’s long form and / or short form text (e.g., summary) promoting or otherwise describing application 126 and / or features thereof, one or more screenshots, icons, or other images, one or moreDocket No.: 1333-900WO01 categorizations (e.g., tags) indicating one or more categories (e.g., productivity, messaging, social networking, game) application 126 should be assigned to.

[0022] One or more of application packages 116 may include applications 126 with machine learning features. As such, application 126 may rely upon one or more machine learning (ML) models 124A–124N (collectively, “ML models 124”) installed at computing device 120 to perform machine learning tasks (e.g., natural language processing / output). ML model 124 may be or include one or more of various different types of machine-learned models. Examples of such different types of machine-learning models are provided below for illustration. Additional models beyond the example models provided below may be used as well. Applications 126 may include and manage operation of ML models 124 to process input, generate application-specific output according to respective use cases of applications 126, or both. For example, application 126 may be a cooking coach or advice application that receives natural language queries and outputs responses in natural language through ML model 124. ML model 124 may be or include one or more of various different types of machine-learned models. Examples of such different types of machine-learning models are provided below for illustration. Additional models beyond the example models provided below may be used as well.

[0023] One or more of the example ML models 124 described below may be used, alone or in combination, to generate output corresponding to particular machine learning tasks performed by applications 126 in response to input data. For example, ML model 124 can be or include one or more of various different types of machine-learned models. For example, ML model 124 may represent various models including recurrent neural networks (RNNs) and / or transformer models (self-attention models). Some examples of ML models 124 include GPT-3, BERT, GEMINI (e.g., Gemini Ultra, Gemini Pro, Gemini Flash, Gemini Nano), ANDROID AICore, and T5. In some examples, machine learning module 310 may perform classification, summarization, name generation, regression, clustering, anomaly detection, recommendation generation, and / or other tasks.

[0024] In some examples, applications 126 may invoke ML model 124 to perform natural language processing (NLP) tasks, such as summarizing, translating, or organizing natural language input. As such, ML model 124 may represent one or more large language models (LLMs) that can interpret natural language input, such as from a user, and generate output associated with a user’s desired application functionality. ML model 124 may perform variousDocket No.: 1333-900WO01 types of NLP tasks based on the natural language input. For example, application 126A may invoke ML model 124 to identify, from NLP input, one or more tasks, in which each of the one or more tasks may map to a respective function of application 126A. Application 126A may execute the mapped function to respond to the user’s natural language input. For example, in response to natural language input to “Find a recipe for scrambled eggs,” ML model 124 may identify tasks for application 126A, including retrieving recipe information including scrambled eggs and presenting such recipe information. Application 126A may execute functions corresponding to such tasks, such as by querying recipes from a collection of stored recipes, provided with application 126A, including “scrambled eggs” and presenting one or more of the resulting recipes, such as via user interface device 128.

[0025] In general, an LLM may accurately perform NLP tasks, such as generating text and other content in response to input (e.g., natural language input). The LLM may be trained on a large and diverse corpus of text. This dataset may cover a wide range of topics and domains to ensure the LLM learns diverse linguistic patterns and contextual relationships. As such, without fine tuning, the LLM may have an increased likelihood of generating output that is non-responsive (e.g., inaccurate, false) to prompts for specific subject matter, or that is undesirable with respect to tone and / or format for the particular purpose of application 126. For example, without fine tuning, an LLM may generate inaccurate output and / or output not tonally suitable for the particular purpose of application 126.

[0026] For instance, application 126 may be a cooking coach or advisor that application developers intend to provide concise formal responses to user queries. By applying adaptation file 118 generated with a fine tuning data set including examples of concise formal responses to the LLM (e.g., ML model 124), the LLM may be fine tuned to generate output corresponding to these examples as compared to informal or other general responses that the LLM would generate by default. In this manner, the LLM may be fine tuned to generate output that satisfies a particular use case that application 126 was developed to service.

[0027] Though described with respect to LLMs, ML model 124 may represent other types of models, including various classification models (e.g., naive Bayes, k-nearest neighbor, support vector machine (SVM), neural networks) or generative models (e.g., diffusion, generative adversarial networks (GAN). In some examples, one or more of ML models 124 may be classification models that receive input and output an indication of a classification to which theDocket No.: 1333-900WO01 input belongs. For instance, application 126 may receive input in the form of photos of food and apply ML model 124 to classify a type or other characteristic of the food.

[0028] In some implementations, ML model 124 may be or include one or more classifier models such as, for example, linear classification models; quadratic classification models; etc. ML model 124 may be or include one or more regression models such as, for example, simple linear regression models; multiple linear regression models; logistic regression models; stepwise regression models; multivariate adaptive regression splines; locally estimated scatterplot smoothing models; etc. In some examples, ML model 124 may be or include one or more generative networks such as, for example, generative adversarial networks. Generative networks may be used to generate new data such as artificial feedback texts.

[0029] In some examples, ML model 124 may be or include one or more artificial neural networks (also referred to simply as neural networks). A neural network may include a group of connected nodes, which also may be referred to as neurons or perceptrons. A neural network may be organized into one or more layers. Neural networks that include multiple layers may be referred to as “deep” networks. A deep network may include an input layer, an output layer, and one or more hidden layers positioned between the input layer and the output layer. The nodes of the neural network may be connected or non-fully connected.

[0030] In some examples, ML models 124 may perform or be subjected to one or more reinforcement learning techniques such as Markov decision processes; dynamic programming; Q functions or Q-learning; value function approaches; deep Q-networks; differentiable neural computers; asynchronous advantage actor-critics; deterministic policy gradient; etc. Computing system 110 may generate ML models 124 using various training techniques, including supervised, unsupervised, semi-supervised, and reinforcement learning techniques, utilizing one or more training data sets.

[0031] ML models 124 of computing device 120 may be provided by various ML model providers (e.g., original equipment manufacturers (OEMs), public / private entities, non-profit entities) and installed at computing device 120. In some examples, ML model 124 may be included with the operating system or otherwise pre-installed at computing device 120. ML model 124 may include an indication of the version of ML model 124. For example, ML model 124 may include metadata, such as in a manifest, or other structured data format, that includes a version number or other version identifier for ML model 124. Examples of version identifiersDocket No.: 1333-900WO01 include string identifiers (e.g., “com.provider_name.model_name:1234”), that may identify other information such as a model name, provider name, or other information (e.g., “com.provider_name.model_name”) as well as a model version (e.g., “1234”), or numeric identifiers (e.g., 1.0, 1.1, 2, 3, etc.).

[0032] In some examples, ML models 124, as provided, may be generalized (e.g., general purpose) models (e.g., untuned LLMs) that may perform general tasks adequately but may not provide the desired output for machine learning tasks particular to application 126. To illustrate, ML model 124, when applied by application 126, may provide responses to cooking questions in general but these responses may not be sufficiently accurate or formatted when application 126 is expected to be a cooking coach or advisor.

[0033] Application 126 may be fine tuned to perform one or more particular machine learning tasks for application 126 through one or more adaptation files 118A–118N (collectively, “adaptation files 118”). For example, application 126 may represent a cooking application that processes natural language (e.g., English) as input and outputs cooking information in response. For instance, computing device 120 may fine tune application 126 by applying adaptation file 118 to ML model 124 so as to output advice, coaching, tips, summaries, or other cooking related information in a style or format corresponding to examples of a fine tuning data set used to generate adaptation file 118. As described above, ML model 124 may be an LLM or other type of model.

[0034] As can be seen, repository 114 may store and store module 112 may provide one or more of adaptation files 118 for each application package 116. Computing device 120, in response to receiving adaptation file 118, may apply adaptation file 118 to ML model 124 during execution of application 126. By applying adaptation file 118 to ML model 124, computing device 120 may fine tune ML model 124 to enhance ML model 124 for one or more particular machine learning tasks, such as those particular to application 126.

[0035] Adaptation file 118 may include a plurality of weights, such as in one or more matrices, that computing device 120 may apply to ML model 124 to enhance the performance of ML model 124 for one or more particular machine learning tasks. The weights of adaptation file 118 may only be compatible with a particular version (e.g., version 1.0) of ML model 124 in that the weights may be nonsensical or incoherent if applied to a version of ML model 124 other than the compatible version.Docket No.: 1333-900WO01

[0036] Store module 112 may receive adaptation files 118 from various sources. For example, store module 112 may receive adaptation files 118 uploaded by application developers of applications 126, such as from development system 104. For instance, store module 112 may receive, such as from development system 104, application package 116A including application 126A and adaptation file 118A from a first application developer and may receive application package 116N including application 126N and adaptation file 118N from an nth application developer. Store module 112 may receive a plurality of adaptation files 118 for each application 126. For example, store module 112 may receive, for application package 116N containing application 126N, a first adaptation file 118C and an nth adaptation file 118N. As can be seen from the example of FIG.1, repository 114 may store application package 116A along with adaptation files 118 (e.g., first adaptation file 118A and second adaptation file 118B) for the corresponding application, in this example, application 126A.

[0037] Adaptation files 118 may include or be received along with an indication of a compatible version (e.g., target version) of ML model 124 (e.g., model name and / or model version number / identifier). For example, adaptation file 118 may include metadata with a version dependency that includes such indication of the compatible version of ML model 124. For instance, at build time, development system 104 may include such metadata into the build of adaptation file 118. In some examples, the metadata may define available delivery types. For instance, store module 212 may deliver adaptation file 118 at install time, on-demand, as a “fast- follow,” or the like, as will be described further below. In these examples, store module 212 may only deliver adaptation file 118 according to the delivery types identified in the metadata for adaptation file 118.

[0038] Store module 112 may receive the indication of the compatible version of ML model 124 from the application developer that generates adaptation files 118 for application 126, such as through the application developer’s development system 104. For example, store module 112 may receive adaptation file 118A including an indication of a version (e.g., version number / identifier) of ML model 124A that adaptation file 118A is compatible with. For instance, store module 112 may receive adaptation file 118A with an indication that adaptation file 118A is compatible with version n of ML model 124A. Repository 114 may store adaptation file 118A along with the indication of the version of ML model 124 that is compatible with adaptation file 118A. For example, and as will be described further below, repository 114 mayDocket No.: 1333-900WO01 maintain an index indicating, for each adaptation file 118, the version of ML model that adaptation file 118 is compatible with.

[0039] For their respective applications 126, application developers may provide (e.g., upload from development system 104), and store module 112 may receive, a plurality of adaptation files 118, such as to allow application 126A to be fine tuned for different versions of ML model 124A. For example, store module 112 may receive first adaptation file 118A that is compatible with version n of ML model 124A and receive second adaptation file 118B that is compatible with version n+1 of ML model 124A. As such, applications 126 may execute with fine tuned ML models 124 on various computing devices 120 with different versions of ML models 124. For example, application 126A may be fine tuned with adaptation file 118A on a first computing device 120A with version n+1 of ML model 124A and be fine tuned with adaptation file 118B on a second computing device 120N with a previous or other different version, such as version n, of ML model 124A. Second computing device 120N may be limited, such as by software and / or hardware limitations, to executing version n of ML model 124A as compared to first computing device 120A which is capable of executing version n+1 of ML model 124A.

[0040] Individual computing devices 120 may include various ML models 124, including different ML models 124 (e.g., ML models from different ML model providers) as well as different versions of the same ML model (e.g., version 1.0, 1.1, 1.2). Each ML model 124 may have a substantial number of versions (e.g., dozens, hundreds), which may make management and distribution of compatible adaptation models 118 difficult or impossible for application developers. As described above, individual computing devices 120 may be limited, such as due to software and / or hardware limitations, to executing a subset of versions of one or more of ML models 124, which may not be the latest versions of the one or more ML models.

[0041] To address these compatibility issues, store module 112 may utilize a targeting dimension based on the version of ML model 124 installed at computing device 120. For example, store module 112 may automatically identify and deliver (e.g., send) an adaptation file of adaptation files 118 that is compatible with ML model 124 installed at computing device 120. For instance, store module 112 may target an adaptation file of adaptation files 118 to the version of ML model 124 installed, based on the indication of the compatible version of ML model 124 included in the adaptation file, such as may be included at build time of the adaptation file.Docket No.: 1333-900WO01

[0042] Continuing the above example for instance, store module 112 may receive an indication of the version of ML model 124 (e.g., version number / identifier) installed at computing device 120. Store module 112 may identify one of adaptation files 118 based on such indication. For example, store module 112 may determine, such as from the index of repository 114, that adaptation file 118A is compatible with the version of ML model 124A (e.g., version 1.0) installed at computing device 120A. Responsive to such selection, store module 112 may deliver (e.g., send), to computing device 120A, adaptation file 118A. By automatically identifying and delivering compatible adaptation files 118 to computing devices 120, store module 112 ensures applications 126 can be properly fine-tuned across numerous computing devices 120 of various configurations and with various versions of one or more ML models 124.

[0043] In this manner, store module 112 reduces utilization of computing resources (e.g., processing, memory resources) in that computing device 120A, with compatible adaptation file 118A, may provide improved responses to user input thereby increasing the likelihood of a satisfactory response and reducing the likelihood of additional or follow up user input and response cycles and the corresponding consumption of processing, memory, or other computing resources. Store module 112 may reduce utilization of network, bandwidth, or other computing resources through the automatic identification and delivery of compatible adaptation files 118. In this manner, computing system 110 and computing devices 120 avoid bandwidth consumption that may otherwise occur in transmission of incompatible or otherwise unnecessary adaptation files 118 relative to particular computing devices 120. In some examples, repository 114 may host, to a large volume of computing devices 120 (e.g., thousands, millions), a large volume of application packages 126 (e.g., thousands, millions) which may each have multiple corresponding adaptation files 118. At this scale, by automatically identifying and delivering compatible adaptation files 118 to respective computing devices 120, store module 112 may substantially reduce the utilization of computing resources, such as those described above.

[0044] Store module 112 may identify, for delivery, an adaptation file of adaptation files 118 for individual applications 126. For example, for application 126A, store module 112 may select either adaptation file 118A or adaptation file 118B because, in this example, adaptation files 118A, 118B correspond to (e.g., are contained within) application package 116A containing application 126A. As another example, for application 126N, store module 112 may identifyDocket No.: 1333-900WO01 either adaptation file 118C or adaptation file 118N for delivery because adaptation files 118C, 118N, in this example, correspond to application package 116N containing application 126N.

[0045] In some examples, development system 104 may generate adaptation files 118 for one or more of applications 126. Development system 104 may be any suitable computing system, such as one or more desktop computers, laptop computers, mainframes, servers, cloud computing systems, virtual machines, etc. capable of sending and receiving information via network 102. In some examples, development system 104 may represent a cloud computing system that provides one or more services via network 102. Development system 104 may be used by developers to develop (e.g., code) applications, generate application packages 116 corresponding to (e.g., containing) respective applications 126, generate adaptation files 118 for applications 126, and perform other development activities. As described above, store module 112 may receive adaptation files 118, application packages 116, and applications 126, or various subsets thereof from development system 104.

[0046] Development system 104 may generate an adaptation file 118 using a fine tuning data set. For example, the fine tuning data set may include examples of prompts and responses to the prompts, such as in question (e.g., prompt) and answer (e.g., response) format. To generate an adaptation file 118 for a machine learning task particular to application 126 (e.g., a cooking application), development system 104 may generate adaptation file 118 with a fine tuning data set including prompts and responses relating to the machine learning task. For example, with respect to a cooking application, development system 104 may generate adaptation file 118 with a fine tuning data set comprising previously validated questions and answers (or other prompts and responses) about cooking. Development system 104 may process the fine tuning data set to generate weights for adaptation file 118 that may be applied to ML model 124, such as by computing device 120, to fine tune ML model 124.

[0047] For example, development system 104 may generate adaptation file 118 including one or more matrices storing weights that may be applied to ML model 124 to fine tune ML model 124 for a particular machine learning task. In some examples, development system 104 may execute a Low-Rank Adaptation (LoRA), Quantized Low-Rank Adaptation (QLoRA), or other suitable fine tuning techniques to generate one or more of adaptation files 118. In general, LoRA techniques may reduce the number of trainable parameters by freezing pre-trained weights of an LLM and injecting small, trainable low-rank matrices that adapt ML model 124 for specificDocket No.: 1333-900WO01 tasks. One or more of adaptation files 118 may represent LoRA, QLoRA, or other artificial intelligence (AI) adaptations or adaptation files. When computing device 120 applies adaptation file 118 to ML model 124, such as during execution of application 126, ML model 124 may generate output that corresponds to examples from the fine tuning data set (e.g., with similar content, styling, and / or formatting) as compared to the output from ML model 124 prior to fine tuning which does not include such correspondence to the fine tuning data set.

[0048] In some examples, development system 104 may include one or more software development kits (SDKs) that enable applications 126 to facilitate automated machine learning adaptation file delivery. For example, using SDK, application 126 may send the indication of the version of ML model 124 installed at computing device 120 to store module 112. In response, store module 112 may send, to computing device 120, adaptation file 118 for application 126 that is compatible with the version of ML model 124 installed at computing device 120. Application 126 may make such a request at install time, on demand, or as a fast-follow. Though described with respect to application 126, various elements of computing device 120 may use the SDK as described with respect to application 126. For example, store client 122 or other element of computing device 120 may use the SDK to send the indication of the compatible version of ML model 124 to store module 112 to obtain, at install time, on demand, or as a fast-follow, adaptation file 118 for one or more of applications 126 that is compatible with the version of ML model 124 installed at computing device 120.

[0049] FIG.2 is a block diagram illustrating an example environment for automated machine learning adaptation file delivery, in accordance with one or more aspects of the present disclosure. As can be seen from the example of FIG. 2, environment 200 may include a computing device 220 that may communicate with a computing system 210 over network 202. Computing system 210, computing device 220, and network 202 of FIG. 2 are respectively described below as an example of computing system 110, computing devices 120, and network 102 as illustrated in FIG. 1.

[0050] Computing system 210 may be any suitable computing system, such as one or more desktop computers, laptop computers, mainframes, servers, cloud computing systems, virtual machines, etc. FIG.2 illustrates only one particular example of computing system 210, and many other examples of computing system 210 may be used in other instances and may include aDocket No.: 1333-900WO01 subset of the components included in example computing system 210 or may include additional components not shown in FIG.2.

[0051] As shown by the example of FIG.2, computing system 210 may include one or more processors 232, one or more input devices 234, one or more output devices 236, one or more communication units 238, and one or more storage devices 240. Communication channels 233 may interconnect each of the components 232, 234, 236, 238 and 240 for inter-component communications (physically, communicatively, and / or operatively). In some examples, communication channels 233 may include a system bus, a network connection, an inter-process communication data structure, or any other method for communicating data. Storage device 240 of computing system 210 may include store module 212 and repository 214.

[0052] Computing system 210 may invoke store module 212 to provide an application store, such as application store 113 of FIG.1, which may facilitate retrieval (e.g., download) of one or more application packages 216A–216N (collectively, “application packages 216”) and adaptation files 218A–218N (collectively, “adaptation files 218”) stored at repository 214 of computing system 210. Store module 212, repository 214, application packages 216, applications 226A–226N (collectively, “applications 226”), and adaptation files 218 of FIG.2 may respectively be examples of store module 112, repository 114, application packages 116, applications 126, and adaptation files 218 as illustrated in FIG.1

[0053] One or more input devices 234 of computing system 210 may receive input. Examples of input are tactile, audio, and video input. Input devices 234 of computing system 210, in one example, includes a presence-sensitive display, touch-sensitive screen, mouse, keyboard, voice responsive system, video camera, microphone or any other type of device for detecting input from a human or machine.

[0054] One or more output devices 236 of computing system 210 may generate output. Examples of output are tactile, audio, and video output. Output devices 236 of computing system 210, in one example, includes a presence-sensitive display, sound card, graphics card, speaker, liquid crystal display (LCD), organic light-emitting diode (OLED) display, or any other type of device for generating output to a human or machine.

[0055] One or more communication units 238 of computing system 210 may communicate with external devices via one or more wired and / or wireless networks by transmitting and / or receiving network signals on the one or more networks. Examples of one or more communication unitsDocket No.: 1333-900WO01 238 include a network interface card (e.g., an Ethernet card), an optical transceiver, a radio frequency transceiver, or any other type of device that can send and / or receive information. Other examples of one or more communication units 238 may include short wave radios, cellular data radios, wireless network radios, as well as universal serial bus (USB) controllers.

[0056] Network 202 may represent any public or private communications network, for example, cellular, WI-FI, and / or other types of networks, for transmitting data between computing systems, servers, and computing devices. Network 202 may include one or more network hubs, network switches, network routers, or any other network equipment, that are operatively inter- coupled thereby providing for the exchange of information between computing system 210 and computing device 220. Computing device 220 and computing system 210 may transmit and receive data across network 202 using any suitable communication techniques. For example, computing system 210 and computing device 220 may communicate (e.g., transmit and receive) application packages 216, adaptation files 218, or both via network 202. Each of computing device 220 and computing system 210 may be operatively coupled to network 202 using respective network links, such as Ethernet, WI-FI, BLUETOOTH or any other types of wired and / or wireless network connections.

[0057] One or more processors 232 may implement functionality and / or execute instructions within computing system 210. For example, one or more processors 232 of computing system 210 may receive and execute instructions stored by one or more storage devices 240 that execute the functionality of operating system 242, store module 212, and repository 214. The instructions executed by one or more processors 232 may cause computing system 210 to store information within one or more storage devices 240 during program execution. Examples of one or more processors 232 include application processors, display controllers, sensor hubs, and any other hardware configured to function as a processing unit. One or more processors 232 may execute instructions of operating system 242, store module 212, and repository 214 to perform actions or functions. That is, operating system 242, store module 212, and repository 214 may be operable by one or more processors 232 to perform various actions or functions of computing system 210.

[0058] Store module 212 may provide an application store, such as application store 113 of FIG. 1, by providing one or more application store services to computing device 220. Store module 212 may publish application packages 216, such as in one or more application listings, forDocket No.: 1333-900WO01 application discovery purposes. Store module 212 may publish application packages 216 such that computing device 220 may download application packages 216, including corresponding applications 226 and adaptation files 218, for installation. Once installed, application packages 216 may be represented as applications 226. Store module 212 may perform operations described herein using software, hardware, firmware, or a mixture of hardware, software, and firmware residing in and / or executing at computing system 210.

[0059] One or more storage devices 240 within computing system 210 may store information for processing during operation of computing system 210. That is, computing system 210 may store data accessed by operating system 242, store module 212, and repository 214 during execution at computing system 210, including application packages 216, adaptation files 218, and one or more indexes 244. Computing system 210 may store application packages 216, adaptation files 218, and index 244, such as to one or more storage devices 240 or repository 214 thereof, for access by store module 212 during execution at computing system 210.

[0060] In some examples, storage device 240 is a temporary memory, meaning that a primary purpose of storage device 240 is not long-term storage. One or more storage devices 240 on computing system 210 may be configured for short-term storage of information as volatile memory and therefore not retain stored contents if powered off. Examples of volatile memories include random access memories (RAM), dynamic random access memories (DRAM), static random access memories (SRAM), and other forms of volatile memories known in the art.

[0061] One or more storage devices 240, in some examples, also include one or more computer- readable storage media. One or more storage devices 240 may be configured to store larger amounts of information than volatile memory. One or more storage devices 240 may further be configured for long-term storage of information as non-volatile memory space and retain information after power on / off cycles. Examples of non-volatile memories include magnetic hard disks, optical disks, flash memories, or forms of electrically programmable memories (EPROM) or electrically erasable and programmable (EEPROM) memories. One or more storage devices 240 may store program instructions and / or information (e.g., data) associated with operating system 242, store module 212, and repository 214.

[0062] In some examples, repository 214 may implement a database, file format, or other structured data format suitable for storage and retrieval of information, such as application packages 216, adaptation files 218, or both. Store module 212 may store application packagesDocket No.: 1333-900WO01 216 and adaptation files 218 received from developers, such as from development system 104 of FIG.1, to repository 214.

[0063] Store module 212 may store index 244 to repository 214 in some examples. As will be described further below, index 244 of repository 214 may include compatibility information for adaptation files 218 that indicates, for each adaptation file 218 in repository 214, a compatible version of one or more ML models 224A–224N (collectively, “ML models 224”). Though illustrated as internal to repository 214, index 244 may be as a separate entity (e.g., separate database) of storage device 240 that is outside of repository 214 in some examples. As can be seen from the example of FIG. 2, storage device 240 may include repository 214.

[0064] Store module 212 may maintain (e.g., add, delete, update), at index 244, units of compatibility information for adaptation files 218. For example, index 244 may include, for each respective adaptation file 218 stored at repository 214, an indication of the version of ML model 224 that the adaptation file is compatible with. In this manner, store module 212 may query index 244 to deliver adaptation file 218 that is compatible with the version of ML model 224 installed at computing device 220. For example, store module 212 may receive an indication of the version of ML model 224 installed at computing device 220. Store module 212 may query index 244 to determine adaptation file 218A of adaptation files 218 is identified as compatible with the version of ML model 224 installed at computing device 220. Store module 212 may retrieve adaptation file 218A from repository 214 and send adaptation file 218A to computing device 220.

[0065] In some examples, store module 212 may deliver adaptation file 218 as part of application package 216A, such when computing device 220 downloads application package 216A to install corresponding application 226A at computing device 220 (e.g., at install time). For example, computing device 220 may send the indication of the version of ML model 224A installed at computing device 220A (e.g., version 1.0), which store module 212 may receive. In response to a download request for application package 216A from computing device 220A, store module 212 may determine, from index 244, adaptation file 218A is compatible with ML model 224A. Store module 212 may deliver (e.g., send), to computing device 220, adaptation file 218A as part of or along with application package 216A.

[0066] In some examples, rather than delivering adaptation file 218A at install time (e.g., as part of or along with application package 216A), store module 212 may deliver adaptation file 218ADocket No.: 1333-900WO01 in as a “fast-follow” or deliver adaptation file 218A on demand (e.g., responsive to a request from computing device 220). For instance, to deliver adaptation file 218A as a fast-follow, store module 212 may deliver adaptation file 218A in a sequence where adaptation file 218 is delivered after application package 216A is delivered. To deliver adaptation file 218 on demand, store module 212 may deliver adaptation file 218A in response to a request or indication from computing device 220 sent by computing device 220 after completing installation of application package 216A.

[0067] Store module 212 may deliver adaptation file 218 in response to an update to computing device 220. For example, computing device 220 may download an updated ML model 224, such as from store module 212, a ML model provider, or other source. Continuing the above example for instance, computing device 220 may be updated so as to include another version of ML model 224A (e.g., computing device 220 may upgrade to version 1.1 of ML model 224A). Store module 212 may receive, from computing device 220, an indication of the updated version of ML model 224A (e.g., version 1.1). In response, store module 212 may determine, from index 244, adaptation file 218B is compatible with ML model 224A, as updated. Store module 212 may deliver adaptation file 218B to cause computing device 220 to fine tune ML model 224A with adaptation file 218B, such as during execution of application 226A.

[0068] Store module 212 may deliver adaptation file 218 in response to an update to adaptation file, such as may be provided by an application developer. For example, the application developer of application 226A may update adaptation file 218A and upload updated adaptation file 218A to store module 212, such as through development system 104 of FIG.1. Store module 212 may replace adaptation file 218A in repository 214 with updated adaptation file 218A. Store module 212 may deliver updated adaptation file 218A to computing device 220 to cause computing device 220 to replace adaptation file 218A previously delivered to computing device 220.

[0069] Computing device 220 may be an example of a smartphone, mobile phone, a tablet computer, a laptop computer, a desktop computer, a wearable device, a gaming system, a media player, an e-book reader, camera device, or a wearable computing device (e.g., a computerized watch, computerized eyewear, etc.), or other computing device. FIG.2 illustrates a particular example of computing device 220, and many other examples of computing device 220 may be used in other instances and may include a subset of the components included in exampleDocket No.: 1333-900WO01 computing device 220 or may include additional components not shown in FIG.2.

[0070] Computing device 220 includes one or more user interface devices 228, one or more processors 252, one or more storage devices 256, and one or more communication units 254. One or more storage devices 256 of computing device 220 may include an operating system that provides an execution environment for store client 222, ML models 224, one or more applications 226, or both. ML models 224 of FIG.2 may be examples of ML models 124 of FIG.1. As described above, applications 226 may represent application packages 216 downloaded from computing system 210 that are installed to and / or executed by computing device 220.

[0071] User interface device 228 of computing device 220 may be hardware that functions as an input and / or output device for computing device 220. For example, user interface device 228 may include a display component (e.g., liquid crystal display (LCD), organic light-emitting diode (OLED) display), which may be a screen at which information is displayed by user interface device 228 and a presence-sensitive input device that may detect an object at and / or near the display component. The presence-sensitive input device may, for example, detect a user’s touch or other input. User interface device 228 may provide tactile, audio, and video output. User interface device 228, in some examples, includes one or more of a presence-sensitive display, speaker, liquid crystal display (LCD), organic light-emitting diode (OLED) display, haptic motors, linear actuating devices, or any other type of device for receiving input or generating output to a human or machine.

[0072] One or more communication units 254 of computing device 220 may communicate with external devices by transmitting and / or receiving communication signals, such as via one or more wireless networks or wireless connections. Examples of one or more communication units 254 include a network interface card (e.g., Ethernet or WI-FI card), an optical transceiver, a radio frequency transceiver, a global positioning system (GPS) receiver, or any other type of device that can send and / or receive information. Other examples of one or more communication units 254 may include short wave radios, cellular data radios, wireless network radios, as well as universal serial bus (USB) controllers or any other type of device that can send and / or receive information over a wired or wireless connection.

[0073] One or more processors 252 may implement functionality and / or execute instructions within computing device 220. For example, one or more processors 252 on computing deviceDocket No.: 1333-900WO01 220 may receive and execute instructions stored by one or more storage devices 256 that execute the functionality of store client 222, applications 226, and ML models 224 as may be fine tuned by adaptation files 218. The instructions executed by one or more processors 252 may cause computing device 220 to store information within one or more storage devices 256 during program execution. Examples of one or more processors 252 include application processors, display controllers, sensor hubs, and any other hardware configured to function as a processing unit.

[0074] One or more storage devices 256 within computing device 220 may store information for processing during operation of computing device 220. That is, computing device 220 may store data, including applications 226, ML models 224, adaptation files 218 received from computing system 210, and other data, or various subsets thereof. In some examples, storage device 256 is a temporary memory, meaning that a primary purpose of storage device 256 is not long-term storage. One or more storage devices 256 on computing device 220 may be configured for short- term storage of information as volatile memory and therefore not retain stored contents if powered off. Examples of volatile memories include random access memories (RAM), dynamic random access memories (DRAM), static random access memories (SRAM), and other forms of volatile memories known in the art.

[0075] One or more storage devices 256, in some examples, also include one or more computer- readable storage media. One or more storage devices 256 may be configured to store larger amounts of information than volatile memory. One or more storage devices 256 may further be configured for long-term storage of information as non-volatile memory space and retain information after power on / off cycles. Examples of non-volatile memories include magnetic hard disks, optical disks, flash memories, or forms of electrically programmable memories (EPROM) or electrically erasable and programmable (EEPROM) memories. One or more storage devices 256 may store program instructions and / or information (e.g., data) associated with applications 226, ML models 224, and store client 222. One or more storage devices 256 may store adaptation files 218 delivered by (e.g., received from) store module 212.

[0076] In some examples, one or more storage devices 256 may store an operating system, a store client 222, and one or more applications 226. Computing device 220 and / or the operating system may provide an execution environment for store client 222 which may send and receive information from computing system 210 to access the application store provided by computingDocket No.: 1333-900WO01 system 210. In some examples, store client 222 may communicate with the store module 212 of computing system 210 to request and receive application store listings including one or more application packages 216. Store client 222 may generate data to output a graphical user interface including one or more user interface elements that displays the application listings, such as through user interface device 228.

[0077] Store client 222 may communicate with store module 212 to obtain, from store module 212, adaptation file 218A that is compatible with ML model 224A installed at computing device 220. For example, store client 222 may transmit, to store module 212, an indication of the version (e.g., version 1.0) of ML model 224 installed at computing device 220. Store client 222 may receive, from store module 212, adaptation file 218A that store module 212 determines is compatible with ML model 224A.

[0078] Store client 222 may perform operations described herein using software, hardware, firmware, or a mixture of hardware, software, and firmware residing in and / or executing at computing device 220 to interact with store module 212 of computing system 210 as well as other functions associated with application packages 216, adaptation files 218, or applications 226, including installing application packages 216 and / or updating or deleting applications 226 and / or adaptation files 218. Computing device 220 may execute or apply applications 226, ML models 224, and adaptation files 218 with multiple processors or multiple devices, as virtual machines executing on underlying hardware, as one or more services of an operating system or computing platform, and / or as one or more executable programs at an application layer of a computing platform of computing device 220.

[0079] FIG.3 is an example of compatibility information, in accordance with one or more aspects of the present disclosure. FIG.3 is described below in the context of FIG. 2. In some examples, compatibility information 362 may constitute (e.g., be part of) an index 344, which may be stored at a repository, such as repository 214 of FIG. 2. Index 344 of FIG. 3 may be an example of index 244 of FIG. 2. Though shown with particular examples, index 344 may include various compatibility information 362, including compatibility information 362 with fewer, additional, and other types (e.g., columns) of information.

[0080] As can be seen from the example of FIG.3, index 344 may include units of compatibility information 362, shown as respective rows of index 344 of FIG.3. Compatibility information 362 may include indications of individual adaptation files 318A–318N (collectively, “adaptationDocket No.: 1333-900WO01 files 318”), indications of individual applications 316A–316N (collectively, “applications 316”), indications of individual ML models 324A–324N (collectively, “ML models 324”), and indications of compatible versions of ML models 324, or various subsets thereof. Adaptation files 318, applications 326, and ML models 324 of FIG.3 are described below as examples of adaptation files 218, applications 226, and ML models 224 of FIG.2.

[0081] Compatibility information 362 may indicate, for each respective adaptation file 318, the version of ML model 324 that is compatible with adaptation file 318. In the example of FIG.3 for instance, compatibility information 362 indicates adaptation file 318A is compatible with version 1.0 of ML model 324A and adaptation file 318B is compatible with version 1.1 of ML model 324A. Compatibility information 362 indicates that adaptation file 318A and adaptation file 318B have been provided, such as by the application developers, for use with application 326A.

[0082] In one example, store module 212 may receive, from computing device 220A, an indication that computing device 220A includes version 1.0 of ML model 324A. In response, store module 212 may determine from compatibility information 362 of index 344, that version 1.0 of ML model 324A is compatible with adaptation file 318A. Store module 212 may accordingly send adaptation file 318A to computing device 220A. In another example, store module 212 may receive, from computing device 220N, an indication that computing device 220N includes version 1.1 of ML model 324A. In response, store module 212 may determine from compatibility information 362 of index 344, that version 1.1 of ML model 324A is compatible with adaptation file 318B. Store module 212 may accordingly send adaptation file 318B, as opposed to adaptation file 318A, to computing device 220A.

[0083] In some examples, computing device 220 may send an indication of application 326 installed at computing device 220. Computing device 220 may send the indication of application 326 along with the indication of the version of ML model 324 installed at computing device 220. Store module 212 may receive these indications and determine from compatibility information 362 an adaptation file of adaptation files 318 to send to computing device 220 based on the application 326 and ML model 324 installed at computing device 220. In this manner, store module 212 may send adaptation files 318 based on applications 326 installed at computing device 220. For example, for computing device 220A, store module 212 may only identify adaptation file 318A or adaptation file 318B are compatible with ML model 324A if storeDocket No.: 1333-900WO01 module 212 receives an indication that application 326A is installed at computing device 220A (since, in this example, adaptation file 318A or adaptation file 318B are intended for use with application 326A as shown in compatibility information 362).

[0084] Some applications 326 may apply different ML models for different languages. Store module 212 may receive an indication of the language used by application 326 (e.g., English, Spanish), and determine adaptation file 318 for delivery based on the same. In the example of FIG.3 for instance, when application 326B is in English and computing device 220 includes version 1.0 or 1.1 of ML model 324B, store module 212 may determine adaptation file 318C or adaptation file 318D, respectively, should be delivered to computing device 220 and that adaptation file 318E should not be delivered. When application 326B is in Spanish, store module 212 may deliver adaptation file 318E, assuming computing device 220 includes version 1.1 of ML model 324C.

[0085] FIG.4 is a flowchart illustrating an example process for automated machine learning adaptation file delivery, in accordance with one or more aspects of the present disclosure. FIG.4 is described below in the context of FIG.2.

[0086] Computing system 210, such as through store module 212, may receive a plurality of adaptation files 218 (402). Each adaptation file from adaptation files 218 may include weights compatible with a version from a plurality of versions of one or more ML models 224. The weights from each of adaptation files 218 may be generated based on a fine tuning data set including one or more sample questions with one or more corresponding sample answers.

[0087] Computing system 210 may store, in a repository 214, for each adaptation file from adaptation files 218, a respective association between adaptation file 218A and a particular version from the plurality of versions of one or more ML models 224 that is compatible with adaptation file 218A (404). For example, computing system 210 may store, in repository 214, an index 244 including associations that indicate, for each adaptation file 218, the particular version of one or more ML models 224 that is compatible with adaptation file 218. These associations may constitute compatibility information, such as compatibility information 362 described above in connection with FIG.3.

[0088] In some examples, computing system 210 may publish a plurality of applications 226 for download, such as in the form of application packages 216, by a computing device 220. In suchDocket No.: 1333-900WO01 examples, the compatibility information may indicate, for each adaptation file 218, a corresponding application 226 that uses or includes adaptation file 218.

[0089] Computing system 210 may receive, from computing device 220, an indication of a version of a ML model 224A (e.g., version 1.0) installed at computing device 220 (406). Computing system 210 may identify, based on the respective associations stored in repository 214, a particular adaptation file 218A from adaptation files 218 that is associated with the particular version that corresponds to the version of ML model 224A installed at computing device 220 (408).

[0090] Computing system 210 may send, to computing device 220, particular adaptation file 218A (410). Sending particular adaptation file 218A to computing device 220 may cause computing device 220 to apply the weights from particular adaptation file 218A to ML model 224A installed at computing device 220 to fine tune ML model 224A installed at computing device 220 for a particular application 226A installed at computing device 220. In some examples, computing system 210 may send particular adaptation file 218A to computing device 220 along with particular application 226A that utilizes particular adaptation file 218A to fine tune ML model 224A installed at computing device 220. For instance, computing system 210 may include particular adaptation file 218A along with an application package 216A so as to provide particular adaptation file 218A during the download of application package 216A. In this manner, particular adaptation file 218A is available at computing device 220 to fine to ML model 224A upon installation of application package 216A at computing device 220.

[0091] Computing system 210 may send a second adaptation file 218B, such as to replace a first adaptation file 218A at computing device 220. For example, computing system 210 may send second adaptation file 218B when computing device 220 upgrades or updates to another version of ML model 224A (e.g., updates from version 1.0 to version 1.1). To illustrate, computing system 210 may receive, from computing device 220, an indication of a second version of ML model 224A installed at computing device 220. The second version of ML model 224A may be different than a first version of ML model 224A installed at computing device 220. For example, the second version may be version n+1 of ML model 224A and the first version may be version n of ML model 224A.

[0092] Similar to above, computing system 210 may identify, based on the respective associations stored in repository 214, a particular adaptation file 218B from adaptation files 218Docket No.: 1333-900WO01 that is associated with the particular version that corresponds to the second version of ML model 224A installed at computing device 220. Computing system 210 may send particular adaptation file 218B to computing device 220. In some examples, computing system 210 may send particular adaptation file 218B to computing device 220 to cause computing device 220 to apply the weights from particular adaptation file 218B to the ML model 224A installed at computing device 220 to fine tune ML model 224A installed at computing device 220 for particular application 226A installed at computing device 220.

[0093] In some examples, one or more processors 232 execute, store module 212, repository 214, or both to provide the functionality described above with respect to the flowchart of FIG.4.

[0094] Aspects of this disclosure include the following examples.

[0095] Example 1: A method includes receiving, by a computing system, a plurality of adaptation files, wherein each adaptation file from the plurality of adaptation files includes a plurality of weights compatible with a version from a plurality of versions of one or more machine learning models; storing, by the computing system and in a repository, for each adaptation file from the plurality of adaptation files, a respective association between the adaptation file and a particular version from the plurality of versions of the one or more machine learning models that is compatible with the adaptation file; receiving, by the computing system and from a computing device, an indication of a version of a machine learning model installed at the computing device; identifying, by the computing system and based on the respective associations stored in the repository, a particular adaptation file from the plurality of adaptation files that is associated with the particular version that corresponds to the version of the machine learning model installed at the computing device; and sending, by the computing system and to the computing device, the particular adaptation file.

[0096] Example 2: The method of example 1, wherein sending the particular adaptation file to the computing device causes the computing device to apply the plurality of weights from the particular adaptation file to the machine learning model installed at the computing device to fine tune the machine learning model installed at the computing device for a particular application installed at the computing device.

[0097] Example 3: The method of any of examples 1 and 2, wherein the version of the machine learning model is a first version of the machine learning model and the particular adaptation file is a first particular adaptation file, the method further includes receiving, by the computingDocket No.: 1333-900WO01 system and from the computing device, an indication of a second version of the machine learning model installed at the computing device, the second version being different from the first version; identifying, by the computing system and based on the respective associations stored in the repository, a second particular adaptation file from the plurality of adaptation files that is associated with the particular version that corresponds to the second version of the machine learning model installed at the computing device; and sending, by the computing system, the second particular adaptation file to the computing device.

[0098] Example 4: The method of example 3, wherein sending the second particular adaptation file to the computing device causes the computing device to apply the plurality of weights from the second particular adaptation file to the machine learning model installed at the computing device to fine tune the machine learning model installed at the computing device for a particular application installed at the computing device.

[0099] Example 5: The method of any of examples 1 through 4, wherein the plurality of weights from each of the plurality of adaptation files are generated based on a fine tuning data set including one or more sample questions with one or more corresponding sample answers.

[0100] Example 6: The method of any of examples 1 through 5, further comprising publishing, by the computing system, a plurality of applications for download by the computing device, wherein each of the plurality of adaptation files is associated with one or more of the plurality of applications.

[0101] Example 7: The method of any of examples 1 through 6, wherein sending the particular adaptation file to the computing device comprises sending the particular adaptation file to the computing device along with a corresponding application that utilizes the particular adaptation file to fine tune the machine learning model installed at the computing device.

[0102] Example 8: A computing system includes a memory that stores instructions; and one or more processors that execute the instructions to: receive a plurality of adaptation files, wherein each adaptation file from the plurality of adaptation files includes a plurality of weights compatible with a version from a plurality of versions of one or more machine learning models; store, in a repository, for each adaptation file from the plurality of adaptation files, a respective association between the adaptation file and a particular version from the plurality of versions of the one or more machine learning models that is compatible with the adaptation file; receive, from a computing device, an indication of a version of a machine learning model installed at theDocket No.: 1333-900WO01 computing device; identify, based on the respective associations stored in the repository, a particular adaptation file from the plurality of adaptation files that is associated with the particular version that corresponds to the version of the machine learning model installed at the computing device; and send, to the computing device, the particular adaptation file.

[0103] Example 9: The computing system of example 8, wherein sending the particular adaptation file to the computing device causes the computing device to apply the plurality of weights from the particular adaptation file to the machine learning model installed at the computing device to fine tune the machine learning model installed at the computing device for a particular application installed at the computing device.

[0104] Example 10: The computing system of any of examples 8 and 9, wherein: the version of the machine learning model is a first version of the machine learning model and the particular adaptation file is a first particular adaptation file; and the one or more processors execute the instructions to: receive, from the computing device, an indication of a second version of the machine learning model installed at the computing device, the second version being different from the first version; identify, based on the respective associations stored in the repository, a second particular adaptation file from the plurality of adaptation files that is associated with the particular version that corresponds to the second version of the machine learning model installed at the computing device; and send the second particular adaptation file to the computing device.

[0105] Example 11: The computing system of example 10, wherein sending the second particular adaptation file to the computing device causes the computing device to apply the plurality of weights from the second particular adaptation file to the machine learning model installed at the computing device to fine tune the machine learning model installed at the computing device for a particular application installed at the computing device.

[0106] Example 12: The computing system of any of examples 8 through 11, wherein the plurality of weights from each of the plurality of adaptation files are generated based on a fine tuning data set including one or more sample questions with one or more corresponding sample answers.

[0107] Example 13: The computing system of any of examples 8 through 12, wherein the one or more processors execute the instructions to publish a plurality of applications for download by the computing device, wherein each of the plurality of adaptation files is associated with one or more of the plurality of applications.Docket No.: 1333-900WO01

[0108] Example 14: The computing system of any of examples 8 through 13, wherein to send the particular adaptation file to the computing device the one or more processors execute the instructions to send the particular adaptation file to the computing device along with a corresponding application that utilizes the particular adaptation file to fine tune the machine learning model installed at the computing device.

[0109] Example 15: Non-transitory computer-readable storage media comprising instructions, that when executed by one or more processors, cause the one or more processors to receive a plurality of adaptation files, wherein each adaptation file from the plurality of adaptation files includes a plurality of weights compatible with a version from a plurality of versions of one or more machine learning models; store, in a repository, for each adaptation file from the plurality of adaptation files, a respective association between the adaptation file and a particular version from the plurality of versions of the one or more machine learning models that is compatible with the adaptation file; receive, from a computing device, an indication of a version of a machine learning model installed at the computing device; identify, based on the respective associations stored in the repository, a particular adaptation file from the plurality of adaptation files that is associated with the particular version that corresponds to the version of the machine learning model installed at the computing device; and send, to the computing device, the particular adaptation file.

[0110] Example 16: The non-transitory computer-readable storage media of example 15, wherein to send the particular adaptation file to the computing device the instructions, when executed by one or more processors, cause the one or more processors to send the particular adaptation file to the computing device along with a corresponding application that utilizes the particular adaptation file to fine tune the machine learning model installed at the computing device.

[0111] Example 17: The non-transitory computer-readable storage media of any of examples 15 and 16, wherein: the version of the machine learning model is a first version of the machine learning model and the particular adaptation file is a first particular adaptation file; and the instructions, when executed by one or more processors, cause the one or more processors to: receive, from the computing device, an indication of a second version of the machine learning model installed at the computing device, the second version being different from the first version; identify, based on the respective associations stored in the repository, a secondDocket No.: 1333-900WO01 particular adaptation file from the plurality of adaptation files that is associated with the particular version that corresponds to the second version of the machine learning model installed at the computing device; and send the second particular adaptation file to the computing device.

[0112] Example 18: The non-transitory computer-readable storage media of example 17, wherein sending the second particular adaptation file to the computing device causes the computing device to apply the plurality of weights from the second particular adaptation file to the machine learning model installed at the computing device to fine tune the machine learning model installed at the computing device for a particular application installed at the computing device.

[0113] Example 19: The non-transitory computer-readable storage media of any of examples 15 through 18, wherein the plurality of weights from each of the plurality of adaptation files are generated based on a fine tuning data set including one or more sample questions with one or more corresponding sample answers.

[0114] Example 20: The non-transitory computer-readable storage media of any of examples 15 through 19, wherein the instructions, when executed by one or more processors, cause the one or more processors to publish a plurality of applications for download by the computing device, wherein each of the plurality of adaptation files is associated with one or more of the plurality of applications.

[0115] Example 21: The non-transitory computer-readable storage media of any of examples 15 through 20, wherein the instructions, when executed by one or more processors, cause the one or more processors to: install an application that utilizes the machine learning model installed at the computing device; and send the particular adaptation file to the computing device responsive to installation of the application.

[0116] Example 22: A computer-program product that includes instructions that cause one or more processors to perform any combination of the methods of examples 1-7.

[0117] In one or more examples, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted over, as one or more instructions or code, a computer-readable medium and executed by a hardware-based processing unit. Computer-readable media may include computer-readable storage media, which corresponds to a tangible medium such as data storage media, or communication media including any medium that facilitates transfer of aDocket No.: 1333-900WO01 computer program from one place to another, e.g., according to a communication protocol. In this manner, computer-readable media generally may correspond to (1) tangible computer- readable storage media, which is non-transitory or (2) a communication medium such as a signal or carrier wave. Data storage media may be any available media that may be accessed by one or more computers or one or more processors to retrieve instructions, code and / or data structures for implementation of the techniques described in this disclosure. A computer program product may include a computer-readable medium.

[0118] By way of example, and not limitation, such computer-readable storage media may comprise RAM, ROM, EEPROM, optical disk storage, magnetic disk storage, or other magnetic storage devices, flash memory, or any other storage medium that may be used to store desired program code in the form of instructions or data structures and that may be accessed by a computer. Also, any connection is properly termed a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. It should be understood, however, that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transient media, but are instead directed to non-transient, tangible storage media. Disk, as used herein, includes magnetic and optical disks, where data may be respectively reproduced magnetically or optically with lasers. Combinations of the above should also be included within the scope of computer- readable media.

[0119] Instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Accordingly, the term "processor," as used herein may refer to any of the foregoing structures or any other structure suitable for implementation of the techniques described herein. In addition, in some aspects, the functionality described herein may be provided within dedicated hardware and / or software modules. Also, the techniques could be fully implemented in one or more circuits or logic elements.Docket No.: 1333-900WO01

[0120] The techniques of this disclosure may be implemented in a wide variety of devices or apparatuses, including a wireless handset, an integrated circuit (IC) or a set of ICs (e.g., a chip set). Various components, modules, or units are described in this disclosure to emphasize functional aspects of devices configured to perform the disclosed techniques, but do not necessarily require realization by different hardware units. Rather, as described above, various units may be combined in a hardware unit or provided by a collection of intraoperative hardware units, including one or more processors as described above, in conjunction with suitable software and / or firmware.

[0121] It is to be recognized that, depending on the example, certain acts or events of any of the methods described herein may be performed in a different sequence, may be added, merged, or left out altogether (e.g., not all described acts or events are necessary for the practice of the method). Moreover, in certain embodiments, acts or events may be performed concurrently, e.g., through multi-threaded processing, interrupt processing, or multiple processors, rather than sequentially.

[0122] In some examples, a computer-readable storage medium comprises a non-transitory medium. The term "non-transitory" indicates that the storage medium is not embodied in a carrier wave or a propagated signal. In certain examples, a non-transitory storage medium may store data that can, over time, change (e.g., in RAM or cache).

[0123] Various examples have been described. These and other examples are within the scope of the following claims.

Claims

Docket No.: 1333-900WO01 CLAIMS:

1. A method comprising: receiving, by a computing system, a plurality of adaptation files, wherein each adaptation file from the plurality of adaptation files includes a plurality of weights compatible with a version from a plurality of versions of one or more machine learning models; storing, by the computing system and in a repository, for each adaptation file from the plurality of adaptation files, a respective association between the adaptation file and a particular version from the plurality of versions of the one or more machine learning models that is compatible with the adaptation file; receiving, by the computing system and from a computing device, an indication of a version of a machine learning model installed at the computing device; identifying, by the computing system and based on the respective associations stored in the repository, a particular adaptation file from the plurality of adaptation files that is associated with the particular version that corresponds to the version of the machine learning model installed at the computing device; and sending, by the computing system and to the computing device, the particular adaptation file.

2. The method of claim 1, wherein sending the particular adaptation file to the computing device causes the computing device to apply the plurality of weights from the particular adaptation file to the machine learning model installed at the computing device to fine tune the machine learning model installed at the computing device for a particular application installed at the computing device.Docket No.: 1333-900WO01 3. The method of any of claims 1 or 2, wherein the version of the machine learning model is a first version of the machine learning model and the particular adaptation file is a first particular adaptation file, the method further comprising: receiving, by the computing system and from the computing device, an indication of a second version of the machine learning model installed at the computing device, the second version being different from the first version; identifying, by the computing system and based on the respective associations stored in the repository, a second particular adaptation file from the plurality of adaptation files that is associated with the particular version that corresponds to the second version of the machine learning model installed at the computing device; and sending, by the computing system, the second particular adaptation file to the computing device.

4. The method of claim 3, wherein sending the second particular adaptation file to the computing device causes the computing device to apply the plurality of weights from the second particular adaptation file to the machine learning model installed at the computing device to fine tune the machine learning model installed at the computing device for a particular application installed at the computing device.

5. The method of any of claims 1-4, wherein the plurality of weights from each of the plurality of adaptation files are generated based on a fine tuning data set including one or more sample questions with one or more corresponding sample answers.

6. The method of any of claims 1-5, further comprising publishing, by the computing system, a plurality of applications for download by the computing device, wherein each of the plurality of adaptation files is associated with one or more of the plurality of applications.Docket No.: 1333-900WO01 7. The method of any of claims 1-6, wherein sending the particular adaptation file to the computing device comprises sending the particular adaptation file to the computing device along with a corresponding application that utilizes the particular adaptation file to fine tune the machine learning model installed at the computing device.

8. A computing system comprising: a memory that stores instructions; and one or more processors that execute the instructions to: receive a plurality of adaptation files, wherein each adaptation file from the plurality of adaptation files includes a plurality of weights compatible with a version from a plurality of versions of one or more machine learning models; store, in a repository, for each adaptation file from the plurality of adaptation files, a respective association between the adaptation file and a particular version from the plurality of versions of the one or more machine learning models that is compatible with the adaptation file; receive, from a computing device, an indication of a version of a machine learning model installed at the computing device; identify, based on the respective associations stored in the repository, a particular adaptation file from the plurality of adaptation files that is associated with the particular version that corresponds to the version of the machine learning model installed at the computing device; and send, to the computing device, the particular adaptation file.

9. The computing system of claim 8, wherein sending the particular adaptation file to the computing device causes the computing device to apply the plurality of weights from the particular adaptation file to the machine learning model installed at the computing device to fine tune the machine learning model installed at the computing device for a particular application installed at the computing device.Docket No.: 1333-900WO01 10. The computing system of any of claims 8 or 9, wherein: the version of the machine learning model is a first version of the machine learning model and the particular adaptation file is a first particular adaptation file; and the one or more processors execute the instructions to: receive, from the computing device, an indication of a second version of the machine learning model installed at the computing device, the second version being different from the first version; identify, based on the respective associations stored in the repository, a second particular adaptation file from the plurality of adaptation files that is associated with the particular version that corresponds to the second version of the machine learning model installed at the computing device; and send the second particular adaptation file to the computing device.

11. The computing system of claim 10, wherein sending the second particular adaptation file to the computing device causes the computing device to apply the plurality of weights from the second particular adaptation file to the machine learning model installed at the computing device to fine tune the machine learning model installed at the computing device for a particular application installed at the computing device.

12. The computing system of any of claims 8-11, wherein the plurality of weights from each of the plurality of adaptation files are generated based on a fine tuning data set including one or more sample questions with one or more corresponding sample answers.

13. The computing system of any of claims 8-12, wherein the one or more processors execute the instructions to publish a plurality of applications for download by the computing device, wherein each of the plurality of adaptation files is associated with one or more of the plurality of applications.Docket No.: 1333-900WO01 14. The computing system of any of claims 8-13, wherein to send the particular adaptation file to the computing device the one or more processors execute the instructions to send the particular adaptation file to the computing device along with a corresponding application that utilizes the particular adaptation file to fine tune the machine learning model installed at the computing device.

15. Non-transitory computer-readable storage media comprising instructions, that when executed by one or more processors, cause the one or more processors to perform any combination of the methods of claims 1-7.

16. A computer-program product that includes instructions that cause one or more processors to perform any combination of the methods of claims 1-7.

Images