Customized on-device network based on a shared foundation network

Abstract

A method includes hosting, by a device, a foundation network that is shared by a plurality of applications. The method also includes receiving, by the device, a notification from a particular application of the plurality of applications. The notification indicates that the particular application will use the foundation network. The method also includes retrieving, by the device, a subnetwork associated with the particular application in response to receiving the notification from the particular application. The method also includes generating, by the device, a customized network for the particular application at least in part by inserting the subnetwork into the foundation network.

Description

CUSTOMIZED ON-DEVICE NETWORK BASED ON A SHARED FOUNDATIONNETWORKCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] The present application claims the benefit of U.S. Provisional Patent Application No. 63 / 519,985, filed August 16, 2023. The contents of which are hereby incorporated by reference in their entirety.BACKGROUND

[0002] Large generative foundation models may be used by a device, such as a mobile phone, to perform multiple different tasks and functions. As non-limiting examples, large generative foundation models can be used to perform an open-domain question and answer task, an open-domain text-to-image generation task, etc. Compared to classic neural network models, large generative foundation models typically offer improved performance and possess unique capabilities, such as improved generalization, in-context learning capabilities, an ability to interact via natural interfaces (text, image, speech, video), etc.

[0003] Running large generative foundation models on the device includes several benefits. For example, by running large generative foundation models on the device, as opposed to using an online large generative foundation model, the large generative foundation model can function when the device is offline, model behaviors can be personalized to a user of the device, and client data remains offline, which leads to increased privacy. However, running large generative foundation models on the device typically requires the device to have increased storage, memory, and computing resources. For example, if multiple applications on the device require a separate large generative foundation model, storing multiple large generative foundation models on the device can occupy a large amount of storage space (e.g., occupy a large amount of memory). Additionally, a large amount of computing resources can be used to run different large generative foundation models for each application, which in turn can utilize a large amount of power at the device.SUMMARY

[0004] According to the techniques described herein, a foundation model can be shared among a plurality of applications and can have interfaces to enable each application to customize the foundation model for its own use. For example, the foundation model can be divided into a shared foundation neural network (hereby network) and a set of interfaces where sub-neural networks (hereby sub-network) can be inserted for updates, enhancements, and customizations. To illustrate, the shared foundation network may include model weights that are shared by a plurality of applications. To customize the shared foundation network for a particular application of the plurality of applications, a particular subnetwork associated with the particular application can be inserted or plugged into the shared foundation network. The particular subnetwork can include network weights that are specific to the particular application such that, when inserted into the shared foundation network, the particular subnetwork changes the shared foundation network into a customized network for the particular application.

[0005] In a first example embodiment, a method includes hosting, by a device, a foundation network that is shared by a plurality of applications. The method also includes receiving, by the device, a notification from a particular application of the plurality of applications. The notification indicates that the particular application will use the foundation network. The method also includes retrieving, by the device, a subnetwork associated with the particular application in response to receiving the notification from the particular application. The method also includes generating, by the device, a customized network for the particular application at least in part by inserting the subnetwork into the foundation network. It should be appreciated that this method ensures the compatibility, safety, authenticity, and freshness of both the foundation network and the subnetwork.

[0006] In a second example, a device includes a memory and a processor coupled to the memory. The processor is configured to host a foundation network that is shared by a plurality of applications. The processor is also configured to receive a notification from a particular application of the plurality of applications. The notification indicates that the particular application will use the foundation network. The processor is also configured to retrieve a subnetwork associated with the particular application in response to receiving the notification from the particular application. The processor is also configured to generate a customized network for the particular application at least in part by inserting the subnetwork into the foundation network.

[0007] In a third example, a non-transitory computer-readable medium includes instructions that, when executed by a processor within a device, cause the processor to performoperations. The operations include hosting a foundation network that is shared by a plurality of applications. The operations also include receiving a notification from a particular application of the plurality of applications. The notification indicates that the particular application will use the foundation network. The operations also include retrieving a subnetwork associated with the particular application in response to receiving the notification from the particular application. The operations further include generating a customized network for the particular application at least in part by inserting the subnetwork into the foundation network.

[0008] In a fourth example embodiment, a system may include various means for carrying out each of the operations of the first example embodiment.

[0009] In a fifth example embodiment, a method includes generating, by a device, a customized network for a particular application based on a foundation network shared by a plurality of applications and a subnetwork associated with the particular application. The subnetwork corresponds to a certified network. Running the customized network on the device ensures the authenticity and integrity of the foundation network and the subnetwork at runtime. At runtime, data used by the customized network is secure and private data is not leaked to unauthorized applications running on the device. At runtime, security measurements are implemented to avoid the foundation network and the subnetwork from being exported, copied, or used in unauthorized environments.

[0010] In a sixth example embodiment, a method includes generating, by a device, a customized network for a particular application based on a foundation network shared by a plurality of applications and a subnetwork associated with the particular application. The subnetwork can be aggregated, certified, distributed, and updated over the internet.

[0011] These, as well as other embodiments, aspects, advantages, and alternatives, will become apparent to those of ordinary skill in the art by reading the following detailed description, with reference where appropriate to the accompanying drawings. Further, this summary and other descriptions and figures provided herein are intended to illustrate embodiments by way of example only and, as such, that numerous variations are possible. For instance, structural elements and process steps can be rearranged, combined, distributed, eliminated, or otherwise changed, while remaining within the scope of the embodiments as claimed.BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 illustrates a device operable to generate a first customized network for a first application using a shared foundation network, in accordance with examples described herein.

[0013] Figure 2 illustrates the device operable to generate a second customized network for a second application using the shared network, in accordance with examples described herein.

[0014] Figure 3 illustrates an example of a process for generating a customized network for an application using a shared foundation network, in accordance with examples described herein.

[0015] Figure 4 illustrates an example of a process for enabling an on-device foundation model, in accordance with examples described herein.

[0016] Figure 5 illustrates an example of another process for enabling an on-device foundation model, in accordance with examples described herein.

[0017] Figure 6 illustrates an example of a process for generating a customized network for an application by modifying a shared foundation network using an embedding lookup matrix, in accordance with examples described herein.

[0018] Figure 7 illustrates an example of a system including a device that is operable to retrieve subnetworks from a store of subnetworks, in accordance with examples described herein.

[0019] Figure 8 is a diagram illustrating training and inference phases of a machinelearning model, in accordance with examples described herein.

[0020] Figure 9 illustrates a flow chart, in accordance with examples described herein.DETAILED DESCRIPTION

[0021] Example methods, devices, and systems are described herein. It should be understood that the words “example” and “exemplary” are used herein to mean “serving as an example, instance, or illustration.” Any embodiment or feature described herein as being an “example,” “exemplary,” and / or “illustrative” is not necessarily to be construed as preferred or advantageous over other embodiments or features unless stated as such. Thus, other embodiments can be utilized and other changes can be made without departing from the scope of the subject matter presented herein.

[0022] Accordingly, the example embodiments described herein are not meant to be limiting. It will be readily understood that the aspects of the present disclosure, as generallydescribed herein, and illustrated in the figures, can be arranged, substituted, combined, separated, and designed in a wide variety of different configurations.

[0023] Further, unless context suggests otherwise, the features illustrated in each of the figures may be used in combination with one another. Thus, the figures should be generally viewed as component aspects of one or more overall embodiments, with the understanding that not all illustrated features are necessary for each embodiment.

[0024] Particular embodiments are described herein with reference to the drawings. In the description, common features are designated by common reference numbers throughout the drawings. In some figures, multiple instances of a particular type of feature are used. Although these features are physically and / or logically distinct, the same reference number is used for each, and the different instances are distinguished by addition of a letter to the reference number. When the features as a group or a type are referred to herein (e.g., when no particular one of the features is being referenced), the reference number is used without a distinguishing letter. However, when one particular feature of multiple features of the same type is referred to herein, the reference number is used with the distinguishing letter. For example, referring to Figure 1, multiple subnetworks are illustrated and associated with reference numbers 140A, 140B, and 140C. When referring to a particular one of these subnetworks, such as the subnetwork 140A, the distinguishing letter “A” is used. However, when referring to any arbitrary one of these subnetworks or to these subnetworks as a group, the reference number 140 is used without a distinguishing letter.

[0025] Additionally, any enumeration of elements, blocks, or steps in this specification or the claims is for purposes of clarity. Thus, such enumeration should not be interpreted to require or imply that these elements, blocks, or steps adhere to a particular arrangement or are carried out in a particular order. Unless otherwise noted, figures are not drawn to scale.I. Overview

[0026] The techniques described herein enable developers to reuse model weights of a large foundation model on a device without sacrificing the foundation model’s performance or customization ability. Reusing the model weights of the large foundation model improves storage capacity at the device.

[0027] For example, according to the techniques described herein, the foundation model can be divided into a shared foundation network (e.g., a large generalized network) and a set of interfaces where subnetworks (e.g., smaller application-specific networks) can be inserted (or plugged) for updates, enhancements, and customizations. To illustrate, a plurality of applications (e.g., system applications and developer-created applications) can sharefoundation model weights that are the basis of the shared foundation network. To customize the shared foundation network for a first application of the plurality of applications, a first subnetwork associated with the first application can be inserted or plugged into the shared foundation network. The first subnetwork can include network weights that are specific to the first application such that, when inserted into the shared foundation network, the first subnetwork “fine-tunes” the shared foundation network into a customized network for the first application. Similarly, to customize the shared foundation network for a second application of the plurality of applications, a second subnetwork associated with the second application can be inserted or plugged into the shared foundation network. The second subnetwork can include network weights that are specific to the second application such that, when inserted into the shared foundation network, the second subnetwork fine-tunes the shared foundation network into a customized network for the second application.

[0028] Thus, as described above, one aspect of the techniques described herein is model weight sharing. In some implementations, the shared foundation network may account for a relatively large amount of the model weights in a customized network for any given application. As a non-limiting example, in a customized network for any given application, a majority of model weights (i.e., 99% or more) may stay in the shared foundation network, while a minority of the model weights (i.e., approximately 0.1% ~ 1.0%) may stay in a subnetwork tailored to the given application. The model weights in the subnetwork may be used for differentiating features specific to the given application. Thus, when the given application invokes a call to the shared foundation network, the model weights of the corresponding subnetwork can be dynamically loaded and combined with the shared foundation network to form the customized network (e.g., a complete model) for the given application. It should be understood that, in some implementations, any given application can include multiple subnetworks (for different tasks) that are loaded (e.g., plugged or inserted) into the shared foundation network.

[0029] Another aspect of the techniques described herein is parameter-efficient model customization. For example, model weights of a subnetwork associated with a given application can be obtained by retraining or fine-tuning foundation models, such as the shared foundation network, on domain-specific samples unique to the given application via a parameter-efficient training process. During the training process, the model weights of the foundation models are frozen and model weights of the subnetwork are updated. Updating the model weights of the subnetwork while the model weights of the foundation models are frozen can achieve comparable performance and generalization as updating models weights of thesubnetwork and model weights of the foundation models; however, only updating the model weights (e.g., parameters) of the subnetwork is more efficient than whole model fine tuning.

[0030] Different interfaces and implementations can be used for inserting a subnetwork into the shared foundation network to generate the customized network at the device for the given application. According to the first implementation of inserting the subnetwork into the shared foundation network, a lightweight parallel branch may be added to a frozen computation graph of the shared foundation network, and the subnetwork can be inserted into the lightweight parallel branch. For example, a Low Rank Adaptation (LoRA) protocol can be used to insert the subnetwork into the shared foundation network according to the first implementation. According to a second implementation of inserting the subnetwork into the shared foundation network, a lightweight sequential branch may be added to a frozen computation graph of the shared foundation network, and the subnetwork can be inserted into the lightweight sequential branch. For example, an adapter protocol or a hyper-network protocol can be used to insert the subnetwork into the shared foundation network according to the second implementation. According to a third implementation of inserting the subnetwork into the shared foundation network, an embedding lookup matrix associated with the shared foundation network may be resized and updated based on the subnetwork. For example, soft prompting can be used to insert the subnetwork into the shared foundation network according to the third implementation.

[0031] According to some aspects of the techniques described herein, a digital signature may be used to ensure that only subnetworks that are compatible with the shared foundation network are loaded into the shared foundation network. In some embodiments, the digital signature for subnetworks may be issued by a creator of an application programming interface (API) that loads the subnetworks into the shared foundation network. By adding the digital signatures to the subnetworks, only subnetworks that are signed with a compatible key can be loaded into the shared foundation model. Thus, by using a digital signature to ensure that compatible subnetworks are loaded into the shared foundation network, incompatible subnetworks and otherwise compatible subnetworks that are maliciously modified can be detected and prevented from being loaded into the shared foundation network. In some embodiments, a store of subnetworks can be used to distribute new subnetworks to the device or upgrade existing subnetworks at the device. Thus, in these embodiments, the store can provide digital signatures to the device to ensure that the subnetworks are compatible with the shared foundation network prior to loading the subnetworks into the shared foundation model.

[0032] Thus, the techniques described herein present system-level and infrastructural - level support for a device to efficiently run on-device large foundation models while easing storage capacity requirements (e.g., saving memory), easing computing resource requirements (e.g., saving computing resources), reducing power usage associated with the computing resources, and improving safety and security. In particular, by using the shared foundation network to facilitate model weight sharing across different applications, storage capacity and computing resource requirements can be eased because each application uses shared model weights that typically account for a majority of the model weights needed to run the application. Storing the shared foundation network on the device (e.g., an “on-device” foundation network) reduces security and privacy concerns as user data does not have to be transmitted to a remote foundation network at a server.

[0033] As described above, the techniques described herein utilize model weight sharing across different applications and parameter-efficient model customization to efficiently run foundation models on the device. There are at least three reasons for utilizing parameterefficient model customization and model weight sharing across different applications to support on-device foundation models. First, training an efficient large foundation model may require internet-scale training data acquisition, curation, and cleaning. Thus, it is typically not practical to train a new large foundation model for every machine learning use case. Second, standard models may not satisfy specific use requirements for applications. For example, an application developer may wish to incorporate private domain knowledge (e.g., private documents and images) or domain-specific enhancements into a model. The private domain knowledge and the domain-specific enhancements may not be included in a standard model and may require parameter-efficient model customization. Third, large foundation models may achieve excellent generalization with a small amount of fine-tuning data. As a result, an application developer can customize the behavior of the foundation models without having to retrain the foundation models from scratch. Fine-tuning requires a relatively small amount of labeled data, which makes it feasible to deploy customized models on the device without duplicating data storage across applications from different owners.

[0034] The shared foundation networks, the subnetworks, and the runtime environment to load such networks are executed on-device in an isolated, secure environment. This secure environment keeps the user’s information on-device and private, even from other applications running on the same device. The secure environment also ensures the compatibility, safety, authenticity, and freshness of both foundation networks and subnetworks. The secureenvironment also prevents the foundation networks and subnetworks from being copied, modified for unauthorized uses.

[0035] The secure on-device environment can be used to prevent features provided by the foundation network and subnetworks, inside the secure environment, from having direct access to the network. Instead, features communicate over a small set of purposeful open- source APIs, which strips out identifying information and uses a set of privacy technologies, to protect users’ private data.

[0036] Although the shared foundation model is a local “on-device” model to support enhanced privacy, in some scenarios, user data can be transmitted to another device or server during use of an application. With respect to embodiments that include using a machine learning model, interactions by the device with cloud-based servers, or otherwise involve sharing information with other devices, a user may be provided with controls allowing the user to make an election as to both if and when systems, programs, or features described herein may enable collection of user information (e.g., information about a user’s social network, social actions, or activities, profession, a user’s preferences, or a user’s current location), and if the user is sent content or communications from a server. In addition, certain data may be treated in one or more ways before it is stored or used, so that personally identifiable information is removed. For example, a user’s identity may be treated so that no personally identifiable information can be determined for the user, or a user’s geographic location may be generalized where location information is obtained (such as to a city, ZIP code, or state level), so that a particular location of a user cannot be determined. Thus, the user may have control over what information is collected about the user, how that information is used, and what information is provided to the user.II. Example Device

[0037] Figure 1 illustrates a device 100 operable to generate a first customized network for a first application using a shared foundation network, in accordance with examples described herein. The device 100 includes a processor 102, a memory 104, and an application programming interface (API) 106. The memory 104 can correspond to a non-transitory computer-readable medium that includes instructions 105 executable by the processor 102 to perform the operations described herein.

[0038] The device 100 can correspond to any user device, such as a laptop computer, a desktop computer, a mobile phone, a tablet, a portable computing device, etc. Although the device 100 depicts three components (e.g., the processor 102, the memory 104, and the API 106), it should be understood that in other embodiments, the device 100 can include additionalcomponents. For example, in other embodiments, the device 100 can include a user interface, a display screen, a keypad, a mouse, a modem, additional processors, additional memories and / or storage devices, etc.

[0039] In one implementation, the memory 104 also stores a plurality of subnetworks 140. For example, in Figure 1, the memory 104 stores a subnetwork 140A, a subnetwork MOB, and a subnetwork 140C. Although three subnetworks 140 are illustrated, in other implementations, the memory 104 can store additional (or fewer) subnetworks 140. As a nonlimiting example, in some implementations, the memory 104 can include thirty (30) subnetworks 140. As another non-limiting example, in some implementations, the memory 104 can include two (2) subnetworks 140. As described below, each subnetwork 140 can be associated with an application 120 and can be used to generate a customized network 150 for the application 120.

[0040] The API 106 can be configured to facilitate communication between the processor 102 and a plurality of different applications 120. For example, the API 106 can facilitate communication between an application 120A and the processor 102. In particular, the application 120 A can share data with the processor 102 through the API 106, and the processor 102 can share data with the application 120A through the API 106. Similarly, the API 106 can facilitate communication between an application 120B and the processor 102. In particular, the application 120B can share data with the processor 102 through the API 106, and the processor 102 can share data with the application 120B through the API 106. Additionally, the API 106 can facilitate communication between an application 120C and the processor 102. In particular, the application 120C can share data with the processor 102 through the API 106, and the processor 102 can share data with the application 120C through the API 106. Although three applications 120 are illustrated, in other implementations, additional (or fewer) applications 120 can communicate with the processor 102 via the API 106. As a non-limiting example, in some implementations, forty (40) applications 120 can communicate with the processor 102 via the API.

[0041] In the illustration of Figure 1, the subnetwork 140A and the subnetwork 140B are associated with the application 120A, and the subnetwork 140C is associated with the application 120B. Thus, in some implementations, multiple subnetworks 140 can be associated with a single application 120. However, in other implementations, a single subnetwork 140 can be associated with a single application 120. Additionally, although the subnetworks 140 are depicted as being stored in the memory 104, in some implementations, the applications 120 can cache corresponding subnetworks 140. As a non-limiting example, in someimplementations, the application 120 A can cache the subnetwork 140 A and the subnetwork 140B, and the application 120B can cache the subnetwork 140C. In a particular implementation, the subnetwork(s) 140 for a particular application 120 can be downloaded when the application 120 is installed at the device 100. For example, in response to installing the application 120 A at the device 100, the subnetworks 140 A, MOB can be downloaded or retrieved from a store of subnetworks that includes subnetworks 140 for different applications 120, as described in greater detail with respect to Figure 7.

[0042] The processor 102 can be configured to host a foundation network 130. The foundation network 130 corresponds to a large generative foundation model that is local to the device 100 (e.g., an “on-device” model). For example, the foundation network 130 is isolated from a server and other devices. By hosting the isolated foundation network 130 on the device 100, as opposed to using an online generative foundation model, the foundation network 130 can function when the device 100 is offline, model behaviors can be personalized, and client data can remain offline, which leads to increased privacy. The foundation network 130 includes a plurality of network weights 132 that, when compared to classical neural networks, provide (i) better generalization, (ii) increased performance, (iii) a more natural interface, (iv) in-context learning, etc. The foundation network 130 is shared by the plurality of applications 120. For example, the network weights 132 (e.g., foundation model weights) of the foundation network 130 are applicable to each application 120A-120C in the plurality of applications 120.

[0043] In Figure 1, the processor 102 can be configured to receive a notification from the application 120A through the API 106. The notification can indicate that the application 120A will use the foundation network 130. For example, receiving the notification from the application 120A can include receiving a call to the foundation network 130 from the application 120A through the API 106. In response to receiving the notification through the API 106, the processor 102 can be configured to retrieve the subnetworks 140 A, MOB associated with the application 120 A. For example, in the embodiment where the subnetworks 140 A, MOB are stored in the memory 104, the processor 102 can retrieve the subnetworks 140 A, MOB from the memory 104 in response to receiving the notification that the associated application 120A will use the foundation network 130. As another example, in the embodiment where the subnetworks 140 A, MOB are cached with the application 120 A, the processor 102 can retrieve the subnetworks 140A, MOB from the application 120A via the API 106.

[0044] The processor 102 includes a customized network generator 110. According to one implementation, the customized network generator 110 can be implemented using dedicated hardware. For example, the customized network generator 110 can be implementedusing an application-specific integrated circuit (ASIC) or a field-programmable gate array (FPGA). According to another implementation, the customized network generator 110 can be implemented using software. For example, the processor 102 can implement the customized network generator 110 by executing the instructions 105 stored in the memory 104. The customized network generator 110 can be configured to generate a customized network 150A for the application 120 A by inserting the subnetworks 140 A, MOB associated with the application 120A into the foundation network 130.

[0045] According to a first implementation, inserting one of the subnetworks 140 A, MOB into the foundation network 130 includes adding a parallel branch to a computation graph of the foundation network 130 and inserting one of the subnetworks 140 A, MOB into the parallel branch. For example, according to the first implementation, one of the subnetworks 140 A, MOB can be inserted into the foundation network 130 according to a subnetwork protocol. The subnetwork protocol defines a) subnetwork architecture; b) compatible foundation networks, loading location, and actual subnetwork parameters. Sample subnetwork architectures include soft prompts, LoRA, I A3, BitFit etc. According to a second implementation, inserting one of the subnetworks 140 A, MOB into the foundation network 130 includes adding a sequential branch to a computation graph of the foundation network 130 and inserting one of the subnetworks 140 A, MOB into the sequential branch. For example, according to the second implementation, one of the subnetworks 140 A, MOB can be inserted into the foundation network 130 according to an adapter protocol or a hyper-network protocol. According to a third implementation, inserting one of the subnetworks 140 A, MOB into the foundation network 130 includes resizing an embedding lookup matrix associated with the foundation network 130 and updating the embedding lookup matrix based on one of the subnetworks 140 A, MOB after resizing the embedding lookup matrix.

[0046] According to one implementation, prior to inserting the subnetworks 140 A, MOB into the foundation network 130, the processor 102 can be configured to determine whether digital signatures 144 A, 144B associated with the subnetworks 140 A, MOB, respectively, indicate that the subnetworks 140 A, MOB are compatible with the foundation network 130. For example, the subnetwork 140A includes a digital signature 144A, and the subnetwork MOB includes a digital signature 144B. As described in greater detail with respect to Figure 7, the digital signatures 144A, 144B may be used to ensure that only subnetworks that are compatible with the foundation network 130 are loaded into the foundation network 130. For example, the digital signature 144A can indicate whether the subnetwork 140A is compatible with the foundation network 130, and the digital signature 144B can indicatewhether the subnetwork MOB is compatible with the foundation network 130. In some embodiments, the digital signatures 144 A, 144B for the subnetworks 140 A, MOB may be issued by a creator of API 106 that loads the subnetworks 140A, MOB into the foundation network 130. The subnetworks 140A, MOB can be inserted into the foundation network 130 in response to a determination that the digital signatures 144 A, 144B indicate that the subnetworks 140A, MOB, respectively, are compatible with the foundation network 130.

[0047] Thus, by using the digital signatures 144 A, 144B to ensure that compatible subnetworks 140 A, MOB are loaded into the foundation network 130, incompatible subnetworks and otherwise compatible subnetworks that are maliciously modified can be detected and prevented from being loaded into the foundation network 130. In some embodiments, a store of subnetworks can be used to distribute new subnetworks to the device 100 or upgrade existing subnetworks at the device 100. Thus, in these embodiments, the store can provide digital signatures to the device 100 to ensure that the subnetworks are compatible with the foundation network 130 prior to loading the subnetworks into the foundation network 130.

[0048] The subnetwork 140 A can correspond to a neural network topology that has associated network weights 142A that are customized for the application 120A. For example, the network weights 142 A in the subnetwork 140 A are applicable to the application 120 A, but the network weights 142 A in the subnetwork 140 A are not applicable to different applications 120B, 120C in the plurality of applications 120. Similarly, the subnetwork MOB can correspond to a neural network topology that has associated network weights 142B that are customized for the application 120 A. For example, the network weights 142B in the subnetwork MOB are applicable to the application 120 A, but the network weights 142B in the subnetwork MOB are not applicable to different applications 120B, 120C in the plurality of applications 120.

[0049] Thus, techniques described with respect to Figure 1 enable developers to reuse the network weights 132 of the foundation network 130 hosted by the device 100 without sacrificing the foundation network’s 130 performance or customization ability. For example, the foundation network 130 may account for a relatively large amount of the network weights 132 in the customized network 150A for the application 120A. As described with respect to Figure 2, these network weights 132 are reusable to create a customized network 150B for the application 120B. Reusing the network weights 132 of the foundation network 130 improves storage capacity at the device 100 (e.g., saves memory at the device 100). For example, duplicating the network weights 132 to create an independent large foundation model for eachapplication 120 may take up a lot of storage, especially when the majority of the network weights used for any given application 120 stay in the foundation network 130. To illustrate, in the customized network 150A for the application 120A, a majority of network weights 132 (i.e., 99.9% or more) may stay in the foundation network 130, while a minority of the network weights 142A, 142B (i.e., approximately 0.1%) may stay in the subnetworks 140A, 140B tailored to the application 120 A. The network weights 142 A, 142B in the subnetworks 140 A, MOB may be used for differentiating features specific to the application 120 A. Thus, when the application 120 A invokes a call to the foundation network 130, the network weights 142 A, 142B of the corresponding subnetworks 140A, 140B can be dynamically loaded and combined with the foundation network 130 to form the customized network 150A (e.g., a complete model) for the application 120 A.

[0050] Figure 2 illustrates the device 100 that is operable to generate a second customized network for a second application using the shared foundation network, in accordance with examples described herein.

[0051] In Figure 2, the processor 102 can be configured to receive a notification from the application 120B through the API 106. The notification can indicate that the application 120B will use the foundation network 130. For example, receiving the notification from the application 120B can include receiving a call to the foundation network 130 from the application 120B through the API 106. In response to receiving the notification through the API 106, the processor 102 can be configured to retrieve the subnetwork 140C associated with the application 120B. For example, in the embodiment where the subnetwork 140C is stored in the memory 104, the processor 102 can retrieve the subnetwork 140C from the memory 104 in response to receiving the notification that the associated application 120B will use the foundation network 130. As another example, in the embodiment where the subnetwork 140C is cached with the application 120B, the processor 102 can retrieve the subnetwork 140C from the application 120B via the API 106.

[0052] The customized network generator 110 can be configured to generate a customized network 150B for the application 120B by inserting the subnetwork 140C associated with the application 120B into the foundation network 130. According to one implementation, prior to inserting the subnetwork 140C into the foundation network 130, the processor 102 can be configured to determine whether a digital signature 144C associated with the subnetwork 140C indicates that the subnetwork 140C is compatible with the foundation network 130. The subnetwork 140C can be inserted into the foundation network 130 in response to a determination that the digital signature 144C indicates that the subnetwork 140Cis compatible with the foundation network 130. The subnetwork 140C can correspond to a neural network topology that has associated network weights 142C that are customized for the application 120B. For example, the network weights 142C in the subnetwork 140C are applicable to the application 120B, but the network weights 142C in the subnetwork 140C are not applicable to different applications 120A, 120C in the plurality of applications 120.

[0053] Thus, techniques described with respect to Figure 2 enable developers to reuse the network weights 132 (used to generate the customized network 150A for the application 120A) to generate the customized network 150B for the application 120B. Reusing the network weights 132 of the foundation network 130 improves storage capacity at the device 100 (e.g., saves memory at the device 100).III. Example Processes

[0054] Figure 3 illustrates an example of a process 300 for generating a customized network for an application using a shared foundation network, in accordance with examples described herein.

[0055] According to the process 300, an application 120 and a corresponding subnetwork 140 can be provided to a natural interface session management module 304. The application 120 can correspond to any of the applications 120A-120C. According to one implementation, the natural interface session management module 304 can correspond to the API 106. For example, in some implementations, the API 106 of can be replaced with the natural interface session management module 304. The natural interface session management module 304 can be configured to facilitate communication between the application 120, the subnetwork 140, and the shared foundation network 130 (e.g., the shared foundation models for the plurality of applications 120. In particular, the natural interface session management module 304 can respond to “natural language” as opposed to function calls that have a particular format, such as Java, Python, etc. Thus, the application 120 can provide text streams to the natural interface session management module 304.

[0056] The natural interface session management module 304 can be configured to facilitate insertion of the subnetwork 140 associated with the application 120 into the shared foundation network 130. For example, the natural interface session management module 304 can identify parameter interfaces 306 that identify parameter interfaces 306 to be used to insert the subnetwork 140 into the shared foundation network 130. As non-limiting examples, the natural interface session management module 304 can identify whether to insert the subnetwork 140 into the shared foundation network 130 using a LoRA protocol 350, an embedding framework 352, an adapter protocol 356, or a hyper-network protocol 358, etc.Using the parameter interfaces 306, the natural interface session management module 304 may generate the customized network 150A, 150B for the application 120 by inserting the subnetwork 140 into the shared foundation network 130.

[0057] As illustrated in Figure 3, in some implementations, the shared foundation network 130 can support different plugins 308. For example, in some scenarios, the shared foundation network 130 may not be trained to process certain types of data. In these scenarios, the shared foundation network 130 can utilize plugins 308 to process these types of data. As a non-limiting example, by utilizing plugins 308, the shared foundation network 130 can process documents 340, math 342, services 344, and personal information 346 that the shared foundation network 130 might not otherwise be able to process. It should be understood that the plugins 308 depicted in Figure 3 are merely for illustrative purposes and should not be construed as limiting. In some implementations, the shared foundation network 130 can be trained to process documents 340, math 342, services 344, and personal information 346 without the use of plugins 308. Because the shared foundation network 130 is an “on-device” network that is local to the device 100 (as opposed to an online network), user information associated with the plugins 308 can remain private and is not subject to malicious actors.

[0058] According to the process 300, the shared foundation network 130 and the inserted subnetwork 140 (e.g., the customized network 150) is provided to a machine-language runtime / compiler 310 to facilitate the use by the application 120.

[0059] Figure 4 illustrates an example of a process 400 for enabling an on-device foundation model, in accordance with examples described herein.

[0060] According to the process 400, generative, creative, and / or exploratory tasks 402 can be provided to one or more natural interfaces 404. The one or more natural interfaces 404 can correspond to the natural interface session management module 304 of Figure 3. In particular, the generative, creative, and / or exploratory tasks 402 can be provided to the one or more natural interfaces 404 as text stream, as opposed to in a programming language format.

[0061] Prompt engineering 406 can be performed such that the description of the generative, creative, and / or exploratory tasks 402 that a foundation network 430 is to accomplish is embedded in an input to the foundation network 430. The foundation network 430 can correspond to the foundation network 130. The foundation network 430 can undergo parameter-efficient fine tuning 408. Based on the parameter-efficient fine tuning 408 of the foundation network 430, the foundation network 430 can perform non-deterministic computing 410 and can generate an output for the generative, creative, and / or exploratory tasks 402. That is, the foundation network 430 can generate an output that varies (e.g., has a non-deterministicposition) because of the nature of the tasks 402 supports non-deterministic computing 410. However, as described below with respect to Figure 5, if the nature of the task is one that requires deterministic computing, the foundation network 430 may have to utilize a plugin to perform the task because the foundation network 430 is trained to perform non-deterministic computing 410.

[0062] Figure 5 illustrates an example of another process 500 for enabling an on-device foundation model, in accordance with examples described herein.

[0063] According to the process 500, factual, transactional, and / or quantitative tasks 502 can be provided to one or more natural interfaces 404. In particular, the factual, transactional, and / or quantitative tasks 502 can be provided to the one or more natural interfaces 404 as text stream, as opposed to in a programming language format.

[0064] Prompt engineering 406 can be performed such that the description of the factual, transactional, and / or quantitative tasks 502 that foundation network 430 is to accomplish is embedded in an input to the foundation network 430. Additionally, the foundation network 430 can undergo parameter-efficient fine tuning 408. However, because the foundation network 430 performs non-deterministic computing, as opposed to deterministic computing 510 (e.g., computing of mathematical, algorithmic, procedural components), the foundation network 430 can utilize plugins, bridges, and / or extensions 508 to perform the deterministic computing 510 associated with the factual, transactional, and / or quantitative tasks 502.

[0065] It should be understood that the foundation network 430 in Figures 4-5 is merely a non-limiting example of a foundation network. In other implementations, the foundation network 430 may be able to perform non-deterministic computing 410 and deterministic computing 510 such that the plugins, bridges, and / or extensions 508 are not needed to perform the deterministic computing 510.

[0066] Figure 6 illustrates an example of a process 600 for generating a customized network for an application by modifying a shared foundation network using an embedding lookup matrix, in accordance with examples described herein.

[0067] According to the process 600, a text encoder 610 includes a tokenizer, an embedding lookup matrix 614, and a transformer 616. A first text prompt 602A (e.g., “A lovely cat in the style of ”) and a second text prompt 602B (e.g., “A lovely dog in the style of ”) are provided to the text encoder 610. The text prompts 602 can correspond to data associated with a subnetwork 140 that is to be inserted into the foundation network 130. The tokenizer 612 can be configured to receive the stream of characters (e.g., the text prompts 602),break the stream of characters into individual tokens or words, and output a stream of tokens or words.

[0068] As illustrated in Figure 6, the embedding lookup matrix 614 includes ninetyseven (97) rows of data. It should be understood that the embedding lookup matrix 614 illustrated in Figure 6 is merely for illustrative purposes and should not be construed as limiting. According to one implementation, the embedding lookup matrix 614 can be associated with the foundation network 130. To insert the text prompts 602 (e.g., the subnetwork 140) into the embedding lookup matrix 614, the embedding lookup matrix 614 can be resized. For example, an additional row and an additional row can be added to the embedding lookup matrix 614. After resizing, the embedding lookup matrix 614 can be updated based on the text prompts 602 (e.g., the subnetwork 140). For example, the tokenized version of the text prompts 602 can be inserted into the additional rows. The transformer 616 can be configured to process the updated embedding lookup matrix 614 and produce a continuous representation, or embedding, of the input.IV. Example System For Subnetwork Retrieval

[0069] Figure 7 illustrates an example of a system 700 including a device that is operable to retrieve subnetworks from a store of subnetworks, in accordance with examples described herein.

[0070] According to the system 700, the device 100 is in communication with a store of subnetworks 702. The store of subnetworks 702 can correspond to an online store or database that enables the device 100 to download subnetworks the subnetworks 140 that can be inserted into the foundation network 130. For example, the subnetwork 140A, the subnetwork 140B, and the subnetwork 140C can be included in the store of subnetworks 702. In some scenarios, the store of subnetworks 702 can also be part of, or included in, an application store. For example, the applications 120 can be included in an online application store, and when the device 100 downloads a particular application 120, the corresponding subnetwork(s) 140 can also be downloaded to the device 100. The store of subnetworks 702 and the store for the applications 120 may be different stores or may be part of the same store. In some scenarios, the same subnetwork 140 can be shared across multiple applications 120. In some scenarios, the subnetwork 140 and the application 120 can be updated independently.

[0071] As described with respect to Figures 1-2, the subnetwork 140A includes network weights 142 A, an indication of the associated application 120 A, and the digital signature 144A. The digital signature 144A can indicate whether the subnetwork 140A is compatible (to be inserted) with the foundation network 130. For example, based on the digitalsignature 144 A, information indicative of the compatibility between the foundation network 130 and the subnetwork 140 A can be communicated. To illustrate, the digital signature 144 A indicates a subnetwork version 710A of the subnetwork 140 A, one or more foundation model types 712A that are compatible with the subnetwork 140 A, one or more foundation model versions 714A that are compatible with the subnetwork 140 A, and parameter interfaces 716A that are compatible with the subnetwork 140 A.

[0072] The subnetwork version 710A may indicate the version of the subnetwork 140 A for the application 120 A. For example, the subnetwork 140 A for the application 120 A can be updated in the store of subnetworks 702. In some scenarios, a first version of the subnetwork 140 A may not be compatible with the foundation network 130, and a second version of the subnetwork 140A may be compatible with the foundation network 130. By indicating the subnetwork version 710A of the subnetwork 140 A in the digital signature 144 A, the device 100 can determine whether the current version 710A of the subnetwork 140A is compatible with the foundation network 130. If the current version 710A of the subnetwork 140A is compatible with the foundation network 130, the device 100 can download (e.g., retrieve) the subnetwork 140A from the store of subnetworks 702 and locally store the subnetwork 140A on the device 100 for use with the foundation network 130 when the application 120 A indicates an intent to use the foundation network 130.

[0073] The one or more foundation model types 712A may indicate different types of foundation models (e.g., foundation networks) that are compatible with the subnetwork 140A. If one of foundation model types 712A match the type of the foundation network 130, the device 100 can download (e.g., retrieve) the subnetwork 140 A from the store of subnetworks 702 and locally store the subnetwork 140A on the device 100 for use with the foundation network 130 when the application 120A indicates an intent to use the foundation network 130.

[0074] The one or more foundation model versions 714A that are compatible with the subnetwork 140A may indicate different versions of foundation models (e.g., foundation networks) that are compatible with the subnetwork 140A. To illustrate, the one or more foundation model versions 714A can indicate whether the subnetwork 140 A is compatible with a first version of a particular foundation network, a second version of the particular foundation network, etc. If the version of the foundation network 130 matches one of the versions in the one or more foundation model versions 714A, the device 100 can download (e.g., retrieve) the subnetwork 140A from the store of subnetworks 702 and locally store the subnetwork 140A on the device 100 for use with the foundation network 130 when the application 120 A indicates an intent to use the foundation network 130.

[0075] The parameter interface 716A can indicate whether the subnetwork 140A is compatible with the subnetwork protocol 350, the embedding framework 352, the adapter protocol 356, the hyper-network protocol 358, etc. If the foundation network 130 has a parameter interface that matches one of the parameter interface 716A associated with the subnetwork 140A, the device 100 can download (e.g., retrieve) the subnetwork 140A from the store of subnetworks 702 and locally store the subnetwork 140A on the device 100 for use with the foundation network 130 when the application 120 A indicates an intent to use the foundation network 130.

[0076] Similar to the digital signature 144 A for the subnetwork 140 A, the digital signature 144B for the subnetwork 140B indicates a subnetwork version 710B of the subnetwork 140B, one or more foundation model types 712B that are compatible with the subnetwork 140B, one or more foundation model versions 714B that are compatible with the subnetwork 140B, and parameter interfaces 716B that are compatible with the subnetwork 140B. Additionally, the digital signature 144C for the subnetwork 140C indicates a subnetwork version 710C of the subnetwork 140C, one or more foundation model types 712C that are compatible with the subnetwork 140C, one or more foundation model versions 714C that are compatible with the subnetwork 140C, and parameter interfaces 716C that are compatible with the subnetwork 140C.

[0077] The device 100 may utilize a particular subnetwork 140 to customize the foundation network 130 if the information in the digital signature 144 of the particular subnetwork indicate that the subnetworks 140 are compatible with the foundation network 130. Thus, by using a digital signature 144 to ensure that compatible subnetworks 140 are loaded into the shared foundation network 130, incompatible subnetworks and otherwise compatible subnetworks that are maliciously modified can be detected and prevented from being loaded into the shared foundation network 130.V. Example Machine-Learning Process For Large Language Models

[0078] Figure 8 shows a diagram 800 illustrating a training phase 802 and an inference phase 804 of trained machine-learning model(s) 832, in accordance with example embodiments. According to some examples, the trained machine-learning model(s) 832 can correspond to the foundation network 130 (e.g., a large generative foundation model) and / or one or more of the subnetworks 140. Some machine-learning techniques involve training one or more machine-learning algorithms on an input set of training data to recognize patterns in the training data and provide output inferences and / or predictions about (patterns in the) training data. The resulting trained machine-learning algorithm can be termed as a trainedmachine-learning model. For example, Figure 8 shows the training phase 802 where machinelearning algorithm(s) 820 are being trained on training data 810 to become trained machinelearning model(s) 832. Then, during the inference phase 804, the trained machine-learning model(s) 832 can receive input data 830 and one or more inference / prediction requests 840 (perhaps as part of the input data 830) and responsively provide as an output one or more inferences and / or prediction(s) 850.

[0079] As such, the trained machine-learning model(s) 832 can include one or more models of machine-learning algorithm(s) 820. The machine-learning algorithm(s) 820 may include, but are not limited to: an artificial neural network (e.g., a herein-described convolutional neural networks, a recurrent neural network, a Bayesian network, a hidden Markov model, a Markov decision process, a logistic regression function, a support vector machine, a suitable statistical machine-learning algorithm, and / or a heuristic machinelearning system). The machine-learning algorithm(s) 820 may be supervised or unsupervised, and may implement any suitable combination of online and offline learning.

[0080] In some examples, the machine-learning algorithm(s) 820 and / or the trained machine-learning model(s) 832 can be accelerated using on-device coprocessors, such as graphic processing units (GPUs), tensor processing units (TPUs), digital signal processors (DSPs), and / or application specific integrated circuits (ASICs). Such on-device coprocessors can be used to speed up the machine-learning algorithm(s) 820 and / or the trained machinelearning model(s) 832. In some examples, the trained machine-learning model(s) 832 can be trained, stored and executed to provide inferences on a particular computing device, and / or otherwise can make inferences for the particular computing device.

[0081] During the training phase 802, the machine-learning algorithm(s) 820 can be trained by providing at least the training data 810 as training input using unsupervised, supervised, semi-supervised, and / or reinforcement learning techniques. Unsupervised learning involves providing a portion (or all) of the training data 810 to the machinelearning algorithm(s) 820 and the machine-learning algorithm(s) 820 determining one or more output inferences based on the provided portion (or all) of the training data 810. Supervised learning involves providing a portion of the training data 810 to the machinelearning algorithm(s) 820, with the machine-learning algorithm(s) 820 determining one or more output inferences based on the provided portion of the training data 810, and the output inference(s) are either accepted or corrected based on correct results associated with the training data 810. In some examples, supervised learning of the machine-learning algorithm(s) 820 can be governed by a set of rules and / or a set of labels for the training input, and the set ofrules and / or set of labels may be used to correct inferences of the machinelearning algorithm(s) 820.

[0082] In some examples, the machine-learning algorithm(s) 820 and / or the trained machine-learning model(s) 832 can use transfer learning techniques. For example, transfer learning techniques can involve the trained machine-learning model(s) 832 being pre-trained on one set of data and additionally trained using the training data 810. More particularly, the machine-learning algorithm(s) 820 can be pre-trained on data from one or more computing devices and a resulting trained machine-learning model provided to a particular computing device, where the particular computing device is intended to execute the trained machinelearning model during the inference phase 804. Then, during the training phase 802, the pretrained machine-learning model can be additionally trained using the training data 810, where the training data 810 can be derived from kernel and non-kernel data of the particular computing device. This further training of the machine-learning algorithm(s) 820 and / or the pre-trained machine-learning model using the training data 810 of the particular computing device’s data can be performed using either supervised or unsupervised learning. Once the machine-learning algorithm(s) 820 and / or the pre-trained machine-learning model has been trained on at least the training data 810, the training phase 802 can be completed. The trained resulting machine-learning model can be utilized as at least one of the trained machine-learning model(s) 832.

[0083] In particular, once the training phase 802 has been completed, the trained machine-learning model(s) 832 can be provided to a computing device, if not already on the computing device. The inference phase 804 can begin after training the machine-learning model(s) 832 are provided to the particular computing device.

[0084] During the inference phase 804, the trained machine-learning model(s) 832 can receive the input data 830 and generate and output one or more corresponding inferences and / or prediction(s) 850 about the input data 830. As such, the input data 830 can be used as an input to the trained machine-learning model(s) 832 for providing corresponding inference(s) and / or prediction(s) 850 to kernel components and non-kernel components. For example, the trained machine-learning model(s) 832 can generate inference(s) and / or prediction(s) 850 in response to one or more inference / prediction requests 840. In some examples, the trained machinelearning model(s) 832 can be executed by a portion of other software. For example, the trained machine-learning model(s) 832 can be executed by an inference or prediction daemon to be readily available to provide inferences and / or predictions upon request. The input data 830 can include data from the particular computing device executing the trained machine-learningmodel(s) 832 and / or input data from one or more computing devices other than the particular computing device.

[0085] If the trained machine-learning model 832 corresponds to the foundation network 130, the input data 830 can include data associated with one or more of the applications 120. Other types of input data are possible as well.

[0086] Inference(s) and / or prediction(s) 650 can include other output data produced by the trained machine-learning model(s) 832 operating on the input data 830 (and the training data 810). In some examples, the trained machine-learning model(s) 832 can use output inference(s) and / or prediction(s) 850 as input feedback 860. The trained machine-learning model(s) 832 can also rely on past inferences as inputs for generating new inferences.

[0087] Convolutional neural networks and / or deep neural networks used herein can be an example of the machine-learning algorithm(s) 820. After training, the trained version of a convolutional neural network can be an example of the trained machine-learning model(s) 832.VI. Additional Example Operations

[0088] Figure 9 illustrates a flow chart of a method 900 related to a new technology. The method 900 may be carried out by the device 100 among other possibilities. The embodiments of Figure 9 may be simplified by the removal of any one or more of the features shown therein. Further, these embodiments may be combined with features, aspects, and / or implementations of any of the previous figures or otherwise described herein.

[0089] The method 900 includes hosting, by a device, a foundation network that is shared by a plurality of applications, at block 902. For example, referring to Figure 1, the device 100 hosts the foundation network 130 that is shared by the plurality of applications 120. The foundation network is isolated from a server and other devices.

[0090] The method 900 also includes receiving, by the device, a notification from a particular application of the plurality of applications, at block 904. The notification indicates that the particular application will use the foundation network. For example, referring to Figure 1, the device 100 receives a notification from the application 120 A. The notification indicates that the application 120A will use the foundation network 130. According to one implementation of the method 900, receiving the notification from the particular application includes receiving, by the device, a call to the foundation network from the particular application.

[0091] The method 900 also includes retrieving, by the device, a subnetwork associated with the particular application in response to receiving the notification from the particular application, at block 906. For example, referring to Figure 1, the device 100 retrieves thesubnetwork 140 A associated with the application 120 A in response to receiving the notification from the application 120 A.

[0092] The method 900 also includes generating, by the device, a customized network for the particular application at least in part by inserting the subnetwork into the foundation network, at block 908. For example, referring to Figure 1, the customized network generator 110 generates the customized network 150A for the application 120A at least in part by inserting the subnetwork 140A into the foundation network 130.

[0093] According to one implementation, the method 900 also includes retrieving, by the device, a second subnetwork associated with the particular application in response to receiving the notification from the particular application. For example, referring to Figure 1, the device 100 retrieves the subnetwork 140B associated with the application 120 A in response to receiving the notification from the application 120 A. Generating the customized network for the particular application further includes inserting, by the device, the second subnetwork into the foundation network. For example, referring to Figure 1, generating the customized network 150A for the application 120A further includes inserting the subnetwork 140B into the foundation network 130.

[0094] According to one implementation, the method 900 includes, prior to inserting the subnetwork into the foundation network, determining, by the device, whether a digital signature associated with the subnetwork indicates that the subnetwork is compatible with the foundation network. For example, referring to Figure 1, prior to inserting the subnetwork 140A into the foundation network 130, the processor 102 determines whether the digital signature 144A associated with the subnetwork 140A indicates that the subnetwork 140A is compatible with the foundation network 130. The subnetwork 140A is inserted into the foundation network 130 in response to a determination that the digital signature 144 A indicates that the subnetwork 140A is compatible with the foundation network 130.

[0095] According to one implementation of the method 900, the subnetwork is retrieved from a store of subnetworks for different applications. According to one implementation of the method 900, the subnetwork corresponds to a neural network topology having associated weights that are customized for the particular application. According to one implementation of the method 900, network weights in the foundation network are applicable to each application in the plurality of applications.

[0096] According to one implementation of the method 900, inserting the subnetwork into the foundation network includes adding a parallel branch to a computation graph of the foundation network and inserting the subnetwork into the parallel branch. In thisimplementation, the subnetwork may be inserted into the foundation network according to a subnetwork protocol.

[0097] According to one implementation of the method 900, inserting the subnetwork into the foundation network includes adding a sequential branch to a computation graph of the foundation network and inserting the subnetwork into the sequential branch. In this implementation, the subnetwork may be inserted into the foundation network according to an adapter protocol or a hyper-network protocol.

[0098] According to one implementation of the method 900, inserting the subnetwork into the foundation network includes resizing an embedding lookup matrix associated with the foundation network and updating the embedding lookup matrix based on the subnetwork after resizing the embedding lookup matrix.

[0099] The method 900 of Figure 9 enables developers to reuse the network weights 132 of the foundation network 130 hosted by the device 100 without sacrificing the foundation network’s 130 performance or customization ability. For example, the foundation network 130 may account for a relatively large amount of the networks weights 132 in the customized network 150A for the application 120A. These network weights 132 are reusable to create the customized network 150B for the application 120B. Reusing the network weights 132 of the foundation network 130 improves storage capacity at the device 100 (e.g., saves memory at the device 100).VII. Conclusion

[0100] The present disclosure is not to be limited in terms of the particular embodiments described in this application, which are intended as illustrations of various aspects. Many modifications and variations can be made without departing from its scope, as will be apparent to those skilled in the art. Functionally equivalent methods and apparatuses within the scope of the disclosure, in addition to those described herein, will be apparent to those skilled in the art from the foregoing descriptions. Such modifications and variations are intended to fall within the scope of the appended claims.

[0101] The above detailed description describes various features and operations of the disclosed systems, devices, and methods with reference to the accompanying figures. In the figures, similar symbols typically identify similar components, unless context dictates otherwise. The example embodiments described herein and in the figures are not meant to be limiting. Other embodiments can be utilized, and other changes can be made, without departing from the scope of the subject matter presented herein. It will be readily understood that the aspects of the present disclosure, as generally described herein, and illustrated in the figures,can be arranged, substituted, combined, separated, and designed in a wide variety of different configurations.

[0102] With respect to any or all of the message flow diagrams, scenarios, and flow charts in the figures and as discussed herein, each step, block, and / or communication can represent a processing of information and / or a transmission of information in accordance with example embodiments. Alternative embodiments are included within the scope of these example embodiments. In these alternative embodiments, for example, operations described as steps, blocks, transmissions, communications, requests, responses, and / or messages can be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved. Further, more or fewer blocks and / or operations can be used with any of the message flow diagrams, scenarios, and flow charts discussed herein, and these message flow diagrams, scenarios, and flow charts can be combined with one another, in part or in whole.

[0103] A step or block that represents a processing of information may correspond to circuitry that can be configured to perform the specific logical functions of a herein-described method or technique. Alternatively or additionally, a block that represents a processing of information may correspond to a module, a segment, or a portion of program code (including related data). The program code may include one or more instructions executable by a processor for implementing specific logical operations or actions in the method or technique. The program code and / or related data may be stored on any type of computer readable medium such as a storage device including random access memory (RAM), a disk drive, a solid state drive, or another storage medium.

[0104] The computer readable medium may also include non-transitory computer readable media such as computer readable media that store data for short periods of time like register memory, processor cache, and RAM. The computer readable media may also include non-transitory computer readable media that store program code and / or data for longer periods of time. Thus, the computer readable media may include secondary or persistent long term storage, like read only memory (ROM), optical or magnetic disks, solid state drives, compact-disc read only memory (CD-ROM), for example. The computer readable media may also be any other volatile or non-volatile storage systems. A computer readable medium may be considered a computer readable storage medium, for example, or a tangible storage device.

[0105] Moreover, a step or block that represents one or more information transmissions may correspond to information transmissions between software and / or hardwaremodules in the same physical device. However, other information transmissions may be between software modules and / or hardware modules in different physical devices.

[0106] The particular arrangements shown in the figures should not be viewed as limiting. It should be understood that other embodiments can include more or less of each element shown in a given figure. Further, some of the illustrated elements can be combined or omitted. Yet further, an example embodiment can include elements that are not illustrated in the figures.

[0107] While various aspects and embodiments have been disclosed herein, other aspects and embodiments will be apparent to those skilled in the art. The various aspects and embodiments disclosed herein are for the purpose of illustration and are not intended to be limiting, with the true scope being indicated by the following claims.

Claims

CLAIMSWhat is claimed is:

1. A method comprising: hosting, by a device, a foundation network that is shared by a plurality of applications; receiving, by the device, a notification from a particular application of the plurality of applications, wherein the notification indicates that the particular application will use the foundation network; retrieving, by the device, a subnetwork associated with the particular application in response to receiving the notification from the particular application; and generating, by the device, a customized network for the particular application at least in part by inserting the subnetwork into the foundation network.

2. The method of claim 1, further comprising, prior to inserting the subnetwork into the foundation network: determining, by the device, whether a digital signature associated with the subnetwork indicates that the subnetwork is compatible with the foundation network, wherein the subnetwork is inserted into the foundation network in response to a determination that the digital signature indicates that the subnetwork is compatible with the foundation network.

3. The method of claim 1, wherein the subnetwork is retrieved from a store of subnetworks that includes subnetworks for different applications.

4. The method of claim 1, wherein the subnetwork corresponds to a neural network topology having associated weights that are customized for the particular application.

5. The method of claim 1, wherein inserting the subnetwork into the foundation network comprises: adding a parallel branch to a computation graph of the foundation network; and inserting the subnetwork into the parallel branch.

6. The method of claim 1, wherein inserting the subnetwork into the foundation network comprises:adding a sequential branch to a computation graph of the foundation network; and inserting the subnetwork into the sequential branch.

7. The method of claim 1, wherein inserting the subnetwork into the foundation network comprises: resizing an embedding lookup matrix associated with the foundation network; and updating the embedding lookup matrix based on the subnetwork after resizing the embedding lookup matrix.

8. The method of claim 1, wherein network weights in the foundation network are applicable to each application in the plurality of applications.

9. The method of claim 1, wherein network weights in the subnetwork are applicable to the particular application, and wherein the network weights in the subnetwork are not applicable to a different application in the plurality of applications.

10. The method of claim 1, wherein the foundation network is isolated from a server and other devices.

11. The method of claim 1, wherein receiving the notification from the particular application comprises receiving, by the device, a call to the foundation network from the particular application.

12. A device comprising: a memory; and a processor coupled to the memory, the processor configured to: host a foundation network that is shared by a plurality of applications; receive a notification from a particular application of the plurality of applications, wherein the notification indicates that the particular application will use the foundation network; retrieve a subnetwork associated with the particular application in response to receiving the notification from the particular application; and generate a customized network for the particular application at least in part by inserting the subnetwork into the foundation network.

13. The device of claim 12, wherein, to insert the subnetwork into the foundation network, the processor is configured to: add a parallel branch to a computation graph of the foundation network; and insert the subnetwork into the parallel branch.

14. The device of claim 12, wherein, to insert the subnetwork into the foundation network, the processor is configured to: add a sequential branch to a computation graph of the foundation network; and insert the subnetwork into the sequential branch.

15. The device of claim 12, wherein, to insert the subnetwork into the foundation network, the processor is configured to: resize an embedding lookup matrix associated with the foundation network; and update the embedding lookup matrix based on the subnetwork after resizing the embedding lookup matrix.

16. The device of claim 12, wherein, prior to inserting the subnetwork into the foundation network, the processor is configured to: determine whether a digital signature associated with the subnetwork indicates that the subnetwork is compatible with the foundation network, wherein the subnetwork is inserted into the foundation network in response to a determination that the digital signature indicates that the subnetwork is compatible with the foundation network.

17. The device of claim 12, wherein the foundation network is isolated from a server and other devices.

18. The device of claim 12, wherein network weights in the subnetwork are applicable to the particular application, and wherein the network weights in the subnetwork are not applicable to a different application in the plurality of applications.

19. A non-transitory computer-readable medium comprising instructions that, when executed by a processor within a device, cause the processor to perform operations comprising:hosting a foundation network that is shared by a plurality of applications; receiving a notification from a particular application of the plurality of applications, wherein the notification indicates that the particular application will use the foundation network; retrieving a subnetwork associated with the particular application in response to receiving the notification from the particular application; and generating a customized network for the particular application at least in part by inserting the subnetwork into the foundation network.

20. The non-transitory computer-readable medium of claim 19, wherein, prior to inserting the subnetwork into the foundation network, the operations comprise: determining, by the device, whether a digital signature associated with the subnetwork indicates that the subnetwork is compatible with the foundation network, wherein the subnetwork is inserted into the foundation network in response to a determination that the digital signature indicates that the subnetwork is compatible with the foundation network.

Images