Personalized neural network pruning

Abstract

A method for generating a personalized model includes receiving one or more personal data samples from a user. A prototype of a personal identity is generated based on the personal data samples. The prototype of the personal identity is trained to reflect personal characteristics of the user. A network graph is generated based on the prototype of the personal identity. One or more channels of a global network are pruned based on the network graph to produce the personalized model.

Description

[0001] Cross-references to related applications

[0002] This application claims priority to U.S. Patent Application No. 17 / 506,646, filed October 20, 2021, entitled “PERSONALIZED NEURAL NETWORK PRUNING,” which claims the benefit of U.S. Provisional Patent Application No. 63 / 094,767, filed October 21, 2020, entitled “PERSONALIZED NEURAL NETWORK PRUNING,” the disclosure of which is expressly incorporated herein by reference in its entirety.

[0003] open field

[0004] The various aspects of this disclosure generally relate to the generation of personalized neural networks.

[0005] background

[0006] Artificial neural networks can include groups of interconnected artificial neurons (e.g., neuron models). Artificial neural networks can be computing devices or represented as methods to be performed by computing devices. A convolutional neural network (CNN) is a type of feedforward artificial neural network. A CNN can include a set of neurons, each with a receptive field and collectively constructing an input space. Convolutional neural networks (CNNs) (such as deep convolutional neural networks (DCNs)) have numerous applications. Specifically, these neural network architectures are used in a variety of technologies, such as image recognition, speech recognition, acoustic scene classification, keyword detection, autonomous driving, and other classification tasks.

[0007] Edge devices, such as smartphones, are widely used. Given the many useful applications of neural networks, the demand for their use on edge devices and for personalized services on such devices is constantly increasing. However, edge devices have limited computational resources, and generalized models can utilize more complex networks and more computation. Thus, conventional personalization methods may not be suitable for edge devices because conventional approaches use limited personal data for retraining or fine-tuning. Unfortunately, these multiple training iterations generally incur significant computational costs during on-device learning, which is an excessive burden for edge devices. Summary of the Invention

[0008] This disclosure is set forth in the independent claims. Some aspects of this disclosure are described in the dependent claims.

[0009] In one aspect of this disclosure, a computer-implemented method is provided. The method includes receiving one or more samples of personal data. The method also includes generating a prototype of a personal identity based on the personal data samples. The prototype of the personal identity is trained to reflect the user's personal characteristics. Additionally, the method includes generating a network graph based on the prototype of the personal identity. Furthermore, the method includes pruning one or more channels of a global network based on the network graph to produce a personalized model.

[0010] In another aspect of this disclosure, an apparatus is provided. The apparatus includes a memory and one or more processors coupled to the memory. The processors are configured to receive one or more samples of personal data. The processors are also configured to generate a prototype of a personal identity based on the personal data samples. The prototype of the personal identity is trained to reflect the user's personal characteristics. Furthermore, the processors are configured to generate a network graph based on the prototype of the personal identity. Additionally, the processors are configured to prune one or more channels of a global network based on the network graph to produce a personalized model.

[0011] In another aspect of this disclosure, an apparatus is provided. The apparatus includes means for receiving one or more samples of personal data. The apparatus also includes means for generating a prototype of a personal identity based on the personal data samples. The prototype of the personal identity is trained to reflect the user's personal characteristics. Additionally, the apparatus includes means for generating a network graph based on the prototype of the personal identity. Furthermore, the apparatus includes means for pruning one or more channels of a global network based on the network graph to generate a personalized model.

[0012] In a further aspect of this disclosure, a non-transient computer-readable medium is provided. Program code is encoded on the computer-readable medium. The program code is executed by a processor and includes code for receiving one or more samples of personal data. The program code also includes code for generating a prototype of an individual identity based on the personal data samples. The prototype of the individual identity is trained to reflect the user's personal characteristics. Additionally, the program code includes code for generating a network graph based on the prototype of the individual identity. Furthermore, the program code includes code for pruning one or more channels of the global network based on the network graph to produce a personalized model.

[0013] Additional features and advantages of this disclosure will be described below. Those skilled in the art will appreciate that this disclosure can be readily used as the basis for modifying or designing other structures for implementing the same purposes as this disclosure. Those skilled in the art will also recognize that such equivalent constructions do not depart from the teachings of this disclosure set forth in the appended claims. Novel features considered characteristic of this disclosure, in both their organization and manner of operation, along with further objects and advantages, will be better understood when considered in conjunction with the accompanying drawings. However, it is to be clearly understood that each drawing is provided for illustrative and descriptive purposes only and is not intended to be a definition of limitation of this disclosure. Brief description of the attached diagram

[0015] The features, nature, and advantages of this disclosure will become more apparent when understood in conjunction with the accompanying drawings, in which the same reference numerals are always used to indicate the subject.

[0016] Figure 1 An example implementation of a neural network using a system-on-a-chip (SoC) (including a general-purpose processor) according to certain aspects of this disclosure is explained.

[0017] Figure 2A , 2B 2C are illustrations explaining various aspects of the neural network according to this disclosure.

[0018] Figure 2D This is a diagram illustrating an exemplary deep convolutional network (DCN) according to various aspects of this disclosure.

[0019] Figure 3 This is a block diagram illustrating an exemplary deep convolutional network (DCN) according to various aspects of this disclosure.

[0020] Figure 4 This is an example of trimming a diagram of a complete or global model based on various aspects of this disclosure to generate a block diagram of a personalized model.

[0021] Figure 5 This is a block diagram illustrating an example implementation of a personalized model based on various aspects of this disclosure.

[0022] Figure 6A and 6B This is a block diagram illustrating a prototype-based personalized pruning (PPP) model according to various aspects of this disclosure.

[0023] Figure 6C This is a block diagram illustrating example residual blocks according to various aspects of this disclosure.

[0024] Figure 7 This is a flowchart illustrating an example method for generating a personalized neural network model according to various aspects of this disclosure.

[0025] Detailed description

[0026] The detailed description that follows, taken in conjunction with the accompanying drawings, is intended as a description of various configurations and is not intended to represent the only configuration in which the concepts described herein can be practiced. This detailed description includes specific details to provide a thorough understanding of the various concepts. However, it will be apparent to those skilled in the art that these concepts can be practiced without these specific details. In some instances, well-known structures and components are shown in block diagram form to avoid obscuring such concepts.

[0027] Based on this teaching, those skilled in the art will appreciate that the scope of this disclosure is intended to cover any aspect of this disclosure, whether implemented independently of or in combination with any other aspect of this disclosure. For example, any number of the aspects described may be used to implement an apparatus or method of practice. Furthermore, the scope of this disclosure is intended to cover such apparatus or methods practiced using other structures, functionalities, or structures and functionalities that complement or differ from the aspects of the described disclosure. It should be understood that any aspect of this disclosure may be implemented by one or more elements of the claims.

[0028] The word “exemplary” is used in this document to mean “serving as an example, instance, or explanation.” Any aspect described as “exemplary” in this document is not necessarily to be construed as superior to or better than the others.

[0029] While specific aspects are described herein, numerous variations and substitutions of these aspects fall within the scope of this disclosure. Although some benefits and advantages of preferred aspects are mentioned, the scope of this disclosure is not intended to be limited to a particular benefit, use, or objective. Rather, the aspects of this disclosure are intended to be broadly applicable to various technologies, system configurations, networks, and protocols, some of which are illustrated as examples in the accompanying drawings and the following description of preferred aspects. The detailed description and accompanying drawings are merely illustrative and not limiting of this disclosure, the scope of which is defined by the appended claims and their equivalents.

[0030] Personalization of deep neural networks is a significant topic in computer vision and signal processing for numerous applications, such as recommender systems, intelligent assistance, speaker verification, and keyword detection. Due to its broad heuristics, generalized models can be distributed to a wide range of users. Each user may have locally generated and / or user-specific data, such as data developed via sensors on the user's camera, microphone, or edge device. This type of data can be referred to as personal data. Personalization of global models can have conflicting objectives: performing well on personal data with a relatively small sample size while continuing to perform well on general data. Training a global model to perform well in many applications relies on training data that covers all situations for generalization. However, this assumption deviates from real-world scenarios.

[0031] One approach to personalizing deep neural networks is to fine-tune a global model using limited personal data. However, storing individual global deep neural network (DNN) models on memory-constrained edge devices is impractical, and optimizing such DNN models with limited personal data is challenging. In other words, deploying DNNs requires substantial computational resources, making it a challenging problem due to memory limitations and computational load. To address this, numerous model compression methods, such as knowledge extraction, model quantization, and model pruning, have been considered. However, these conventional methods involve additional training phases or retraining. Therefore, the additional computational cost may be unavoidable.

[0032] Various aspects of this disclosure relate to prototype-based personalization pruning (PPP), which uses prototype representations to learn a metric space. PPP considers both model complexity and personalization. PPP uses prototypes of individual identities driven by limited personalization data to select subset graphs with prototype modules. For this reason, the model structure of PPP can be dynamically customized to enhance personalization and reduce model complexity. According to various aspects of this disclosure, prototypes of individual identities are provided for each group of individual data. The prototypes can represent characteristics of the individual data. Dynamic path networks can be trained for personalization pruning to produce personalized models that are pruned versions of generalized or full models without fine-tuning using individual data.

[0033] Figure 1 An example implementation of a system-on-a-chip (SoC) 100 is described, which may include a central processing unit (CPU) 102 or a multi-core CPU configured for personalized neural network pruning. Variables (e.g., neural signals and synaptic weights), system parameters associated with computing devices (e.g., weighted neural networks), latency, frequency slot information, and task information may be stored in memory blocks associated with a neural processing unit (NPU) 108, a memory block associated with the CPU 102, a memory block associated with a graphics processing unit (GPU) 104, a memory block associated with a digital signal processor (DSP) 106, a memory block 118, or may be distributed across multiple blocks. Instructions executed at the CPU 102 may be loaded from the program memory associated with the CPU 102 or from memory block 118.

[0034] SOC 100 may also include additional processing blocks tailored to specific functions, such as GPU 104, DSP 106, connectivity block 110 (which may include fifth-generation (5G) connectivity, fourth-generation LTE (4G) connectivity, Wi-Fi connectivity, USB connectivity, Bluetooth connectivity, etc.), and multimedia processor 112, for example, capable of detecting and recognizing gestures. In one implementation, NPU 108 is implemented within CPU 102, DSP 106, and / or GPU 104. SOC 100 may also include sensor processor 114, image signal processor (ISP) 116, and / or navigation module 120 (which may include a global positioning system).

[0035] The SOC 100 may be based on the ARM instruction set. In one aspect of this disclosure, instructions loaded into the general-purpose processor 102 may include code for receiving one or more samples of personal data. The general-purpose processor 102 may also include code for generating a prototype of an individual identity based on the personal data samples. The prototype of the individual identity is trained to reflect the user's personal characteristics. The general-purpose processor 102 may also include code for generating a network graph based on the prototype of the individual identity. The general-purpose processor 102 may further include code for pruning one or more channels of the global network based on the network graph to produce a personalized model.

[0036] Deep learning architectures perform object recognition tasks by learning to represent inputs at progressively higher levels of abstraction in each layer, thereby constructing useful feature representations of the input data. In this way, deep learning addresses a major bottleneck in traditional machine learning. Before deep learning, machine learning methods for object recognition problems often relied heavily on human-engineered features, perhaps combined with shallow classifiers. Shallow classifiers could be two-class linear classifiers, where a weighted sum of feature vector components is compared to a threshold to predict which class the input belongs to. Human-engineered features could be templates or kernels customized for a specific problem domain by engineers with domain expertise. In contrast, deep learning architectures can learn to represent features similar to those that human engineers might design, but this learning is achieved through training. Furthermore, deep networks can learn to represent and recognize novel types of features that humans might not have considered before.

[0037] Deep learning architectures can learn hierarchical levels of features. For example, if visual data is presented to the first layer, it can learn to recognize relatively simple features (such as edges) in the input stream. In another example, if auditory data is presented to the first layer, it can learn to recognize spectral power at specific frequencies. A second layer, taking the output of the first layer as input, can learn to recognize combinations of features, such as recognizing simple shapes in visual data or sound combinations in auditory data. For example, higher layers can learn to represent complex shapes in visual data or words in auditory data. Even higher layers can learn to recognize common visual objects or spoken phrases.

[0038] Deep learning architectures can perform particularly well when applied to problems with a naturally hierarchical structure. For example, the classification of motor vehicles can benefit from first learning to identify wheels, windshields, and other features. These features can then be combined in different ways at higher levels to identify cars, trucks, and airplanes.

[0039] Neural networks can be designed with various connectivity patterns. In feedforward networks, information is passed from lower layers to higher layers, where each neuron in a given layer communicates to neurons in higher layers. As mentioned above, hierarchical representations can be constructed in successive layers of a feedforward network. Neural networks can also have backflow or feedback (also known as top-down) connections. In a backflow connection, the output from a neuron in a given layer can be communicated to another neuron in the same layer. Backflow architectures can help identify patterns across more than one block of input data sequentially delivered to the neural network. Connections from neurons in a given layer to neurons in lower layers are called feedback (or top-down) connections. Networks with many feedback connections can be beneficial when the recognition of higher-level concepts can help discern specific lower-level features of the input.

[0040] The connections between layers of a neural network can be fully connected or partially connected. Figure 2A An example of a fully connected neural network 202 is explained. In a fully connected neural network 202, a neuron in the first layer can pass its output to each neuron in the second layer, so that each neuron in the second layer receives input from each neuron in the first layer. Figure 2BAn example of a locally connected neural network 204 has been explained. In the locally connected neural network 204, neurons in the first layer can connect to a finite number of neurons in the second layer. More generally, the locally connected layers of the locally connected neural network 204 can be configured such that each neuron in a layer will have the same or similar connectivity pattern, but its connection strength can have different values ​​(e.g., 210, 212, 214, and 216). The connectivity pattern of locally connected networks may produce spatially dissimilar receptive fields in higher layers because neurons in higher layers in a given region can receive inputs that are tuned to a restricted portion of the total input to the network through training.

[0041] An example of a locally connected neural network is a convolutional neural network. Figure 2C An example of a convolutional neural network 206 has been explained. Convolutional neural network 206 can be configured such that the connection strength associated with the input for each neuron in the second layer is shared (e.g., 208). Convolutional neural networks may be well-suited for problems where the spatial location of the input is meaningful.

[0042] One type of convolutional neural network is the deep convolutional network (DCN). Figure 2D A detailed example of a DCN 200 designed to recognize visual features from an image 226 input from an image capture device 230 (such as an in-vehicle camera) is explained. The DCN 200 of this example can be trained to identify traffic signs and the numbers provided on them. Of course, the DCN 200 can be trained for other tasks, such as identifying lane markings or traffic lights.

[0043] The DCN 200 can be trained using supervised learning. During training, images (such as image 226 of a speed limit sign) can be presented to the DCN 200, and a "forward pass" can then be computed to produce output 222. The DCN 200 may include feature extraction segments and classification segments. Upon receiving image 226, a convolutional layer 232 may apply a convolutional kernel (not shown) to image 226 to generate a first set of feature maps 218. As an example, the convolutional kernel of the convolutional layer 232 may be a 5x5 kernel that generates a 28x28 feature map. In this example, since four different feature maps are generated in the first set of feature maps 218, four different convolutional kernels are applied to image 226 at the convolutional layer 232. The convolutional kernel may also be referred to as a filter or convolutional filter.

[0044] The first set of feature maps 218 can be subsampled by a max-pooling layer (not shown) to generate a second set of feature maps 220. The max-pooling layer reduces the size of the first set of feature maps 218. That is, the size of the second set of feature maps 220 (e.g., 14x14) is smaller than the size of the first set of feature maps 218 (e.g., 28x28). The reduced size provides similar information to subsequent layers while reducing memory consumption. The second set of feature maps 220 can be further convolved via one or more subsequent convolutional layers (not shown) to generate one or more subsequent sets of feature maps (not shown).

[0045] exist Figure 2D In the example, the second set of feature maps 220 is convolved to generate a first feature vector 224. Furthermore, the first feature vector 224 is further convolved to generate a second feature vector 228. Each feature of the second feature vector 228 may include a number corresponding to a possible feature of the image 226 (such as "sign", "60", and "100"). A softmax function (not shown) converts the numbers in the second feature vector 228 into probabilities. Thus, the output 222 of the DCN 200 is the probability that the image 226 includes one or more features.

[0046] In this example, the probabilities of “sign” and “60” in output 222 are higher than the probabilities of other features in output 222 (such as “30”, “40”, “50”, “70”, “80”, “90”, and “100”). Before training, output 222 generated by DCN 200 is likely incorrect. Therefore, the error between output 222 and the target output can be calculated. The target output is the ground truth value of image 226 (e.g., “sign” and “60”). The weights of DCN 200 can then be adjusted so that the output 222 of DCN 200 is more closely aligned with the target output.

[0047] To adjust the weights, the learning algorithm can compute gradient vectors for the weights. This gradient indicates how much the error will increase or decrease as the weights are adjusted. At the top layers, this gradient directly corresponds to the values ​​of the weights connecting the activated neurons in the penultimate layer to the neurons in the output layer. In lower layers, the gradient depends on the values ​​of the weights and the error gradients computed from the higher layers. The weights can then be adjusted to reduce the error. This method of adjusting weights is called "backpropagation" because it involves a "back pass" in neural networks.

[0048] In practice, the error gradient of the weights may be calculated on a small number of examples, thus approximating the true error gradient. This approximation method is called stochastic gradient descent. Stochastic gradient descent can be repeated until the error rate achievable by the entire system stops decreasing or until the error rate reaches the target level. After learning, new images can be presented to the DCN, and the forward pass in the network produces an output 222, which can be considered an inference or prediction of the DCN.

[0049] Deep Belief Networks (DBNs) are probabilistic models that include multiple layers of hidden nodes. DBNs can be used to extract hierarchical representations of training datasets. DBNs can be obtained by stacking multiple layers of Restricted Boltzmann Machines (RBMs). RBMs are a class of artificial neural networks that can learn probability distributions on an input set. Because RBMs can learn probability distributions without information about which class each input should be classified into, they are often used in unsupervised learning. Using a hybrid unsupervised and supervised paradigm, the bottom RBM of a DBN can be trained unsupervised and used as a feature extractor, while the top RBM can be trained supervised (on the joint distribution of inputs from previous layers and the target class) and used as a classifier.

[0050] Deep convolutional networks (DCNs) are networks of convolutional networks configured with additional pooling and normalization layers. DCNs have achieved state-of-the-art performance on many tasks. DCNs can be trained using supervised learning, where both the input and output targets are known for many paradigms and are used to modify the network's weights using gradient descent.

[0051] DCNs can be feedforward networks. Furthermore, as mentioned above, the connections from neurons in the first layer of a DCN to the neuron group in the next higher layer are shared across neurons in the first layer. The feedforward and shared connections of a DCN can be used for fast processing. The computational burden of a DCN can be much smaller than, for example, a similarly sized neural network that includes backflow or feedback connections.

[0052] The processing of each layer in a convolutional network can be considered as a spatially invariant template or a fundamental projection. If the input is first decomposed into multiple channels, such as the red, green, and blue channels of a color image, then a convolutional network trained on that input can be considered three-dimensional, having two spatial dimensions along the axis of the image and a third dimension capturing color information. The output of the convolutional connections can be considered as forming a feature map in subsequent layers, where each element receives input from a range of neurons in the previous layer (e.g., feature map 220) and from each of those multiple channels. The values ​​in the feature map can be further processed non-linearly (such as correction, max(0, x)). Values ​​from neighboring neurons can be further pooled (which corresponds to downsampling) and provide additional local invariance and dimensionality reduction. Normalization, corresponding to whitening, can also be applied through lateral inhibition between neurons in the feature map.

[0053] The performance of deep learning architectures can improve as more labeled data points become available or as computational power increases. Modern deep neural networks are routinely trained with thousands of times more computational resources than were available to a typical researcher just fifteen years ago. New architectures and training paradigms can further boost the performance of deep learning. Corrected linear units reduce the training problem known as vanishing gradients. New training techniques reduce overfitting and thus enable larger models to achieve better generalization. Encapsulation techniques abstract data from a given receptive field and further improve overall performance.

[0054] Figure 3 This is a block diagram illustrating a Deep Convolutional Network 350. A Deep Convolutional Network 350 can include multiple layers of different types based on connectivity and weight sharing. For example... Figure 3 As shown, the deep convolutional network 350 includes convolutional blocks 354A and 354B. Each of the convolutional blocks 354A and 354B may be configured with a convolutional layer (CONV) 356, a normalization layer (LNorm) 358, and a max pooling layer (MAX POOL) 360.

[0055] Convolutional layer 356 may include one or more convolutional filters that can be applied to the input data to generate feature maps. Although only two convolutional blocks 354A and 354B are shown, this disclosure is not limited thereto, and any number of convolutional blocks 354A and 354B may be included in the deep convolutional network 350 according to design preferences. Normalization layer 358 may normalize the output of the convolutional filters. For example, normalization layer 358 may provide whitening or lateral suppression. Max pooling layer 360 may provide spatial downsampling aggregation to achieve local invariance and dimensionality reduction.

[0056] For example, the parallel filter set of the deep convolutional network can be loaded onto the CPU 102 or GPU 104 of the SOC 100 to achieve high performance and low power consumption. In an alternative embodiment, the parallel filter set can be loaded onto the DSP 106 or ISP 116 of the SOC 100. Additionally, the deep convolutional network 350 can access other processing blocks that may exist on the SOC 100, such as the sensor processor 114 and navigation module 120, respectively dedicated to sensors and navigation.

[0057] The deep convolutional network 350 may also include one or more fully connected layers 362 (FC1 and FC2). The deep convolutional network 350 may further include logistic regression (LR) layers 364. Weights (not shown) to be updated are located between each layer 356, 358, 360, 362, and 364 of the deep convolutional network 350. The output of each layer (e.g., 356, 358, 360, 362, and 364) can be used as input to a subsequent layer in the deep convolutional network 350 (e.g., 356, 358, 360, 362, and 364) to learn a hierarchical feature representation from the input data 352 (e.g., image, audio, video, sensor data, and / or other input data) supplied from the first convolutional block 354A. The output of the deep convolutional network 350 is a classification score 366 for the input data 352. The classification score 366 may be a set of probabilities, where each probability is the probability that the input data includes features from a feature set.

[0058] Various aspects of this disclosure relate to a prototype-based personalized pruning (PPP) model that uses prototype representations to learn a metric space. PPP considers both model complexity and personalization. PPP uses prototypes of individual identities driven by limited personalized data to select a subset of graphs with prototype modules. For this reason, the model structure of PPP can be dynamically customized to enhance personalization and reduce model complexity. According to various aspects of this disclosure, prototypes of individual identities can be generated for each individual (which can be defined as each identity or group of personal data). The prototypes of individual identities represent characteristics of the personal data. Dynamic path networks can be trained for personalized pruning to produce personalized models that are pruned versions of generalized or full models without fine-tuning using personal data.

[0059] During the training phase, the gating module can be trained to select appropriate channels for pruning. Personalization is also considered. A prototype of the individual identity can be defined using the output of the gating module. The output of the gating module is a binary embedding pattern for each convolutional module, indicating whether each channel should be pruned. To account for performance after pruning using the prototype of the individual identity during training, the output of the gating module can be regularized to resemble the output of the prototype of the individual identity. As a result, after training, the global model can be pruned more easily using, for example, a graph generator and a network pruner, the prototype of the individual identity derived from the given constrained individual data. Furthermore, after training, PPP can operate without a global model (e.g., a complete model) because channel selection is based on the prototype of the individual identity representing the individual data, rather than the incoming input used in a regular dynamic path network.

[0060] The architecture of PPP can be, for example, a convolutional neural network (e.g., Figure 3 (As shown in 350). A convolutional neural network can be configured with gating modules that control each channel of the convolutional module. It can be provided for each convolutional layer, and , ,in It is the input of the gating module. It is the embedding vector in the gating embedding space, and and These are the dimensions of the input and output channels of the convolutional layer associated with this gating module, respectively. A global average pooling layer can be used as the input... and have The broadcast operation therefore has no spatial dimension. Embedded vector Can we learn whether to use probability? Using channels, this probability It is a random variable in the range [0,1]. The softmax operation (e.g., Gumbel softmax) can be applied along with a pass-through estimator to train the gating module. Therefore, in the forward pass, it can be expressed with probability... Applying the argmax function for hard attention However, softmax can be used instead of argmax to relax the gradient, so that the sample can be represented by a vector. As expressed below: ,in yes The first in One element, It is a sequence of independent and identically distributed random variables, and It is the temperature parameter of the softmax function.

[0061] The PPP model can be trained based on a global model. Training dataset. Where K represents the number of classes. Parameter Marking and the p The first identity (e.g., the identity of an individual client) corresponds to the first i Data pairs. Convolutional layers can be specified as... ,in h and w These correspond to the height and width of the active area, respectively. x It is input activation and y This is the output of the convolutional layer:

[0062] (1)

[0063] in and They are the first l Parameters of convolutional layers and gating modules.

[0064] The output feature map and embedding vector can be calculated based on the individual identity of each mini-batch. To train the gating module for channel selection, a broadcast operator is used for each convolutional layer. To calculate the trimmed channel.

[0065] A prototype can represent an individual's identity. Within each mini-batch, a specific... p The number of identity data is Then, calculate all The average value of the embedding vectors within. Parameters Defined as all in the same batch p The average vector of each identity, which represents p Personality:

[0066] (2)

[0067] It can compute all of the same batch for all convolutional layers. p The average vector of each identity A collection. The set is p The prototype of an individual identity is not discrete due to averaging. During the testing phase using the pruned model, the prototype of an individual identity can be a discrete vector. In some respects, the prototype of an individual identity can be represented using an element-wise step function. and threshold To convert into a discrete vector: (3)

[0068] The personalized regularization loss can be calculated as follows:

[0069]

[0070] Where C is the set of convolutional layers in the network. B It includes small batches of several identities, and P It is the number of individual identities.

[0071] loss Regularize the data for each group of individuals to select similar gating patterns (such as prototypes of personal identity). Therefore, neural networks can be based on... To prune. PPP can rely on prototypes representing the individual identities of the group's data rather than the incoming input. Furthermore, regularization loss can be computed without a distance metric between prototypes. .

[0072] The target loss can be used as a soft constraint on network utilization. It can be calculated by counting all active channels (filters) in each convolutional layer, and the utilization can be regularized using a specific objective. Thus, the target loss can be given by:

[0073]

[0074] Therefore, target loss Tools can be provided for adjusting or controlling network complexity.

[0075] Using standard cross-entropy loss The total training loss can be expressed as:

[0076]

[0077] in and It is a hyperparameter used to balance the loss.

[0078] Unlike conventional techniques that use data-dependent paths (which always use the entire network, e.g., because it selects the forward graph based on the input)), PPP is independent of the input after training. Instead, according to various aspects of this disclosure, the forward graph can depend on a prototype of a given individual's data. Therefore, unlike conventional personalization schemes (such as fine-tuning a global model using individual data), PPP dynamically generates individual models, thereby reducing computational complexity.

[0079] During the testing phase, when given limited individual data, Equation 2 can be used to compute a prototype of an individual's identity. Using this computed prototype, a personalized, pruned model can be generated by using a graph generator and a network pruner to eliminate or prune convolutional layer filters for a given binary pattern specific to each individual, without additional training. In some respects, pruning can be repeated to further enhance the personalized model.

[0080] Figure 4 This is a block diagram illustrating an example of trimming a diagram of a complete or global model 400 based on various aspects of this disclosure to generate a personalized model 402. (See reference) Figure 4 The graph of the complete model 400 may include multiple nodes (e.g., 406a), with interconnections (e.g., 408a) between the nodes (e.g., 406a). The graph of the complete model 400 may represent a convolutional neural network (e.g., Figure 3 (as shown in 350). For example, each column node of the complete model 400 (e.g., 406a) can correspond to a convolutional layer of a convolutional neural network (e.g., ...). Figure 3 (As shown in 356). Each interconnect 408a can represent a channel to a node. If all channels to or from a node (e.g., 406z) are pruned or removed, that node (e.g., 406z) can be considered removed.

[0081] According to various aspects of this disclosure, personal data can be used to prune the complete model 400 to remove or eliminate interconnects 408a or channels. As described, the convolutional neural network can be configured with gating modules that control each channel of the convolutional module. It can be provided for each convolutional layer, and , ,in x It is the input of the gating module. z It is the embedding vector in the gating embedding space, and n and m These are the dimensions of the input and output channels of the convolutional layer associated with the gating module, respectively. Spatial global average pooling of the input x can be used. 。 Using embedding vectors z The gating module can learn whether to use probability. p Using channels, this probability p It is a random variable in the range [0,1]. In some respects, Gumblesoftmax and pass-through estimators can be used to train the gating module. That is, in the forward pass of the neural network, argmax is applied with probability p for hard attention. The gradient can be calculated via Gumble softmax.

[0082] Furthermore, each node not connected to another node (shown by dashed line 406z) can also be removed. By removing interconnects 408a and nodes 406a, a customized model 402 can be generated. Doing so reduces network size and computational complexity.

[0083] Figure 5 This is a block diagram 500 illustrating an example implementation of a personalized model based on various aspects of this disclosure. (See reference) Figure 5 For example, smartphone 502 can register for voice-assisted activation. Prototype generator 504 can create a user prototype based on registration samples. Graph generator 506 can determine the network graph (structure) based on the personally identifiable user prototype. The graph structure can be provided to network pruner 508, which can then generate a personalized model by pruning the network based on the determined network graph. In some aspects, the pruned model can also be fine-tuned using user samples. Therefore, a user's voice can be processed using a small, personalized network.

[0084] Figure 6A and 6B These are block diagrams illustrating prototype-based Personalized Pruning (PPP) models 600 and 650, respectively, based on various aspects of this disclosure. (See references.) Figure 6A The PPP model 600 includes a prototype generator 602, a graph generator 604, and a network pruner 606. The prototype generator 602 can generate a prototype of an individual's identity based on personal data samples, such as those obtained via sensor 608 (e.g., a camera or microphone). The prototype generator 602 can provide the personal prototype to the graph generator 604. The graph generator 604 can then determine the network graph structure based on the personal prototype. Subsequently, the graph structure can be provided to the network pruner 606, for example, as meta-information. Using the full model 610, the network pruner 606 performs binary determination, removing channels from the full model 610 based on the determined graph to generate a smaller, personalized model (shown as the pruned model 612). During operation, input data can be provided to the pruned model 612 via sensor 608 to generate predictions (e.g., image classification or indications about detected keywords).

[0085] refer to Figure 6B The PPP model 650 can be further configured for knowledge transfer from the full model 652. The PPP model 650 may have a similar configuration and functionality to the PPP model 600. During the teaching phase, data from sensor 656 (e.g., a camera or microphone) can be provided to the full model 652. The full model 652 can generate predictions, which can then be provided to the pruned model 654 to further train the personalized model.

[0086] Figure 6CThis is a block diagram illustrating an example residual block 670 of a neural network according to various aspects of this disclosure. (See reference) Figure 6C Neural networks (e.g.) Figure 3 The example residual block 670 shown (350) includes gating modules 672a, 672b and prototype generators 674a, 674b at each convolutional layer 676a, 676b of the neural network. Although two convolutional layers are shown, this is merely an example for ease of explanation. In this configuration, gating modules 672a, 672b control each channel of the convolutional block, rather than the entire residual block 670.

[0087] For example, based on Equation 2, prototype generator 674 uses the output of gating module 672 to generate a continuous prototype for one or more personal identity data. For example, prototype generator 674 can also convert a continuous prototype into a discrete prototype using the step function of Equation 3.

[0088] Figure 7 This is a flowchart illustrating an example method 700 for generating personalized network models according to various aspects of this disclosure. For example... Figure 7 As shown, in box 702, a personal data sample is received. In one example, the personal data sample may be generated via a sensor such as a camera or microphone.

[0089] In box 704, a prototype of a personal identity is generated based on a sample of personal data. This prototype is trained to reflect the user's personal characteristics. As described, for example, refer to... Figure 6A The prototype generator 602 can generate prototypes of personal identities based on personal data samples, such as those obtained via sensor 608 (e.g., a camera or microphone). The prototypes of personal identities can be generated for each individual (which can be defined as each identity or group of personal data). That is, the prototype can represent an individual identity. In each mini-batch, a specific... p The number of identity data is Then, calculate all The average value of the embedding vectors within. Parameters Defined as all in the same batch p The average vector of each identity, which represents p The individuality. Subsequently, all convolutional layers in the same batch can be computed. p The average vector of each identity A collection. The set is p The prototype.

[0090] In box 706, a network graph is generated based on the prototype of this individual identity. For example, see reference... Figure 6AAs described, graph generator 604 can receive prototypes from prototype generator 602. Graph generator 604 can determine the network graph structure based on the individual prototypes.

[0091] In box 708, channels of the global network are pruned based on this network graph to produce a personalized model. For example, as referenced... Figure 6A The graph structure described above, generated by graph generator 604, can be provided, for example, as metadata to network pruner 606. Using the full model 610, network pruner 606 performs binary determination, which removes channels of the full model 610 based on the determined graph to generate a smaller, personalized model (shown as pruned model 612).

[0092] Examples of implementations are provided in the following numbered clauses:

[0093] 1. A computer-implemented method, comprising:

[0094] Receive one or more samples of personal data;

[0095] A prototype of an individual identity is generated based on samples of personal data. This prototype is trained to reflect the user's personal characteristics.

[0096] Generate network graphs based on the prototype of this individual identity; and

[0097] Based on this network graph, one or more channels of the global network are pruned to produce a personalized model.

[0098] 2. The computer-implemented method of Clause 1, wherein the network graph is generated based on the task and objective size of the personalized model.

[0099] 3. A computer-implemented method as described in Clause 1 or 2, wherein the prototype of the personal identity is learned by minimizing the distance between features of a sample of personal data and the prototype of the personal identity corresponding to the user.

[0100] 4. The computer-implemented method of any one of clauses 1-3 further includes:

[0101] Input is received via this personalized model; and

[0102] Inferences are generated based on this input via the personalized network.

[0103] 5. The computer-implemented method of any one of clauses 1-4 further includes:

[0104] Receive output from the global network; and

[0105] This output can be used to fine-tune the personalized network.

[0106] 6. A computer-implemented method as described in any of Clauses 1-5, wherein the prototype of the individual identity is the average vector of individual data samples in the same batch.

[0107] 7. A computer-implemented method as described in any of Clauses 1-6, wherein the one or more channels are trimmed based on a binary determination of the individual's identity prototype.

[0108] 8. An apparatus comprising:

[0109] Memory; and

[0110] At least one processor coupled to the memory, the at least one processor being configured to:

[0111] Receive one or more samples of personal data;

[0112] A prototype of an individual identity is generated based on samples of personal data. This prototype is trained to reflect the user's personal characteristics.

[0113] Generate network graphs based on the prototype of this individual identity; and

[0114] Based on this network graph, one or more channels of the global network are pruned to produce a personalized model.

[0115] 9. The apparatus of Clause 8, wherein the at least one processor is further configured to generate the network graph based on the task and target size of the personalized model.

[0116] 10. The apparatus of claim 8 or 9, wherein the at least one processor is further configured to learn the prototype of the personal identity by minimizing the distance between features of the personal data sample and the prototype of the personal identity corresponding to the user.

[0117] 11. The apparatus of any one of claims 8-10, wherein the at least one processor is further configured to:

[0118] Input is received via this personalized model; and

[0119] Inferences are generated based on this input via the personalized network.

[0120] 12. The apparatus of any one of claims 8-11, wherein the at least one processor is further configured to:

[0121] Receive output from the global network; and

[0122] This output can be used to fine-tune the personalized network.

[0123] 13. The apparatus of any of Clauses 8-12, wherein the prototype of the personal identity is the average vector of personal data samples in the same batch.

[0124] 14. The apparatus of any of clauses 8-13, wherein the one or more channels are trimmed based on binary determination according to the prototype of the individual's identity.

[0125] 15. An apparatus comprising:

[0126] Device for receiving one or more samples of personal data;

[0127] A device for generating a prototype of a personal identity based on samples of personal data, the prototype of which is trained to reflect the user’s personal characteristics.

[0128] A device for generating a network graph based on a prototype of the individual's identity; and

[0129] A device for pruning one or more channels of a global network based on the network graph to produce a personalized model.

[0130] 16. The device as described in Clause 15 further includes means for generating the network graph based on the task and target size of the personalized model.

[0131] 17. The device as described in Clause 15 or 16 further includes means for learning a prototype of a personal identity by minimizing the distance between features of a sample of personal data and a prototype corresponding to that personal identity of the user.

[0132] 18. The equipment as described in any of Clauses 15-17, further comprising:

[0133] Device for receiving input via the personalized model; and

[0134] A device for generating inferences based on the input via the personalized network.

[0135] 19. The equipment as described in any of Clauses 15-18, further comprising:

[0136] A means for receiving output from the global network; and

[0137] A device for fine-tuning the personalized network based on the output.

[0138] 20. An apparatus as described in any of Clauses 15-19, wherein the prototype of the personal identity is the average vector of personal data samples in the same batch.

[0139] 21. An apparatus as described in any of Clauses 15-20, wherein the one or more channels are trimmed based on a binary determination of the prototype of the individual's identity.

[0140] 22. A non-transitory computer-readable medium having program code encoded thereon, the program code being executed by a processor and comprising:

[0141] Program code used to receive one or more samples of personal data;

[0142] Program code used to generate a prototype of an individual identity based on samples of personal data, the prototype of which is trained to reflect the user's personal characteristics.

[0143] Program code used to generate network graphs based on the prototype of this individual's identity; and

[0144] Program code used to prune one or more channels of the global network based on the network graph to produce a personalized model.

[0145] 23. The non-transient computer-readable medium as described in Clause 22 further includes program code for generating the network graph based on the task and objective size of the personalized model.

[0146] 24. The non-transient computer-readable medium of Clause 22 or 23 further includes program code for learning a prototype of an individual identity by minimizing the distance between features of a sample of personal data and a prototype corresponding to that individual identity of the user.

[0147] 25. A non-transient computer-readable medium as described in any of Clauses 22-24, further comprising:

[0148] The program code used to receive input via this personalized model; and

[0149] Program code used to generate inferences based on the input via the personalized network.

[0150] 26. A non-transient computer-readable medium as described in any of Clauses 22-25, further comprising:

[0151] Program code used to receive output from this global network; and

[0152] The program code used to fine-tune the personalized network based on this output.

[0153] 27. A non-transient computer-readable medium such as that in any of Clauses 22-26, wherein the prototype of the personal identity is the average vector of personal data samples in the same batch.

[0154] 28. A non-transient computer-readable medium as described in any of Clauses 22-27, wherein the one or more channels are trimmed based on a binary determination of the prototype of the individual's identity.

[0155] In one aspect, the receiving device, the device for generating a prototype of an individual identity, the device for generating a network graph, and / or the pruning device may be a CPU 102, a program memory associated with the CPU 102, a dedicated memory block 118, and / or an NPU 108 configured to perform the functions described herein. In another configuration, the aforementioned devices may be any module or any equipment configured to perform the functions described herein.

[0156] The various operations of the methods described above can be performed by any suitable means capable of performing the corresponding functions. These means may include various hardware and / or software components and / or modules, including but not limited to circuits, application-specific integrated circuits (ASICs), or processors. Generally, in the cases where operations are illustrated in the accompanying drawings, those operations may have corresponding paired means with similar numbers plus functional components.

[0157] As used herein, the term "determine" encompasses a wide variety of actions. For example, "determine" can include calculation, computation, processing, derivation, research, searching (e.g., looking in a table, database, or other data structure), ascertaining, and the like. Additionally, "determine" can include receiving (e.g., receiving information), accessing (e.g., accessing data in memory), and similar actions. Furthermore, "determine" can include parsing, selecting, choosing, establishing, and similar actions.

[0158] As used in this article, the phrase “at least one of” referring to a list of items means any combination of those items, including a single member. As an example, “at least one of a, b, or c” is intended to cover: a, b, c, ab, ac, bc, and abc.

[0159] The various illustrative logic blocks, modules, and circuits described in this disclosure can be implemented or executed using a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA) or other programmable logic device (PLD), discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the described functions. The general-purpose processor may be a microprocessor, but in alternatives, the processor may be any commercially available processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors cooperating with a DSP core, or any other such configuration.

[0160] The steps of the methods or algorithms described in this disclosure can be implemented directly in hardware, in a software module executed by a processor, or in a combination of both. The software module can reside in any form of storage medium known in the art. Some examples of usable storage media include random access memory (RAM), read-only memory (ROM), flash memory, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, removable disks, CD-ROMs, and so on. The software module may include a single instruction or many instructions, and may be distributed across several different code segments, across different programs, and across multiple storage media. The storage medium may be coupled to the processor so that the processor can read and write information from / to the storage medium. In an alternative, the storage medium may be integrated into the processor.

[0161] The methods disclosed herein include one or more steps or actions for achieving the described methods. These method steps and / or actions may be interchanged with each other without departing from the scope of the claims. In other words, unless a specific order of steps or actions is specified, the order and / or use of specific steps and / or actions may be modified without departing from the scope of the claims.

[0162] The described functionality can be implemented in hardware, software, firmware, or any combination thereof. If implemented in hardware, an example hardware configuration may include a processing system within the device. The processing system can be implemented using a bus architecture. Depending on the specific application and overall design constraints of the processing system, the bus may include any number of interconnect buses and bridges. The bus can link together various circuits, including processors, machine-readable media, and bus interfaces. The bus interface can be used, in particular, to connect network adapters to the processing system via the bus. The network adapter can be used to implement signal processing functions. In some respects, user interfaces (e.g., keypads, displays, mice, joysticks, etc.) may also be connected to the bus. The bus can also link various other circuits, such as timing sources, peripherals, regulators, power management circuits, and similar circuits, which are well known in the art and will not be described further.

[0163] A processor is responsible for managing the bus and general processing, including executing software stored on a machine-readable medium. A processor may be implemented using one or more general-purpose and / or special-purpose processors. Examples include microprocessors, microcontrollers, DSP processors, and other circuit systems capable of executing software. Software should be interpreted broadly as instructions, data, or any combination thereof, whether referred to as software, firmware, middleware, microcode, hardware description language, or otherwise. As examples, a machine-readable medium may include random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, disks, optical disks, hard drives, or any other suitable storage medium, or any combination thereof. The machine-readable medium may be implemented in a computer program product. This computer program product may include packaging materials.

[0164] In hardware implementations, machine-readable media can be a separate part of the processing system from the processor. However, as those skilled in the art will readily appreciate, machine-readable media or any part thereof can be external to the processing system. As examples, machine-readable media may include transmission lines, carrier waves modulated by data, and / or computer components separate from the device, all accessible to the processor via a bus interface. Alternatively or additionally, machine-readable media or any part thereof may be integrated into the processor, such as caches and / or general-purpose register files. While the various components discussed may be described as having a specific location, such as local components, they can also be configured in various ways, such as certain components being configured as part of a distributed computing system.

[0165] The processing system can be configured as a general-purpose processing system having one or more microprocessors providing processor functionality, and external memory providing at least a portion of machine-readable medium, all linked to other supporting circuitry via an external bus architecture. Alternatively, the processing system may include one or more neuromorphic processors for implementing the neuron and nervous system models described herein. As another alternative, the processing system can be implemented using an application-specific integrated circuit (ASIC) with a processor, bus interface, user interface, supporting circuitry, and at least a portion of machine-readable medium integrated on a single chip, or using one or more field-programmable gate arrays (FPGAs), programmable logic devices (PLDs), controllers, state machines, gated logic, discrete hardware components, or any other suitable circuitry, or any combination of circuitry capable of performing the various functionalities described throughout this disclosure. Depending on the specific application and the overall design constraints imposed on the system, those skilled in the art will recognize how best to implement the functionality described with respect to the processing system.

[0166] Machine-readable media may include several software modules. These software modules include instructions that, when executed by a processor, cause the processing system to perform various functions. These software modules may include transfer modules and receive modules. Each software module may reside in a single storage device or be distributed across multiple storage devices. As an example, when a trigger event occurs, a software module may be loaded from a hard drive into RAM. During the execution of a software module, the processor may load some instructions into a cache to improve access speed. One or more cache lines may subsequently be loaded into a general-purpose register file for processor execution. In the context of the functionality of the software modules described below, it will be understood that such functionality is implemented by the processor when the processor executes the instructions from the software module. Furthermore, it should be understood that aspects of this disclosure result in improvements to the functionality of a processor, computer, machine, or other system implementing such aspects.

[0167] If implemented in software, the functions can be stored or transmitted as one or more instructions or codes on or through a computer-readable medium. Computer-readable media includes both computer storage media and communication media, encompassing any medium that facilitates the transfer of a computer program from one location to another. Storage media can be any available medium accessible to a computer. By way of example and not limitation, such computer-readable media may include RAM, ROM, EEPROM, CD-ROM or other optical disc storage, disk storage or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible to a computer. Additionally, any connection is also legitimately referred to as computer-readable media. For example, if the software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technology (such as infrared (IR), radio, and microwave), then that coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technology (such as infrared, radio, and microwave) is included in the definition of medium. As used herein, disks and discs include compact discs (CDs), laser discs, optical discs, digital multi-purpose discs (DVDs), floppy disks, and Blu-ray® discs, where disks typically reproduce data magnetically, while discs optically reproduce data using lasers. Therefore, in some aspects, computer-readable media may include non-transient computer-readable media (e.g., tangible media). Additionally, in other aspects, computer-readable media may include transient computer-readable media (e.g., signals). Combinations of the above should also be included within the scope of computer-readable media.

[0168] Therefore, some aspects may include a computer program product for performing the given operations. For example, such a computer program product may include a computer-readable medium on which instructions are stored (and / or encoded) that can be executed by one or more processors to perform the described operations. In some aspects, the computer program product may include packaging material.

[0169] Furthermore, it should be understood that modules and / or other suitable means for performing the described methods and techniques may be downloaded and / or otherwise obtained by the user terminal and / or base station where applicable. For example, such devices can be coupled to a server to facilitate the transfer of means for performing the described methods. Alternatively, the various methods described can be provided via storage devices (e.g., RAM, ROM, physical storage media such as CDs or floppy disks) so that the device can acquire the various methods once the storage device is coupled to or provided to the user terminal and / or base station. In addition, any other suitable techniques suitable for providing the described methods and techniques to the device may be utilized.

[0170] It will be understood that the claims are not limited to the precise configurations and components described above. Various modifications, substitutions, and variations can be made to the layout, operation, and details of the methods and apparatus described above without departing from the scope of the claims.

Claims

1. A computer-implemented method, comprising: One or more samples of a user's personal data are generated by sensors or input devices on an edge device; A prototype of a personal identity is generated based on one or more personal data samples, and the prototype of the personal identity is trained to reflect the user's personal characteristics. Generate a network graph based on the prototype of the individual identity; as well as Based on the network graph, one or more channels of the global network are pruned to generate a personalized model for deployment on the edge device. The method further includes receiving input by the sensor or the input device of the edge device, the input including an image associated with the user or voice from the user; And based on the input, the personalized model is used at the edge device to generate inferences, the inferences responding to the input being the image rather than image classification, or responding to the input including the voice rather than speech processing or keyword detection.

2. The computer-implemented method of claim 1, wherein the network graph is generated based on the task and target size of the personalized model.

3. The computer-implemented method of claim 1, wherein the prototype of the personal identity is learned by minimizing the distance between features of the one or more personal data samples and the prototype of the personal identity corresponding to the user.

4. The computer-implemented method as described in claim 1, further comprising: Input is received via the personalized model; as well as Inferences are generated based on the input via the personalized model.

5. The computer-implemented method of claim 1, wherein the prototype of the personal identity is the average vector of personal data samples in the same batch.

6. The computer-implemented method of claim 5, wherein the one or more channels are pruned based on a binary determination of the individual's identity prototype.

7. An apparatus comprising: Memory; as well as At least one processor coupled to the memory, the at least one processor being configured to: One or more samples of a user's personal data are generated through sensors or input devices on edge devices; A prototype of a personal identity is generated based on one or more personal data samples, and the prototype of the personal identity is trained to reflect the user's personal characteristics. Generate a network graph based on the prototype of the individual identity; as well as Based on the network graph, one or more channels of the global network are pruned to generate a personalized model for deployment on the edge device. The at least one processor is configured to: receive input via the sensor or input device of the edge device, the input including an image associated with the user or voice from the user; and generate an inference at the edge device using the personalization model based on the input, the inference being image classification in response to the input being the image, or speech processing or keyword detection in response to the input including the voice.

8. The apparatus of claim 7, wherein the at least one processor is further configured to generate the network graph based on the task and target size of the personalized model.

9. The apparatus of claim 7, wherein the at least one processor is further configured to learn the prototype of the personal identity by minimizing the distance between features of the one or more personal data samples and a prototype of the personal identity corresponding to the user.

10. The apparatus of claim 7, wherein the at least one processor is further configured to: Input is received via the personalized model; and Inferences are generated based on the input via the personalized model.

11. The apparatus of claim 7, wherein the prototype of the personal identity is the average vector of personal data samples in the same batch.

12. The apparatus of claim 7, wherein the one or more channels are trimmed based on binary determination according to the prototype of the individual identity.

13. An apparatus comprising: A device for generating one or more samples of a user's personal data via sensors or input devices on an edge device; Apparatus for generating a prototype of a personal identity based on one or more samples of personal data, the prototype of the personal identity being trained to reflect the user's personal characteristics. Apparatus for generating network graphs based on prototypes of the personal identities; as well as A means for pruning one or more channels of the global network based on the network graph to generate a personalized model for deployment on the edge device. The device further includes means for receiving input via the sensor or the input device of the edge device, the input including an image associated with the user or voice from the user; And means for generating inferences at the edge device using the personalized model based on the input, the inferences being in response to the input being the image rather than image classification, or in response to the input including the voice rather than speech processing or keyword detection.

14. The apparatus of claim 13, further comprising means for generating the network graph based on the task and target size of the personalized model.

15. The device of claim 13, further comprising means for learning the prototype of the personal identity by minimizing the distance between features of the one or more personal data samples and a prototype of the personal identity corresponding to the user.

16. The apparatus of claim 13, further comprising: Device for receiving input via the personalized model; as well as A device for generating inferences based on the input via the personalized model.

17. The device of claim 13, wherein the prototype of the personal identity is the average vector of personal data samples in the same batch.

18. The device of claim 13, wherein the one or more channels are trimmed based on binary determination according to the prototype of the individual identity.

19. A non-transitory computer-readable medium having program code encoded thereon, the program code being executed by a processor and comprising: Program code used to generate one or more samples of a user's personal data via sensors or input devices on an edge device; Program code for generating a prototype of an individual identity based on the personal data sample, the prototype of the individual identity being trained to reflect the user's personal characteristics; Program code used to generate a network graph based on the prototype of the individual's identity; as well as Program code used to prune one or more channels of the global network based on the network graph to generate a personalized model for deployment on the edge device. The program code includes: program code for receiving input via the sensor or input device of the edge device, the input including an image associated with the user or voice from the user; And program code for generating inferences at the edge device using the personalized model based on the input, the inferences responding to the input being the image rather than image classification, or responding to the input including the voice rather than speech processing or keyword detection.

20. The non-transient computer-readable medium of claim 19, further comprising: Program code for generating the network graph based on the task and target size of the personalized model.

21. The non-transient computer-readable medium of claim 19, further comprising program code for learning a prototype of the personal identity by minimizing the distance between features of the one or more personal data samples and a prototype of the personal identity corresponding to the user.

22. The non-transient computer-readable medium of claim 19, further comprising: Program code used to receive input via the personalized model; as well as Program code used to generate inferences based on the input via the personalized model.

23. The non-transient computer-readable medium of claim 19, wherein the prototype of the personal identity is the average vector of personal data samples in the same batch.

24. The non-transient computer-readable medium of claim 19, wherein the one or more channels are trimmed based on binary determination according to the prototype of the personal identity.

Images