Variable parameters of machine learning models during runtime

Abstract

The present disclosure relates to variable parameters of machine learning models during runtime. The subject technology receives code corresponding to a neural network (NN) model and a set of weights for the NN model. The subject technology determines a set of variable layers in the NN model. The subject technology determines information for mapping a second set of weights to the set of weights for the NN model. The subject technology generates metadata corresponding to the set of variable layers and the information for mapping the second set of weights to the set of weights for the NN model, where the generated metadata enables updating of the set of variable layers during execution of the NN model.

Description

[0001] Cross-reference to related applications

[0002] This patent application claims the benefit of U.S. Provisional Patent Application Serial No. 62 / 855,898, filed May 31, 2019, entitled “MUTABLE PARAMETERS FORMACHINE LEARNING MODELS DURING RUNTIME”, which is incorporated herein by reference in its entirety and forms part of this U.S. Utility Patent Application for all purposes. Technical Field

[0003] This specification pertains to providing neural network models that execute on the target platform. Background Technology

[0004] Software engineers and scientists have been using computer hardware across various industries to improve machine learning applications, including image classification, video analytics, speech recognition, and natural language processing. Notably, neural networks are being used more and more frequently to create systems capable of performing different computational tasks based on training on large amounts of data. Attached Figure Description

[0005] Some features of this subject matter are set forth in the appended claims. However, for illustrative purposes, several embodiments of this subject matter are illustrated in the following figures.

[0006] Figure 1 An example network environment based on one or more specific implementations is shown.

[0007] Figure 2 An example software stack implemented on an electronic device according to one or more specific implementations is shown, which is used to compile source code locally, generate metadata for the variable parameters of a neural network model, and load the model in an application running on the electronic device.

[0008] Figure 3 An example structure of a variable weight file provided by a user or application, based on one or more specific implementations, is shown for updating the variable weights of a neural network model.

[0009] Figure 4 An example structure of a compiler-generated metadata section, based on one or more specific implementations, is shown. This metadata section will be included as part of the compiled binary of a neural network model to facilitate the updating of the neural network model's variable weights during runtime.

[0010] Figure 5A flowchart of an example process according to one or more specific implementations is shown, which is used to generate metadata for a neural network to update parameters during runtime.

[0011] Figure 6 A flowchart is shown, illustrating an example process according to one or more specific implementations, for using Figure 5 The generated metadata, as described in the document, is used to compile the neural network.

[0012] Figure 7 A flowchart of an example process 700 according to one or more specific implementations is shown, which is used to update the weights of a currently executing neural network model.

[0013] Figure 8 An electronic system that can be used to implement one or more specific embodiments of the subject matter technology is shown. Detailed Implementation

[0014] The specific embodiments listed below are intended to describe various configurations of the subject matter and are not intended to represent the only configuration in which the subject matter can be practiced. The accompanying drawings are incorporated herein and form part of the detailed description. The detailed description includes specific details intended to provide a thorough understanding of the subject matter. However, the subject matter is not limited to the specific details set forth herein, but can be practiced in one or more other specific embodiments. In one or more specific embodiments, structures and components are shown in block diagram form to avoid obscuring the concepts of the subject matter.

[0015] The popularity of machine learning has surged in recent years, particularly due to the availability of vast amounts of training data and advancements in more powerful and efficient computing hardware. A common machine learning technique involves using deep neural networks to perform a set of machine learning tasks. A common approach to training deep neural networks is to utilize graphics processing units (GPUs), which are also used to execute the deep neural network on new input data after training. However, in some cases, when executing a given deep neural network, depending on the machine learning task, it may be impossible to update certain parameters of the network. For example, some types of parameter data are included as part of the compiled binary of the neural network, which is executed by the machine learning application on the target device. Additionally, in some runtime environments, the machine learning application may not have direct access to the neural network. Therefore, it may be impossible to update the parameter data in the neural network during runtime.

[0016] The implementation of the techniques described herein enables the parameters (e.g., weights) of a neural network to be updated during execution by an electronic device, and improves the computational capabilities of the electronic device by eliminating the need for additional computational resources when recompiling the neural network to update the parameters. Advantageously, the neural network can adapt to changing conditions (e.g., environment) more quickly and perform machine learning tasks that respond faster to such changing conditions. Therefore, these beneficial effects are understood to improve the computational capabilities of a given electronic device, such as an end-user device, which typically has fewer available computational resources than, for example, one or more cloud-based servers.

[0017] Figure 1 An example network environment 100 according to one or more embodiments is illustrated. However, not all depicted components are usable in all embodiments, and one or more embodiments may include additional or different components compared to those shown in the figures. Variations in the arrangement and type of these components may be made without departing from the spirit or scope of the claims set forth herein. Additional components, different components, or fewer components may be provided.

[0018] Network environment 100 includes electronic devices 110, 115, and server 120. Network 106 may communicatively (directly or indirectly) couple electronic devices 110 and / or server 120, electronic devices 115 and / or server 120 and / or electronic devices 110 and / or electronic devices 115. In one or more embodiments, network 106 may be an interconnection network of devices that may include the Internet or be communicatively coupled to the Internet. For illustrative purposes, network environment 100 is described in... Figure 1 The diagram shows electronic devices 110, 115, and server 120; however, network environment 100 may include any number of electronic devices and any number of servers.

[0019] Electronic device 110 can be, for example, a desktop computer, portable computing devices such as laptops, smartphones, peripherals (e.g., digital cameras, headphones), tablets, wearable devices such as watches, wristbands, etc. Figure 1 In this context, by way of example, electronic device 110 is depicted as a desktop computer. Electronic device 110 may be and / or may include, as described below relative to... Figure 8 The electronic system under discussion, in whole or in part.

[0020] In one or more embodiments, electronic device 110 may provide a system for generating metadata to enable variable parameters in a given neural network model. As mentioned herein, variable parameters refer to the parameters of a machine learning model that can be updated when the neural network model is executed. Specifically, the subject system may include a neural network compiler for compiling code corresponding to the neural network model. In one example, by using the compiled code, the subject system may create an executable software package to be deployed on a target platform such as electronic device 115 under the guidance of server 120. When the compiled code is executed, the target platform may perform one or more given operations of the neural network model.

[0021] Electronic device 115 can be, for example, a portable computing device such as a laptop computer, a smartphone, peripheral devices (e.g., a digital camera, headphones), a tablet device, a wearable device such as a watch, a wristband, etc., or any electronic device. The electronic device may also include processors with different computing capabilities, including, for example, CPUs, GPUs, and / or neural processors. Figure 1 In this example, electronic device 115 is described as a smartphone device. In one or more embodiments, electronic device 115 may be and / or may include the following relative to... Figure 8 The electronic system under discussion, in whole or in part.

[0022] In one or more specific implementations, server 120 deploys compiled code, included in an executable software package, to a target device for execution. In one example, electronic device 115 may be a target device for receiving a software package containing compiled neural network code and for executing the compiled code within the runtime environment of electronic device 115. Electronic device 115 (or any electronic device serving as a target device) includes a framework enabled to perform operations within the compiled code of the neural network. A framework can refer to a software environment that provides specific functionality as part of a larger software platform to facilitate the deployment of software applications.

[0023] Figure 2 An example software stack implemented on an electronic device (e.g., electronic device 115) according to one or more specific embodiments is illustrated. This software stack is used to compile source code locally, generate metadata for variable parameters of a neural network model, and load the model in an application executing on the electronic device. The software stack may include different layers corresponding to different address spaces in the memory of the electronic device. In some specific embodiments, the electronic device may include its own neural network compiler, which enables the device to support compiling the source code of a neural network model. In this way, the device can load and execute a locally compiled neural network model without involving another device (e.g., server 120 or electronic device 110) to compile such source code. Although electronic device 115 is mentioned, it should be understood that... Figure 2 The software stack shown can be implemented using any suitable device, including neural processors, to support the execution of neural network models outside of CPUs and / or GPUs.

[0024] In one or more embodiments, electronic device 115 may include one or more portions of a software stack for running a neural network model, but may not include one or more portions of a software stack for compiling a neural network model.

[0025] As shown in the figure, the software stack includes application 210 at its first layer. Application 210 may include components such as a first machine learning software library 212, a second machine learning software library 214, and a neural processor framework 216. In one implementation, the first machine learning software library 212 may be publicly available for use by third parties (e.g., developers who write code for application 210), while the second machine learning software library 214 may not be accessible to third parties and may only be used internally by some components shown in the software stack. Below the first layer is an intermediate layer (“System”) that includes a neural processor daemon 240, a neural processor compiler service 250, and a model cache 260. In one example, the neural processor daemon 240 is a secure background process that the neural processor framework 216 can communicate with to perform operations for compiling, loading, and / or unloading neural network models. As further shown, a neural processor driver 270 resides in the layer corresponding to the kernel (e.g., the operating system running on the electronic device 115), and below the neural processor driver 270 is the neural processor firmware 280. In one example, the neural processor driver 270 allows other software (e.g., application 210 and / or neural processor daemon 240) to communicate with the neural processor firmware 280, which enables such software to control the neural processor included in the electronic device 115 (e.g., via executing commands).

[0026] In one example, a user (e.g., a developer and / or application 210) provides code and information for a neural network (NN) model with layers and weights (e.g., scaling, bias, kernel weights, activation parameters), which may include weights and layers that are variable during runtime. In one example, weights may be assigned to specific layers of the NN model. Additionally, one or more layers of the NN model include bias values ​​and scaling values ​​(e.g., weights) that can be updated at runtime to modify the layer's execution. In the specific implementation described herein, only certain types of layers in the neural network, such as gain-offset control (GOC) layers, may include variable parameters, while other layers in the neural network may include immutable parameters that cannot be updated during runtime. However, similar techniques can be generalized to other types of layers, such as convolutional layers where kernel weights can be changed, or activation layers where activation parameters can also be changed using similar mechanisms.

[0027] The aforementioned code and information (e.g., including weights and layers) for the NN model are provided to a neural processor daemon 240 (e.g., a security daemon), which then sends the NN model's code and information, along with the weights, to a compiler 252 provided by a neural processor compiler service 250. The neural processor compiler service 250 will be discussed in further detail below.

[0028] Compiler 252 generates a compiled binary file of the NN model and is configured during compilation to generate metadata that enables the weights of the NN model to be updated at runtime, as will be discussed in further detail herein. In one specific implementation, the NN model involves a style transfer network that utilizes environmental conditions and / or sensor information (e.g., camera, motion, etc.). As mentioned herein, a style transfer network refers to a machine learning network that provides software algorithms for manipulating, for example, digital images or videos to adopt the appearance or visual style of another image. Compiler 252 creates variable kernel data sections in the binary file to enable the driver component to change the weights of the NN model at runtime. The driver component can update the corresponding variable weights of the NN model at runtime based on a weight file provided by the application and metadata information generated at compile time and embedded in the compiled network binary file. Alternatively, compiler 252 includes weights (e.g., scaling parameters and / or bias parameters) in information separate from the compiled binary file (e.g., in a file). Compiler 252 generates metadata to enable the neural processor driver 270 to update the weights at runtime. In one or more specific implementations, one or more techniques may be applied to another type of network that requires updates based on some environmental parameters (e.g., networks other than style transfer networks). For example, one or more techniques may be used to personalize a network to a user-private environment (e.g., scene semantic networks, where an application can detect certain objects such as chairs or tables, but is tuned to a user-private environment).

[0029] As an example, such a neural network model as described above can be a convolutional neural network. Each convolutional layer of a given neural network model detects features in an input image by comparing a sub-region of the image to a set of kernels, and then using at least one convolutional operation to determine the similarity between the sub-region and the kernel. For example, each kernel can represent a feature that may be present in the image, and such kernels can represent image features as numerical values ​​(e.g., matrices) and can be stored in a specific part of a compiled binary file (e.g., a kernel data section). In one example, the kernel can be represented as a matrix with the same dimensions as the sub-region.

[0030] After compilation, compiler 252 places the compiled binary file of the NN model in model cache 260. Additionally, compiler 252 sends a handle corresponding to the NN model to neural processor daemon 240. As mentioned herein, the handle is a reference to the NN model (e.g., a pointer in memory) that facilitates access to the NN model stored in model cache 260.

[0031] In one implementation, during the runtime of the neural network (NN) model, a client application, such as application 210 designed to execute the NN model's binary file, can pass a weight file (e.g., containing updated weight values) to the neural processor driver 270. However, due to the system's security architecture, the client application does not have permission to access the NN binary file, but only has a reference (e.g., a handle) to it. The client application uses the handle to pass the new weight file to the neural processor driver and receives the results of a machine learning task performed by the NN binary file using the updated weights from the weight file. An example of a weight file is provided in... Figure 3 Further discussion is needed.

[0032] The neural processor framework 216 facilitates communication with the neural processor daemon 240 to invoke commands related to managing neural network models, including at least compiling, loading, and / or unloading the neural network model. In one example, the neural processor daemon 240 may receive a notification indicating that the application 210 has been installed on the electronic device. The neural processor daemon 240 is able to traverse the components of the application 210 (e.g., contained in an application bundle or package) to locate the neural network model that is part of the application. Once located, the neural processor daemon 240 sends commands to the neural processor compiler service 250 to compile the source code associated with the neural network model. As shown, the neural processor compiler service 250 includes a compiler 252 that compiles the source code corresponding to the neural network model. The compiler 252 may store the compiled neural network model in a model cache 260, which may be stored in memory (e.g., RAM provided by the electronic device 115). In one implementation, the model cache 260 is stored in a memory address space (e.g., the system memory address space) separate from the memory address space of the application 210. The neural processor daemon 240 also includes a driver controller 242 that communicates directly with the neural processor driver 270 (e.g., via a device driver client), which will be discussed in more detail below.

[0033] In one implementation, when application 210 is executed (e.g., after installation), application 210 may load a compiled neural network model now stored in model cache 260 and store the neural network model source in machine learning (ML) model storage device 218 in the memory address space of application 210. The application may utilize at least one of the first ML software library 212 and the second ML software library 214 and / or the neural processor framework 216 to send commands to the neural processor daemon 240 to load the cached neural network model into the memory address space of the neural processor driver (e.g., by storing it in ML model storage device 218). After being loaded into the memory address space, application 210 may use the loaded neural network model to invoke commands in various ways discussed herein.

[0034] However, in another specific implementation, application 210 is not permitted to store the cached neural network model in ML model storage device 218, and the neural network model will be accessed by using the aforementioned handle. For some applications, security requirements are more stringent, and application 210 is only permitted to access the model via a handle provided to the neural network model (e.g., for updating the parameters of the neural network model using a weight file during runtime).

[0035] In one example, application 210 includes a driver controller 220 that communicates with neural processor driver 270. During execution, application 210 may execute inference commands against a compiled neural network model loaded in the memory space of the neural processor driver. Neural processor driver 270 enables application 210 to indirectly send commands to neural processor firmware 280 for execution on the neural processor. For example, application 210 utilizes driver controller 220 to send commands to initiate style transfer (e.g., prediction instructions 230 using a device driver client, as shown in this example) using a compiled neural network model loaded in the memory space of the neural processor driver. In this respect, driver controller 220 sends prediction command 230 to neural processor driver 270, which in turn sends commands to neural processor firmware 280 for execution. The result of executing prediction command 230 is returned to application 210 by neural processor driver 270.

[0036] Alternatively, application 210 may use the loaded neural network model to invoke commands such as style transfer by communicating with neural processor daemon 240. In this example, neural processor daemon 240 (e.g., using a device driver client invoked by driver controller 242) sends commands to neural processor driver 270, which then communicates with neural processor firmware 280 that ultimately runs the commands on the neural processor. The results of the commands are sent from neural processor driver 270 back to neural processor daemon 240. Neural processor daemon 240 then sends the command results to application 210.

[0037] Recently, specialized (e.g., dedicated) hardware has been developed, optimized for performing specific operations from a given neural network (NN). A given electronic device may include a neural processor, which can be implemented as a circuit that performs various machine learning operations based on computations including multiplication, addition, and accumulation. Such computations may be arranged to perform, for example, convolutions on input data. In one example, the neural processor is specifically configured to execute machine learning algorithms, typically by operating on a predictive model such as an NN. In one or more implementations, the electronic device may also include a neural processor in addition to a CPU and / or GPU.

[0038] As discussed in this paper, a CPU can refer to the main processor in a given electronic device that performs the following operations: basic arithmetic, logic, control, and input / output operations specified by instructions from a computer program or application, including some operations for neural network models. As discussed in this paper, a GPU can refer to a special-purpose electronic circuit designed to perform operations for rendering graphics; it is also used in many cases to handle computational workloads of machine learning operations (e.g., operations specified by instructions from a computer program or application). CPUs, GPUs, and neural processors can each have different computational specifications and capabilities depending on their specific implementation, and each of these components can provide different levels of performance for certain operations compared to the other components.

[0039] As discussed in this paper, a convolutional neural network (CNN) refers to a specific type of neural network that uses different types of layers. These layers consist of nodes existing in three dimensions, and the dimensions can vary between layers. In a CNN, nodes in a layer can only connect to a subset of nodes in the previous layer. The final output layer can be fully connected and its size can be determined based on the number of classifiers. As described in this paper, a fully connected layer means that each node in that layer receives input from every node in the previous layer. CNN models can include various combinations and, in some cases, multiple instances and orders of each of the following types of layers: input layer, convolutional layer, pooling layer, rectified linear unit (ReLU) layer, and fully connected layer. Part of the operation performed by a CNN involves obtaining a set of filters (or kernels) that iterate over the input data based on one or more parameters. In one example, the depth of a convolutional layer can be equal to the number of filters used. It should be understood that the size of different convolutions at each layer can be determined mathematically, taking into account the hyperparameters of the CNN.

[0040] Figure 3 An example structure of a variable weight file 310 provided by a user or application, according to one or more embodiments, is shown for updating the variable weights of a neural network model. However, not all depicted components are usable in all embodiments, and one or more embodiments may include additional or different components compared to those shown in the figures. Variations in the arrangement and type of these components may be made without departing from the spirit or scope of the claims set forth herein. Additional components, different components, or fewer components may be provided.

[0041] As shown in the figure, the variable weight file 310 includes a set of weights corresponding to scaling values ​​320, bias values ​​330, scaling values ​​340, and bias values ​​350. In one implementation, the variable weight file 310 includes information presented in a format provided by a given client but incompatible with the structure of the compiled binary file corresponding to the neural network model. For example, the set of weights may include data in a layout that has contiguous addresses for each of the aforementioned scaling and bias values. However, in one example, such contiguous addresses of the variable weight file 310 do not provide the neural processor driver 270 with information about the layout of such information in the compiled binary file during runtime, in order to be able to update the weights in the compiled binary file. In the compiled binary file, variable parameters may be located in different parts of the binary file (e.g., at different address offsets) that are different from the addresses of the corresponding parameters in the variable weight file 310.

[0042] In one specific implementation, a weight file with variable parameters is provided for each operation from a given neural network model.

[0043] In one implementation, compiler 252 generates metadata to enable neural processor driver 270 to update weights during runtime. This metadata includes: 1) information about variable and updatable layers, and 2) information about transformations performed by compiler 252 during compilation for the variable layers to generate a compiled binary that meets the hardware requirements of the target device, enabling the hardware to execute the NN. This metadata is included in the compiled binary of the neural network, and in one implementation, the metadata is stored in a portion of the binary accessible to neural processor driver 270 during runtime (e.g., a specific portion of the binary without specific security protection). Furthermore, the metadata information may include updatable header information, listing procedures (further described herein), and information regarding the expected size of weight data provided by the client (e.g., in one or more weight files), enabling neural processor driver 270 to verify data provided by the client at runtime.

[0044] During compilation, compiler 252 performs several transformations of the neural network model, including but not limited to: 1) fusing scaling and bias operations; 2) fusing one scaling and bias layer with another scaling and bias layer; 3) other transformations, including flattening layers (e.g., converting a tensor to a single dimension that is forwarded to another layer). In one implementation, the gain offset control (GOC) layer includes variable scaling and / or bias parameters that can be updated during runtime. Compiler 252 generates sufficient information (e.g., metadata) to enable the reconstruction of the final result based on the initial weights provided by the client; for example, the client or application provides two values ​​for two corresponding scaling and bias layers (e.g., similar to the variable weight file 310), and the metadata should include sufficient information for reconstructing the final result, even if the two layers have been fused (e.g., combined) as part of the compilation process. Additionally, compiler 252 generates information (e.g., additional metadata) about specific data that is immutable within the neural network model (e.g., layers with scalar scaling and scalar bias may be immutable; in one implementation, scaling and bias offsets are immutable).

[0045] Additionally, compiler 252 generates rasterization information to conform to the hardware requirements of the target device that will execute the compiled neural network. For example, such rasterization information includes: 1) information about offsets within the variable kernel data section of the compiled binary; 2) information about aligning data (e.g., layout) for hardware compatibility; and 3) information about the location of the corresponding amount of data entering the variable kernel data section.

[0046] The following text combines Figure 4 An example describing the aforementioned metadata generated by compiler 252.

[0047] Figure 4 An example structure of a metadata section 400 generated by compiler 252 according to one or more embodiments is shown. This metadata section will be included as part of the compiled binary file of the neural network model to facilitate the updating of the variable weights of the neural network model during runtime. However, not all depicted components are usable in all embodiments, and one or more embodiments may include additional or different components compared to those shown in the figures. Variations in the arrangement and type of these components may be made without departing from the spirit or scope of the claims set forth herein. Additional components, different components, or fewer components may be provided.

[0048] As mentioned herein, the term "procedure" refers to a set of operations from a given neural network model, where the set of operations may originate from a single layer of the neural network model or from multiple layers of the neural network model. In one example, when the operations correspond to a layer with variable parameters (such as a variable GOC layer as described above), each operation in the procedure includes a separate variable weight file. Specifically, variable scaling and bias data may be stored in separate weight files at offsets specified by the metadata further described below. It should be understood that the metadata described below may include one or more procedures for a neural network model.

[0049] For example, at runtime, the user or application provides a corresponding variable weight file for each operation in a given process. However, in one specific implementation, the neural processor driver 270 requires information to match each weight file with a specific operation in the process. This information is provided by the metadata described herein.

[0050] In practice, a process (including the corresponding operation performed by a given neural network model) is uniquely identified by a corresponding symbol (e.g., proc0, proc1, etc.) and / or an index in a data structure such as a list. Similarly, each operation in each process can be uniquely identified by a specific symbol (e.g., op0, op1, etc.) or a corresponding index in an array.

[0051] The metadata section 400 begins with header information, which includes a version number corresponding to the metadata layout version and information indicating the hardware architecture type (this information is for future reference to determine whether the metadata section 400 is compatible with specific hardware). In one implementation, header information 405 may include the following information:

[0052] *version

[0053] *cpu_type

[0054] *cpu_subtype

[0055] *procedure_count

[0056] In one implementation, metadata section 400 includes information about the offset of each “ProcList” section for the corresponding procedure (discussed further below), even if some procedures may be immutable. For a given immutable procedure, the offset is 0, and in one implementation, no-ops are included. In one example, no-ops refers to information indicating that no corresponding operation is associated with the immutable procedure.

[0057] Additionally, the metadata section 400 includes information about “InitInfo”, which includes information about a set of transformation and rasterization data applicable to a set of one or two scaling / bias vectors from a variable weight file provided by a client (e.g., application 210) and / or neural processor driver 270 during runtime.

[0058] like Figure 4 As shown, list 410 (“ProcLists”) includes a count 430 indicating the number of corresponding ProcList sections. Specifically, each ProcList section in metadata section 400 includes information about a list of offsets for each of the aforementioned “InitInfo” (e.g., information about a set of transformation and rasterization data). In one example, each ProcList section may correspond to a specific procedure associated with a specific procedure identifier. When a given procedure with a specific identifier (e.g., proc_id) is executed, all corresponding “InitInfo” data are processed and used to patch / update runtime copies of the variable kernel data sections before running the procedure.

[0059] In one implementation, list 410 (“ProcLists”) may include information in the following format:

[0060] • Count the number of corresponding ProcList sections

[0061] • An array of offsets for (N) ProcList sections

[0062] • Section offset of the Nth ProcList

[0063] · Array of ProcList sections

[0064] • Count of the InitInfo section offset of this ProcList

[0065] • An array of offsets for (N) InitInfo sections

[0066] • Section offset of the Nth InitInfo

[0067] exist Figure 4 In the example, list 410 includes a count 430 indicating the number of ProcList sections (e.g., 2), an offset value 434 for each ProcList section, and an offset value 433. Offset value 434 corresponds to the position of ProcList section 435, and offset value 433 corresponds to the position of ProcList section 437. For example... Figure 4 As shown, the ProcList section 435 includes a count 440 indicating the number of InitInfo sections (e.g., 3), offset values ​​442, 444, and 446 corresponding to the respective InitInfo sections. For example, offset value 442 corresponds to the position of InitInfo section 450, offset value 444 corresponds to the position of InitInfo section 460, and offset value 446 corresponds to the position of InitInfo section 470.

[0068] As further shown, project 420 (“Initialization Project”) includes information about each “InitInfo” section included in metadata section 400. For example, project 420 includes InitInfo section 450, InitInfo section 460, and InitInfo section 470. As described above, the corresponding offsets may be included in the ProcList section corresponding to each InitInfo section. Each InitInfo section in project 420 may include the following information for each InitInfo section:

[0069] • Count the number of InitInfo sections

[0070] ·Array of InitInfo sections

[0071] Boolean value HasScale flag

[0072] Boolean value HasBias flag

[0073] • Boolean value HasWeight flag

[0074] • Boolean value HasActivation flag

[0075] • Optional: Scaling vector file information (if HasScale == 1)

[0076] • Optional: Bias vector file information (if HasBias == 1)

[0077] • Optional: Weight file information (if HasWeight == 1)

[0078] • Optional: Activate file information (if HasActivation == 1)

[0079] • The conversion of the InitInfo section

[0080] ·Conversion count

[0081] • A list of (N) sequential transformations (each of which has a variable size)

[0082] • Projects in this InitInfo section

[0083] • Count of projects

[0084] • A sequential list of (N) items (each with a variable size)

[0085] ◆Project conversion information

[0086] ◆Rasterization information of the project

[0087] In one example, the "Items in this InitInfo section" includes information about the values ​​of the scaling and / or bias parameters. The "Item Count" information refers to the number of scaling and / or bias parameters. The "(N) Items Sequential List" information refers to information about each parameter, including the corresponding transformation and rasterization information. In one example, the "(N) Items Sequential List" information may include N instances of the corresponding transformation and rasterization information.

[0088] Figure 5 A flowchart of an example process 500 according to one or more specific implementations is shown, which is used to generate metadata for a neural network for updating parameters during runtime. For illustrative purposes, this document primarily refers to... Figure 2 The software architecture is used to describe process 500, which can be described by components of the software architecture. Figure 1 The process 500 is executed by one or more processors of electronic device 115. However, process 500 is not limited to electronic device 115, and one or more blocks (or operations) of process 500 may be executed by one or more other components of other suitable devices (such as electronic device 110). Further for illustrative purposes, the blocks of process 500 are described herein as occurring sequentially or linearly. However, multiple blocks of process 500 may occur in parallel. Moreover, the blocks of process 500 need not be executed in the order shown, and / or one or more blocks of process 500 need not be executed and / or may be replaced by other operations.

[0089] Compiler 252 receives code corresponding to a neural network (NN) model and a set of weights for the NN model (510). Compiler 252 determines a set of variable layers in the NN model (512). Compiler 252 determines a set of transformations for each layer in the set of layers, which transforms each layer into code that conforms to the hardware requirements of the target platform running the NN model (514). Generated metadata may include information corresponding to the set of transformations (e.g., rasterization information conforming to hardware requirements).

[0090] At operation 516, compiler 252 determines information for mapping a second set of weights (e.g., provided by the application client at execution time) to that set of weights used in the NN model (e.g., in the compiled binary file). In one example, mapping the second set of weights involves mapping a logical representation of the weights (e.g., provided by a first format of a given weight file) to a hardware representation of the weights required by the hardware of the target device. Such mapping may involve information (e.g., the layout of the data in the hardware's memory) corresponding to the offsets and / or alignments of the weights required by the hardware, so that the weights can be accessed from the binary file. The first format may refer to the data layout provided in the weight file, for example, as previously stated in Figure 3 As described, it includes contiguous addresses for weights. To ensure hardware compatibility of weights from the weight file, specific offset and alignment information (e.g., mappings) is determined by compiler 252 and included in the metadata generated by compiler 252. Figure 4 This was also discussed in China.

[0091] In addition, compiler 252 generates metadata corresponding to the set of variable layers, the set of transformations, and information for mapping the second set of weights (518).

[0092] Figure 6 A flowchart of an example process 600 according to one or more specific implementations is shown, which is used to use Figure 5 The generated metadata described in [the document] is used to compile the neural network. For illustrative purposes, this article primarily refers to [the document's source]. Figure 2 The software architecture components describe process 600, which can be described by... Figure 1 The process 600 is executed by one or more processors of electronic device 115. However, process 600 is not limited to electronic device 115, and one or more blocks (or operations) of process 600 may be executed by one or more other components of other suitable devices (such as electronic device 110). Further for illustrative purposes, the blocks of process 600 are described herein as occurring sequentially or linearly. However, multiple blocks of process 600 may occur in parallel. Moreover, the blocks of process 600 need not be executed in the order shown, and / or one or more blocks of process 600 need not be executed and / or may be replaced by other operations.

[0093] Compiler 252 compiles the code and generated metadata to create a compiled binary file of the NN model (610). Compiler 252 provides the compiled binary file stored in a cache to the neural processor compiler service (612). In addition, the neural processor compiler service provides a handle to a security application, wherein the handle includes a reference to the compiled binary file stored in the cache (614).

[0094] Figure 7 A flowchart of an example process 700 according to one or more specific implementations is shown, which is used to update the weights of a currently executing neural network model. For illustrative purposes, this document primarily refers to... Figure 2 The software architecture components describe process 700, which can be described by... Figure 1 The process 700 is executed by one or more processors of electronic device 115. However, process 700 is not limited to electronic device 115, and one or more blocks (or operations) of process 700 may be executed by one or more other components of other suitable devices (such as electronic device 110). Further for illustrative purposes, the blocks of process 700 are described herein as occurring sequentially or linearly. However, multiple blocks of process 700 may occur in parallel. Moreover, the blocks of process 700 need not be executed in the order shown, and / or one or more blocks of process 700 need not be executed and / or may be replaced by other operations.

[0095] The neural processor driver 270 receives a weight file (710) and an inference request from an application client. In one example, the weight file includes information corresponding to a set of values ​​used to update a set of weights of a neural network model currently being executed on an electronic device (e.g., electronic device 115).

[0096] The neural processor driver 270 determines metadata that is used to update the set of weights of the neural network model (712) based on information provided in the binary file of the neural network model. In one example, the metadata includes mappings corresponding to offsets and alignments so that the weights from the weight file are compatible with the hardware provided by the electronic device.

[0097] The neural processor driver 270 updates the set of weights of the neural network model at least in part based on the metadata and weight file (714). In one example, the weights can be updated by applying numerical transformations and rasterization mappings from the metadata to the weights in the weight file.

[0098] Figure 8 An electronic system 800 is shown that can be used to implement one or more specific embodiments of the subject matter technology. The electronic system 800 may be... Figure 1The illustrated electronic device 110, electronic device 115, and / or server 120 may be or may be part of them. The electronic system 800 may include various types of computer-readable media and interfaces for various other types of computer-readable media. The electronic system 800 includes a bus 808, one or more processing units 812, system memory 804 (and / or cache), ROM 810, persistent storage device 802, input device interface 814, output device interface 806, and one or more network interfaces 816, or subsets and variations thereof.

[0099] Bus 808 generally represents all system buses, peripheral bus, and chipset bus that communicatively connect numerous internal devices of electronic system 800. In one or more embodiments, bus 808 communicatively connects one or more processing units 812 to ROM 810, system memory 804, and permanent storage device 802. One or more processing units 812 retrieve instructions to be executed and data to be processed from these various memory units in order to perform the processes disclosed in this subject matter. In different embodiments, one or more processing units 812 may be a single processor or a multi-core processor.

[0100] ROM 810 stores static data and instructions required by one or more processing units 812 and other modules of the electronic system 800. On the other hand, persistent storage device 802 can be a read-write memory device. Persistent storage device 802 can be a non-volatile memory cell that stores instructions and data even when the electronic system 800 is powered off. In one or more embodiments, mass storage devices (such as magnetic disks or optical disks and their corresponding disk drives) can be used as persistent storage device 802.

[0101] In one or more embodiments, a removable storage device (such as a floppy disk, flash drive, and its corresponding disk drive) may be used as persistent storage device 802. Like persistent storage device 802, system memory 804 may be a read-write memory device. However, unlike persistent storage device 802, system memory 804 may be volatile read-write memory, such as random access memory. System memory 804 may store any instructions and data that one or more processing units 812 may need during operation. In one or more embodiments, the processes disclosed in this subject matter are stored in system memory 804, persistent storage device 802, and / or ROM 810. One or more processing units 812 retrieve instructions to be executed and data to be processed from these various memory units to execute the processes of one or more embodiments.

[0102] Bus 808 is also connected to input device interface 814 and output device interface 806. Input device interface 814 enables a user to transmit information and select commands to electronic system 800. Input devices that can be used with input device interface 814 may include, for example, an alphanumeric keypad and pointing devices (also known as "cursor control devices"). Output device interface 806 may, for example, enable the display of images generated by electronic system 800. Output devices that can be used with output device interface 806 may include, for example, printers and display devices such as liquid crystal displays (LCDs), light-emitting diode (LED) displays, organic light-emitting diode (OLED) displays, flexible displays, flat panel displays, solid-state displays, projectors, or any other device for outputting information. One or more embodiments may include devices that act as both input and output devices, such as touchscreens. In these embodiments, the feedback provided to the user can be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, voice, or tactile input.

[0103] Finally, as Figure 8 As shown, bus 808 also couples electronic system 800 to one or more networks and / or to one or more network nodes via one or more network interfaces 816, such as Figure 1 The electronic device 115 shown is an example. In this way, the electronic system 800 can be part of a computer network (such as a LAN, wide area network (“WAN”), or intranet), or it can be part of a network of networks (such as the Internet). Any or all components of the electronic system 800 may be used in conjunction with the subject matter disclosed herein.

[0104] One aspect of the present invention may include collecting and using data obtained from specific and legitimate sources to perform machine learning operations, such as those provided in applications utilizing machine learning models (e.g., neural networks), to provide specific functionality that may be useful to a user. This disclosure contemplates that, in some instances, the collected data may include personal information data that uniquely identifies or can be used to identify a specific person. Such personal information data may include demographic data, location-based data, online identifiers, telephone numbers, email addresses, home addresses, data or records related to a user's health or fitness level (e.g., vital sign measurements, medication information, exercise information), date of birth, or any other personal information.

[0105] This disclosure recognizes that the use of such personal information data in the techniques of this invention can benefit users. For example, personal information data can be used to perform machine learning tasks useful to users (e.g., prediction, classification, similarity determination, anomaly detection, etc.). Therefore, the use of such personal information data enables users to have greater control over the content delivered. Furthermore, this disclosure also anticipates other uses of personal information data that benefit users. For example, health and fitness data can be used according to user preferences to provide insights into their overall health status, or it can be used as positive feedback to individuals using the technology to pursue health goals.

[0106] This disclosure assumes that entities responsible for the collection, analysis, disclosure, transmission, storage, or other use of such personal information data will comply with established privacy policies and / or privacy practices. Specifically, it is expected that such entities will implement and consistently apply privacy practices generally recognized as meeting or exceeding industry or governmental requirements for protecting user privacy. Such information regarding the use of personal data should be prominently displayed and easily accessible to the user, and should be updated as the data is collected and / or used. Users' personal information should be collected only for lawful use. Furthermore, such collection / sharing should only occur after receiving user consent or other lawful grounds provided for in applicable law. In addition, such entities should consider taking any necessary steps to protect and safeguard access to such personal information data and ensure that others with access to personal information data comply with their privacy policies and procedures. Additionally, such entities may be subject to third-party evaluations to demonstrate their compliance with widely accepted privacy policies and practices. Furthermore, policies and practices should be tailored to specific types of personal information data that are convenient to collect and / or access, and made applicable to applicable laws and standards, including specific considerations specific to jurisdictions where higher standards can be imposed. For example, in the United States, the collection or access to certain health data may be governed by federal and / or state laws, such as the Health Insurance Portability and Accountability Act (HIPAA); while health data in other countries may be subject to other regulations and policies and should be handled accordingly.

[0107] Regardless of the foregoing, this disclosure also envisions implementation schemes for users to selectively prevent the use or access to personal information data. That is, this disclosure anticipates providing hardware and / or software components to prevent or block access to such personal information data. For example, with regard to advertising delivery services, the inventive technology can be configured to allow users to opt-in or opt-out at any time during or after service registration to participate in the collection of personal information data. In another example, users can choose not to provide emotion-related data for a targeted content delivery service. Furthermore, users can choose to limit the duration for which emotion-related data is retained, or completely prevent the development of underlying emotional states. In addition to providing opt-in and opt-out options, this disclosure envisions providing notifications related to access to or use of personal information. For example, users can be notified when downloading an application that their personal information data will be accessed, and then reminded again just before the application accesses the personal information data.

[0108] Furthermore, the purpose of this disclosure is to manage and process personal information data to minimize the risk of unintentional or unauthorized access or use. Once data is no longer needed, this risk can be minimized by limiting data collection and deleting data. Additionally, and where applicable, including in certain health-related applications, data deidentification can be used to protect user privacy. Deidentification can be facilitated, where appropriate, by removing identifiers, controlling the amount or specificity of stored data (e.g., collecting location data at the city level rather than the address level), controlling how data is stored (e.g., aggregating data among users), and / or other methods such as differentiated privacy.

[0109] Therefore, while this disclosure broadly covers the use of personal information data to implement one or more of the various disclosed embodiments, it also contemplates that various embodiments can be implemented without access to such personal information data. That is, various embodiments of the present invention will not be rendered inoperable due to the absence of all or part of such personal information data. For example, content can be selected and delivered to the user based on aggregated non-personal information data or an absolute minimum amount of personal information, such as content processed only on the user's device or other non-personal information available for content delivery services.

[0110] The embodiments within the scope of this disclosure may be implemented in part or in whole using a tangible computer-readable storage medium (or multiple tangible computer-readable storage media of one or more types) on which one or more instructions are written. The tangible computer-readable storage medium may also be substantially non-transitory.

[0111] Computer-readable storage media can be any storage medium that can be read, written, or otherwise accessed by general-purpose or special-purpose computing devices, including any processing electronics and / or processing circuits capable of executing instructions. For example, without limitation, computer-readable media can include any volatile semiconductor memory, such as RAM, DRAM, SRAM, T-RAM, Z-RAM, and TTRAM. Computer-readable media can also include any non-volatile semiconductor memory, such as ROM, PROM, EPROM, EEPROM, NVRAM, flash memory, nvSRAM, FeRAM, FeTRAM, MRAM, PRAM, CBRAM, SONOS, RRAM, NRAM, track memory, FJG, and Millipede memory.

[0112] Additionally, computer-readable storage media may include any non-semiconductor memory, such as optical disc storage devices, magnetic disk storage devices, magnetic tape, other magnetic storage devices, or any other medium capable of storing one or more instructions. In one or more embodiments, the tangible computer-readable storage medium may be directly coupled to a computing device, while in other embodiments, the tangible computer-readable storage medium may be indirectly coupled to a computing device, for example, via one or more wired connections, one or more wireless connections, or any combination thereof.

[0113] Instructions can be directly executable or can be used to develop executable instructions. For example, instructions can be implemented as executable or non-executable machine code, or as high-level language instructions that can be compiled to produce executable or non-executable machine code. Furthermore, instructions can also be implemented as data, or may include data. Computer executable instructions can also be organized in any format, including routines, subroutines, programs, data structures, objects, modules, applications, applets, functions, etc. As those skilled in the art will recognize, details including, but not limited to, the number, structure, sequence, and organization of instructions can vary significantly without altering the underlying logic, functionality, processing, and output.

[0114] While the above discussion primarily concerns microprocessors or multi-core processors that execute software, one or more specific implementations are executed by one or more integrated circuits such as ASICs or FPGAs. In one or more specific implementations, such integrated circuits execute instructions stored on the circuit itself.

[0115] Those skilled in the art will recognize that the various exemplary blocks, modules, elements, components, methods, and algorithms described herein can be implemented as electronic hardware, computer software, or a combination of both. To illustrate this interchangeability between hardware and software, the various exemplary blocks, modules, elements, components, methods, and algorithms have been generally described above in terms of functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. Those skilled in the art can implement the described functionality in different ways for each specific application. Various components and blocks can be arranged differently (e.g., in different orders or divided in different ways) without departing from the scope of the subject matter.

[0116] It should be understood that any particular order or hierarchical structure of the boxes in the process disclosed in this invention is an example of an exemplary method. Based on design preferences, it should be understood that the particular order or hierarchical structure of the boxes in the process may be rearranged or all illustrated boxes may be executed. Any box within these boxes may be executed simultaneously. In one or more embodiments, multitasking and parallel processing may be advantageous. Furthermore, the division of various system components in the above embodiments should not be construed as requiring such division in all embodiments, and it should be understood that program components and systems may generally be integrated together in a single software product or packaged into multiple software products.

[0117] As used in this specification and any claim of this patent application, the terms "base station," "receiver," "computer," "server," "processor," and "memory" refer to electronic devices or other technical devices. These terms exclude persons or groups of persons. For the purposes of this specification, the terms "display" or "being displayed" mean displaying on an electronic device.

[0118] As used herein, the phrase "at least one of" following a series of items separated by the terms "and" or "or" modifies the list as a whole, not each member of the list (i.e., each item). The phrase "at least one of" does not require selection of at least one of each of the listed items; rather, it allows for the inclusion of at least one of any one item and / or at least one of any combination of items and / or at least one of each item. For example, the phrases "at least one of A, B, and C" or "at least one of A, B, or C" respectively refer to only A, only B, or only C; any combination of A, B, and C; and / or at least one of each of A, B, and C.

[0119] The predicates “configured to,” “operable to,” and “programmed to” do not imply any specific tangible or intangible modification to a particular subject matter but are intended to be used interchangeably. In one or more embodiments, a processor configured to monitor and control operations or components can also mean that the processor is programmed to monitor and control operations or that the processor is operable to monitor and control operations. Similarly, a processor configured to execute code can be interpreted as a processor programmed to execute code or operable to execute code.

[0120] Phrases such as aspect, that aspect, on the other hand, some aspects, one or more aspects, implementation, that implementation, another implementation, some implementations, one or more implementations, implementation scheme, that implementation scheme, another implementation scheme, some implementation schemes, one or more implementation schemes, configuration, that configuration, other configuration, some configurations, one or more configurations, subject matter technology, disclosure, this disclosure, other variations, etc., are for convenience only and do not imply that disclosures involving one or more such phrases are essential to the subject matter technology, or that such disclosures apply to all configurations of the subject matter technology. Disclosures involving one or more such phrases may apply to all configurations or one or more configurations. Disclosures involving one or more such phrases may provide one or more examples. Phrases such as aspect or some aspects may refer to one or more aspects, and this applies similarly to other foregoing phrases.

[0121] The term “exemplary” is used herein to mean “serving as an example, instance, or illustration.” Any embodiment described herein as “exemplary” or “example” is not necessarily to be construed as preferred or superior to other specific embodiments. Furthermore, the terms “comprising,” “having,” etc., as used in the specification or claims, are intended to be inclusive, similar to how the term “comprising” is interpreted when used as a transitional word in the claims.

[0122] All structural and functional equivalents of elements throughout the various aspects described herein that are known or later become apparent to those skilled in the art are expressly incorporated herein by reference and are intended to be covered by the claims. Furthermore, nothing disclosed herein is intended to be made public, regardless of whether such disclosure is expressly stated in the claims. No claim element should be interpreted in accordance with 35 U.S.SC §112(f) unless the element is expressly stated using the phrase “means for…” or, in the case of a method claim, using the phrase “step for…”.

[0123] The preceding description is provided to enable those skilled in the art to practice the various aspects described herein. Various modifications to these aspects will be apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects. Therefore, this claim is not intended to be limited to the aspects shown herein, but rather to be consistent with the language of the claim, wherein references to elements in singular values ​​are not intended to mean “only one,” but rather “one or more,” unless specifically indicated. Unless otherwise specifically stated, the term “some” means one or more. Male pronouns (e.g., his) include female and neutral (e.g., her and its), and vice versa. Titles and subtitles (if any) are used for convenience only and do not limit the disclosure of this subject matter.

Claims

1. A method for generating metadata, comprising: Receive code corresponding to a neural network (NN) model and a set of weights for the NN model, wherein the input data of the NN model includes at least one of image data or video data; Determine a set of variable layers in the NN model; Determine the information used to map the second set of weights to the set of weights used in the NN model; as well as Generate metadata corresponding to the set of variable layers, and information for mapping the second set of weights to the set of weights for the NN model, wherein the metadata includes a mapping that maps the second set of weights provided in a first format to a hardware representation required by the hardware of the target platform, and wherein the generated metadata enables the set of variable layers to be updated during the execution of the compiled binary file corresponding to the code of the NN model and the metadata.

2. The method according to claim 1, further comprising: Provide the compiled binary file for storage in the cache; as well as A handle is provided to a security application, wherein the handle includes a reference to the compiled binary file stored in the cache.

3. The method of claim 1, wherein the code further comprises parameters corresponding to the bias and scaling values ​​for the weights used in the NN model.

4. The method of claim 1, wherein the second set of weights adopts a first format corresponding to a file, the file including the second set of weights having a first set of addresses, the first set of addresses being different from the second set of addresses used for the set of weights in the NN model.

5. The method according to claim 1, further comprising: A set of transformations is determined for each of the set of variable layers, which changes each layer into code that conforms to the hardware requirements of the target platform running the NN model, wherein the generated metadata includes information corresponding to the set of transformations.

6. The method of claim 5, wherein the set of conversions comprises: Combine scaling and bias operations, merge scaling and bias layers with another scaling and bias layer, or flatten a layer.

7. The method of claim 1, wherein the metadata includes information corresponding to an offset of a set of operations performed by a corresponding variable layer of the NN model.

8. The method of claim 1, wherein the metadata includes information about variable kernel data sections offset to the compiled binary of the NN model, the variable kernel sections including variable corresponding weights during the execution of the NN model.

9. The method of claim 8, wherein the driver component updates the corresponding variable weights of the NN model during runtime based on a weight file provided by the application.

10. The method of claim 9, wherein the weight file comprises a set of vectors, the set of vectors comprising data corresponding to the respective variable weights.

11. The method of claim 1, wherein the mapping includes memory layout information, the memory layout information including offsets and alignment constraints required by the hardware of the target platform.

12. A system for compiling code, comprising: processor; A memory device comprising instructions that, when executed by the processor, cause the processor to: Receive code corresponding to a neural network (NN) model and a set of weights for the NN model, wherein the input data of the NN model includes at least one of image data or video data; Determine a set of variable layers in the NN model; Determine the information used to map the second set of weights to the set of weights used in the NN model; as well as Generate metadata corresponding to the set of variable layers, and information for mapping the second set of weights to the set of weights for the NN model, wherein the metadata includes a mapping that maps the second set of weights provided in a first format to a hardware representation required by the hardware of the target platform, and wherein the generated metadata enables the set of variable layers to be updated during the execution of the NN model. as well as The code and the generated metadata are compiled to create a compiled binary file of the NN model.

13. The system of claim 12, wherein the memory device further comprises instructions that, when executed by the processor, cause the processor to: Provide the compiled binary file for storage in the cache; and A handle is provided to a security application, wherein the handle includes a reference to the compiled binary file stored in the cache.

14. The system of claim 12, wherein the code further includes parameters for bias values, scaling values, weight values, and activation parameters corresponding to the NN model.

15. The system of claim 12, wherein the second set of weights adopts a first format corresponding to a file, the file including the second set of weights having a first set of addresses, the first set of addresses being different from the second set of addresses used for the set of weights in the NN model.

16. The system of claim 12, wherein the memory device further comprises instructions that, when executed by the processor, cause the processor to: A set of transformations is determined for each of the set of variable layers, which changes each layer into code that conforms to the hardware requirements of the target platform running the NN model, wherein the generated metadata includes information corresponding to the set of transformations.

17. The system of claim 16, wherein the set of conversions comprises: Combine scaling and bias operations, merge scaling and bias layers with another scaling and bias layer, or flatten a layer.

18. The system of claim 12, wherein the metadata includes information corresponding to an offset of a set of operations performed by a respective variable layer of the NN model.

19. The system of claim 18, wherein the metadata includes information about variable kernel data sections offset to the compiled binary of the NN model, the variable kernel sections including corresponding weights, scaling, bias, or activation parameters that are variable during the execution of the NN model.

20. The system of claim 19, wherein the driver component updates the corresponding variable weights of the NN model during runtime, based on a weight file provided by the application.

21. A non-transitory computer-readable medium comprising instructions, which, when executed by a computing device, cause the computing device to perform an operation comprising: A driver provided by the computing device receives code and a weight file corresponding to a neural network (NN) model, the weight file including information corresponding to a set of values ​​used to update a set of weights of the neural network model, the neural network model currently being executed on the computing device, wherein the input data of the neural network model includes at least one of image data or video data; The driver determines metadata used to update the set of weights of the neural network model based on information provided in the binary file of the neural network model, wherein the metadata includes a mapping that maps a second set of weights provided in a first format to a hardware representation required by the hardware of the target platform. The driver updates the set of weights of the neural network model based at least in part on the metadata and the weight file; as well as The code and metadata are compiled to create a compiled binary file of the NN model.

Images