System and method for encoding and decoding parameters of machine learning model

Abstract

A method for decoding parameters of a machine learning model is provided. The method comprises: decoding a code stream to determine values of a plurality of samples in image data; determining one or more parameter values based on values of a plurality of samples in the image data; and determining one or more parameters of the machine learning model based on the one or more parameter values.

Description

Technical Field

[0001] This application relates to the field of machine learning, and more specifically, to methods and systems for encoding and decoding parameters of machine learning models for transmission or storage. Background Technology

[0002] Machine learning models are now widely used. In particular, neural networks are a popular class of machine learning models. Current neural networks can have millions or even billions of parameters, and their size continues to grow. Typically, large neural networks are trained on computing systems with abundant computing resources. After training, the neural network can be transferred to user devices for deployment. Given the number and scale of these neural networks' parameters, efficient methods are often needed to transfer and store the parameters of neural networks and other machine learning models. Summary of the Invention

[0003] Embodiments of this application provide methods, systems, and computer-readable media for encoding and decoding parameters of machine learning models for transmission and storage, which overcome the problems of conventional solutions.

[0004] According to a first aspect, a method for decoding parameters of a machine learning model is provided. The method includes: decoding a bitstream to determine values ​​of a plurality of samples in image data; determining one or more parameter values ​​based on the values ​​of the plurality of samples in the image data; and determining one or more parameters of a machine learning model based on the one or more parameter values.

[0005] In some embodiments, determining one or more parameter values ​​based on the values ​​of multiple samples in image data includes mapping the values ​​of multiple samples in the image data from a range of sample values ​​to a range of parameter values.

[0006] In some embodiments, one or more parameter values ​​are determined based on the bit depth associated with the image data.

[0007] In some embodiments, determining one or more parameter values ​​based on the values ​​of multiple samples in image data includes: determining the spatial location of one of the multiple samples in image data; and determining one parameter value based on the spatial location of the sample in image data and the value of the sample.

[0008] In some embodiments, determining a parameter value based on the spatial location of the sample in the image data and the value of the sample includes: determining the parameter value by mapping the value of the sample in the image data to a parameter value based on the spatial location of the sample in the image data.

[0009] In some embodiments, determining the spatial location of one of a plurality of samples in image data is based on the coding block size.

[0010] In some embodiments, one or more parameter values ​​are floating-point numbers that include a sign bit, an exponent, and a mantissa.

[0011] In some embodiments, the image data includes a first dedicated image or channel for sign bits representing one or more parameter values.

[0012] In some embodiments, the image data includes a second dedicated image or channel for representing an index of one or more parameter values.

[0013] In some embodiments, the image data includes a third dedicated image or channel for representing the mantissa of one or more parameter values.

[0014] In some embodiments, determining one or more parameter values ​​based on the values ​​of multiple samples in the image data includes applying a left shift operation to the values ​​of multiple samples of a third dedicated image or channel.

[0015] In some embodiments, image data is received from a remote computing device.

[0016] In some embodiments, image data is retrieved from a local storage device.

[0017] In some embodiments, image data is encoded using an image or video compression algorithm.

[0018] In some embodiments, the compression algorithm is a lossy compression algorithm.

[0019] In some embodiments, the compression algorithm is a lossy compression algorithm.

[0020] In some embodiments, image data is encoded according to PNG, JPEG, or WebP image formats.

[0021] In some embodiments, image data is encoded using algorithms based on H.264 / AVC, H.265 / HEVC, H.266 / VVC, VP9, ​​AV1, or AVS.

[0022] In some embodiments, the method further includes: acquiring encoding information associated with a compression algorithm; and wherein decoding the bitstream to determine the values ​​of multiple samples in the image data is based on the encoding information.

[0023] In some embodiments, the machine learning model is a neural network.

[0024] In some embodiments, the image data includes: a first image or channel representing one or more parameters of a first layer of a neural network, and a second image or channel representing one or more parameters of a second layer of a neural network.

[0025] According to a second aspect, a computer-readable medium is provided. The computer-readable medium includes computer-executable instructions stored thereon, which, when executed by a computing device, cause the computing device to perform any of the methods described in the first aspect. The computer-readable medium is non-transitory.

[0026] According to a third aspect, a decoder is provided. The decoder includes: one or more processors; and a computer-readable medium. The computer-readable medium includes computer-executable instructions stored thereon, which, when executed by the one or more processors, cause the one or more processors to perform any of the methods described in the first aspect.

[0027] According to the fourth aspect, a method for encoding parameters of a machine learning model is provided. The method includes: obtaining one or more parameters of the machine learning model; determining one or more parameter values ​​based on the one or more parameters of the machine learning model; determining values ​​of multiple samples in image data based on the one or more parameter values; and encoding the image data to generate a bitstream.

[0028] In some embodiments, the method further includes sending the generated bitstream to reconstruct the one or more parameters at another computing device.

[0029] In some embodiments, the method further includes sending the generated image data to reconstruct the one or more parameters at another computing device.

[0030] In some embodiments, the method further includes storing the generated image data.

[0031] In some embodiments, determining the values ​​of multiple samples in image data based on one or more parameter values ​​includes mapping one or more parameter values ​​from a parameter value range to a sample value range.

[0032] In some embodiments, the method further includes sending data indicating a range of parameter values.

[0033] In some embodiments, determining the values ​​of multiple samples in image data based on one or more parameter values ​​is based on the target bit depth.

[0034] In some embodiments, the method further includes determining the spatial location of one of a plurality of samples in the image data.

[0035] In some embodiments, determining the spatial location of one of a plurality of samples in image data includes: determining a mapping relationship between the spatial location of the sample in the image data and a parameter index that identifies one of the one or more parameters.

[0036] In some embodiments, determining the spatial location of one of a plurality of samples in image data is based on the coding block size.

[0037] In some embodiments, the value of one or more parameters is a floating-point number that includes a sign bit, an exponent, and a mantissa.

[0038] In some embodiments, the image data includes a first dedicated image or channel for sign bits representing one or more parameter values.

[0039] In some embodiments, the image data includes a second dedicated image or channel for representing an index of one or more parameter values.

[0040] In some embodiments, the image data includes a third dedicated image or channel representing the tail of one or more parameter values.

[0041] In some embodiments, determining the value of multiple samples in image data based on one or more parameter values ​​includes applying a right shift operation to the mantissa of one of the parameter values.

[0042] In some embodiments, lossless compression algorithms are used to encode image data.

[0043] In some embodiments, image data is encoded using a lossy compression algorithm.

[0044] In some embodiments, image data is encoded according to PNG, JPEG, or WebP image formats.

[0045] In some embodiments, image data is encoded using H.264 / AVC, H.265 / HEVC, H.266 / VVC, VP9, ​​AV1, or AVS-based algorithms.

[0046] In some embodiments, the method further includes sending encoded information associated with the compression algorithm to a remote device.

[0047] In some embodiments, the machine learning model is a neural network.

[0048] In some embodiments, the image data includes: a first image or channel for representing one or more parameters of a first layer of a neural network, and a second image or channel for representing one or more parameters of a second layer of a neural network.

[0049] In some embodiments, the image data includes: a first time image or channel representing a first parameter value associated with a first time step, and a second time image or channel representing a second parameter value associated with a second time step.

[0050] According to a fifth aspect, a computer-readable medium is provided. The computer-readable medium includes computer-executable instructions stored thereon, which, when executed by a computing device, cause the computing device to perform any of the methods described in the fourth aspect. The computer-readable medium is non-transitory.

[0051] According to a sixth aspect, an encoder is provided. The encoder includes: one or more processors; and a computer-readable medium. The computer-readable medium includes computer-executable instructions stored thereon, which, when executed by the one or more processors, cause the one or more processors to perform any of the methods described in the fourth aspect.

[0052] According to a seventh aspect, a method for transmitting parameters of a machine learning model is provided. The method includes: acquiring one or more parameters of a first machine learning model by a first computing device; determining one or more parameter values ​​by the first computing device based on the one or more parameters of the machine learning model; determining values ​​of a plurality of samples in image data based on the one or more parameter values; encoding the image data to generate a bitstream; transmitting the bitstream by the first computing device; receiving the bitstream by a second computing device; decoding the bitstream by the second computing device to determine the values ​​of the plurality of samples in the image data; determining one or more parameter values ​​based on the values ​​of the plurality of samples in the image data; and determining one or more parameters of the machine learning model based on the one or more parameter values.

[0053] According to an eighth aspect, a computer-readable medium is provided. The computer-readable medium includes computer-executable instructions stored thereon, which, when executed by a plurality of computing devices, cause the computing devices to perform any of the methods described in the seventh aspect. The computer-readable medium is non-transitory.

[0054] According to a ninth aspect, a system is provided that includes a plurality of computing devices, each computing device including one or more processors and memory, the plurality of computing devices being configured to perform any of the methods described in the seventh aspect.

[0055] According to a tenth aspect, an apparatus for decoding parameters of a machine learning model is provided, comprising: a decoding unit and a parameter determination unit. The decoding unit is configured to: decode a bitstream to determine values ​​of a plurality of samples in image data. The parameter determination unit is configured to determine one or more parameter values ​​based on the values ​​of the plurality of samples in the image data, and to determine one or more parameters of a machine learning model based on the one or more parameter values.

[0056] According to an eleventh aspect, an apparatus for encoding parameters of a machine learning model is provided, comprising: an acquisition unit, an image data determination unit, and an encoding unit. The acquisition unit is configured to acquire one or more parameters of the machine learning model. The image data determination unit is configured to determine one or more parameter values ​​based on the one or more parameters of the machine learning model, and to determine values ​​of multiple samples in the image data based on the one or more parameter values. The encoding unit is configured to encode the image data to generate a bitstream.

[0057] The features described in the context of one aspect can be combined with features described in the context of one or more other aspects. Specifically, the features involving encoding and decoding in the first through sixth aspects can be combined with the seventh through ninth aspects.

[0058] These and other aspects of this application will become more apparent from the following description of the embodiments. Attached Figure Description

[0059] The embodiments will now be described by way of example only, with reference to the accompanying drawings, in which: Figure 1 A flowchart of the decoding operation according to an embodiment is shown; Figure 2 A flowchart of the encoding operation according to an embodiment is shown; Figure 3 A flowchart illustrating the transfer of parameters of a machine learning model according to an embodiment is shown; Figure 4 A block diagram of an example encoding system according to an embodiment is shown; Figure 5 A block diagram of an example decoding system according to an embodiment is shown; Figure 6 A schematic diagram of one layer of a neural network is shown; Figure 7 A schematic diagram of an example decoder according to various embodiments is shown; Figure 8 A schematic diagram of an example encoder according to various embodiments is shown; and Figure 9 A schematic diagram of an example encoding and decoding computing device according to various embodiments is shown. Detailed Implementation

[0060] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0061] An encoder is a device that encodes data into a bitstream, while a decoder is a device that decodes the bitstream to obtain the encoded data or an approximation of the encoded data. A bitstream consists of a sequence of bits.

[0062] As used herein, “image data” includes a single image, which may also be referred to as an “image frame,” or multiple images or “multiple image frames.” Since video data comprises multiple images / multiple image frames, the term “image data” as used herein also includes video data.

[0063] Machine learning models can be transferred from one computing device to another. For example, a machine learning model can be trained on a computing system with abundant computing resources and then transferred to other computing devices, such as user devices, for deployment. In another example, the computing device can be part of a distributed system used to train the machine learning model. Machine learning models can be transferred between computing devices to synchronize the model during training. Machine learning models can be very large, in some cases having millions or even billions of parameters. Transferring and storing such models requires significant amounts of bandwidth, memory, and storage space.

[0064] In the techniques described herein, the parameters of a machine learning model are represented (e.g., converted to or mapped to) image data, and the image data is encoded using an image or video compression algorithm. The encoded image data is smaller in size than the original parameters, thus significantly reducing the bandwidth required to transmit the parameters and the memory or storage space required to store the machine learning model parameters. A receiving computing device can decode the encoded image data to reconstruct the parameters of the machine learning model.

[0065] Figure 1 A flowchart of the decoding operation according to an embodiment is shown. Figure 2 A flowchart of the corresponding encoding operation is shown. Figure 1 The operation can be performed by one or more processors at one or more locations. For example, Figure 1 The operation can be performed by Figure 7 The decoder 10 shown is used for implementation.

[0066] Figure 1 The flowchart begins at step 101, where the bitstream is decoded to determine the values ​​of multiple samples in the image data. The image data represents one or more parameter values ​​used in a machine learning model. The bitstream can be obtained from a remote computing device via a network connection. In an alternative embodiment, the image data can also be obtained from local storage or memory.

[0067] Typically, the bitstream is decoded based on a specific compression algorithm used to encode the image data. Image data can be encoded using any suitable image or video compression algorithm and can be encoded according to any image or video file format. The compression algorithm can be lossless or lossy. In one example, the image data is encoded according to the PNG image format. Other suitable image formats include JPEG or WebP. In another example, the image data is encoded based on the H.265 / HEVC algorithm. In yet another example, the image data is encoded based on the H.266 / VVC algorithm. Other suitable algorithms include H.264 / AVC, VP9, ​​AV1, or AVS-based algorithms. The embodiments are not limited to any particular compression algorithm or format.

[0068] The machine learning model can be of any type, such as neural networks, support vector machines, decision trees, random forests, linear regression, K-means clustering, reinforcement learning systems, etc. This machine learning model can be applied to any type of task, such as image recognition, object detection, image segmentation, image captioning, video object tracking, action recognition, speech recognition, speaker detection, natural language processing, question answering, and control of autonomous vehicles or electromechanical agents. The embodiments are not limited to any specific machine learning model or application of the machine learning model.

[0069] In step 102, one or more parameter values ​​are determined based on the values ​​of multiple samples in the image data. The values ​​of multiple samples in the image data can be mapped from a range of sample values ​​to a range of parameter values ​​used by the machine learning model. This mapping can be performed on the decompressed image data. A "sample" of the image data can be a "pixel" of the image data.

[0070] In one example, determining one or more parameters may be based on the bit depth associated with the image data. For example, the bit depth may be 8 bits, and the sample values ​​of the image data may range from 0 to 255. The original range of parameter values ​​may differ from the range of sample values, and the determination step may include mapping the sample values ​​from the range of sample values ​​to the range of parameter values. Data indicating the range of parameter values ​​may be included in the image data used for encoding or may be encoded separately. It should be noted that the embodiments are not limited to an 8-bit bit depth; any suitable bit depth and range of parameter values ​​may be used.

[0071] As part of step 102, a correspondence between multiple samples and multiple parameters can be determined. That is, step 102 may include: determining the spatial location of one of multiple samples in the image data, and determining one of the parameter values ​​based on the spatial location of that sample in the image data and the value of that sample. Specifically, step 102 may include: determining a parameter value based on the spatial location of the sample in the image data by mapping the value of that sample in the image data to a parameter value. For example, determining the mapping between the spatial location of a sample in the image data and parameter indices that identify one or more parameters. In one example, the machine learning model is a neural network, and one or more parameters include weights of a specific layer of the neural network. Weights are associated with edges connecting units in the first layer and units in the second layer. Weights can be identified using tuples, each tuple having a first value indexed for units in the first layer and a second value indexed for units in the second layer. Samples of the image data can be identified by their spatial coordinates. A mapping can be determined to establish a mapping between sample coordinates and parameter indices, thereby clarifying which values ​​of the image data correspond to which parameters of the machine learning model. In one example, a direct correspondence can be used, where the parameter is indexed by two integer values ​​and treated as coordinates in the image data. In other examples, further transformations may be needed to convert from sample coordinates to parameter indices, or vice versa.

[0072] It should be understood that the embodiments are not limited to sample coordinates, and other types of spatial locations, such as blocks or regions of image data, can also be used. This can be the case in scenarios where compression algorithms use coded blocks. In one example, the above mapping is based on the size of the coded block.

[0073] In step 103, one or more parameters of the machine learning model are determined based on one or more parameter values ​​obtained from step 102. For example, the parameter can be determined based on sample values ​​at the corresponding spatial location in the image data, which are then converted to a range of parameter values.

[0074] In one example, the machine learning model can be an existing machine learning model, with one or more parameter values ​​used to specify updates to some or all of the parameters of the existing machine learning model. In another example, one or more parameter values ​​obtained in step 102 can be used to create a machine learning model. Image data can also represent the architecture, activation function selection, or other hyperparameters used to create the machine learning model, or this data can be obtained separately. In some cases, the architecture can be inferred from image data; for example, the size of a layer of a neural network can be inferred from the dimensions of the image data. Image data can represent parameters of different layers of a neural network in different images or channels. Thus, in one example, image data can include a first image or channel representing one or more parameters of a first layer of a neural network, and a second image or channel representing one or more parameters of a second layer of a neural network. However, the embodiments are not limited to any specific example.

[0075] In one example, one or more parameter values ​​are floating-point numbers, including the sign bit, exponent, and mantissa. Image data may include: a dedicated image or channel representing the sign bit of one or more parameters; and / or, a dedicated image or channel representing the exponent of one or more parameters; and / or, a dedicated image or channel representing the mantissa of one or more parameters. Each of these images or channels can be decoded to appropriately reconstruct the sign bit, exponent, and mantissa. A more detailed example is described below.

[0076] Figure 2 It shows the relationship with Figure 1 The flowchart of the encoding operation corresponding to the operation. Figure 2 The operation can be performed by one or more processors at one or more locations. For example, Figure 2 The operation can be performed by Figure 8 The encoder 20 shown is used for implementation.

[0077] In step 201, one or more parameters of the machine learning model are obtained. Parameters can be a subset of the parameters of the machine learning model. For example, parameters can be parameters of a single layer of a neural network or parameters of certain specific layers. Parameters can include only the weights of the neural network (excluding bias), or only the bias (excluding weights). Alternatively, all parameters of the machine learning model can be obtained.

[0078] The machine learning model can be of any type, such as neural networks, support vector machines, decision trees, random forests, linear regression, K-means clustering, reinforcement learning systems, etc. This machine learning model can be applied to any type of task, such as image recognition, object detection, image segmentation, image captioning, video object tracking, action recognition, speech recognition, speaker detection, natural language processing, question answering, and control of autonomous vehicles or electromechanical agents. The embodiments are not limited to any specific machine learning model or application of the machine learning model.

[0079] In step 202, one or more parameter values ​​are determined based on one or more parameters of the machine learning model obtained in step 201.

[0080] In step 203, the values ​​of multiple samples in the image data are determined based on one or more parameter values. Specifically, there may be a mapping relationship between one or more parameter values ​​and the values ​​of multiple samples in the image data. In other words, one or more parameter values ​​are mapped from a range of parameter values ​​to a range of sample values ​​to obtain the values ​​of multiple samples in the image data.

[0081] The determination (in step 203) can be based on a target bit depth. The target bit depth can be predetermined or provided by the user. In one example, the target bit depth is 8 bits, and the sample values ​​are in the range of 0 to 255. The values ​​of one or more parameters can be in different ranges, and the determination can include mapping multiple parameter values ​​from parameter value ranges to sample value ranges to generate sample values ​​for each parameter. Data indicating the parameter value ranges can be determined in the generated image data or provided as separate data for decoding. It should be understood that other bit depths, such as 10 bits, 12 bits, or any other suitable bit depth, can be used.

[0082] Prior to step 203, the method may further include determining the spatial location of one of a plurality of samples in the image data. Step 203 may include determining the value of the sample in the image data by mapping the value of the sample in the image data to one of the parameter values, based on the spatial location of the sample in the image data. For example, determining the spatial location of one of a plurality of samples in the image data includes determining a mapping relationship between the spatial locations of the plurality of samples in the image data and a plurality of parameter indices identifying the one or more parameters. As described above, in one example, the machine learning model is a neural network, and one or more parameters include weights of a particular layer of the neural network. The weights are associated with edges connecting units in a first layer and units in a second layer. The weights may be identified using tuples having a first value indexed to a unit in the first layer and a second value indexed to a unit in the second layer. Samples of the image data may be identified by their spatial coordinates. A mapping relationship may be determined to achieve the mapping between sample coordinates and parameter indices, thereby determining the location in the image data used to determine the parameter value. In one example, a direct correspondence may be used, where the parameter is indexed by two integer values ​​and treated as coordinate values ​​in the image data. In other examples, further transformations may be required to convert from parameter indices to sample coordinates, or vice versa.

[0083] In one example, initial image data containing sample values ​​is generated by mapping parameter values ​​to sample values ​​and storing the sample values ​​at a defined spatial location in the initial image data corresponding to the parameters. An image or video compression algorithm is then applied to the initial image data to generate output encoded image data for transmission or storage. However, the embodiments are not limited to this specific example.

[0084] It should also be understood that the embodiments are not limited to sample coordinates, and other types of spatial locations, such as blocks or regions of image data, can also be used. This can be the case in scenarios where the compression algorithm uses coded blocks. In one example, the above mapping is based on the size of the coded block.

[0085] In one example, one or more parameter values ​​are floating-point numbers including a sign bit, exponent, and mantissa. The generated image data may include: a dedicated image or channel for representing the sign bit of one or more parameters; and / or, a dedicated image or channel for representing the exponent of one or more parameters; and / or, a dedicated image or channel for representing the mantissa of one or more parameters. A more detailed example is described below.

[0086] In one example, the machine learning model can be a neural network, and the image data represents the parameters of each layer of the neural network in different images or channels. Therefore, the image data can include: a first image or channel representing one or more parameters of the first layer of the neural network, and a second image or channel representing one or more parameters of the second layer of the neural network. The image data can also represent the architecture, activation function selection, or other hyperparameters for the machine learning model, or these data can be represented separately. In some cases, the architecture can be inferred from the image data; for example, the size of the layers of the neural network can be inferred from the dimensions of the image data.

[0087] In one example, the machine learning model may be in the process of training. Storing the parameters of the machine learning model can be useful as training progresses. For example, it can be used to examine how the parameters change during training, or to restore the parameters to a previous time step if training fails to improve the machine learning model. In this regard, the image data may include: a first time image or channel representing the first parameter value associated with a first time step, and a second time image or channel representing the second parameter value associated with a second time step.

[0088] In step 204, the image data is encoded to generate a bitstream. For example, the image data may be encoded according to an image or video compression algorithm. As mentioned above, any suitable image or video compression algorithm can be used to encode the image data. Any suitable image or video file format can be used to encode the image data. The compression algorithm can be a lossless or lossy compression algorithm. In one example, the image data is encoded according to the PNG image format. Other suitable image formats include JPEG and WebP. In another example, the image data is encoded based on the H.265 / HEVC algorithm. In yet another example, the image data is encoded based on the H.266 / VVC algorithm. Other suitable compression algorithms include H.264 / AVC, VP9, ​​AV1, or AVS-based algorithms. However, the embodiments are not limited to any specific algorithm or format.

[0089] The bitstream can be sent to another computing device for reconstructing one or more parameters. Alternatively, the bitstream can be stored on a local storage device for subsequent reconstruction of one or more parameters.

[0090] Figure 3 This is a flowchart illustrating the operations used to transfer parameters of a machine learning model. Figure 3The flowchart begins at step 301, where the first computing device acquires one or more parameters of the first machine learning model. The first computing device can be any type of computing device, such as a server, desktop computer, mobile device (such as a laptop computer, smartphone, or tablet), or any other suitable computing device.

[0091] In step 302, the first computing device determines one or more parameter values ​​based on one or more parameters of the machine learning model.

[0092] In step 303, the first computing device determines the values ​​of multiple samples in the image data based on one or more parameter values.

[0093] In step 304, the first computing device encodes the image data to generate a bitstream.

[0094] The above can be based on the information above. Figure 2 The described operations are used to complete this.

[0095] In step 305, the first computing device sends the bitstream, and in step 306, the second computing device receives the bitstream. The second computing device can be any type of computing device, such as a server, desktop computer, mobile device (such as a laptop computer, smartphone, or tablet), or any other suitable computing device. Image data can be sent via any type of network connection, such as a local area network (LAN), a wide area network (WAN) including the Internet, or a cellular network.

[0096] In step 307, the second computing device decodes the bitstream to determine the values ​​of multiple samples in the image data.

[0097] In step 308, the second computing device determines one or more parameter values ​​based on the values ​​of multiple samples in the image data.

[0098] In step 309, the second computing device determines one or more parameters of the machine learning model based on one or more parameter values.

[0099] Based on the above information Figure 1 The described operations are used to perform steps 307-309.

[0100] The second computing device determines one or more parameters of a second machine learning model corresponding to one or more parameters of the first machine learning model, based on the determined parameter values. In one example, the parameters are transmitted to deploy a copy of the first machine learning model on the second computing device. Therefore, the second machine learning model is initialized using the same architecture / hyperparameters as the first machine learning model. This data may be represented in image data, or may be sent separately, or may exist pre-existing on the second computing device. Subsequently, the parameters of the second machine learning model may be set based on the determined parameter values ​​to create a copy of the first machine learning model for deployment on the second computing device.

[0101] In another example, the transfer of parameters can be used to update a copy of an existing machine learning model on a second computing device. For example, the first machine learning model may have already undergone further training or fine-tuning. The second computing device can then update the corresponding parameters of the second machine learning model based on the determined parameter values.

[0102] In another example, the first and second computing devices can be part of a distributed training system for training a machine learning model. One or both of the first and second computing devices can train one or more copies of the machine learning model. The transfer of parameter values ​​can be used to synchronize the parameters of the machine learning model between the first and second computing devices during training.

[0103] It should be understood that the embodiments are not limited to the specific examples described above.

[0104] Now refer to Figure 4 The exemplary encoding system embodiments shown are and Figure 5 The exemplary decoding system embodiments shown are used to describe specific encoding and decoding examples. The example encoding and decoding systems can be appropriately implemented according to the above descriptions. Figures 1 to 3 Any operation discussed.

[0105] Figure 4 The encoding system 400 is configured to process one or more parameters 401 of an input machine learning model. The encoding system 400 includes an image generation subsystem 402 configured to generate initial image data 403 based on one or more parameter values ​​401. This may include mapping parameter values ​​to sample values ​​and / or determining sample locations in the image data to represent the parameters. The encoding system 400 also includes an image / video encoder 404 configured to encode the initial image data 403 using an image or video compression algorithm to generate output encoded image data 405 for transmission or storage.

[0106] Figure 5The decoding system 500 is configured to process input encoded image data 501. The decoding system 500 includes an image / video decoder 502 configured to decode the encoded image data 501 according to an appropriate image / video decompression algorithm to generate decoded image data 503. The decoding system 500 also includes a parameter decoding subsystem 504 configured to reconstruct one or more parameters from the decoded image data 504. This may include mapping sample values ​​to parameter values ​​and / or determining sample locations in the decoded image data 503 to represent the parameters. The parameter decoding subsystem 504 provides one or more parameters 505 of the machine learning model represented in the encoded image data 501 as output.

[0107] Three examples of parameter encoding and decoding will now be described. For ease of illustration, each example uses the weights of a single layer of a neural network as parameters and the neural network as a machine learning model. However, it should be understood that the embodiments are not limited to this. Figure 6 As shown, the weights are associated with the connections between cells in the first layer 601 and cells in the second layer 602. Cells in the first layer 601 are indexed by x, and cells in the second layer 602 are indexed by y. Each weight w is identified by a tuple x,y, that is, the weight is associated with the connection between a cell x in the first layer 601 and a cell y in the second layer 602. Therefore, the weight w 1,2 This represents the weight associated with the connection between a cell in the first layer (x = 1) and a cell in the second layer (y = 2). The first layer 601 has X cells, and the second layer 602 has Y cells. For clarity, Figure 6 Some units and connections have been omitted.

[0108] In the first example, the weight values ​​range from w min arrive w max An 8-bit target depth is selected, providing sample values ​​ranging from 0 to 255. For encoding, weights are provided as input to the encoding system 400. The image generation subsystem 401 then converts the weight values ​​from... w min arrive w max The range is mapped to the range of 0 to 255. Since the weights have double-integer indices, these weights can be directly used as sample coordinates for the image data. Therefore, each sample in the initial image 403(V) (with coordinates (x, y)) can be generated as follows: V[x, y] = round((w x,y – w min * 255.0) / (w max– w min )) (1) The asterisk (*) represents multiplication, and round(.) rounds to the nearest integer value.

[0109] Then, the image / video encoder 404 encodes the initial image data 403(V) using the PNG image format to generate output encoded image data 405. This is related to the weight value range. w min and w max Together they are sent from the first computing device carrying the encoding system 400 to the second computing device.

[0110] The second computing device receives the encoded image data 405 / 501 and provides it as input to the decoding system 500. The image / video decoder 502 decodes the encoded image data 501 according to the PNG image format to generate decoded image data 503. Then, the parameter decoding subsystem 504 performs a reverse mapping on the decoded image data 503 to reconstruct the weights 505 as follows: w x,y = (V[x, y] / 255.0) * (w max – w min ) + w min (2).

[0111] In the second example, the range of weight values ​​remains the same. w min arrive w max And an 8-bit target bit depth is selected. However, in this example, the image / video encoder 404 uses the H.265 / HEVC compression algorithm with a CTU size of N×N. In this case, the image generation subsystem 402 generates the initial image 403(V) as follows: V[i, j] = round((w x,y – w min * 255.0) / (w max – w min )) (3) in, i = y / / N ;and j = y % N + x * N ; / / indicates integer division (rounding down to zero), % indicates the modulo operator. In this formula, the first row of CTUs corresponds to the weights at x=0. Specific samples within a CTU correspond to weights arranged in a zigzag pattern, pointing to continuous y-cells.

[0112] Then, the image / video encoder 404 encodes the initial image data 403(V) using the H.265 / HEVC compression algorithm to generate output encoded image data 405. This data is sent from the first computing device carrying the encoding system 400 to the second computing device. Weight value range w min and w max It is also sent to the second computing device. The CTU block size can also be sent, or it can be agreed upon in advance with the second computing device.

[0113] The second computing device receives encoded image data 405 / 501 and decompresses the encoded image data 501 using an image / video decoder 502. Then, the parameter decoding subsystem 504 performs reverse mapping on the decompressed image data 503 to reconstruct the weights 505 as follows: w x,y = (V[i, j] / 255.0) * (w max – w min ) + w min (4).

[0114] In the third example, the weights are 32-bit floating-point numbers represented according to the IEEE 754 format. In this format, the weight values ​​are expressed as: w x,y = sign x,y * 2 ^ (exponent x,y – 128) * (1.mantissa x,y ) (5) One bit is used to represent the sign, eight bits are used to represent the exponent, and 23 bits are used to represent the mantissa.

[0115] The sign bit, exponent, and mantissa are represented in their respective dedicated images. In this example, the target bit depth for each image is 8 bits. The initial sign bit image S generated using the image generation subsystem 402 is: S[x,y] = sign x,y * 255 (6) The initial image E of the exponent generated using the image generation subsystem 402 is: E[x,y] = exponent x,y (7) The initial image M generated using the image generation subsystem 402 is: M[x,y] = mantissa x,y >>15 (8) Here, >> is the right bit shift operator. In this case, only the 8 most significant bits of the mantissa are retained. The remaining bits are quantized away. The image / video encoder 404 encodes the three initial images S, E, and M that form the initial image data 403 according to the PNG image format to generate output encoded image data 405. This encoded image data 405 is sent to the second computing device.

[0116] The second computing device receives the encoded image data 405 / 501 and provides it as input to the decoding system 500. The image / video decoder 502 decodes the encoded image data 501 according to the PNG image format to generate decoded image data 503 comprising three images. Then, the parametric decoding subsystem 504 performs a reverse mapping on each image to reconstruct the sign, exponent, and mantissa as follows: sign x,y = round(S[x, y] / 255) (9) exponent x,y = E[x,y] (10) mantissa x,y = M[x,y]<<15 (11) Here, << is the left shift operator.

[0117] It should be understood that the embodiments are not limited to the examples described above. Furthermore, while the examples above describe encoding / decoding single-layer weights of a neural network, it should be understood that multi-layer weights can be encoded / decoded, and / or subsets of parameters can be encoded / decoded. In addition to neural networks, parameters of other types of machine learning models (such as support vector machines, decision trees, random forests, linear regression, k-means clustering, and reinforcement learning systems) can also be encoded / decoded.

[0118] Figure 7 A schematic diagram of decoder 10 according to an embodiment is shown. Specifically, Figure 7 A schematic diagram of a decoder 10 configured to perform any of the decoding operations described herein is shown, for example, referring to... Figure 1 , Figure 3 and Figure 5The operations described above are omitted here for the sake of brevity.

[0119] like Figure 7 As shown, the decoder 10 includes a processor 11 and a computer-readable medium 12. The processor 11 and the computer-readable medium 12 may be connected via a bus system. The computer-readable medium is configured to store a program, instructions, or code. The processor 11 is configured to execute the program, instructions, or code in the computer-readable medium 12 to perform any of the decoding method embodiments disclosed herein.

[0120] Therefore, in this embodiment, the computer-readable medium 12 is configured to store a computer program that can run in the processor 11, and the processor 11 is configured to run the computer program to perform the steps of any of the decoding methods described herein.

[0121] Figure 8 A schematic diagram of an encoder 20 according to an embodiment is shown. Specifically, Figure 8 A schematic diagram of an encoder 20 configured to perform any of the encoding operations described herein is shown, for example, referring to... Figure 2 , Figure 3 and Figure 4 The operations described above are omitted here for the sake of brevity.

[0122] like Figure 8 As shown, encoder 20 includes processor 21 and computer-readable medium 22. Processor 21 and computer-readable medium 22 may be connected via a bus system. The computer-readable medium is configured to store programs, instructions, or code. Processor 21 is configured to execute the programs, instructions, or code in computer-readable medium 22 to perform the steps of any of the encoding methods described herein.

[0123] Therefore, in this embodiment, the computer-readable medium 22 is configured to store a computer program that can run in the processor 21, and the processor 21 is configured to run the computer program to perform the steps of any of the encoding methods described herein.

[0124] Figure 9 A schematic diagram of multiple computing devices connected via network 90 is shown. The multiple computing devices include, according to... Figure 8 The encoder 20 is configured with an encoding computing device 20A. Multiple computing devices also include, according to... Figure 7The decoder 10 is configured with decoding computing devices 10A-10C. Encoding computing device 20A is configured to represent one or more parameters of a machine learning model as image data and encode the image data using an image or video compression algorithm according to the operations described above. Encoding computing device 20A is configured to send the encoded image data to one or more of the decoding computing devices 10A-10C. Each of the decoding computing devices 10A-10C is configured to decode the encoded image data according to the operations described above to obtain one or more parameter values ​​for determining one or more parameters of the machine learning model. Although Figure 9 A certain number of encoding computing devices and decoding computing devices are shown, but it should be understood that the embodiments are not limited to any particular number of encoding computing devices or decoding computing devices.

[0125] The embodiments may also provide a computer-readable medium containing computer-executable instructions for causing one or more processors of a computing device to perform the method of any embodiment.

[0126] Examples of computer-readable media include volatile and non-volatile media, removable and non-removable media, and include, but are not limited to: solid-state storage, removable disks, hard disk drives, magnetic media, and optical disks. Generally, computer-readable media includes any type of media suitable for storing, encoding, or carrying a series of instructions executable by one or more computers to perform any one or more of the processes and features described herein.

[0127] It should be understood that the functions of the various components can be combined in multiple ways, and are not limited to those described above. For example, in some embodiments, the functions of multiple devices can be integrated into a single device. In other embodiments, the functions of the at least one device can be split into multiple independent (or distributed) devices.

[0128] Conditional language such as “may” is typically used to indicate features / steps used in a particular embodiment, but alternative embodiments may include alternative features or omit these features entirely.

[0129] Furthermore, the method steps are not limited to the specific order described, and it should be understood that these steps can be combined in any other suitable order. In some embodiments, this can allow some method steps to be executed in parallel. Additionally, in some embodiments, specific method steps may be omitted entirely.

[0130] While some embodiments have been discussed, it should be understood that these embodiments are intended to illustrate the general teachings of the invention, and various modifications may be made without departing from the scope of the invention. The scope of the invention should be interpreted in accordance with the appended claims and any equivalents thereof.

[0131] Many other variations and modifications will arise in those skilled in the art upon reference to the foregoing exemplary embodiments. The foregoing embodiments are given by way of example only and are not intended to limit the scope of the invention, which is defined by the appended claims.

Claims

1. A method for decoding parameters of a machine learning model, performed by a first computing device, the method comprising: Decode the bitstream to determine the values ​​of multiple samples in the image data; One or more parameter values ​​are determined based on the values ​​of the multiple samples in the image data; and One or more parameters of the machine learning model are determined based on the one or more parameter values.

2. The method according to claim 1, wherein, Determining one or more parameter values ​​based on the values ​​of the plurality of samples in the image data includes: mapping the values ​​of the plurality of samples in the image data from a range of sample values ​​to a range of parameter values.

3. The method according to claim 1 or 2, wherein, The one or more parameter values ​​are determined based on the bit depth associated with the image data.

4. The method according to any one of the preceding claims, wherein, Determining the one or more parameter values ​​based on the values ​​of the plurality of samples in the image data includes: Determine the spatial location of one of the plurality of samples in the image data; and One of the parameter values ​​is determined based on the spatial location of the sample in the image data and the value of the sample.

5. The method according to claim 4, wherein, Determining a parameter value based on the spatial location and value of the sample in the image data includes: determining the parameter value by mapping the value of the sample in the image data to a parameter value based on the spatial location of the sample in the image data.

6. The method according to claim 4 or 5, wherein, The spatial location of one of the plurality of samples in the image data is determined based on the coding block size.

7. The method according to any one of the preceding claims, wherein, The one or more parameter values ​​are floating-point numbers that include a sign bit, an exponent, and a mantissa.

8. The method according to claim 7, wherein, The image data includes a first dedicated image or channel for representing the sign bits of the one or more parameter values.

9. The method according to claim 7 or 8, wherein, The image data includes a second dedicated image or channel for representing an index of the one or more parameter values.

10. The method according to any one of claims 7 to 9, wherein, The image data includes a third dedicated image or channel for representing the tail of the one or more parameter values.

11. The method according to claim 10, wherein, Determining the one or more parameter values ​​based on the values ​​of the plurality of samples in the image data includes applying a left shift operation to the values ​​of the plurality of samples in the third dedicated image or channel.

12. The method according to any one of the preceding claims, wherein, The image data was received from a remote computing device.

13. The method according to any one of the preceding claims, wherein, The image data is encoded using an image or video compression algorithm.

14. The method according to claim 13, wherein, The compression algorithm is a lossless compression algorithm.

15. The method according to claim 13, wherein, The compression algorithm is a lossy compression algorithm.

16. The method according to claim 13, wherein, The image data is encoded in PNG, JPEG, or WebP image formats.

17. The method according to claim 13, wherein, The image data is encoded based on algorithms such as H.264 / AVC, H.265 / HEVC, H.266 / VVC, VP9, ​​AV1, or AVS-based algorithms.

18. The method according to any one of claims 13 to 17, further comprising: Obtain the encoding information associated with the compression algorithm; Furthermore, decoding the bitstream to determine the values ​​of the plurality of samples in the image data is based on the encoding information.

19. The method according to any one of the preceding claims, wherein, The machine learning model is a neural network.

20. The method according to claim 19, wherein, The image data includes: a first image or channel representing one or more parameters of the first layer of the neural network, and a second image or channel representing one or more parameters of the second layer of the neural network.

21. A method for encoding parameters of a machine learning model, performed by a second computing device, the method comprising: Obtain one or more parameters from a machine learning model; One or more parameter values ​​are determined based on one or more parameters of the machine learning model; The values ​​of multiple samples in the image data are determined based on the one or more parameter values; and The image data is encoded to generate a bitstream.

22. The method according to claim 21, further comprising: The generated bitstream is sent to reconstruct the one or more parameters at another computing device.

23. The method according to claim 21 or 22, wherein, Determining the values ​​of the plurality of samples in the image data based on the one or more parameter values ​​includes: mapping the one or more parameter values ​​from a parameter value range to a sample value range.

24. The method according to any one of claims 21 to 23, further comprising: Send data indicating the range of parameter values.

25. The method according to any one of claims 21 to 24, wherein, The value of the plurality of samples in the image data is determined based on the target bit depth, according to one or more parameter values.

26. The method according to any one of claims 21 to 25, further comprising: Determine the spatial location of one of the plurality of samples in the image data, and Determining the values ​​of the plurality of samples in the image data based on the one or more parameter values ​​includes: determining the values ​​of the samples in the image data by mapping the values ​​of the samples in the image data to one of the parameter values ​​based on the spatial location of the samples in the image data.

27. The method according to claim 26, wherein, Determining the spatial location of one of the plurality of samples in the image data includes: determining the mapping relationship between the spatial location of the sample in the image data and a parameter index that identifies one of the one or more parameters.

28. The method according to claim 26 or 27, wherein, The spatial location of one of the plurality of samples in the image data is determined based on the coding block size.

29. The method according to any one of claims 21 to 28, wherein, The one or more parameter values ​​are floating-point numbers that include a sign bit, an exponent, and a mantissa.

30. The method according to claim 29, wherein, The image data includes a first dedicated image or channel for representing the sign bits of the one or more parameter values.

31. The method according to claim 29 or 30, wherein, The image data includes a second dedicated image or channel for representing an index of the one or more parameter values.

32. The method according to any one of claims 29 to 31, wherein, The image data includes a third dedicated image or channel representing the tail of the one or more parameter values.

33. The method according to claim 32, wherein, Determining the values ​​of the plurality of samples in the image data based on the one or more parameter values ​​includes applying a right shift operation to the mantissa of one of the parameter values.

34. The method according to any one of claims 21 to 33, wherein, The image data is encoded using a lossless compression algorithm.

35. The method according to any one of claims 21 to 33, wherein, The image data is encoded using a lossy compression algorithm.

36. The method according to any one of claims 21 to 33, wherein, The image data is encoded according to PNG, JPEG, or WebP image formats.

37. The method according to any one of claims 21 to 33, wherein, The image data is encoded using H.264 / AVC, H.265 / HEVC, H.266 / VVC, VP9, ​​AV1, or an AVS-based algorithm.

38. The method according to any one of claims 21 to 37, the method further comprising: Send encoded information associated with the compression algorithm to a remote device.

39. The method according to any one of claims 21 to 38, wherein, The machine learning model is a neural network.

40. The method according to claim 39, wherein, The image data includes: a first image or channel representing one or more parameters of a first layer of the neural network, and a second image or channel representing one or more parameters of a second layer of the neural network.

41. The method according to any one of claims 21 to 40, wherein, The image data includes: a first time image or channel for representing a first parameter value associated with a first time step, and a second time image or channel for representing a second parameter value associated with a second time step.

42. A computer-readable medium comprising computer-executable instructions stored thereon, which, when executed by a computing device, cause the computing device to perform the method according to any one of claims 1 to 41.

43. A method for transmitting parameters of a machine learning model, the method comprising: One or more parameters of the first machine learning model are obtained from the first computing device; The first computing device determines one or more parameter values ​​based on one or more parameters of the machine learning model; The first computing device determines the values ​​of multiple samples in the image data based on the one or more parameter values; The first computing device encodes the image data to generate a bitstream; The bitstream is sent by the first computing device; The bitstream is received by a second computing device; The second computing device decodes the bitstream to determine the values ​​of the plurality of samples in the image data; The one or more parameter values ​​are determined based on the values ​​of the plurality of samples in the image data; and One or more parameters of the machine learning model are determined based on the one or more parameter values.

44. An apparatus for decoding parameters of a machine learning model, comprising: The decoding unit is configured to decode the bitstream to determine the values ​​of multiple samples in the image data; and The parameter determination unit is configured to: determine one or more parameter values ​​based on the values ​​of the plurality of samples in the image data, and determine one or more parameters of the machine learning model based on the one or more parameter values.

45. A decoder, comprising: One or more processors; and A computer-readable medium includes computer-executable instructions stored thereon, which, when executed by the one or more processors, cause the one or more processors to perform the method according to any one of claims 1 to 20.

46. ​​An apparatus for encoding parameters of a machine learning model, comprising: The acquisition unit is configured to acquire one or more parameters of a machine learning model; The image data determination unit is configured to: determine one or more parameter values ​​based on one or more parameters of the machine learning model, and determine the values ​​of multiple samples in the image data based on the one or more parameter values; and The encoding unit is configured to encode the image data to generate a bitstream.

47. An encoder, comprising: One or more processors; and A computer-readable medium includes computer-executable instructions stored thereon, which, when executed by the one or more processors, cause the one or more processors to perform the method according to any one of claims 21 to 41.

Images