Processing Method of Neural Network and Electronic Device

Abstract

Disclosed are a method and electronic device for processing a neural network. The electronic device includes: a host processor configured to receive a request for an accelerator to process a neural network, generate an instruction executable by the accelerator in response to the request, and send the instruction to the accelerator; and an accelerator configured to execute the neural network in response to the instruction and send a result of executing the neural network to the host processor.

Description

[0001] This application claims the benefit of Korean Patent Application No. 10-2020-0074268, filed on June 18, 2020, in the Korean Intellectual Property Office, the disclosure of which is incorporated herein by reference in its entirety for all purposes. Technical Field

[0002] The following description relates to a method and electronic device for processing a neural network. Background Art

[0003] Independent hardware can be dedicated to artificial intelligence (AI) technology. For example, AI can perform inference and learning through predetermined operations. In this way, various devices can be dedicated hardware for implementing and executing AI.

[0004] Specialized hardware for AI can be implemented by, for example, a graphics processing unit (GPU), or by repurposed field programmable gate arrays (FPGAs) and application-specific integrated circuits (ASICs). Summary of the Invention

[0005] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.

[0006] In one general aspect, an electronic device includes: a host processor configured to receive a request for an accelerator to process a neural network, generate an instruction executable by the accelerator in response to the request, and send the instruction to the accelerator; and an accelerator configured to execute the neural network in response to the instruction and send a result of executing the neural network to the host processor, wherein the accelerator is configured to determine an intermediate tensor corresponding to a result of executing an operation to be executed by performing a convolution based on a target tensor and a kernel in a channel-axis direction, the target tensor being the target of which the operation to be executed is to be executed, the kernel having a number of input channels and a number of output channels determined based on the target tensor and including an element having a scaling value determined based on the target tensor.

[0007] In one general aspect, an electronic device includes: a host processor configured to receive a request for obtaining a data inference result, generate an instruction in response to the request, and send the instruction to an accelerator; and an accelerator configured to execute a neural network in response to the instruction, and send a result of executing the neural network to the host processor as a data inference result, wherein the accelerator is configured to perform an operation to be performed by performing a convolution based on a target tensor and a kernel, the kernel having a number of input channels and a number of output channels determined based on the target tensor and including an element with a scaling value determined based on the target tensor.

[0008] In one general aspect, a method for processing a neural network includes: receiving, by a host processor, a request for an accelerator to process the neural network; generating, by the host processor, an instruction executable by the accelerator in response to the request, and sending the instruction to the accelerator; executing, by the accelerator, the neural network in response to the instruction, and sending a result of executing the neural network to the host processor; and wherein the step of executing the neural network includes: determining an intermediate tensor corresponding to a result of executing an operation to be executed by performing a convolution based on a target tensor and a kernel in a channel-axis direction, the target tensor being to be executed on which the operation to be executed is to be executed, the kernel having a number of input channels and a number of output channels determined based on the target tensor and including an element of a scaling value determined based on the target tensor.

[0009] In one general aspect, an electronic device for recognizing an image includes: a host processor configured to: receive a request for recognizing an image to be recognized, generate an instruction executable by an accelerator in response to the request, and send the instruction to the accelerator; and an accelerator configured to: execute a neural network with the image to be recognized as input in response to the instruction, and send a result of executing the neural network as a recognition result of the image to be recognized to the host processor, wherein the accelerator is configured to: determine an intermediate tensor corresponding to a result of performing the operation to be performed by performing a convolution based on a target tensor and a kernel in a channel-axis direction, the target tensor being the target of which the operation to be performed is to be performed, the kernel having a number of input channels and a number of output channels determined based on the target tensor and including an element of a scaling value determined based on the target tensor.

[0010] In one general aspect, a processor-implemented tensor processing method includes: receiving a request for an accelerator to process a neural network including a normalization layer; and generating instructions executable by the accelerator in response to the request, wherein, by executing the instructions, the accelerator is configured to: determine an intermediate tensor corresponding to a result of performing a portion of an operation included in the normalization layer by performing a convolution in a channel-axis direction, the convolution being based on a target tensor and a kernel, the target tensor being the target tensor on which the portion of the operation is to be performed, the kernel having a number of input channels and a number of output channels determined based on the target tensor and including an element of a scaling value determined based on the target tensor.

[0011] The accelerator may be configured to determine the intermediate tensor by extracting diagonal elements from a result tensor determined by convolution based on the target tensor and the kernel.

[0012] The number of input channels of the kernel can be determined based on the number of elements of the normalization unit applied to the target tensor.

[0013] The number of elements of the normalization unit applied to the target tensor may be equal to the number of channels of the input tensor, and the number of input channels of the kernel may be equal to the number of channels of the input tensor.

[0014] The number of output channels of the kernel is determined based on the width and length of the target tensor.

[0015] The scaling value for each element included in the kernel may include a runtime value corresponding to the target tensor.

[0016] The scaled value of each element included in the kernel may be equal to the value of the corresponding element in the target tensor.

[0017] The target tensor may be determined based on: a mean subtraction tensor including a value determined by subtracting a value of each element from a mean value of elements included in an input tensor of the normalization layer; and a constant value determined based on the number of elements of a normalization unit applied to the target tensor.

[0018] The target tensor can be determined by performing a convolution in the channel axis direction, the convolution being based on: the mean subtraction tensor; and a second kernel having a number of input channels and a number of output channels determined based on the mean subtraction tensor and including diagonal elements with scaling values determined based on the constant value.

[0019] The number of input channels and the number of output channels of the second kernel may be equal to the number of elements of the normalization unit, and diagonal elements in the second kernel may have scaling values different from scaling values of the remaining elements.

[0020] The constant value may be equal to a square root of the number of elements of the normalization unit applied to the target tensor, and the scaling value of the second kernel may be equal to the inverse of the square root.

[0021] The normalization layer may be configured to perform normalization using either or both of a mean and a variance determined based on values of one or more elements included in a target tensor.

[0022] Convolution may be performed between a kernel and an input tensor transformed so that elements included in the same channel are aligned in a row, and an intermediate tensor may be determined by transforming elements determined as a result of the convolution into the same form as that of the input tensor.

[0023] The convolution may be performed in the accelerator such that the target tensor is not sent outside the accelerator to perform operations according to the normalization layer.

[0024] The accelerator may be included in any one of: a user terminal to which data to be inferred using a neural network is input; and a server to receive the data to be inferred from the user terminal.

[0025] A non-transitory computer-readable storage medium may store instructions that, when executed by a processor, configure the processor to perform the method.

[0026] In another general aspect, an accelerator includes: one or more processors configured to: determine a target tensor in which a portion of an operation included in a normalization layer in a neural network is to be performed; determine a kernel having a number of input channels and a number of output channels determined based on the target tensor and including an element with a scaling value determined based on the target tensor; and determine an intermediate tensor corresponding to a result of performing the portion of the operation by performing a convolution based on the target tensor and the kernel in a channel-axis direction.

[0027] The one or more processors may be configured to determine the intermediate tensor by extracting diagonal elements from a result tensor determined by convolution based on the target tensor and the kernel.

[0028] The determined number of input channels of the kernel may be based on the number of elements of the normalization unit applied to the target tensor.

[0029] The scaling value for each element included in the kernel may include a runtime value corresponding to the target tensor.

[0030] The scaled value of each element included in the kernel may be equal to the value of the corresponding element in the target tensor.

[0031] In another general aspect, an electronic device includes: a host processor configured to: generate an instruction executable by the accelerator in response to a request for the accelerator to process a neural network including a normalization layer; and the accelerator configured to: determine, by executing the instruction: an intermediate tensor corresponding to a result of performing a portion of an operation included in the normalization layer by performing convolution in a channel-axis direction, the convolution being based on a target tensor and a kernel, the target tensor being where the portion of the operation is to be performed, the kernel having a number of input channels and a number of output channels determined based on the target tensor and including an element of a scaling value determined based on the target tensor.

[0032] In another general aspect, an electronic device includes: a host processor configured to: generate an instruction in response to a request for a data inference result; and an accelerator configured to: by executing the instruction, perform an operation of a normalization layer of a neural network by performing a convolution based on a target tensor and a kernel, the kernel having a number of input channels and a number of output channels determined based on the target tensor and including an element with a scaling value determined based on the target tensor; and determine a data inference result based on the steps of performing the operation.

[0033] The data inference result can be any of speech recognition, machine translation, machine interpretation, object recognition, pattern recognition, and computer vision.

[0034] Other features and aspects will be apparent from the following detailed description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 An example of an electronic device is shown.

[0036] Figure 2 and Figure 3 Shows an example of processing a tensor for the first normalization layer.

[0037] Figure 4 and Figure 5 Shows an example of processing tensors for the second normalization layer.

[0038] Figure 6 Shows an example of a neural network including a normalization layer.

[0039] Figure 7 Shows an example of a normalization layer.

[0040] Figure 8 and Figure 9 An example of a tensor processing method is shown.

[0041] Figure 10 and Figure 11 An example of an electronic device is shown.

[0042] Throughout the drawings and detailed description, unless otherwise described or provided, the same drawing reference numerals will be understood to refer to the same elements, features, and structures. The drawings may not be to scale, and the relative sizes, proportions, and depictions of the elements in the drawings may be exaggerated for clarity, illustration, and convenience. DETAILED DESCRIPTION

[0043] The following detailed description is provided to help the reader gain a comprehensive understanding of the methods, devices and / or systems described herein. However, various changes, modifications and equivalents of the methods, devices and / or systems described herein will be clear after understanding the disclosure of the application. For example, the order of operations described herein is merely an example and is not limited to those orders set forth herein, but can be changed as will be clear after understanding the disclosure of the application, except for operations that must occur in a specific order. In addition, for greater clarity and conciseness, descriptions of features known in the art after understanding the disclosure of the application may be omitted.

[0044] Although the terms "first" or "second" are used herein to describe various members, components, regions, layers, or portions, these members, components, regions, layers, or portions should not be limited by these terms. Instead, these terms are used only to distinguish one member, component, region, layer, or portion from another member, component, region, layer, or portion. Therefore, without departing from the teachings of the examples described herein, what is referred to as a first member, first component, first region, first layer, or first portion may also be referred to as a second member, second component, second region, second layer, or second portion.

[0045] Throughout the specification, when an element (such as a layer, region, or substrate) is described as being “on,” “connected to,” or “coupled to” another element, the element may be directly “on,” “connected to,” or “coupled to” the other element, or one or more other elements may be present between them. Conversely, when an element is described as being “directly on,” “directly connected to,” or “directly coupled to” another element, there may not be other elements present between them. Similarly, expressions such as “between” and “immediately between,” and “adjacent to” and “immediately adjacent to” may also be interpreted as described above.

[0046] The terms used herein are only used to describe specific examples and will not limit the disclosure. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. As used herein, the term "and / or" includes any one of the relevant listed items and any combination of any two or more. As used herein, the terms "include", "comprise" and "have" illustrate the presence of stated features, quantities, operations, elements, components and / or combinations thereof, but do not exclude the presence or addition of one or more other features, quantities, operations, elements, components and / or combinations thereof. The term "may" is used herein for examples or embodiments (e.g., what may be included or achieved for an example or embodiment) to indicate that there is at least one example or embodiment that includes or achieves such features, and all examples are not limited thereto.

[0047] Unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by those skilled in the art to which the present disclosure pertains and as generally understood after understanding the present disclosure. Unless expressly defined as such herein, terms (such as those defined in commonly used dictionaries) will be interpreted as having a meaning consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted in an idealized or overly formal sense.

[0048] Hereinafter, examples will be described in detail with reference to the accompanying drawings. The specific structural or functional descriptions below are illustrative for the purpose of describing examples only, and the scope of the examples is not limited to the descriptions provided in this specification. Those skilled in the art may make various changes and modifications based on their understanding of the disclosure of this application. The same reference numerals in the drawings represent the same elements, and known functions or configurations will be omitted herein.

[0049] Figure 1 An example of an electronic device is shown.

[0050] Reference Figure 1 , the electronic device 100 may include a host processor 110 (eg, one or more processors), an off-chip memory 120, a memory controller 130, and an accelerator 140. The host processor 110, the off-chip memory 120, the memory controller 130, and the accelerator 140 may communicate with each other via a bus.

[0051] The host processor 110 may be a device that controls the operation of components included in the electronic device 100 and may include, for example, a central processing unit (CPU). The host processor 110 may receive a request for processing a neural network in the accelerator 140 and, in response to the request, may generate an instruction that can be executed by the accelerator 140. The request may be used for data inference based on the neural network, and the accelerator 140 may execute the neural network in response to the generated instruction to obtain a data inference result for speech recognition, machine translation, machine interpretation, object recognition, pattern recognition, and / or computer vision, etc. The host processor 110 may send parameters of the neural network and inference target data to the accelerator 140.

[0052] The off-chip memory 120 may be a memory provided outside the accelerator 140, and may be, for example, a dynamic random access memory (DRAM) used as a main memory of the electronic device 100. The off-chip memory 120 may be accessed through the memory controller 130. The off-chip memory 120 may store parameters of a neural network to be executed by the accelerator 140, and is utilized when the on-chip memory in the accelerator 140 is insufficient for the accelerator 140 to execute the neural network.

[0053] The off-chip memory 120 may have a larger memory capacity than the on-chip memory in the accelerator 140. However, when the neural network is executed, the memory access cost of the accelerator 140 accessing the off-chip memory 120 may be greater than the memory access cost of the accelerator 140 accessing the internal on-chip memory. The memory access cost may be the power and / or time required to access the corresponding memory and read or write data.

[0054] The accelerator 140 may be an AI accelerator that infers input data by executing a neural network according to instructions of the host processor 110, and may be an independent processor different from the host processor 110. For example, the accelerator 140 may be a neural processing unit (NPU) (or neural processor), a graphics processing unit (GPU), or a tensor processing unit (TPU).

[0055] Due to the characteristics of the operation according to the neural network, the accelerator 140 can process tasks that can be more efficiently processed by a separate dedicated processor (i.e., the accelerator 140) rather than by the general host processor 110. In this example, the on-chip memory and one or more processing elements (PEs) included in the accelerator 140 can be used by the accelerator 140 to execute the neural network. The on-chip memory can be a global buffer included in the accelerator 140 and can be distinguished from the off-chip memory 120 provided outside the accelerator 140. For example, the on-chip memory can be a scratchpad memory, a static random access memory (SRAM), etc. that can be accessed through the address space.

[0056] A neural network may include multiple layers. A neural network may include an input layer, multiple hidden layers, and an output layer. Each layer may include multiple nodes (also called artificial neurons). Although a node may be referred to as an "artificial neuron," such a term is not intended to convey any relevance to how a neural network structure is computationally mapped or intuitively identifies information with how human neurons operate. In other words, the term "artificial neuron" is simply a technical term for a hardware-implemented node representing a neural network. Each node may be a computing unit with one or more inputs and outputs, and the nodes may be connected to each other. Weights may be set for the connections between nodes, and the weights may be adjusted or changed. Weights may amplify, reduce, or maintain relevant data values, thereby determining the extent to which the data values influence the final result. The weighted inputs of the nodes included in the previous layer may be input to each node included in the output layer. The process of inputting weighted data from a predetermined layer to the next layer may be referred to as propagation.

[0057] In order to improve the performance of neural networks, normalization can be applied to data transmitted between multiple layers. In other words, the data distribution can be normalized by a series of operations (such as obtaining the mean and / or variance of the data transmitted from one layer to the next layer, and dividing the result of subtracting the value of each element of the corresponding data from the mean by the variance). In one example, the normalization layer can perform normalization using any one or both of the mean and variance determined based on the values of one or more elements included in the input tensor. Normalization applied to neural networks may include, for example, layer normalization and instance normalization. Layer normalization can be a technique that performs normalization in the channel direction regardless of the batch size, and can be applied to models in language fields such as speech recognition, machine translation, and machine interpretation. Instance normalization can be a technique that normalizes the entire image in units of channels by performing normalization in the width and height directions, and can be applied to, for example, generative adversarial networks (GANs).

[0058] In addition, when the attention technique is applied to a neural network for image processing, global average pooling in which the size of the pooling window is equal to the size of the entire image can be performed, and the neural network to which global average pooling is applied may include, for example, a spatial pyramid pooling network (SPPNet) and a squeeze and excitation network (SENet).

[0059] As described above, the operation for performing normalization of the neural network may include an operation for calculating the mean and / or variance each time data is transferred between multiple layers. For such an operation, a typical electronic device may transfer the data to be normalized to a special operator (such as a host processor outside the accelerator), and the result calculated by the special operator may be received again by the accelerator, so such data movement of a typical electronic device may increase latency and power consumption. Therefore, in order to improve the performance of the electronic device 100, in one example, when the operation to be performed belongs to the operation of the normalization layer, the accelerator 140 may detect the operation to be performed as an operation that can be performed by the host processor 110, and convert the operation to be performed into an operation that can be performed by the accelerator 140 to be executed on the accelerator 140. In one example, the electronic device 100 of one or more embodiments may minimize data movement outside the accelerator 140 by performing the operation for performing normalization of the neural network inside the accelerator 140.

[0060] For example, in order to perform an operation that is not directly supported by the accelerator 140, a sum of squares operation and / or a mean square operation may be replaced with a convolution operation. The kernel for the convolution operation may be determined based on the data to be normalized. In this way, even when the convolution operation does not match the operation specified by the normalization layer, the accelerator 140 may obtain output data based on the convolution operation that is identical to the output that would be generated by directly executing the operation specified by the normalization layer. In short, even without modifying the hardware architecture, the accelerator 140 may obtain output data equivalent to the output of the normalization layer without sending the data to a special operator (such as the host processor 110).

[0061] Hereinafter, a detailed description will be provided with reference to the following drawings.

[0062] Figure 2 and Figure 3 Shows an example of processing a tensor for the first normalization layer.

[0063] Reference Figure 2 , a process of dividing the value of each element of the input tensor by a constant (e.g., a predetermined value) obtained based on the number of elements belonging to the normalization unit to obtain a mean square operation result according to the first normalization layer is shown. The first normalization layer may be based on layer normalization, and convolution may be performed to attenuate in the channel axis direction. Such a convolution operation may be performed based on a convolution layer. For ease of description, Figure 2 The examples of tensor shapes, kernel shapes, and element values shown in are provided, and various shapes and values may be applicable without limitation.

[0064] For data normalization, operations according to the first normalization layer may be performed, and the tensor of the portion on which such operations have been performed may correspond to Figure 2 For example, Figure 2 The input tensor of may be an average subtraction tensor obtained by subtracting the value of each element from the average value of the elements included in the tensor input to the first normalization layer. Figure 2 In the example of , the input tensor has a shape of (1, 4, 3, 1), and the value of each element can be represented by any one of a to l. Here, the tensor structure or shape is represented as (batch, channel, width, height).

[0065] Figure 2The kernel shown in may have a number of input channels (e.g., a total number) and a number of output channels determined based on the input tensor. The kernel may have a number of input channels and a number of output channels that are equal to the number of elements belonging to the normalization unit applied to the input tensor. For example, the kernel may have a number of input channels and a number of output channels that are equal to the number of channels of the input tensor. In Figure 2 In the example of , the kernel has a shape of (4, 4, 1, 1), where the kernel shape is represented as (input channels, output channels, width, height). Furthermore, in the kernel, diagonal elements may have a scaling value corresponding to the inverse of the square root of the number of elements belonging to the normalization unit applied to the input tensor (e.g., the number of channels of the input tensor), and the remaining elements may have a scaling value of "0." In this specification, for ease of description, scaling values may also be referred to as weight values.

[0066] The convolution between the input tensor and the kernel is performed in the channel-axis direction. Figure 2 In the convolution operation, the channel axis direction can be indicated by Ch. For example, among the first element values a, b, c, and d in each channel of the input tensor and the values 1 / 2, 0, 0, and 0 of the first column of the kernel, a can be multiplied by 1 / 2, and b, c, and d can be multiplied by "0". Based on their sum, the value of the first element in the output tensor can be determined to be a / 2. Similarly, convolution can also be performed on the remaining elements. Each element in the output tensor can have a value obtained by multiplying the value of the corresponding element in the input tensor by the inverse of the square root of the number of channels of the input tensor. The output tensor has the same shape as the input tensor, (1, 4, 3, 1).

[0067] According to another example, in an accelerator including a shifter and / or a divider, an operation of dividing by the square root of the number of elements belonging to a normalization unit applied to an input tensor may be performed in the shifter and / or the divider without the above-mentioned convolution operation.

[0068] Reference Figure 3 , an example of a convolution operation for obtaining a mean square operation result according to the first normalization layer is shown. For ease of description, Figure 3 The tensor shapes, kernel shapes, and element values shown in are examples, and various shapes and element values may be applied without limitation.

[0069] Figure 3 The input tensor in the example can be Figure 2 The output tensor obtained in the example of and may have a shape of (1, 4, 3, 1).

[0070] Figure 3The kernel shown in may have a number of input channels and a number of output channels determined based on the input tensor. The kernel may have a number of input channels determined based on the number of elements belonging to the normalization unit applied to the input tensor. For example, the kernel may have a number of input channels equal to the number of channels of the input tensor. Additionally, the kernel may have a number of output channels equal to the width and length of the input tensor. Figure 3 In the example of , the kernel may have a shape of (4,3,1,1).

[0071] In addition, each element in the kernel may have a scaling value determined based on the input tensor. The scaling value of each element included in the kernel may include a runtime value (e.g., the value of the target tensor at runtime) corresponding to the target tensor (e.g., the input tensor). For example, the scaling value of each element included in the kernel may be equal to the value of the corresponding element in the target tensor.

[0072] The convolution between the input tensor and the kernel may be performed in the channel axis direction. First, the scaling values a / 2, b / 2, c / 2, and d / 2 included in the first column of the kernel may be convolved with the input tensor, so that the elements in the first channel of the initial tensor (e.g., the result tensor) may be determined. For example, the first element values a / 2, b / 2, c / 2, and d / 2 in each channel of the input tensor may be multiplied by the corresponding scaling values a / 2, b / 2, c / 2, and d / 2 of the first column of the kernel, respectively, and based on their sum, the value of the first element in the first channel of the initial tensor may be determined as (a / 2, b / 2, c / 2, and d / 2). 2 +b 2 +c 2 +d 2 ) / 4. Through the above-mentioned convolution operation, the sum of the squares of the first element values a / 2, b / 2, c / 2 and d / 2 in each channel of the input tensor can be obtained. In the initial tensor, the value of the second element in the first channel can be determined by the convolution operation between the second element values e / 2, f / 2, g / 2 and h / 2 in each channel of the input tensor and the corresponding scaling values a / 2, b / 2, c / 2 and d / 2 of the kernel. However, since the convolution operation is performed to obtain the sum of squares, the value of the second element in the first channel of the initial tensor can be discarded, which will be described later. Similarly, the value of the third element in the first channel of the initial tensor can also be discarded.

[0073] In addition, the scaling values e / 2, f / 2, g / 2, and h / 2 included in the second column of the kernel may be convolved with the input tensor, so that an element in the second channel of the initial tensor may be determined. For example, the second element values e / 2, f / 2, g / 2, and h / 2 in each channel of the input tensor may be multiplied by the corresponding scaling values e / 2, f / 2, g / 2, and h / 2 of the second column of the kernel, respectively, and based on their sum, the value of the second element in the second channel of the initial tensor may be determined as (e2 +f 2 +g 2 +h 2 ) / 4. Similarly, the remaining element values can be determined. In addition, the scaling values i / 2, j / 2, k / 2, and l / 2 included in the third column of the kernel can be convolved with the input tensor, so that the elements in the third channel of the initial tensor can be determined.

[0074] exist Figure 3 In an example, the initial tensor may have a shape of (1, 3, 3, 1), and the output tensor may be determined by extracting the diagonal elements from the initial tensor. No additional hardware is required to extract the diagonal elements from the initial tensor. The output tensor may have a shape of (1, 3, 1, 1). In one example, Figure 3 The input tensor, kernel, and output tensor of may correspond to the further described below. Figure 8 and Figure 9 The target tensor, kernel, and intermediate tensors in any graph in .

[0075] Belong to applied to Figure 3 The sum of the squares of the elements of the normalized unit of the input tensor can be obtained by Figure 3 The convolution operation performed in is used to calculate. Figure 2 The convolution operation and Figure 3 When the convolution operation is performed continuously, Figure 3 The final output tensor obtained in may include the tensors applied to Figure 2 The mean square of the elements of the normalized unit of the input tensor . In addition, as mentioned above, when Figure 2 When the input tensor of is the average subtraction tensor, Figure 3 The final output tensor obtained in may include the variance of the elements belonging to the normalized unit. This variance can be used for normalization, which will be described in detail later.

[0076] In this way, the variance of the tensor input to the normalization layer through the two convolutional layers can be obtained by the multiplier-accumulator (MAC) included in the accelerator without the tensor being transmitted outside the accelerator, thereby reducing the memory access cost. When the accelerator includes a shifter and / or a divider, even in the absence of a convolutional layer, Figure 2 The operations described in can also be performed, so the variance can be calculated using only one convolutional layer.

[0077] According to another example, Figure 2 The operations described in Figure 3 The operations described in are then performed. In this example, Figure 3 Each of the element values of the output tensor calculated in can be divided by the number of elements belonging to the normalization unit. However, when the input range is restricted to be small, as described above in Figure 2 The operations described in Figure 3 The operations described in may be convenient in terms of quantization and thus may be preferred depending on the embodiment.

[0078] Although the height of the input tensor is "1" in the previous non-limiting example, when the width or height is not "1", the input tensor can be reshaped so that the width or height is "1", and then the above operation can be performed, and the obtained output tensor can be reshaped again into the original input tensor shape.

[0079] Furthermore, although the batch of the input tensor is “1” in the previous description, when the batch is not “1”, the above-described operation may be performed independently for each batch.

[0080] Figure 4 and Figure 5 Shows an example of processing tensors for the second normalization layer.

[0081] Reference Figure 4 , a process of dividing the value of each element of the input tensor by a constant (e.g., a predetermined value) obtained based on the number of elements belonging to the normalization unit to obtain a mean square operation result according to the second normalization layer is shown. The second normalization layer may be based on instance normalization, and after transforming the shape of the tensor by reshaping and transposing, the convolution may be performed to attenuate in the channel axis direction to normalize the entire image in units of channels. Such a convolution operation may be performed based on a convolution layer. For ease of description, Figure 4 The tensor shapes, kernel shapes, and element values shown in are examples, and various shapes and values may be applied without limitation.

[0082] For data normalization, operations according to the second normalization layer should be performed, and the tensor of the portion on which such operations have been performed may correspond to Figure 4 The input tensor in . For example, Figure 4 The input tensor of can be a mean subtraction tensor consisting of the mean of the elements included in the tensor input to the second normalization layer minus the value of each element. Figure 4 In the example of , the input tensor has a shape of (1, 2, 2, 2), and the value of each element can be represented by any one of a to h. Here, the tensor structure is represented as (batch, channel, width, height).

[0083] In order to refer to the above Figure 2 and Figure 3The convolution operation described above, performed in the channel axis direction, is applied to the instance normalization according to the second normalization layer, and the shape of the input tensor can be transformed. First, the two-dimensional elements in each channel of the input tensor can be reshaped into one-dimensional elements. For example, the elements a to d in the first channel of the input tensor can be arranged in a row, and the elements e to h in the second channel can also be arranged in a row. The reshaped input tensor can be transposed. The transposed input tensor can have a shape of (1, 4, 2, 1) and is convolved with the kernel as described below.

[0084] Figure 4 The kernel shown in may have a number of input channels and a number of output channels determined based on the input tensor. The kernel may have a number of input channels and a number of output channels that are equal to the number of elements belonging to the applied normalization unit. The number of elements belonging to the normalization unit applied for instance normalization may be the number of elements included in the same channel in the input tensor. In addition, the diagonal elements and the remaining elements in the kernel may have different scaling values. For example, the diagonal elements of the kernel may include the inverse of the square root of the number of elements in the applied normalization unit as a scaling value. The remaining elements may include a scaling value of "0".

[0085] The convolution between the input tensor and the kernel is performed in the channel axis direction and Figure 4 In the above description (e.g., Figure 2 The description of the convolution between the input tensor and the kernel) can be applied to the convolution operation.

[0086] According to another example, if the accelerator includes a shifter and / or a divider, the convolution operation requiring the convolution layer can be replaced with a division operation to be performed by the shifter and / or the divider. By utilizing the shifter and / or the divider, the number of convolution layers used to perform normalization can be reduced by "1".

[0087] Reference Figure 5 , an example of a convolution operation for obtaining a mean square operation result according to the second normalization layer is shown. For ease of description, Figure 5 The tensor shapes, kernel shapes, and element values shown in are examples, and various shapes and element values may be applied without limitation.

[0088] Figure 5 The input tensor in the example can be Figure 4 The output tensor obtained in the example of and may have a shape of (1, 4, 2, 1).

[0089] Figure 5The kernel shown in may have a number of input channels and a number of output channels determined based on the input tensor. The kernel may have a number of input channels determined based on the number of elements belonging to the normalization unit applied to the input tensor. The elements included in the same channel used for normalization according to the instance of the second normalization layer correspond to the normalization unit. Figure 4 As described in , since the tensor shape is transformed by reshaping and transposing, the tensor is transformed to have the same number of channels as the number of elements included in the same channel, so in Figure 5 In the example of , the kernel can have a number of input channels equal to the number of channels of the input tensor. In addition, the kernel can have a number of output channels equal to the width and length of the input tensor. Figure 5 In the example, the kernel may have a shape of (4,2,1,1).

[0090] Furthermore, each element in the kernel may have a scaling value determined based on the input tensor. The scaling value of each element included in the kernel may include a runtime value corresponding to the target tensor. For example, the scaling value of each element included in the kernel may be equal to the value of the corresponding element in the target tensor.

[0091] The convolution between the input tensor and the kernel can be performed in the channel axis direction. Figure 3 The description of the convolution between the input tensor and the kernel) can be applied to the convolution operation.

[0092] exist Figure 5 In an example, an initial tensor (e.g., a result tensor) determined by a convolution operation may have a shape of (1, 2, 2, 1), and an output tensor may be determined by extracting diagonal elements from the initial tensor. No additional hardware is required to extract diagonal elements from the initial tensor, and the output tensor may have a shape of (1, 2, 1, 1). In another example, the output tensor may be determined by transforming elements determined as a result of the convolution (e.g., elements in the initial tensor) into a form that is the same as that of the input tensor. In one example, Figure 5 The output tensor, input tensor, and kernel of may correspond to the further described below. Figure 8 and Figure 9 The intermediate tensors, target tensors, and kernels in any graph in .

[0093] In this way, when Figure 4 The convolution operation and Figure 5 When the convolution operation is performed continuously, Figure 5 The final output tensor obtained in may include the tensors applied to Figure 4 The mean square of the elements of the normalized unit of the input tensor . In addition, as mentioned above, when Figure 4 When the input tensor of is the average subtraction tensor, Figure 5The final output tensor obtained in may include the variance of the elements belonging to the normalized unit. The obtained variance can be used for normalization.

[0094] when Figure 5 When the convolution operation is performed, Figure 4 The reshape and transpose operations described in can be performed before performing the convolution. The sum of the squares of the elements belonging to the normalization unit can be obtained by Figure 5 It is calculated by the convolution operation.

[0095] According to another example, Figure 4 The operations described in Figure 5 In this example, reshaping and transposing the input tensor can be done in Figure 5 The convolution operation is performed before, and Figure 5 Each of the element values of the output tensor computed in can be divided by the number of elements belonging to the normalization unit.

[0096] Although the batch of the input tensor is "1" in the previous non-limiting example, when the batch is not "1", the above operation can be performed independently for each batch.

[0097] Figure 6 Shows an example of a neural network including a normalization layer.

[0098] Reference Figure 6 , showing a bidirectional encoder representations from transformers (BERT) model. Among the multiple layers included in the BERT model, there is a normalization layer 610, wherein when the corresponding model is repeated 12 times, the normalization layer 610 is executed a total of 12 times. Since the normalization layers 610 are provided separately from each other, it is difficult for a typical BERT model to process the normalization layer 610 at one time outside the accelerator, and since the normalization layer 610 is processed outside the accelerator each time in the typical BERT model, the cost of data movement (e.g., memory access cost) may be large. However, when such a typical normalization layer 610 is replaced with the above reference Figures 1 to 5 When describing the convolutional layer according to one or more embodiments, such data movement outside the accelerator for processing the normalization layer 610 is prevented, and thus such costs can be effectively avoided. Figure 6 In the example of , an attention mask (attention_mask) having a shape of (1, 128, 128) represented as (minbatch, length, length) and another input having a shape of (128, 768) represented as (length, hidden layer size) may be input to the attention layer.

[0099] Figure 7 Shows an example of a normalization layer.

[0100] Reference Figure 7 , showing the internal structure of the normalization layer. Figure 7 The normalization layer shown in FIG can be used to perform the Figure 6 The layer normalization layer in the BERT model of . The mean subtraction tensor can be obtained from the input tensor at the first part 710 of the normalization layer, and the variance tensor can be obtained at the second part 720. However, the second part 720 can be replaced by the above reference Figures 1 to 5 One or two convolutional layers of one or more embodiments described. The inverse of the square root of the variance may be obtained at the third portion 730 and then multiplied by the mean subtraction tensor at the fourth portion 740. In this case, normalization may be performed by the neural network.

[0101] Figure 8 and Figure 9 An example of a tensor processing method is shown.

[0102] Reference Figure 8 , showing a tensor processing method performed by a host processor. In operation 810, the host processor may receive a request for processing a neural network (e.g., a neural network including a normalization layer) through an accelerator. In operation 820, the host processor may generate an instruction that can be executed by the accelerator in response to the request, and send the instruction to the accelerator. In one example, the accelerator may execute the neural network in response to the instruction, and send the result of executing the neural network to the host processor. For example, the accelerator may detect whether an operation to be executed in the operation included in the neural network belongs to an operation that can be executed by the accelerator or an operation that can be executed by the host processor; when it is detected that the operation to be executed belongs to an operation that can be executed by the host processor, the operation to be executed is converted into an operation that can be executed by the accelerator for execution on the accelerator. The accelerator may convert the operation to be performed into an operation that can be performed by the accelerator for execution on the accelerator by performing the following steps: determining an intermediate tensor corresponding to the result of performing the operation to be performed by performing a convolution based on the target tensor and the kernel in the channel axis direction, the target tensor being used to perform the operation to be performed, the kernel having the number of input channels and the number of output channels determined based on the target tensor and including an element of a scaling value determined based on the target tensor. In one example, when the instruction is executed by the accelerator, the accelerator may determine an intermediate tensor corresponding to the result of performing a portion of the operation included in the normalization layer by performing a convolution based on the target tensor and the kernel in the channel axis direction. In one example, Figure 8 The intermediate tensor, target tensor and kernel of Figure 3 and Figure 5 The output tensors, input tensors, and kernels in any graph in .

[0103] Reference Figure 9 , shows a tensor processing method performed by an accelerator (for example, a tensor processing method performed by the accelerator when the operation to be performed is part of the operation of the normalization layer). In operation 910, the accelerator may obtain (for example, determine) a target tensor for performing the part of the operation in the normalization layer included in the neural network. In operation 920, the accelerator may obtain a kernel having the number of input channels and the number of output channels determined based on the target tensor and including an element with a scaling value determined based on the target tensor. In operation 930, the accelerator may determine an intermediate tensor corresponding to the result of performing the part of the operation by performing a convolution based on the target tensor and the kernel in the channel axis direction. In one example, Figure 9 The intermediate tensor, target tensor and kernel of Figure 3 and Figure 5 The output tensor, input tensor, and kernel in any of the graphs in [ 1 ]. The tensor processing method can be applied to training or inference on the algorithm side, the compiler side, or network hardware using multi-head attention (such as automatic speech recognition (ASR) or transformers). In addition, even when a low-precision quantization scheme is applied to the tensor processing method, performance gains can be ensured while maintaining system accuracy.

[0104] Reference above Figures 1 to 7 The description provided can be applied to Figure 8 and Figure 9 The operations shown in .

[0105] Figure 10 and Figure 11 An example of an electronic device is shown.

[0106] Reference Figure 10 , electronic devices (e.g., Figure 1 The electronic device 100) can be implemented as a server 1000.

[0107] The server 1000 may be an independent device different from the user terminal controlled by the user, and may communicate with the user terminal via a wired and / or wireless network. In one example, the server 1000 may be an electronic device for recognizing an image. Data to be inferred using a neural network (e.g., an image to be recognized) may be collected from the user terminal and sent to the server 1000 via the network, and the server 1000 may be configured according to the above reference. Figures 1 to 5The described tensor processing method processes layers (e.g., normalization layers) included in a neural network through accelerator 1020. For example, host processor 1010 may receive a request to identify an image to be recognized, generate instructions executable by accelerator 1020 in response to the request, and send the instructions to accelerator 1020. In response to the instructions, accelerator 1020 may execute the neural network using the image to be recognized as input, and send the results of executing the neural network to host processor 1010 as recognition results for the image to be recognized. In this example, tensors may not be sent to host processor 1010 outside of accelerator 1020 to perform operations based on the normalization layer. Server 1000 may then obtain inference results (e.g., image recognition results) from the neural network and return the inference results to the user terminal. In this example, since tensors are not sent outside of accelerator 1020, the latency and power consumption of tensor transmission can be reduced, and the efficiency of server 1000 in recognizing images can be improved. For example, user terminals may include various computing devices (such as smart phones, tablets, laptops, and personal computers), various wearable devices (such as smart watches and smart glasses), various home appliances (such as smart speakers, smart TVs, and smart refrigerators), smart cars, smart automatic service terminals (kiosks), and Internet of Things (IoT) devices.

[0108] The user terminal may simply provide the user with the inference result received from the server 1000 , or perform subsequent operations based on the inference result.

[0109] Reference Figure 11 , electronic devices (e.g., Figure 1 The electronic device 100) can be implemented as a user terminal 1100. Figure 11 , for ease of description, the user terminal 1100 is shown as a smartphone. However, any device controlled by a user may be applicable hereto without limitation. The user terminal 1100 may obtain data to be inferred using a neural network directly from the user, and process the normalization layer included in the neural network through the accelerator 1120 according to the above-mentioned tensor processing method. In this example, the tensor may not be sent to the host processor 1110 outside the accelerator 1120 to perform the operation according to the normalization layer, and the accelerator 1120 may determine the data inference result based on the steps of performing the operation according to the normalization layer. In addition, the user terminal 1100 may obtain the inference result from the neural network and simply provide the inference result to the user or perform subsequent operations based on the inference result.

[0110] In this regard Figures 1 to 11The electronic devices, host processors, off-chip memories, memory controllers, accelerators, servers, user terminals, electronic devices 100, host processors 110, off-chip memories 120, memory controllers 130, accelerators 140, servers 1000, host processors 1010, accelerators 1020, user terminals 1100, host processors 1110, accelerators 1120, and other devices, devices, units, modules, and components described herein are implemented by hardware components or are representatives of hardware components. Examples of hardware components that can be used to perform the operations described herein include, where appropriate, controllers, sensors, generators, drivers, memories, comparators, arithmetic logic units, adders, subtractors, multipliers, dividers, integrators, and any other electronic components configured to perform the operations described herein. In other examples, one or more of the hardware components that perform the operations described herein are implemented by computing hardware (e.g., by one or more processors or computers). The processor or computer can be implemented by one or more processing elements (such as logic gate arrays, controllers and arithmetic logic units, digital signal processors, microcomputers, programmable logic controllers, field programmable gate arrays, programmable logic arrays, microprocessors, or any other device or combination of devices configured to respond and execute instructions in a limited manner to achieve the desired result). In one example, the processor or computer includes or is connected to one or more memories storing instructions or software executed by the processor or computer. The hardware components implemented by the processor or computer can execute instructions or software (such as an operating system (OS) and one or more software applications running on the OS) for performing the operations described in this application. The hardware components can also access, manipulate, process, create and store data in response to the execution of instructions or software. For simplicity, the singular term "processor" or "computer" can be used in the description of the examples described in this application, but in other examples, multiple processors or computers can be used, or the processor or computer can include multiple processing elements, or multiple types of processing elements, or both. For example, a single hardware component, or two or more hardware components can be implemented by a single processor, or two or more processors, or a processor and a controller. One or more hardware components may be implemented by one or more processors, or a processor and a controller, and one or more other hardware components may be implemented by one or more other processors, or another processor and another controller. One or more processors, or a processor and a controller may implement a single hardware component, or two or more hardware components.The hardware components may have any one or more of different processing configurations, examples of which include: a single processor, independent processors, parallel processors, single instruction single data (SISD) multiprocessing, single instruction multiple data (SIMD) multiprocessing, multiple instruction single data (MISD) multiprocessing, and multiple instruction multiple data (MIMD) multiprocessing.

[0111] Perform the operations described in this application Figures 1 to 11 The method shown in is performed by computing hardware (e.g., by one or more processors or computers), which is implemented as executing instructions or software as described above to perform the operations performed by the method described in this application. For example, a single operation, or two or more operations may be performed by a single processor, or two or more processors, or a processor and a controller. One or more operations may be performed by one or more processors, or a processor and a controller, and one or more other operations may be performed by one or more other processors, or another processor and another controller. One or more processors, or a processor and a controller may perform a single operation, or two or more operations.

[0112] Instructions or software for controlling computing hardware (e.g., one or more processors or computers) to implement hardware components and perform the methods described above may be written as computer programs, code segments, instructions, or any combination thereof to individually or collectively instruct or configure one or more processors or computers to operate as a machine or special-purpose computer to perform the operations performed by the hardware components and methods described above. In one example, the instructions or software include machine code (such as machine code generated by a compiler) that is directly executed by one or more processors or computers. In another example, the instructions or software include high-level code that is executed by one or more processors or computers using an interpreter. The instructions or software may be written in any programming language based on the block diagrams and flow charts shown in the accompanying drawings and the corresponding descriptions used herein, which disclose algorithms for performing the operations performed by the hardware components and methods described above.

[0113] Instructions or software for controlling computing hardware (e.g., one or more processors or computers) to implement hardware components and perform the methods described above, and any associated data, data files, and data structures, may be recorded, stored, or fixed in or on one or more non-transitory computer-readable storage media. Examples of non-transitory computer-readable storage media include read-only memory (ROM), programmable random access read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disc storage, hard disk drive (HDD), solid state drive (SSD), card memory (such as, multimedia card or micro card (for example, Secure Digital (SD) or Extreme Digital (XD))), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk and any other device, any other device is configured to store instructions or software and any associated data, data files and data structures in a non-transitory manner and provide the instructions or software and any associated data, data files and data structures to one or more processors or computers so that one or more processors or computers can execute the instructions. In one example, the instructions or software and any associated data, data files and data structures are distributed on a networked computer system so that the instructions and software and any associated data, data files and data structures are stored, accessed and executed in a distributed manner by one or more processors or computers.

[0114] Although the present disclosure includes specific examples, it will be clear after understanding the disclosure of the present application that various changes in form and detail can be made in these examples without departing from the spirit and scope of the claims and their equivalents. The examples described herein should be considered as descriptive only and not for limiting purposes. The description of the features or aspects in each example should be considered to be applicable to similar features or aspects in other examples. If the described techniques are performed in a different order and / or if the components in the described systems, architectures, devices or circuits are combined in different ways and / or replaced or supplemented by other components or their equivalents, suitable results can be achieved.

Claims

1. An electronic device comprising: a host processor configured to: receive a request for the accelerator to process a neural network, generate instructions executable by the accelerator in response to the request, and send the instructions to the accelerator; and an accelerator configured to: execute the neural network in response to the instruction and send a result of executing the neural network to the host processor, The accelerator is configured to: when the operation to be performed in the operations included in the neural network belongs to an operation that can be executed by the host processor, convert the operation to be performed into an operation that can be executed by the accelerator through the following steps to be executed on the accelerator: by performing a convolution based on the target tensor and the kernel in the channel axis direction, an intermediate tensor corresponding to the result of executing the operation to be performed is determined, the target tensor will be used to execute the operation to be performed, the kernel has the number of input channels and the number of output channels determined based on the target tensor and includes an element of a scaling value determined based on the target tensor.

2. The electronic device according to claim 1, wherein The accelerator is configured to determine the intermediate tensor by extracting diagonal elements from a result tensor determined by convolution based on the target tensor and the kernel.

3. The electronic device according to claim 1, wherein The number of input channels of the kernel is determined based on the number of elements of the normalization unit applied to the target tensor.

4. The electronic device according to claim 3, wherein: The number of elements of the normalization unit applied to the destination tensor is equal to the number of channels of the destination tensor, and the number of input channels of the kernel is equal to the number of channels of the destination tensor.

5. The electronic device according to claim 1, wherein The number of output channels of the kernel is determined based on the width and length of the target tensor. The electronic device according to claim 1 , wherein: The scaling value for each element included in the kernel includes the runtime value corresponding to the target tensor.

7. The electronic device according to claim 1, wherein Each element included in the kernel is scaled to the value of the corresponding element in the destination tensor.

8. The electronic device according to any one of claims 1 to 7, wherein: The target tensor is determined based on the following: a mean subtraction tensor including a value determined by subtracting a value of each element from a mean of the elements included in an input tensor of a layer of the neural network corresponding to the operation to be performed; and The constant value is determined based on the number of elements of the normalization unit applied to the target tensor.

9. The electronic device according to claim 8, wherein: The target tensor is determined by performing a convolution along the channel axis based on: Average subtraction tensor; and A second kernel has a number of input channels and a number of output channels determined based on the mean subtraction tensor and includes diagonal elements with scaling values determined based on the constant value.

10. The electronic device according to claim 9, wherein: The number of input channels and the number of output channels of the second kernel are equal to the number of elements of the normalization unit, and The diagonal elements in the second kernel have scaling values that are different from the scaling values of the remaining elements.

11. The electronic device according to claim 9, wherein: The constant value is equal to the square root of the number of elements of the normalization unit applied to the target tensor, and the scaling value of the diagonal elements in the second kernel is equal to the inverse of the square root.

12. The electronic device according to any one of claims 1 to 7, wherein: When the operation to be performed belongs to an operation of the normalization layer, the accelerator detects the operation to be performed as an operation that can be performed by the host processor, and The normalization layer is configured to perform normalization using any one or both of a mean and a variance determined based on values of one or more elements included in the target tensor.

13. The electronic device according to claim 1, wherein The accelerator is configured as: Perform a convolution between the kernel and the target tensor transformed so that elements included in the same channel are arranged in a row, and An intermediate tensor is determined by transforming elements determined as a result of the convolution into the same form as that of the target tensor.

14. The electronic device according to any one of claims 1 to 7, wherein: The convolution is performed in the accelerator so that the target tensor is not sent outside the accelerator to perform the operation.

15. The electronic device according to any one of claims 1 to 7, wherein: The accelerator is included in any of the following: A user terminal is input with data to be inferred using a neural network; and The server receives the data to be inferred from the user terminal.

16. A method for processing a neural network, comprising: receiving, by the host processor, a request for processing the neural network by the accelerator; generating, by the host processor in response to the request, an instruction executable by the accelerator, and sending the instruction to the accelerator; and executing the neural network in response to the instruction through the accelerator, and sending a result of executing the neural network to the host processor; Among them, the step of executing the neural network includes: when the operation to be executed among the operations included in the neural network belongs to an operation that can be executed by the host processor, the operation to be executed is converted into an operation that can be executed by the accelerator through the following steps to be executed on the accelerator: by performing a convolution based on the target tensor and the kernel in the channel axis direction, an intermediate tensor corresponding to the result of executing the operation to be executed is determined, the target tensor will be used to execute the operation to be executed, the kernel has the number of input channels and the number of output channels determined based on the target tensor and includes an element of a scaling value determined based on the target tensor.

17. An electronic device for recognizing an image, comprising: a host processor configured to: receive a request for identifying an image to be identified, generate an instruction executable by the accelerator in response to the request, and send the instruction to the accelerator; and an accelerator configured to: execute a neural network with the image to be recognized as input in response to the instruction, and send a result of executing the neural network as a recognition result of the image to be recognized to the host processor, The accelerator is configured to: when the operation to be performed in the operations included in the neural network belongs to an operation that can be executed by the host processor, convert the operation to be performed into an operation that can be executed by the accelerator through the following steps to be executed on the accelerator: by performing a convolution based on the target tensor and the kernel in the channel axis direction, an intermediate tensor corresponding to the result of executing the operation to be performed is determined, the target tensor will be used to execute the operation to be performed, the kernel has the number of input channels and the number of output channels determined based on the target tensor and includes an element of a scaling value determined based on the target tensor.

Images