Computing system for processing neural network and method of operating computing system

Abstract

The invention provides a computing system for processing a neural network and a method of operating the computing system. The computing system may include an operating component including at least one convolution block; and a controller configured to control the operation component to perform a convolution operation, in which the at least one convolution block includes: a first convolution layer configured to perform a first convolution operation on the input data based on a 1 * 1 kernel to generate first result data; a second convolution layer configured to perform a second convolution operation on each channel of the first result data based on an n * n core, where n is a natural number of 2 or more, and summing result values of the convolution operation to generate second result data; and a third convolution layer configured to perform a third convolution operation on the second result data based on the 1 * 1 core to generate final result data.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to Korean Patent Application No. 10-2023-0097281, filed on Jul. 26, 2023, the disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0003] Various embodiments of the present disclosure generally relate to a semiconductor device, and more particularly, to a computing system for processing a neural network and a method of operating the computing system. Background Art

[0004] A neural network can utilize artificial neurons obtained by simplifying the functions of biological neurons, and these artificial neurons can be interconnected through connection lines with connection weights. The connection weights (i.e., parameters of the neural network) can be specific values ​​of the connection lines, and can also be referred to as "connection strengths". A neural network can perform human cognitive functions or learning (or training) processes through artificial neurons. Training a neural network can be understood as training the parameters of a neural network. In addition, a trained neural network can be understood as a neural network to which trained parameters are applied. Each artificial neuron can also be referred to as a "node".

[0005] Convolution operations may occupy a large part of the operations required in a neural network model. Convolution operations may be performed by a multiplication-accumulation (MAC) operation device implemented as a storage area in which a plurality of units are formed into an array structure. When a depth-wise convolution operation, which is one of the convolution operations, is applied to the multiplication-accumulation (MAC) operation device, unit utilization may be reduced. Therefore, a neural network architecture capable of improving unit utilization is needed. Summary of the invention

[0006] Various embodiments of the present disclosure relate to a computing system capable of improving neural network processing performance and a method of operating the computing system.

[0007] Embodiments of the present disclosure may provide a computing system. The computing system may include: an operation component including at least one convolution block configured to perform a convolution operation on input data based on weight data to generate final result data; and a controller configured to control the operation component to perform a convolution operation, wherein at least one convolution block includes: a first convolution layer configured to perform a first convolution operation on input data based on a 1×1 size core to generate first result data; a second convolution layer configured to perform a second convolution operation on each channel of the first result data based on an n×n size core, and summing the result values ​​of the convolution operation on each channel of the first result data to generate second result data, wherein n is a natural number of 2 or greater; and a third convolution layer configured to perform a third convolution operation on the second result data based on a 1×1 size core to generate final result data.

[0008] Embodiments of the present disclosure may provide a method for operating a computing system for processing a convolution block including multiple convolution layers. The method may include: performing a first convolution operation on input data based on a 1×1 size core by a first convolution layer among the multiple convolution layers to generate first result data with a first size of channels; performing a second convolution operation on each channel of the first result data based on an n×n size core by a second convolution layer among the multiple convolution layers, where n is a natural number of 2 or greater; summing the result values ​​of the second convolution operation on each channel in the second convolution layer by the second convolution layer to generate second result data with a second size of channels; and performing a third convolution operation on the second result data based on a 1×1 size core by a third convolution layer among the multiple convolution layers to generate final result data.

[0009] These and other features and advantages of the present disclosure will become apparent through a detailed description of embodiments of the present disclosure and the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Embodiments will now be described more fully below with reference to the accompanying drawings; however, the embodiments may be implemented in different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete and will convey the scope of the embodiments to those skilled in the art.

[0011] In the accompanying drawings, dimensions may be exaggerated for clarity of illustration. It should be understood that when an element is referred to as being "between" two elements, it may be the only element between the two elements, or one or more intermediate elements may also be present. Throughout the specification, the same reference numerals refer to the same elements.

[0012] Figure 1 is a diagram illustrating a computing system according to an embodiment of the present disclosure.

[0013] Figure 2 is a diagram illustrating operating components according to an embodiment of the present disclosure.

[0014] Figure 3 is a diagram illustrating a convolution block according to an embodiment of the present disclosure.

[0015] Figure 4 is a diagram illustrating a convolution operation in a second convolutional layer according to an embodiment of the present disclosure.

[0016] Figure 5 is a diagram illustrating a bottleneck structure according to an embodiment of the present disclosure.

[0017] Figure 6 is a diagram illustrating an operation of mapping weight data according to an embodiment of the present disclosure.

[0018] Fig. 7A is a flowchart illustrating a method of determining the number of channels of first result data according to an embodiment of the present disclosure.

[0019] Figure 7B is a diagram illustrating an operation of mapping weight data based on the determined number of channels of first result data according to an embodiment of the present disclosure.

[0020] Fig. 8A is a flowchart illustrating a method of determining the number of channels of second result data according to an embodiment of the present disclosure.

[0021] Figure 8B is a diagram illustrating an operation of mapping weight data based on the determined number of channels of second result data according to an embodiment of the present disclosure.

[0022] Fig. 9 is a diagram illustrating a quantization operation according to an embodiment of the present disclosure.

[0023] Fig.10 is a flow chart illustrating a method of operating a computing system according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0024] The specific structural or functional descriptions in the embodiments according to the concept of the present disclosure introduced in this specification are only used for describing the embodiments according to the concept of the present disclosure. The embodiments according to the concept of the present disclosure can be practiced in various forms and should not be interpreted as being limited to the embodiments described in the specification.

[0025] Figure 1 is a diagram illustrating a computing system 100 according to an embodiment of the present disclosure.

[0026] Reference Figure 1, the computing system 100 may include an interface 110, a memory 120, an operation component 130, and a controller 140. In an embodiment, the interface 110, the memory 120, the operation component 130, and the controller 140 may communicate with each other through a communication bus.

[0027] In an embodiment, the computing system 100 may be a neural network dedicated hardware accelerator itself or a device including a hardware accelerator. For example, the computing system 100 may be a device capable of performing a weight stationary-based two-dimensional (2D) matrix multiplication operation.

[0028] In an embodiment, the neural network may be implemented as a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), etc.

[0029] The interface 110 may perform communication with an external device. For example, the interface 110 may receive input data, weight data, etc. required for processing a neural network from an external device, or may transmit final result data provided from the operation component 130 to an external device.

[0030] The memory 120 may be a volatile memory or a non-volatile memory and may store instructions or program codes required to process a neural network.

[0031] The operation component 130 may include a storage area CA for performing operations.

[0032] In an embodiment, the operation component 130 may perform a convolution operation, such as an element-wise multiply-accumulate (MAC) operation. The operation component 130 may provide a result of the MAC operation to the memory 120 or the controller 140.

[0033] The controller 140 may control the operation of the computing system 100. The controller 140 may control the operation of the interface 110, the memory 120, and the operation component 130. For example, the controller 140 may set and manage parameters related to a neural network operation (e.g., a convolution operation) so that the operation component 130 may operate the layers of the neural network normally.

[0034] The controller 140 may be implemented as hardware, software (or firmware), or a combination of hardware and software. In an embodiment, the controller 140 may be implemented as hardware logic designed to perform the above functions. In an embodiment, the controller 140 may be implemented as at least one of various processors such as a central processing unit (CPU), a microprocessor, etc., and may run a program including instructions or program codes constituting the above functions.

[0035] In an embodiment, the neural network may include at least one convolution block, for example, in the operation component 130. The convolution block may include a plurality of convolution layers forming a bottleneck structure. For example, one of the plurality of convolution layers may extract a feature map by summing the result values ​​of performing a convolution operation on each channel of the input data based on a kernel of size n×n (i.e., a channel-by-channel convolution operation), where n is a natural number of 2 or greater. Each of the remaining convolution layers may extract a feature map by performing a convolution operation based on a kernel of size 1×1. Here, the remaining convolution layers may be located before or after the one convolution layer in the order of the convolution operation.

[0036] Figure 2 The present invention is a diagram showing an embodiment of the present invention. Figure 1 A diagram of operating assembly 130 is shown in detail.

[0037] Reference Figure 2 , the operating component 130 may include an input circuit 131, an output circuit 132, and a storage area CA.

[0038] The input circuit 131 may apply input data to the storage area CA. In an embodiment, the input circuit 131 may be implemented using a buffer, a driving circuit, a decoder, etc. configured to apply the input data.

[0039] The memory area CA may include a plurality of cells forming an array structure.

[0040] In an embodiment, each unit may be referred to as a “processing element (PE).” Each of the plurality of units may perform a multiply-accumulate (MAC) operation on input data and weight data.

[0041] In an embodiment, the storage area CA may store weight data. The weight data may be converted into a weight array to be stored in the storage area CA.

[0042] The output circuit 132 may output the result of the operation on the storage area CA as final result data. In an embodiment, the output circuit 132 may be implemented using a buffer, a driving circuit, a decoder, etc. configured to output the final result data.

[0043] and Figure 2 Unlike the illustrated embodiment, according to an embodiment, the operation component 130 may further include a weight buffer that inputs weight data to the storage area CA. In addition, in an embodiment, the input circuit 131 and the output circuit 132 may be implemented as a single component that performs input and output functions.

[0044] Figure 3 is a diagram illustrating a convolution block CBLK according to an embodiment of the present disclosure.

[0045] Reference Figure 3 , the convolution block CBLK may include a first convolution layer CONV1, a second convolution layer CONV2, and a third convolution layer CONV3.

[0046] In an embodiment, the convolution block CBLK may receive input data INPUT and may perform a convolution operation on the input data INPUT using a first convolution layer CONV1, a second convolution layer CONV2, and a third convolution layer CONV3. The convolution block CBLK may output final result data OUTPUT as a result of the convolution operation.

[0047] The convolution operation may be performed in the order of a first convolution layer CONV1, a second convolution layer CONV2, and a third convolution layer CONV3.

[0048] In an embodiment, data input to or output from each of the first convolutional layer CONV1, the second convolutional layer CONV2, and the third convolutional layer CONV3 may be referred to as a “feature map”.

[0049] The first convolution layer CONV1 may receive input data INPUT. In an embodiment, the first convolution layer CONV1 may output first result data RESULT1 by performing a convolution operation on the input data INPUT based on a kernel of size 1×1 (1×1 kernel). In an embodiment, the first convolution layer CONV1 may perform a point-by-point convolution operation.

[0050] The second convolution layer CONV2 may receive the first result data RESULT1 from the first convolution layer CONV1. In an embodiment, the second convolution layer CONV2 may output the second result data RESULT2 by performing a standard convolution operation on the first result data RESULT1. The standard convolution operation may be an operation of performing a convolution operation on each channel of the input data based on a kernel of size n×n and summing the result values ​​of the convolution operation on each channel, where n is a natural number of 2 or greater.

[0051] The third convolution layer CONV3 may receive the second result data RESULT2 from the second convolution layer CONV2. In an embodiment, the third convolution layer CONV3 may output final result data OUTPUT by performing a convolution operation on the second result data RESULT2 based on a kernel of size 1×1 (1×1 kernel). In an embodiment, the third convolution layer CONV3 may perform a point-by-point convolution operation.

[0052] In an embodiment, the convolution block CBLK may further include a batch normalization layer BN and an activation layer ACT.

[0053] For example, a batch normalization layer BN and an activation layer ACT may be located after each of the first convolution layer CONV1, the second convolution layer CONV2, and the third convolution layer CONV3 in their operation order.

[0054] In an embodiment, batch normalization may refer to an operation of performing normalization using a mean and covariance of each batch unit even if data segments have various distributions for each batch unit. A batch normalization layer BN may refer to a layer that performs batch normalization.

[0055] The activation layer ACT may be a layer that converts the weighted sum of input data into result data using an activation function. For example, the activation layer ACT may use a rectified linear unit (ReLU) function to output 0 when the input data is less than 0, and output the input data unchanged when the input data is greater than 0.

[0056] Figure 4 is a diagram illustrating a convolution operation in a second convolutional layer according to an embodiment of the present disclosure.

[0057] Reference Figure 4 , the second convolutional layer CONV2 can receive the first result data RESULT1 having three channels.

[0058] In an embodiment, the second convolution layer CONV2 may perform a convolution operation on each of a first channel RESULT1_C1 of the first result data RESULT1 , a second channel RESULT1_C2 of the first result data RESULT1 , and a third channel RESULT1_C3 of the first result data RESULT1 based on a kernel K.

[0059] The second convolution layer CONV2 may output second result data RESULT2 by summing result values ​​R1 , R2 , and R3 of the convolution operation of each channel of the first result data RESULT1 .

[0060] The convolution operation in the second convolution layer CONV2 may be a standard convolution operation, and may refer to an operation of performing a convolution operation on each channel of the first result data RESULT1 and summing the results R1 , R2 , and R3 of the convolution operation.

[0061] Therefore, the second convolutional layer CONV2 performs a standard convolution operation, thereby increasing the unit utilization of the storage area CA for storing weight data.

[0062] Figure 5 is a diagram illustrating a bottleneck structure according to an embodiment of the present disclosure.

[0063] Reference Figure 5, the convolution block can be formed into a bottleneck structure, in which the first convolution layer CONV1, the second convolution layer CONV2 and the third convolution layer CONV3 are sequentially located.

[0064] In an embodiment, the first convolutional layer CONV1 and the third convolutional layer CONV3 may perform point-by-point convolution operations.

[0065] In an embodiment, the first convolution layer CONV1 may perform a convolution operation so that the number of channels of the input data is reduced. Figure 3 , the first convolution layer CONV1 may perform a convolution operation on the input data INPUT so that the number of channels of the first result data RESULT1 becomes smaller than the number of channels of the input data INPUT. For example, when the data input to the first convolution layer CONV1 has C1 channels, the first convolution layer CONV1 may reduce the number of channels of the data to C2 which is smaller than C1. That is, the first convolution layer CONV1 may perform a convolution operation so that the number of channels of the data input to the second convolution layer CONV2 is reduced.

[0066] In an embodiment, the third convolution layer CONV3 may perform a convolution operation so that the number of channels of the input data increases. Figure 3 , the third convolution layer CONV3 may perform a convolution operation on the second result data RESULT2 so that the number of channels of the final result data OUTPUT becomes greater than the number of channels of the second result data RESULT2. For example, when the data input to the third convolution layer CONV3 has C3 channels, the third convolution layer CONV3 may increase the number of channels of the data to C4 which is greater than C3. That is, the third convolution layer CONV3 may perform a convolution operation so that the number of channels of the data output from the second convolution layer CONV2 increases.

[0067] Therefore, the computing system 100 can adjust the number of channels of data input to the second convolution layer CONV2 or the number of channels of data output from the second convolution layer CONV2 by using the first convolution layer CONV1 and the third convolution layer CONV3.

[0068] Figure 6 is a diagram illustrating the operation of mapping weight data according to an embodiment of the present disclosure. In an embodiment, Figure 6 The storage area CA shown may be configured to process convolution operations in the second convolutional layer CONV2.

[0069] Reference Figure 6 , the weight data DATA_W may be mapped to the weight array WA. The mapped weight array WA may be stored in the storage area CA. The elements forming the weight array WA may correspond to the cells in the storage area CA, respectively.

[0070] The number of rows Ah of the storage area CA may represent the height of the storage area CA, and the number of columns Aw of the storage area CA may represent the width of the storage area CA.

[0071] In an embodiment, the weight data DATA_W may be represented by a plurality of kernels K1 to Km having a plurality of channels. The plurality of kernels K1 to Km may be applied to the convolution operation in the second convolution layer CONV2. The number of channels of each of the kernels K1 to Km may be the same as the number of channels IC of the input data. In addition, the number of kernel types may be the same as the number of channels OC of the output data.

[0072] In an embodiment, the number of rows WA_h of the weight array WA may be calculated by multiplying the number of channels IC of the input data by the n×n size of each core. The number of rows WA_h of the weight array WA may represent the height of the weight array WA.

[0073] For example, the number of rows WA_h of the weight array WA may be calculated using the following equation (1):

[0074] WA_h=Kh×Kw×IC (1)

[0075] In equation (1), WA_h may represent the number of rows of the weight array WA, Kh may represent the height of each kernel, Kw may represent the width of each kernel, and IC may represent the number of channels of input data.

[0076] In an embodiment, the number of columns of the weight array WA may correspond to the number of channels OC of the output data. The number of columns WA_w of the weight array WA may represent the width of the weight array WA.

[0077] Fig. 7A is a flowchart illustrating a method of determining the number of channels of first result data according to an embodiment of the present disclosure. Figure 7B is a diagram illustrating an operation of mapping weight data based on the determined number of channels of first result data according to an embodiment of the present disclosure.

[0078] Fig. 7A The method shown can be used, for example, Figure 1 The controller 140 shown in FIG. 1 is executed. In an embodiment, the controller 140 may determine the number of channels of the first result data so that the number of rows WA_h' of the weight array WA is the same as the number of rows Ah of the storage area CA.

[0079] In operation S701, the controller 140 may calculate a minimum channel number IC_MIN of the first result data. The minimum channel number IC_MIN of the first result data may be calculated as a value obtained by dividing the number of rows Ah of the storage area CA by the size of each kernel applied to the second convolutional layer CONV2.

[0080] For example, the minimum channel number IC_MIN of the first result data may be calculated using the following equation (2):

[0081] IC_MIN=ceil(Ah÷(Kh×Kw)) (2)

[0082] In equation (2), IC_MIN may be the minimum number of channels of the first result data, Ah may be the number of rows of the storage area CA, and ceil may be a function of rounding off the decimal point.

[0083] In an embodiment, the controller 140 may determine the smaller value of the minimum number of channels IC_MIN of the first result data and the number of columns Aw of the storage area CA as the final number of channels IC' of the first result data. The number of columns Aw of the storage area CA is used as the comparison target in order to consider the number of channels of the data output by the first convolution layer CONV1 previously operated.

[0084] In operation S703 , the controller 140 may determine whether the minimum channel number IC_MIN of the first result data is less than the column number Aw of the storage area CA.

[0085] When the minimum channel number IC_MIN of the first result data is less than the column number Aw of the storage area CA (ie, in case of “yes” in operation S703 ), the controller 140 may determine the minimum channel number IC_MIN of the first result data as the final channel number IC′ of the first result data (in operation S705 ).

[0086] When the minimum channel number IC_MIN of the first result data is equal to or greater than the column number Aw of the storage area CA (ie, in case of "No" in operation S703), the controller 140 may determine the column number Aw of the storage area CA as the final channel number IC' of the first result data (in operation S707).

[0087] In an embodiment, the controller 140 may use the first convolution layer CONV1 capable of adjusting the number of channels to set the number of channels of the first result data so that the number of channels of the first result data becomes the final number of channels IC′.

[0088] In an embodiment, the number of rows WA_h' of the weight array WA may be calculated by multiplying the final number of channels IC' of the first result data by the size of the kernel. Therefore, the number of rows WA_h' of the weight array WA may be the same as the number of rows Ah of the storage area CA.

[0089] Fig. 8A is a flowchart illustrating a method of determining the number of channels of second result data according to an embodiment of the present disclosure. Figure 8B is a diagram illustrating an operation of mapping weight data based on the determined number of channels of second result data according to an embodiment of the present disclosure.

[0090] Fig. 8A The method shown can be used, for example, Figure 1 The controller 140 shown in FIG. 1 is executed. In an embodiment, the controller 140 may determine the number of channels of the second result data so that the number of columns WA_w' of the weight array WA is the same as the number of columns Aw of the storage area CA.

[0091] In an embodiment, the controller 140 may determine the smaller value of the number of rows Ah of the storage area CA and the number of columns Aw of the storage area CA as the number of channels of the second result data. The reason why the number of rows Ah of the storage area CA is used as the comparison target is to consider the number of channels of the data input to the third convolution layer CONV3 of the subsequent operation.

[0092] In operation S801, the controller 140 may determine whether the number of rows Ah of the memory area CA is less than the number of columns Aw of the memory area CA.

[0093] When the row number Ah of the storage area CA is less than the column number Aw of the storage area CA (ie, in case of “Yes” in operation S801 ), the controller 140 may determine the row number Ah of the storage area CA as the final channel number OC′ of the second result data (in operation S803 ).

[0094] When the row number Ah of the storage area CA is equal to or greater than the column number Aw of the storage area CA (ie, in case of "No" in operation S801), the controller 140 may determine the column number Aw of the storage area CA as the final channel number OC' of the second result data (in operation S805).

[0095] In an embodiment, the controller 140 may set the number of channels of the second result data by adjusting the kernel type of the second convolution layer CONV2 so that the number of channels of the second result data becomes the final number of channels OC′.

[0096] In an embodiment, the number of columns WA_w' of the weight array WA may correspond to the final number of channels OC' of the second result data. Therefore, the number of columns WA_w' of the weight array WA may be the same as the number of columns Aw of the storage area CA.

[0097] Fig. 9 is a diagram illustrating a quantization operation according to an embodiment of the present disclosure.

[0098] Reference Fig. 9 , the controller 140 may perform a quantization operation on each of the first convolution layer CONV1, the second convolution layer CONV2, and the third convolution layer CONV3 included in the convolution block CBLK. Here, the quantization operation may be a process of converting a real number type variable into an integer type variable, and may reduce the size of the neural network model and increase the processing speed.

[0099] In an embodiment, the controller 140 may perform a channel-by-channel quantization operation on the second convolutional layer CONV2. The channel-by-channel quantization operation may be an operation of performing quantization for each channel. For example, the convolution block CBLK may include a channel-by-channel quantization layer C_QUANT and a channel-by-channel dequantization layer C_DEQUANT before and after the second convolutional layer CONV2. The channel-by-channel quantization layer C_QUANT may perform channel-by-channel quantization on the input data of the second convolutional layer CONV2. In addition, the channel-by-channel dequantization layer C_DEQUANT may perform channel-by-channel dequantization on the result data of the second convolutional layer CONV2. Dequantization may be an operation of converting quantized data back to the original variable form.

[0100] In an embodiment, the controller 140 may perform a layer-by-layer quantization operation on each of the first convolutional layer CONV1 and the third convolutional layer CONV3. The layer-by-layer quantization operation may be an operation of performing quantization on each layer. For example, the convolution block CBLK may include a layer-by-layer quantization layer L_QUANT and a layer-by-layer dequantization layer L_DEQUANT before and after each of the first convolutional layer CONV1 and the third convolutional layer CONV3. The layer-by-layer quantization layer L_QUANT may perform layer-by-layer quantization on the input data segments of the first convolutional layer CONV1 and the third convolutional layer CONV3, respectively. In addition, the layer-by-layer dequantization layer L_DEQUANT may perform layer-by-layer dequantization on the result data segments of the first convolutional layer CONV1 and the third convolutional layer CONV3, respectively.

[0101] Fig.10 is a flow chart illustrating a method of operating a computing system according to an embodiment of the present disclosure.

[0102] Fig.10 The method shown can be used, for example, Figure 1 The computing system 100 shown is used to execute.

[0103] Reference Fig.10 In operation S1001, the computing system 100 may generate first result data having channels of a first size by performing a convolution operation on input data based on a kernel of a size of 1×1 through a first convolution layer.

[0104] In an embodiment, the computing system 100 may perform a convolution operation such that the first size becomes smaller than a size of a channel of the input data.

[0105] In an embodiment, the first size may be determined as a smaller value of a value obtained by dividing the number of rows of the storage area where the convolution operation is performed by the n×n size and the number of columns of the storage area.

[0106] In an embodiment, channel-by-channel quantization may be performed on the first result data.

[0107] In operation S1003 , the computing system 100 may perform a convolution operation on each channel of the first result data based on a kernel of a size of n×n through a second convolution layer.

[0108] In operation S1005, the computing system 100 may generate second result data of channels having a second size by summing result values ​​of the convolution operation of the respective channels through the second convolution layer.

[0109] In an embodiment, the second size may be determined as a smaller value of the number of rows of the storage area where the convolution operation is performed and the number of columns of the storage area.

[0110] In operation S1007 , the computing system 100 may generate final result data by performing a convolution operation on the second result data based on a kernel of a size of 1×1 through a third convolution layer.

[0111] In an embodiment, the computing system 100 may perform a convolution operation such that a size of a channel of result data becomes larger than a second size.

[0112] According to the present disclosure, a computing system with improved neural network processing performance and a method of operating the computing system are provided.

[0113] Although the present disclosure has been shown and described with reference to certain embodiments of the present disclosure, it will be understood by those skilled in the art that various changes may be made in form and detail without departing from the scope of the present disclosure and any equivalents. Therefore, the scope of the present disclosure should not be limited to the above-described embodiments, but should include equivalents thereof.

[0114] In the above-described embodiments, all operations may be selectively performed or some operations may be omitted. In each embodiment, the operations are not necessarily performed in the order described, but may be rearranged. The embodiments disclosed in this specification and the accompanying drawings are only examples for facilitating understanding of the present disclosure, and the present disclosure is not limited thereto. That is, it is obvious to those skilled in the art that various modifications may be made based on the technical scope of the present disclosure.

[0115] Embodiments of the present disclosure have been described in the drawings and the specification. Although specific terms are used herein, this is only for the purpose of describing embodiments of the present disclosure. Therefore, the present disclosure is not limited to the above-described embodiments, but many variations may exist within the scope of the present disclosure. It is obvious to those skilled in the art that, in addition to the embodiments disclosed herein, various modifications may also be made based on the technical scope of the present disclosure. In addition, the embodiments may be combined to form additional embodiments.

Claims

1. A computing system comprising: An operation component, comprising at least one convolution block, wherein the operation component performs a convolution operation on input data based on weight data to generate final result data; as well as a controller, controlling the operation component to perform the convolution operation, Wherein, the at least one convolution block comprises: A first convolution layer, performing a first convolution operation on the input data based on a kernel of size 1×1 to generate first result data; a second convolution layer that performs a second convolution operation on each channel of the first result data based on a kernel of size n×n, and sums result values ​​of the convolution operation on each channel of the first result data to generate second result data, wherein n is a natural number of 2 or greater; and The third convolution layer performs a third convolution operation on the second result data based on a kernel of size 1×1 to generate the final result data.

2. The computing system of claim 1, wherein: The at least one convolutional block further comprises: Batch Normalization layers; and Activate the layer.

3. The computing system of claim 1, wherein: The at least one convolution block is formed as a bottleneck structure, and the first convolution layer, the second convolution layer and the third convolution layer are sequentially located in the bottleneck structure.

4. The computing system of claim 3, wherein: According to the first convolution operation, the number of channels of the first result data becomes smaller than the number of channels of the input data.

5. The computing system of claim 3, wherein: According to the third convolution operation, the number of channels of the final result data becomes greater than the number of channels of the second result data.

6. The computing system of claim 1, wherein: Each of the first convolution operation and the third convolution operation comprises a point-by-point convolution operation.

7. The computing system of claim 1, wherein: The operation component further includes a storage area in which a plurality of cells storing a weight array corresponding to the weight data are formed in an array structure.

8. The computing system of claim 7, wherein: The controller determines the number of channels of the first result data so that the number of rows of the weight array is equal to the number of rows of the storage area.

9. The computing system of claim 8, wherein: The controller determines a smaller value of a value obtained by dividing the number of rows of the storage area by the n×n size and the number of columns of the storage area as the number of channels of the first result data.

10. The computing system of claim 8, wherein: The number of rows of the weight array is calculated by multiplying the number of channels of the first result data by the n×n size.

11. The computing system of claim 7, wherein: The controller determines the number of channels of the second result data so that the number of columns of the weight array is equal to the number of columns of the storage area.

12. The computing system of claim 11, wherein: The controller determines a smaller value of the number of rows of the storage area and the number of columns of the storage area as the number of channels of the second result data.

13. The computing system of claim 1, wherein: The controller performs a channel-by-channel quantization operation on the second convolutional layer.

14. The computing system of claim 1, wherein: The controller performs a layer-by-layer quantization operation on each of the first convolutional layer and the third convolutional layer.

15. A method of operating a computing system that processes a convolutional block comprising a plurality of convolutional layers, the method comprising: Performing a first convolution operation on input data based on a kernel of size 1×1 by a first convolution layer among the plurality of convolution layers to generate first result data having channels of a first size; A second convolution layer among the plurality of convolution layers performs a second convolution operation on each channel of the first result data based on a kernel of size n×n, where n is a natural number of 2 or greater; The second convolution layer sums result values ​​of the second convolution operation on each channel in the second convolution layer to generate second result data of channels having a second size; as well as A third convolutional layer among the plurality of convolutional layers performs a third convolution operation on the second result data based on a kernel of size 1×1 to generate final result data.

16. The method according to claim 15, wherein: As a result of performing the first convolution operation, the first size becomes smaller than a size of a channel of the input data, and Due to performing the third convolution operation, the size of the channel of the final result data becomes larger than the second size.

17. The method according to claim 15, wherein: The first size is determined as a smaller value of a value obtained by dividing the number of rows of the storage area on which the first convolution operation is performed by the n×n size and the number of columns of the storage area.

18. The method according to claim 15, wherein: The second size is determined as a smaller value between the number of rows of the storage area on which the second convolution operation is performed and the number of columns of the storage area.

19. The method according to claim 15, further comprising: Before performing the second convolution operation on each channel of the first result data, a channel-by-channel quantization operation is performed on the first result data.

Images