Electronic device and control method thereof

Abstract

An electronic device for executing an artificial intelligence algorithm is provided. The electronic device includes a memory and a processor, the memory stores input data and a plurality of second kernel data obtained from first kernel data, and the processor obtains amplified data, in which at least a portion of the input data is amplified according to the first kernel data. The data is amplified by performing a convolution operation on each of the plurality of second kernel data and the input data. Each of the plurality of second kernel data includes a different first kernel element from a plurality of first kernel elements in the first kernel data.

Description

Technical Field

[0001] The present disclosure relates to an electronic device and a control method thereof, and more particularly, to an electronic device for amplifying input data based on kernel data and a control method thereof. Background Art

[0002] Unlike traditional rule-based intelligent systems, artificial intelligence (AI) systems are systems that achieve human-level intelligence, in which machines train themselves, make judgments, and become intelligent. The more an AI system is used, the higher its recognition rate and the better its understanding of user preferences. Therefore, traditional rule-based intelligent systems have been replaced by deep learning-based AI systems.

[0003] Artificial intelligence technologies consist of machine learning (e.g., deep learning) and elemental technologies that use machine learning.

[0004] Machine learning is an algorithmic technology that classifies / trains itself based on the features of input data. Elemental technology is a technology that uses machine learning algorithms such as deep learning and includes language understanding, visual understanding, reasoning / prediction, knowledge representation, motion control, etc.

[0005] Artificial intelligence technology can be applied to various fields, examples of which are described below. Language understanding is a technology for recognizing and applying / processing human language / characters, including natural language processing, machine translation, dialogue systems, query response, speech recognition / synthesis, etc. Visual understanding is a technology for recognizing and processing objects like human perception, including object recognition, object tracking, image search, human recognition, scene understanding, spatial understanding, image enhancement, etc. Reasoning prediction is a technology for judging, logically inferring and predicting information, including knowledge / probability-based reasoning, optimization prediction, preference-based planning and recommendation. Knowledge representation is a technology for automating human experience information into knowledge data, including knowledge construction (data generation / classification) and knowledge management (data utilization). Motion control is a technology for controlling the autonomous driving of vehicles and the movement of robots, including motion control (navigation, collision and driving), operation control (behavior control), etc.

[0006] In particular, a neural network that increases the resolution of an input image (which is called super-resolution) consists of a convolutional layer for extracting features of the image, a nonlinear layer (ReLu, pooling, element-wise summation, batch normalization, etc.) for calculating the extracted features, and an upscaling layer for doubling the resolution of the input image.

[0007] The upscaling layers may be formed in various ways using, for example, bilinear, sub-pixel shuffling, transposed convolution, Gaussian filters, etc. However, the additional upscaling layers that may be formed using the above methods may increase the hardware area.

[0008] Therefore, there is a need for a method for performing upscaling operations while keeping the hardware area small.

[0009] Recently, high-resolution image services and real-time streaming services are in high demand. However, for transmitting high-resolution images and streaming images, compression loss is inevitable, which leads to image quality degradation.

[0010] The above information is presented as background information only to assist in understanding the present disclosure. No determination has been made, and no assertion is made, as to whether any of the above may be applicable as prior art with respect to the present disclosure. Summary of the invention

[0011] Technical issues

[0012] An embodiment of the present disclosure is directed to providing an electronic device having a reduced hardware area by eliminating hardware for performing a specific operation and a control method thereof.

[0013] Technical Solution

[0014] According to one aspect of the present disclosure, an electronic device is provided, the electronic device having: a memory configured to store input data and a plurality of second kernel data obtained from first kernel data; and a processor configured to obtain data in which a portion of the input data is amplified according to the first kernel data by performing a convolution operation on each of the plurality of second kernel data and the input data. Each of the plurality of second kernel data includes a different first kernel element from a plurality of first kernel elements in the first kernel data.

[0015] Each of the plurality of second kernel data can be obtained from the expanded first kernel data based on the plurality of first kernel elements separated by an enlargement factor (r). The expanded first kernel data is obtained by expanding the first kernel data based on the size and factor (r) of the first kernel data.

[0016] The electronic device may further include a communicator including a circuit. The processor is further configured to receive the plurality of second kernel data from the server through the communicator, and store the received plurality of second kernel data in the memory.

[0017] The electronic device may further include a communicator including a circuit. The processor is further configured to receive the first kernel data from the server through the communicator, obtain the plurality of second kernel data from the first kernel data, and store the plurality of second kernel data in the memory.

[0018] The processor may also be configured to perform a convolution operation on each of the plurality of second kernel data and the element to be amplified in the portion of the input data and the plurality of peripheral elements surrounding the element to be amplified to obtain a plurality of amplified elements about the element to be amplified. The sum of the element to be amplified and the plurality of peripheral elements is the same as the sum of the plurality of second elements respectively included in the plurality of second kernel data.

[0019] The processor may be further configured to determine positions of the plurality of enlarged elements relative to the enlarged data based on positions of the elements to be enlarged relative to the input data.

[0020] The processor may include a convolution array including a circuit configured to perform a convolution operation on each of the plurality of second kernel data and the input data, and a line memory including a circuit configured to store the amplified data.

[0021] The convolution array may include a plurality of processing elements each including a plurality of register files. Each of the plurality of processing elements performs a multiplication operation by using a second kernel element of the plurality of processing elements input to the plurality of second kernel data, and accumulates and stores a result of the multiplication operation in a register file of the plurality of register files corresponding to the second kernel element.

[0022] The processor may further include a shuffler including a circuit and disposed between the convolution array and the line memory. The shuffler shuffles a plurality of operation results output from the plurality of processing elements and provides the shuffled plurality of operation results to the line memory.

[0023] The shuffler may further include a plurality of buffer groups and a FIFO memory, wherein the plurality of buffer groups include circuits, and the FIFO memory includes circuits and is configured to receive a plurality of processing elements output from corresponding register files included in the plurality of processing elements, and output a plurality of operation results to the plurality of buffer groups. The plurality of buffer groups store each of the plurality of operation results in a buffer corresponding to each of the plurality of buffer groups, and based on the plurality of operation results being stored in all buffers included in the plurality of buffer groups, the plurality of operation results stored in one of the plurality of buffer groups are provided to the line memory in a pre-set order.

[0024] According to one aspect of the present disclosure, an electronic system is provided, including: a server, configured to obtain multiple second kernel data including different first kernel elements among multiple first kernel elements in the first kernel data; and an electronic device, configured to receive the second kernel data from the server, and obtain such data by performing a convolution operation on each of the multiple second kernel data and input data, in which a portion of the input data is amplified according to the first kernel data.

[0025] The server may expand the first kernel data based on the size and magnification factor (r) of the first kernel data, and obtain each of the plurality of second kernel data based on a plurality of first kernel elements separated by the factor (r) in the expanded first kernel data.

[0026] According to one aspect of the present disclosure, a method for controlling an electronic device is provided, the method comprising performing a convolution operation on each of a plurality of second kernel data obtained from first kernel data and input data, and acquiring data based on the convolution operation, in which a portion of the input data is amplified according to the first kernel data.

[0027] Each of the plurality of second kernel data can be obtained from the expanded first kernel data based on the plurality of first kernel elements separated by an enlargement factor (r). The expanded first kernel data is obtained by expanding the first kernel data based on the size and factor (r) of the first kernel data.

[0028] The method may further include receiving the plurality of second kernel data from a server.

[0029] The method may further include receiving first kernel data from a server, and obtaining the plurality of second kernel data from the first kernel data.

[0030] The acquisition of the amplified data may also include performing a convolution operation on each of the plurality of second kernel data and an element to be amplified in a portion of the input data and a plurality of peripheral elements surrounding the element to be amplified, and obtaining a plurality of amplified elements about the element to be amplified. The sum of the element to be amplified and the plurality of peripheral elements is the same as the sum of the plurality of second elements included in the plurality of second kernel data.

[0031] The obtaining of the magnified data may include determining positions of the plurality of magnified elements relative to the magnified data based on positions of the elements to be magnified relative to the input data.

[0032] The acquiring of the enlarged data may include performing a convolution operation on each of the plurality of second kernel data and the input data through a convolution array, and storing the enlarged data in an online memory.

[0033] The convolution array may include a plurality of processing elements each including a plurality of register files. The execution of the convolution operation includes: performing a multiplication operation by using a second kernel element of a plurality of processing elements input to the plurality of second kernel data through each processing element of the plurality of processing elements, and accumulating and storing a plurality of operation results in a register file corresponding to the second kernel element of the plurality of register files.

[0034] Beneficial Effects

[0035] According to the above aspects, the electronic device performs a magnification operation through a convolution array and eliminates additional hardware for performing the magnification operation, thereby reducing the hardware area. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The above and other aspects, features and advantages of certain embodiments of the present disclosure will become more apparent from the following description taken in conjunction with the accompanying drawings, in which:

[0037] Figure 1 is a block diagram showing a configuration of an electronic device according to an embodiment;

[0038] Figure 2a to Figure 2c is a diagram illustrating a method of obtaining a plurality of second kernel data according to an embodiment;

[0039] Figure 3a to Figure 3b is a diagram showing second kernel data according to an embodiment;

[0040] Figure 4a and Figure 4b is a diagram illustrating an operation of a processor according to an embodiment according to a relationship between input data and enlarged data;

[0041] Figure 5 is a diagram illustrating a method of identifying a size of second kernel data according to an embodiment;

[0042] Figures 6a to 6c is a diagram illustrating a method of obtaining a plurality of second kernel data according to an embodiment;

[0043] Figures 7a to 7c is a diagram illustrating a method of obtaining a plurality of second kernel data according to another embodiment;

[0044] Figure 8 is a diagram showing an internal structure of a processor according to an embodiment;

[0045] Fig. 9 is a diagram illustrating a convolution array according to an embodiment;

[0046] Fig.10a and Fig.10bis a diagram illustrating a shuffling method of output data according to an embodiment;

[0047] Figures 11a to 11d is a diagram illustrating a shuffling operation according to an embodiment;

[0048] Fig.12 is a diagram illustrating the operation of an electronic system according to an embodiment; and

[0049] Fig.13 is a flowchart illustrating a method for controlling an electronic device according to an embodiment. DETAILED DESCRIPTION

[0050] The embodiments may be modified differently. Therefore, specific embodiments are shown in the drawings and described in detail in the specific embodiments. However, it should be understood that the present disclosure is not limited to specific embodiments, but includes all modifications, equivalents and substitutions without departing from the scope and spirit of the present disclosure. In addition, well-known functions or structures are not described in detail because they will obscure the present disclosure with unnecessary details.

[0051] Hereinafter, each embodiment will be described in detail with reference to the drawings.

[0052] Figure 1 is a block diagram showing a configuration of an electronic device according to an embodiment. Figure 1 , the electronic device 100 may include a storage 110 such as a memory and a processor 120.

[0053] The electronic device 100 may amplify a portion of the input data based on the kernel data. For example, the electronic device 100 may amplify the content to correspond to the resolution of the display device, and provide the amplified content to a display device such as a set-top box (STB), a server, etc. The electronic device 100 may include a display and amplify the content to correspond to the resolution of the display, such as a desktop PC, a notebook, a smart phone, a tablet PC, etc. However, the present disclosure is not limited thereto, and the electronic device 100 may be any type of device as long as it can perform the amplification operation.

[0054] The memory 110 may be provided separately from the processor 120 , and implemented as a hard disk, a nonvolatile memory, a volatile memory, or the like.

[0055] The memory 110 may store input data. The input data may include amplified data. The memory 110 may store a portion of the input data. The input data may include data to be amplified, or the input data may be data obtained by filling the data to be amplified by the processor 120. For ease of explanation, it will be assumed that the input data includes data to be amplified and filled data. However, the present disclosure is not limited to this. The input data may include only data to be amplified. The processor 120 may fill the data to be amplified and store the filled data in the memory 110.

[0056] The memory 110 may store a plurality of second kernel data obtained from the first kernel data. The first kernel data may be data for amplifying a portion of the input data. For example, the first kernel data may be a Gaussian filter for amplifying a portion of the input data. Each of the plurality of second kernel data may include a different first kernel element among the plurality of first kernel elements included in the first kernel data.

[0057] The electronic device 100 may receive a plurality of second kernel data from the server. The server may obtain a plurality of second kernel data from the first kernel data and send the plurality of second kernel data to the electronic device 100. The electronic device 100 may receive the first kernel data from the server and directly obtain the plurality of second kernel data from the first kernel data.

[0058] The memory 110 may include a larger storage capacity than a line memory provided in the processor 120 , and provide data to the line memory, or receive data from the line memory.

[0059] For example, the input data and the first kernel data stored in the memory 110 may be temporarily stored in the online memory. The input data and the first kernel data stored in the memory 110 may be partially stored in the online memory. The amplified data may be stored in the online memory, and the memory 110 may receive the amplified data from the online memory.

[0060] The processor 120 may control the overall operations of the electronic device 100 .

[0061] The processor 120 may obtain data in which a portion of the input data is amplified by the first kernel data by performing a convolution operation on each of the plurality of second kernel data and the input data.

[0062] A plurality of second kernel data may be obtained from the first kernel data. Each of the plurality of second kernel data may include a different first kernel element among the plurality of first kernel elements included in the first kernel data.

[0063] Each of the plurality of second kernel data can be obtained from the expanded first kernel data based on a plurality of first kernel elements separated by an enlargement multiple (r). The expanded first kernel data is obtained by expanding the first kernel data based on the size and multiple (r) of the first kernel data.

[0064] For example, when the first kernel data is in a 7×7 matrix form, nine second kernel data in a 3×3 matrix form can be obtained from the first kernel data. The size of each of the plurality of second kernel data can be determined based on the size of the first kernel data. In addition, the number of the plurality of second kernel data can be determined based on the magnification (r). The method for obtaining the plurality of second kernel data will be further described in detail below.

[0065] The electronic device 100 may further include a communicator (not shown), and the processor 120 may receive a plurality of second kernel data from the server through the communicator and store the received plurality of second kernel data in the memory 110 .

[0066] The processor 120 may also receive the first kernel data from the server through the communicator, obtain a plurality of second kernel data from the first kernel data, and store the plurality of second kernel data in the memory 110 .

[0067] The communicator may be configured to perform communication with various types of external devices using various types of communication methods. The communicator may include a Wi-Fi chip, a Bluetooth chip, a wireless communication chip, an NFC chip, etc. The processor 120 may perform communication with various types of external devices using the communicator.

[0068] A Wi-Fi chip or a Bluetooth chip can perform communication using a Wi-Fi method and a Bluetooth method, respectively. When a Wi-Fi chip or a Bluetooth chip is used, various connectivity information such as an SSID and a session key may be first sent and received. A communication connection may be established based on the connectivity information, and various information may be sent and received in the established communication connection. A wireless communication chip refers to a chip that performs communication according to various communication standards (such as IEEE, ZigBee, third generation (3G), third generation partnership project (3GPP), long term evolution (LTE), etc.). An NFC chip refers to a chip that operates in NFC (near field communication) mode using the 13.56MHz band in various RF-ID bands (such as 135kHz, 13.56MHz, 433MHz, 860 to 960MHz, and 2.45GHz, etc.).

[0069] The communicator can perform unidirectional communication or bidirectional communication with the external device. When performing unidirectional communication, the communicator can receive the first kernel data or the plurality of second kernel data from the external device. When performing bidirectional communication, the communicator can receive the first kernel data or the plurality of second kernel data from the external device, or send the amplified data to the external device.

[0070] The processor 120 may perform a convolution operation on each of the plurality of second kernel data with the element to be amplified in the partial input data and a plurality of peripheral elements surrounding the element to be amplified to obtain a plurality of amplified elements about the element to be amplified. The sum of the element to be amplified and the plurality of peripheral elements may be the same as the sum of the plurality of second elements respectively included in the plurality of second kernel data.

[0071] For example, when the element to be amplified and the plurality of peripheral elements surrounding the element to be amplified are in a 3×3 matrix form, and each of the plurality of second kernel data is in a 3×3 matrix form, the processor 120 may perform a convolution operation on the element to be amplified and the plurality of peripheral elements surrounding the element to be amplified with one of the plurality of second kernel data to obtain an amplified element. The processor 120 may perform the operation on the remaining portion of the plurality of second kernel data in the same manner. In other words, the number of the plurality of amplified elements may be the same as the number of the plurality of second kernel data.

[0072] The enlarged data may be in a matrix form, and the processor 120 may determine the position of the enlarged element based on the second kernel data used in the convolution operation. For example, when the first second kernel data among the plurality of second kernel data is used for the operation, the enlarged element obtained by the operation of the first second kernel data may be set in the first row and first column of the enlarged data, and when the second second kernel data among the plurality of second kernel data is used for the operation, the enlarged element obtained by the operation of the second second kernel data may be set in the first row and second column of the enlarged data.

[0073] As described above, according to an embodiment, the processor 120 may obtain enlarged data of the element to be enlarged. For example, the processor 120 may convert the element to be enlarged (one pixel value) into enlarged data having 3×3 pixel values.

[0074] The processor 120 may determine positions of the plurality of enlarged elements relative to the enlarged data based on positions of the to-be-enlarged elements of the input data.

[0075] For example, the processor 120 may convert the first enlarged object element as a pixel value included in the input data into enlarged first data having a 3×3 pixel value. When the first enlarged object element of the plurality of enlarged object elements of the input data is set at the leftmost top, the processor 120 may arrange the first enlarged data at the leftmost top of the enlarged data.

[0076] The processor 120 may determine the positions of the plurality of zoom-in elements relative to the zoom-in data based on the relative positions of the zoom-in target elements.

[0077] For example, the processor 120 may convert a first enlargement object element as one pixel value included in the input data into first enlargement data having 3×3 pixel values, and convert a second enlargement object element as one pixel value included in the input data into second enlargement data having 3×3 pixel values. When the second enlargement object element is disposed on the right side of the first enlargement object element, the processor 120 may arrange the second enlargement data on the right side of the first enlargement data.

[0078] The processor 120 may include a convolution array for performing a convolution operation on each of the plurality of second kernel data and the input data, and a line memory for storing the amplified data.

[0079] The convolution array may include multiple processing elements, each processing element having multiple register files, and each processing element of the multiple processing elements may perform a multiplication operation by using a second kernel element of the multiple processing elements input to the multiple processing elements in the multiple second kernel data, and accumulate and store the operation results in a register file corresponding to the second kernel element in the multiple register files.

[0080] The processor 120 may further include a shuffler disposed between the convolution array and the line memory, and the shuffler may shuffle a plurality of operation results output from a plurality of processing elements and provide the results to the line memory.

[0081] The shuffler may receive a plurality of operation results output from register files included in a plurality of buffer groups, and a plurality of corresponding processing elements include FIFO memories that output the plurality of operation results to the plurality of buffer groups. The plurality of buffer groups may store each of the plurality of operation results in a buffer corresponding to each of the plurality of buffer groups, and when the operation results are stored in all buffers included in the plurality of buffer groups, the plurality of operation results stored in one of the plurality of buffer groups may be provided to the line memory in a preset order.

[0082] The electronic device 100 may upscale input data by using a convolution array without requiring additional dedicated hardware.

[0083] Hereinafter, according to an embodiment, the operation of the processor 120 will be described in further detail with reference to the accompanying drawings.

[0084] Figure 2a to Figure 2c 1 is a diagram illustrating a method of obtaining a plurality of second kernel data according to an embodiment. It will be assumed that a server, rather than the electronic device 100, obtains the plurality of second kernel data.

[0085] Reference Figure 2a , the server may multiply each of the 25 elements of the 5×5 input data 210 by the 49 elements of the 7×7 first kernel data 220 to obtain a plurality of intermediate data.

[0086] For example, the server may obtain the first intermediate data 230 by multiplying the elements in the first row and first column of the input data 210 by the 49 elements included in the first kernel data 220, obtain the second intermediate data 231 by multiplying the elements in the first row and second column of the input data 210 by the 49 elements in the first kernel data 220, and obtain the third intermediate data 232 by multiplying the elements in the second row and first column by the 49 elements in the first kernel data 220. In this way, the server may obtain Figure 2a A plurality of intermediate data 230 on the right.

[0087] The server may arrange a plurality of intermediate data at predetermined intervals and also include overlapping portions. Figure 2b , the server may arrange the second intermediate data 231 three spaces to the right of the first intermediate data 230, and arrange the third intermediate data 232 three spaces below the first intermediate data 230. In this way, the server may overlap and arrange a plurality of intermediate data. The left-right or up-and-down distances between adjacent intermediate data may all be the same.

[0088] Figure 2c is a diagram showing the Figure 2b A diagram showing the number of overlaps of each region when a plurality of intermediate data are overlapped in the same manner. For ease of explanation, the number of overlaps of some regions will be omitted.

[0089] Figure 2c The 9×9 center area 240 in the overlapping area of ​​​​the input data 210 can be used as the enlarged data, and the area to be enlarged can be the 3×3 center area in the input data 210. In other words, when the 3×3 center area in the input data 210 is the data to be enlarged, and the elements around the data to be enlarged are filled, the 3×3 center area in the input data 210 can be used as the enlarged data. Figure 2a to Figure 2c The process shown in FIG. 2 is magnified to a 9×9 central area 240 .

[0090] The first element 211 in the 3×3 exact center region of the input data 210 may be enlarged to the 3×3 sub-region 241 on the upper left side in the 9×9 center region 240. The elements in the 3×3 exact center region of the input data 210 may be enlarged to the 3×3 sub-regions, and each sub-region may be set in the 9×9 center region 240 according to the position of the corresponding element.

[0091] Depending on the position of the elements, each of the nine sub-areas may have the same number of operations. Specifically, the number of overlaps of the first row and first column of the sub-area 241 on the upper left side may be 4 (four), and the number of overlaps of the first row and first column of the remaining sub-areas may also be 4 (four). The four first kernel elements of the first kernel elements used to generate the first kernel data 220 of the first row and first column of the sub-area 241 on the upper left side may be the same as the four first kernel elements of the first kernel elements used to generate the first kernel data 220 of the first row and first column of the remaining sub-areas. In this case, the same four first kernel elements may be used, but the position of the sub-area may be determined based on the position of the elements of the input data to be operated with the four first kernel elements. The server may obtain one of the multiple second kernel data by using the four first kernel elements.

[0092] The number of overlaps of the first row and the second column of each sub-region may be 6 (six), and six kernel elements may be used. The server may obtain another second kernel data of the plurality of second kernel data by using the six first kernel elements.

[0093] Based on such regularity, the server can obtain a plurality of second kernel data from the first kernel data. Since the sub-region includes nine elements, the plurality of second kernel data can be nine.

[0094] The server can identify which region of the input data 210 the plurality of second kernel data are to be operated on, and will refer to Figure 4a and Figure 4b A further detailed description thereof is given.

[0095] Figure 3a to Figure 3b is a diagram illustrating second kernel data according to an embodiment.

[0096] With Figure 2c In the same manner as shown in , the server may generate the second kernel data for each element in the sub-region. Specifically, the server may divide the plurality of first kernel elements in the first kernel data into a plurality of groups, and add 0 (zero) to each of the plurality of groups to have a predetermined number of elements, thereby obtaining a plurality of second kernel data. Since the number of elements in the sub-region is 9 (nine), the reference Figure 3a , the server can generate nine second kernel data k0, k1, ... and k8.

[0097] For example, when the first kernel data is in a 7×7 matrix form, the server may divide 49 first kernel elements in the first kernel data into 9 groups of 4, 6, 4, 6, 9, 6, 4, 6, and 4. The server may add 5, 3, 5, 3, 0, 3, 5, 3, and 5 0s to each of the nine groups so that each group becomes 9 (nine).

[0098] The second kernel data may be in a matrix form, and the server may determine a position where 0 is to be added in the corresponding second kernel data based on at least one first kernel element in each of the plurality of groups.

[0099] Reference Figure 3a , the number of nine in the second kernel data refers to the first kernel element in the first kernel data 220. For example, the second kernel data k0 may include the first kernel elements 17, 20, 38, and 41 in the first kernel data 220, and the remaining area may be 0 (zero).

[0100] Reference Figure 3b , the server may generate the second kernel data k0 in a 3×3 matrix form. The remaining second kernel data of FIG. 3 may be similarly generated in a 3×3 matrix form.

[0101] Figure 4a and Figure 4b is a diagram illustrating an operation of a processor according to a relationship between input data and enlarged data according to an embodiment.

[0102] According to an embodiment, Figure 2b As shown, the server may recognize how to generate elements in the 9×9 central area 240 when a plurality of intermediate data are overlapped.

[0103] For example, the server may recognize that a convolution operation is performed between the first region 410 - 1 of the input data 210 and the second kernel data k0 to generate an element 241 - 1 in the sub-region 241 of the 9×9 central region 240 .

[0104] The convolution operation between the first region 410 - 1 of the input data 210 and the second kernel data k0 is as follows.

[0105] 1×a+2×b+3×c+4×d+5×e+6×f+7×g+8×h+9×i

[0106] The above formula is a general expression for explaining the convolution operation. Since the second kernel data k0 is Figure 3b As shown, the above formula can be expressed as follows.

[0107] (41)×a+(38)×b+0×c+(20)×d+(17)×e+0×f+0×g+0×h+0×i=(41)×a+(38)×b+(20)×d+(17)×e

[0108] The server can recognize that a convolution operation can be performed between the first area 410-1 of the input data 210 and the second kernel data k1, and the right element 241-1 can be obtained in the sub-area 241 of the 9×9 central area 240. The server can recognize that a convolution operation is performed between the first area 410-1 of the input data 210 and the second kernel data k8, and the lower right element can be obtained in the sub-area 241 of the 9×9 central area 240.

[0109] The server can recognize that a sub-region can be obtained according to a convolution operation between a region of the input data and each of the plurality of second kernel data.

[0110] When one area of ​​the input data is changed, the server may recognize that another sub-area is available in the 9×9 center area 240 .

[0111] For example, refer to Figure 4b , the server can recognize that the sub-region 242 can be obtained based on the convolution operation between the second region 410-2 of the input data 210 and each of the multiple second kernel data, and the sub-region 249 can be obtained based on the convolution operation between the ninth region 410-9 of the input data 210 and each of the multiple second kernel data.

[0112] The input data 210 may have nine 3×3 regions, and nine sub-regions of the sub-regions may be obtained accordingly.

[0113] The server may execute the operation from the input data 210 and the first kernel data 220 ( Figure 2c )Get the central area of ​​9×9 240( Figure 4a ) method.

[0114] The electronic device 100 may receive the enlargement method from the server and store the enlargement method in the memory 110. The enlargement method may be implemented as a partial configuration of the processor 120 in a hardware format or stored in a line memory in the processor 120.

[0115] The processor 120 may perform a convolution operation between each of the plurality of second kernel data and the input data according to the enlargement method, and acquire data obtained by enlarging a portion of the input data according to the first kernel data.

[0116] Reference Figure 4a and Figure 4b The processor 120 may perform a convolution operation between a region of the input data 210 and a plurality of second kernel data to obtain a 9×9 central region 240 , where the 9×9 central region 240 is obtained by enlarging the input data 210 . Figures 2a to 4bAn embodiment is shown in which second kernel data in a 3×3 matrix form is obtained when first kernel data is in a 7×7 matrix form. Figures 2a to 4b A method of obtaining second kernel data from a result of a Gaussian filter according to an embodiment is shown. Figures 5 to 7c A more general method of acquiring a plurality of second kernel data from a first kernel data is shown.

[0117] It is assumed that the first kernel data in the form of a k×k matrix and the magnification factor (r) are input by the user.

[0118] Figure 5 is a diagram illustrating a method for identifying a size of second kernel data according to an embodiment. It is assumed that first kernel data in a k×k matrix form and a magnification (r) are input by a user.

[0119] Reference Figure 5 , F0, F1, and F2 represent one pixel and may be a pixel to be enlarged. Figure 5 Data 521 , 522 , and 523 magnified by a factor (r) from the first kernel data 510 of F0 , F1 , and F2 are shown.

[0120] The relationship with the pixel affected by the first kernel data 510 may be the size of the second kernel data. Figure 5 , the second kernel data based on the F2 pixel influence to reach F1 may be in a 3×3 matrix form, and the first kernel data 510 may be in a range between r+2 and r+r / 2. That is, the size of the first kernel data 510 may be expressed as follows.

[0121] r <k<=3r

[0122] Reference Figure 5 , the second kernel data based on the F2 pixel influence to reach F0 may be in a 5×5 matrix form, and the first kernel data 510 may be in a range from r+r / 2 to 2r+r / 2. That is, the size of the first kernel data 510 may be expressed as follows.

[0123] 3r <k<=5r

[0124] In the same manner, when the second kernel data is in a 7×7 matrix form, the size of the first kernel data 510 will be as follows.

[0125] 5r <k<=7r

[0126] When the user inputs the first kernel data and the magnification (r), the server may obtain the size of the second kernel data based on the size of the first kernel data and the magnification (r).

[0127] For example, when the user inputs the first kernel data in a 7×7 matrix form and the magnification is 3, the first equation (3<7<=9) may be satisfied, and the size of the second kernel data may be 3×3. When the user inputs the first kernel data in a 7×7 matrix form and the magnification is 2, the second equation (6<7<=10) may be satisfied, and the size of the second kernel data may be 5×5.

[0128] Figures 6a to 6c is a diagram illustrating a method of obtaining a plurality of second kernel data according to an embodiment. Figure 6a , it will be assumed that the first kernel data 610 in the form of a 5×5 matrix and a magnification (r) of 2 are applied.

[0129] Reference Figure 6a , the server can expand the first kernel data 610. The server can expand the first kernel data 610 based on the size and magnification factor (r) of the second kernel data. Figure 5 As shown, the size of the second kernel data is obtained, and based on Figure 5 According to the first equation (2<5<=6), the size of the second kernel data can be 3×3.

[0130] The server may determine the size of the first kernel data by multiplying the size of the second kernel data by the magnification factor. For example, the server may determine the size of the first kernel data to be 6×6 by multiplying the size of the second kernel data 3 by the magnification factor 2.

[0131] The server may obtain 6×6 expanded first kernel data 620 by performing zero padding to the right side and bottom side of the 5×5 first kernel data. However, the present disclosure is not limited thereto, and the server may obtain 6×6 expanded first kernel data by performing zero padding to the left side and top side of the 5×5 first kernel data. Hereinafter, for ease of explanation, it will be assumed that the right side and bottom side of the first kernel data are padded with zero.

[0132] Reference Figure 6b , the server can obtain each of the plurality of second kernel data based on a plurality of first kernel elements separated by an interval of a magnification (r).

[0133] For example, the server may obtain k0 of the second kernel data 630 based on 1, 3, 5, 11, 13, 15, 21, 23, and 25 of the first kernel data 610. The server may obtain k1 of the second kernel data 630 based on 2, 4, 12, 14, 22, and 24 of the first kernel data 610. The server may add 0 to obtain 3×3 second kernel data. Each of the plurality of second kernel data 630 may include a different first kernel element among the plurality of first kernel elements in the first kernel data 610.

[0134] The server may identify each of the plurality of second kernel data as two-dimensional information of (i, j) (i and j are natural numbers less than or equal to the magnification (r)), identify the size of the second kernel data based on the size and multiple (r) of the first kernel data, expand the first kernel data based on the size and multiple (r) of the second kernel data, and obtain the second kernel data of (i, j) among the plurality of second kernel data based on the first kernel element located in the (r×a+i)th row and the (r×b+j)th column of the expanded first kernel data. Here, a and b may be natural numbers of 0 or more, respectively.

[0135] Reference Figure 6c , the server may obtain each of the plurality of second kernel elements 650 by changing the arrangement of the first kernel element 640. That is, the server may sequentially arrange the first kernel element 640 located at the (r×a+i)th row and the (r×b+j)th column in the extended first kernel data including 0 (zero) by performing zero padding, and obtain each of the plurality of second kernel elements 650 by changing the arrangement order.

[0136] For example, the server may set 1, 3, 5; 11, 13, 15; 21, 23, 25 to 25, 23, 21; 15, 13, 11; 5, 3, 1 to obtain k0 of the second kernel data.

[0137] Figures 7a to 7c FIG. 2 is a diagram showing a method for obtaining a plurality of second kernel data according to another embodiment. Figure 7a As shown in the matrix 710 on the left, according to an embodiment, it is assumed that the first kernel data 710 in the form of a 7×7 matrix and a magnification factor of 3 are applied.

[0138] Reference Figure 7a , the server may expand the first kernel data 710 to obtain the expanded first kernel data 720. The server may expand the first kernel data 710 based on the size and magnification (r) of the second kernel data. Figure 5 As shown, the size of the second kernel data is obtained, and based on Figure 5 According to the first equation (3<7<=9), the size of the second kernel data can be 3×3.

[0139] The server may determine the size of the first kernel data 710 by multiplying the size of the second kernel data by the magnification factor. For example, the server may determine the size of the first kernel data to be 9×9 by multiplying the size of the second kernel data 3 by the magnification factor 3.

[0140] The server may obtain 9×9 extended first kernel data 720 by performing zero padding of 7×7 first kernel data 710 in all directions.

[0141] Reference Figure 7b , the server may obtain each of the plurality of second kernel data 730 based on a plurality of first kernel elements separated by intervals of a magnification (r).

[0142] For example, the server may obtain k0 of the second kernel data 730 based on 17, 20, 38, and 41 of the first kernel data. The server may obtain k1 of the second kernel data 730 based on 15, 18, 21, 36, 39, and 42 of the first kernel data. Each of the plurality of second kernel data may include a different first kernel element of the plurality of first kernel elements in the first kernel data.

[0143] That is, the server may identify each of the plurality of second kernel data as two-dimensional information of (i, j) (where i and j are natural numbers less than or equal to the magnification (r)), identify the size of the second kernel data based on the size and multiple (r) of the first kernel data, expand the first kernel data based on the size and multiple (r) of the second kernel data, and obtain the second kernel data j of (i, j) among the plurality of second kernel data based on the first kernel element located in the (r×a+i)th row and the (r×b+j)th column of the expanded first kernel data. Here, a and b may be natural numbers greater than 0 (zero), respectively.

[0144] like Figure 7c As shown, the server can obtain each of the plurality of second kernel elements by changing the arrangement of the first kernel element. That is, the server can sequentially arrange the first kernel element 740 located in the (r×a+i)th row and the (r×b+j)th column of the extended first kernel data including 0 (zero) by performing zero padding, and obtain each of the plurality of second kernel elements 750 by changing the arrangement order.

[0145] As described above, although the first kernel data and the magnification (r) are changed, the server can obtain a plurality of second kernel data.

[0146] Figures 2a to 7c 1 is a diagram showing that the server obtains a plurality of second kernel data from the first kernel data, but the present disclosure is not limited thereto. For example, according to an embodiment, the electronic device 100 may receive the first kernel data from the server, and the processor 120 may obtain the plurality of second kernel data in a manner similar to that of the first kernel data. Figures 2a to 7c A plurality of second kernel data are obtained from the first kernel data in the same manner as shown in .

[0147] Figure 8 is a diagram showing an internal structure of a processor according to an embodiment.

[0148] Reference Figure 8 , the processor 120 may include a convolution array 121, a line memory 122, and a shuffler 123. According to an embodiment, Figure 8 The configuration is similar to Figure 1 Therefore, its detailed description will be omitted to avoid redundancy.

[0149] The convolution array 121 may perform a convolution operation on each of the plurality of second kernel data and the input data 210. The convolution array 121 may receive the input data 210 and each of the plurality of second kernel data from the memory 110. The memory 110 may store the input data 210 and the first kernel data 220, and the processor 120 may read a first kernel element corresponding to a second kernel element required in a current cycle from the memory 110 and provide the first kernel element to the convolution array 121.

[0150] The processor 120 may convert the first kernel data 220 stored in the memory 110 into a plurality of second kernel data and store the plurality of second kernel data in the memory 110. The convolution array 121 may receive a second kernel element required in a current cycle from the memory 110 among the plurality of second kernel data stored in the memory 110.

[0151] The convolution array 121 may receive the input data 210 and the second kernel element required in the current cycle from the line memory 122. The line memory 122 may receive the input data 210 and the first kernel data 220 from the memory 110, and store the input data 210 and the first kernel data 220. The line memory 122 may receive the input data 210 and a plurality of second kernel data from the memory 110, and store the input data 210 and the plurality of second kernel data.

[0152] The convolution array 121 can accumulate and store the convolution operation results, and the following will refer to the embodiment. Fig. 9 A further detailed description of the convolution array 121 is provided.

[0153] The line memory 122 may store the amplified data. The line memory 122 may store the input data 210 and a plurality of second kernel data.

[0154] The shuffler 123 may be provided between the convolution array 121 and the line memory 122. The shuffler 123 may shuffle the operation results output from the convolution array 121 and provide the results to the line memory 122. Figures 11a to 11d A further detailed description thereof is provided.

[0155] Fig. 9 is a diagram showing a convolution array 121 according to an embodiment.

[0156] Reference Fig. 9 , the convolution array 121 may include a plurality of processing elements each including a plurality of register files.

[0157] Each processing element may basically include a multiplier and an arithmetic logic unit (ALU), and the ALU may include at least one adder. The processing element may use the multiplier and the ALU to perform arithmetic operations. However, the present disclosure is not limited thereto, and any other structure may be used as long as it can perform functions such as arithmetic operations and shift operations.

[0158] Each processing element may include multiple register files. Fig. 9 As shown, each processing element may include nine register files (RFs).

[0159] Each processing element can store the operation result in a specific cycle in one of the multiple register files. Each processing element can shift the operation result in a specific cycle to an adjacent processing element and include an additional register for storing the shifted operation result from the adjacent processing element.

[0160] Each of the plurality of processing elements may perform a multiplication operation using a second kernel element of the plurality of processing elements input to the plurality of second kernel data, and accumulate and store the operation results in a register file corresponding to the second kernel element of the plurality of register files.

[0161] For example, the plurality of processing elements may receive five elements in a first row of the 5×5 input data 210 in Figure 2. The plurality of processing elements may sequentially receive second kernel elements from one of the plurality of second kernel data and perform a convolution operation.

[0162] Each of the plurality of processing elements may store the operation result in a register file corresponding to the second kernel element. For example, nine register files respectively included in the plurality of processing elements may correspond to the second kernel data k0 to k8 respectively, and when the second kernel element in the second kernel data k0 is used for operation, each of the plurality of processing elements may store the operation result in a register file corresponding to the second kernel data k0. Therefore, although the second kernel element included in the second kernel data other than k0 is used, the operation result can be recognized.

[0163] Reference Figure 8 Since the convolution operation method of the ID convolution array 121 is a well-known technique, a detailed description is omitted. The number of register files respectively included in a plurality of processing elements is shown as an example, but the number may vary.

[0164] Convolution array 121 may perform parallel operations through multiple processing elements and adjust the order of data output accordingly.

[0165] Fig.10a and Fig.10b is a diagram illustrating a shuffling method of output data according to an embodiment.

[0166] like Fig.10a As shown in the first cycle shown in , multiple processing elements can output 5 parallel operation results through the second kernel data k0. Fig.10a In the second cycle shown in , multiple processing elements can output 5 operation results through the second kernel data. Fig.10a The first cycle shown refers to a cycle until the convolution operation of one second kernel data is completed.

[0167] As described above, five parallel processing elements of the second kernel data k0 may be output according to parallel operations, and when stored in the line memory 122 as is, errors may occur in enlargement.

[0168] Therefore, according to Fig.10b The order shown will be stored in the line memory 122 according to the output data of the parallel operation, and this shuffling operation can be performed by the shuffler 123. Fig.10b The size of the parallel operations performed by processor 120 is shown to be 6 (six).

[0169] Figures 11a to 11d is a diagram illustrating a shuffling operation according to an embodiment.

[0170] Reference Fig.11a The shuffler 123 may receive a plurality of operation results output from register files respectively corresponding to each other and included in a plurality of buffer groups 1110 and a plurality of processing elements, and includes a FIFO memory 1120 that outputs the plurality of processing elements to the plurality of buffer groups 1110 .

[0171] The plurality of buffer groups 1110 may store each of the plurality of operation results in a buffer corresponding to each of the plurality of buffer groups. Fig.11a As shown in , multiple buffer groups can store multiple operation results in the first buffer of each group. Fig.11b , multiple buffer groups can store multiple processing elements in the second buffer of each group. Fig.11c As shown, multiple buffer groups may store multiple processing elements in a third buffer of each group.

[0172] When the operation results are stored in all buffers included in the plurality of buffer groups, the plurality of buffer groups may provide a plurality of operation results stored in one buffer group among the plurality of buffer groups in a pre-stored order.

[0173] For example, refer to Fig.11c , the plurality of buffer groups may output 1131 all data stored in a buffer group disposed at the top of the plurality of buffer groups to the line memory 122. Fig.11d , the plurality of buffer groups may output 1132 all data stored in a buffer group disposed at a center among the plurality of buffer groups to the line memory 122 .

[0174] The shuffler 123 may shuffle the data output from the convolution array 121 and provide the shuffled data to the line memory 122 .

[0175] Fig.12 is a diagram illustrating the operation of an electronic system according to an embodiment.

[0176] The electronic system 1000 may include an electronic device 100 and a server 200 .

[0177] The server 200 may obtain a plurality of second kernel data, the plurality of second kernel data including different first kernel elements among a plurality of first kernel elements included in the first kernel data. Specifically, the server 200 may expand the first kernel data based on the size and magnification (r) of the first kernel data, and obtain each of the plurality of second kernel data based on a plurality of first kernel elements separated by a multiple (r) in the expanded first kernel data.

[0178] The electronic device 100 may receive the second kernel data from the server 200, and perform a convolution operation on each of the plurality of second kernel data and the input data and acquire data obtained by amplifying a portion of the input data through the first kernel data.

[0179] Fig.13 is a flowchart illustrating a method of controlling an electronic device according to an embodiment.

[0180] In operation S1310, a convolution operation may be performed between each of a plurality of second kernel data obtained from the first kernel data and input data. In operation S1320, data in which a portion of the input data is amplified according to the first kernel data may be acquired based on the convolution operation.

[0181] Each of the plurality of second kernel data can be obtained from the expanded first kernel data based on the size of the first kernel data and the magnification multiple (r). The expanded first kernel data is obtained by expanding the first kernel data based on the size of the first kernel data and the multiple (r).

[0182] The method may also include receiving a plurality of second kernel data from the server.

[0183] The method may further include an operation of receiving first kernel data from a server and an operation of obtaining a plurality of second kernel data from the first kernel data.

[0184] The operation of obtaining the amplified data of operation S1320 may include: performing a convolution operation on each of the multiple second kernel data with an element to be amplified in a part of the input data and a plurality of peripheral elements surrounding the element to be amplified and obtaining a plurality of amplified elements about the element to be amplified, and the sum of the element to be amplified and the plurality of peripheral elements may be the same as the sum of the plurality of second elements respectively included in the plurality of second kernel data.

[0185] The operation of obtaining the enlarged data S1320 may include determining a plurality of enlarged elements with respect to the enlarged data based on a position of the element to be enlarged relative to the input data.

[0186] The operation of obtaining the enlarged data S1320 may include performing a convolution operation on each of the plurality of second kernel data and the input data through a convolution array, and storing the enlarged data in a line memory.

[0187] The convolution array may include multiple processing elements that respectively include multiple register files, and the operation of performing the convolution operation S1310 may include performing a multiplication operation using second kernel elements of the multiple processing elements input to the multiple processing elements in the multiple second kernel data through the corresponding multiple processing elements, and accumulating and storing the operation results in the register files corresponding to the second kernel elements in the multiple register files.

[0188] The method may further include shuffling a plurality of operation results output from a plurality of processing elements through a shuffler disposed between the convolution array and the line memory and providing the operation results to the line memory.

[0189] The operation of providing the operation results to the line memory may include receiving a plurality of operation results output from respectively corresponding register stacks included in a plurality of processing elements of a first-in-first-out (FIFO) memory, and outputting the plurality of operation results to a plurality of buffer groups, storing each of the plurality of operation results in a buffer corresponding to each of the plurality of buffer groups, and providing the plurality of operation results stored in one of the plurality of buffer groups to the line memory in a preset order based on the operation results being stored in all buffers including the plurality of buffer groups.

[0190] According to various embodiments, the electronic device may remove additional hardware for performing a magnification operation to minimize hardware area by performing the magnification operation by a convolution array.

[0191] As described above, Gaussian kernel data has been described as first kernel data, but even when the first kernel data is not Gaussian kernel data, enlargement can be performed in the same manner. For example, the processor can perform enlargement in various ways such as using bilinear, sub-pixel shuffling, transposed convolution, etc., and in this case, the enlargement can be performed by a convolution operation. When transposed convolution is used, enlargement can be performed by extending the transposed convolution in the z direction based on the convolution operation.

[0192] Various embodiments may be implemented as software including one or more instructions stored in a machine-readable storage medium. A machine may be a device that calls one or more instructions stored in a storage medium and is operable according to the one or more instructions called, including an electronic device according to various embodiments (e.g., electronic device (A)). When a command is executed by a processor, the processor may use other components to perform functions corresponding to the command directly or under the control of the processor. One or more instructions may include a code made by a compiler or a code executable by an interpreter. A machine-readable storage medium may be provided in the form of a non-temporary storage medium. "Non-temporary" means that the storage medium does not include a signal but is tangible, but does not distinguish whether the data is stored semi-permanently or temporarily on the storage medium.

[0193] According to an embodiment, the method according to various embodiments disclosed herein may be provided in a computer program product. The computer program product may be traded as a commodity between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., a compact disc read-only memory (CD-ROM)) or through an application store (e.g., PlayStore). TM ) Online distribution. In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a storage medium such as a manufacturer's server, an application store's server, or a memory of a relay server.

[0194] The various embodiments described above can be implemented in a recording medium that can be read by a computer or similar device using software, hardware, or a combination thereof. In some cases, the embodiments described herein can be implemented by the processor itself. According to software implementation, embodiments such as the processes and functions described herein can be implemented in separate software modules. According to some embodiments, software (e.g., program) may include one or more instructions stored in a machine (i.e., computer) readable storage medium (i.e., internal memory) or an external memory. Each of the software modules can perform one or more of the functions and operations described herein.

[0195] A non-transitory computer-readable storage medium refers to a medium that stores data semi-permanently rather than for a very short time, such as a register, a cache, and a memory, and can be read by a device. Specifically, the above-mentioned various applications or programs can be stored in a non-transitory computer-readable medium such as a compact disc (CD), a digital versatile disc (DVD), a hard disk, a Blu-ray disc, a universal serial bus (USB) memory stick, a memory card, and a read-only memory (ROM), and can be provided.

[0196] Each of the components (e.g., modules or programs) according to various embodiments may be composed of a single entity or multiple entities, and some of the above-mentioned subcomponents may be omitted, or other components may be further included in various embodiments. Alternatively or additionally, some components (e.g., modules or programs) may be integrated into one entity to perform the same or similar functions performed by each component before integration. According to various embodiments, the operations performed by modules, programs or other components may be performed sequentially, in parallel, repeatedly or heuristically, or at least some operations may be performed in a different order, or omitted, or another function may be further added.

[0197] Although embodiments have been shown and described, it will be appreciated by those skilled in the art that changes may be made to these embodiments without departing from the principles and spirit of the present disclosure. Therefore, the scope of the present disclosure is not to be interpreted as being limited to the described embodiments, but is to be defined by the appended claims and their equivalents.

Claims

1. An electronic device, comprising: A memory storing input data and a plurality of second kernel data obtained from the first kernel data, so that each of the plurality of second kernel data includes a different first kernel element among the plurality of first kernel elements in the first kernel data; as well as a processor, performing a convolution operation on each of the plurality of second kernel data and the input data, shuffling the performed convolution operation and obtaining data based on the plurality of shuffle operation results, wherein the data is obtained by amplifying at least a portion of the input data according to the first kernel data, wherein the processor is configured to divide a plurality of first kernel elements in the first kernel data into a plurality of groups, add zeros to each of the plurality of groups based on at least one first kernel element included in each of the plurality of groups so that each of the plurality of groups has a predetermined number of second kernel data, and obtain the plurality of second kernel data based on the plurality of groups having zeros, The size of each of the plurality of second kernel data is determined based on the size of the first kernel data, and Wherein, the processor comprises: A convolution array includes a circuit, and the convolution array is configured to perform a convolution operation on each of the plurality of second kernel data and the input data by performing parallel operations via a plurality of processing elements, A line memory includes a circuit, and the line memory stores amplified data, and A shuffler includes a circuit, and the shuffler is located between the convolution array and the line memory and is configured to shuffle a plurality of operation results output from the convolution array and output the plurality of shuffled operation results to the line memory.

2. The electronic device according to claim 1, wherein: Each of the plurality of second kernel data is obtained from the expanded first kernel data based on the plurality of first kernel elements separated by an interval of a magnification factor r, wherein r is a natural number, and The expanded first kernel data is obtained by expanding the first kernel data based on the size of the first kernel data and the magnification factor r.

3. The electronic device according to claim 1, further comprising: Communicator, including circuitry, The processor is further configured to receive the plurality of second kernel data from the server via the communicator, and control the memory to store the received plurality of second kernel data.

4. The electronic device according to claim 1, further comprising: Communicator, including circuitry, The processor is further configured to receive the first kernel data from the server via the communicator, obtain the plurality of second kernel data from the first kernel data, and control the memory to store the plurality of second kernel data.

5. The electronic device according to claim 1, wherein: The processor is further configured to perform a convolution operation on each of the plurality of second kernel data and the element to be amplified in the at least one portion of the input data and a plurality of peripheral elements surrounding the element to be amplified, so as to obtain a plurality of amplified elements about the element to be amplified. The first sum of the element to be amplified and the plurality of peripheral elements is the same as the second sum of the plurality of second elements respectively included in the plurality of second kernel data.

6. The electronic device according to claim 5, wherein: The processor is further configured to determine positions of the plurality of enlarged elements relative to the element to be enlarged based on the position of the element to be enlarged relative to the input data.

7. The electronic device according to claim 1, wherein: The convolution array includes a plurality of processing elements each including a plurality of register files, and Each of the plurality of processing elements performs a multiplication operation of a second kernel element of the plurality of second kernel data input to the plurality of processing elements, and accumulates and stores a result of the multiplication operation in a register file corresponding to the second kernel element in the plurality of register files.

8. The electronic device according to claim 1, wherein: The shuffler includes: a plurality of buffer groups including circuits; and A first-in-first-out FIFO memory includes a circuit, and the FIFO memory receives at least two processing elements among the plurality of processing elements output from corresponding register files among the plurality of register files, and outputs the plurality of operation results to a plurality of buffer groups, wherein the plurality of buffer groups store each of the plurality of operation results in a buffer corresponding to each of the plurality of buffer groups, and Wherein, based on the fact that the plurality of operation results are stored in all buffers included in the plurality of buffer groups, the plurality of operation results stored in one buffer group among the plurality of buffer groups are output to the line memory in a preset order.

9. An electronic system comprising: The server obtains a plurality of second kernel data, wherein the plurality of second kernel data include different first kernel elements among the plurality of first kernel elements in the first kernel data; and The electronic device receives the plurality of second kernel data from the server, performs a convolution operation on each of the plurality of second kernel data and the input data, and obtains data based on the convolution operation, wherein the data is obtained by amplifying at least a portion of the input data according to the first kernel data, The server is further configured to divide the plurality of first kernel elements in the first kernel data into a plurality of groups, add zeros to each of the plurality of groups based on at least one first kernel element included in each of the plurality of groups so that each of the plurality of groups has a predetermined number of second kernel data, and obtain the plurality of second kernel data based on the plurality of groups having zeros, The size of each of the plurality of second kernel data is determined based on the size of the first kernel data, and Wherein, the electronic equipment comprises: A convolution array includes a circuit, and the convolution array is configured to perform a convolution operation on each of the plurality of second kernel data and the input data by performing parallel operations via a plurality of processing elements, A line memory includes a circuit, and the line memory stores amplified data, and A shuffler includes a circuit, and the shuffler is located between the convolution array and the line memory and is configured to shuffle a plurality of operation results output from the convolution array and output the plurality of shuffled operation results to the line memory.

10. The electronic system according to claim 9, wherein: The server expands the first kernel data based on the size and magnification r of the first kernel data, and obtains each second kernel data of the multiple second kernel data based on multiple first kernel elements separated by the magnification r in the expanded first kernel data, where r is a natural number.

11. A method for controlling an electronic device, the electronic device storing input data and a plurality of second kernel data obtained from first kernel data, so that each of the plurality of second kernel data includes a different first kernel element of a plurality of first kernel elements in the first kernel data, the method comprising: The processor divides a plurality of first kernel elements in the first kernel data into a plurality of groups, adds zero to each of the plurality of groups based on at least one first kernel element included in each of the plurality of groups so that each of the plurality of groups has a predetermined number of second kernel data, and obtains the plurality of second kernel data based on the plurality of groups having zeros; The processor performs a convolution operation on each of the plurality of second kernel data and the input data, shuffles the performed convolution operation, and obtains data based on the plurality of shuffle operation results, wherein the data is obtained by amplifying at least a portion of the input data according to the first kernel data; and The processor obtains amplified data based on a convolution operation, wherein the amplified data is obtained by amplifying at least a portion of the input data according to the first kernel data, The size of each of the plurality of second kernel data is determined based on the size of the first kernel data, and Wherein, the processor comprises: A convolution array includes a circuit, and the convolution array is configured to perform a convolution operation on each of the plurality of second kernel data and the input data by performing parallel operations via a plurality of processing elements, A line memory includes a circuit, and the line memory stores amplified data, and A shuffler includes a circuit, and the shuffler is located between the convolution array and the line memory and is configured to shuffle a plurality of operation results output from the convolution array and output the plurality of shuffled operation results to the line memory.

12. The method according to claim 11, further comprising: each of the plurality of second kernel data is obtained from the expanded first kernel data based on the plurality of first kernel elements separated by an interval of a magnification factor r, wherein r is a natural number, and The expanded first kernel data is obtained by expanding the first kernel data based on the size of the first kernel data and the magnification factor r.

13. The method according to claim 11, further comprising: The plurality of second kernel data are received from the server.

Images