Electronic device for performing deconvolution calculation and control method thereof
By designing a processor to perform convolution calculation and accumulation operations in electronic devices, the problem of large hardware occupancy is solved, efficient deconvolution calculation is realized, and space utilization is improved.
Patent Information
- Application Number
- CN202080081142.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2020-02-11
- Filing Date
- 2020-12-18
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2040-12-18
AI Technical Summary
In embedded environments, the area occupied by hardware is difficult to minimize, especially when performing convolutional calculations and deconvolution calculations, large hardware areas need to be allocated, resulting in waste of resources.
By designing an electronic device, its processor can generate calculation information based on the input data of deconvolution calculation, use the address generation module to obtain the memory address value, use the convolution calculation module to perform the convolution calculation, and accumulate the intermediate value to obtain the inverse volume Accumulate calculation results, thereby reducing the area occupied by hardware.
It realizes efficient execution of deconvolution calculation without increasing hardware resources, reducing the space occupied by hardware and improving space efficiency.
Smart Images

Figure CN114730330B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure generally relates to an electronic device, and more particularly, to an electronic device that performs a deconvolution calculation and a control method thereof. Background Art
[0002] Super-resolution (SR) may refer to converting a low-resolution image into a high-resolution image. An artificial intelligence (AI) model may be used to perform the SR operation and may include a convolutional layer, a nonlinear layer, and an upscaling layer.
[0003] A hardware accelerator (HWA) supporting an SR network may include hardware for performing convolution operations, hardware for supporting layers composed of nonlinear functions, and hardware responsible for upscaling.
[0004] The upscaling method may vary depending on the calculation method or the implementation method. However, when separate hardware is configured according to the upscaling method, a problem tends to occur in which an excessively large area becomes occupied by the hardware.
[0005] In addition, an electronic device including hardware for performing a convolution calculation operation may have separate hardware for performing a deconvolution calculation operation. Therefore, there tends to be a problem that an electronic device should allocate a large area to have hardware for performing both a convolution calculation operation and a deconvolution calculation operation. Summary of the invention
[0006] Technical issues
[0007] Accordingly, there is a need in the art for a method and electronic device that minimizes the area occupied by hardware in an embedded environment.
[0008] Problem Solution
[0009] The present disclosure is provided to address at least the above problems and / or disadvantages and to provide at least the advantages described below.
[0010] Accordingly, an aspect of the present disclosure is to provide an electronic device for obtaining a result value of a deconvolution calculation through a convolution calculation process and a control method thereof.
[0011] According to one aspect of the present disclosure, an electronic device includes: a memory; and a processor connected to the memory, wherein the processor is configured to control the electronic device, wherein the processor is configured to: based on input data of a deconvolution calculation being input, obtain calculation information; based on the obtained calculation information, obtain the size of output data; use an address generation module to obtain multiple memory address values corresponding to the size of the output data; use a convolution calculation module to perform convolution calculation based on the calculation information to obtain an intermediate value in the convolution calculation process; use the address generation module to obtain a memory address value corresponding to the obtained intermediate value among the multiple obtained memory address values; store the obtained intermediate value in the memory address value corresponding to the intermediate value; use an accumulation calculation module to accumulate at least one intermediate value based on the memory address value corresponding to the intermediate value; and obtain a deconvolution calculation value for the input data based on the accumulated at least one intermediate value.
[0012] According to another aspect of the present disclosure, a method for controlling an electronic device includes: input data based on a deconvolution calculation is input to obtain calculation information; the size of output data is obtained based on the obtained calculation information; a plurality of memory address values corresponding to the size of the output data are obtained using an address generation module; a convolution calculation module is used to perform a convolution calculation based on the calculation information to obtain an intermediate value in a convolution calculation process; a memory address value corresponding to an obtained intermediate value among a plurality of obtained memory address values is obtained using the address generation module; the obtained intermediate value is stored in a memory address value corresponding to the intermediate value; an accumulation calculation module is used to accumulate at least one intermediate value based on the memory address value corresponding to the intermediate value; and a deconvolution calculation value for the input data is obtained based on the accumulated at least one intermediate value. BRIEF DESCRIPTION OF THE DRAWINGS
[0013] The above and other aspects, features and advantages of certain embodiments of the present disclosure will become more apparent from the following description in conjunction with the accompanying drawings, in which:
[0014] Figure 1 is a block diagram showing an electronic device according to an embodiment;
[0015] Figure 2 shows a convolution calculation operation according to an embodiment;
[0016] Figure 3 Shows Figure 2 Partial operation of the convolution operation;
[0017] Figure 4 Shows Figure 2 Partial operation of the convolution operation;
[0018] Figure 5 Shows Figure 2Partial operation of the convolution operation;
[0019] Figure 6 shows convolution calculation and deconvolution calculation according to an embodiment;
[0020] Figure 7 shows a deconvolution calculation operation according to an embodiment;
[0021] Figure 8 Shows Figure 7 Partial operation of the deconvolution calculation operation;
[0022] Fig. 9 Shows Figure 7 Partial operation of the deconvolution calculation operation;
[0023] Fig.10 Shows Figure 7 Partial operation of the deconvolution calculation operation;
[0024] Fig.11a Shows Figure 7 Partial operation of the deconvolution calculation operation;
[0025] Fig.11b Shows Figure 7 Partial operation of the deconvolution calculation operation;
[0026] Fig.12 It is shown that a deconvolution calculation operation is performed based on the size of the stride according to an embodiment;
[0027] Fig.13 A method for controlling an electronic device according to a first embodiment is shown;
[0028] Fig.14 A method for controlling an electronic device according to a second embodiment is shown;
[0029] Fig.15 A method for controlling an electronic device according to a third embodiment is shown;
[0030] Fig.16 A method for controlling an electronic device according to a fourth embodiment is shown;
[0031] Fig.17 A method for controlling an electronic device according to a fifth embodiment is shown;
[0032] Fig.18 An operation of assigning a result value of a deconvolution calculation operation to a memory address is shown;
[0033] Fig.19 An operation of assigning intermediate values obtained from a deconvolution calculation operation to a buffer address and a memory address according to the first embodiment is shown;
[0034] Fig. 20 An operation of assigning intermediate values obtained from a deconvolution calculation operation to a buffer address and a memory address according to a second embodiment is shown;
[0035] Fig.21 shows a hardware diagram according to an embodiment;
[0036] Fig. 22 shows input activated movement according to an embodiment;
[0037] Fig.23 shows the movement of weight values according to an embodiment;
[0038] Fig.24 The operation of a multiply and accumulate (MAC) processing array according to an embodiment is shown;
[0039] Fig.25 shows the unit calculation operation of the MAC processing unit (MPU) according to the first embodiment;
[0040] Fig.26 shows a unit calculation operation of the MPU according to the second embodiment;
[0041] Fig. 27 illustrates the operation of a post-processing array according to an embodiment;
[0042] Fig.28 shows mathematical equations used in convolution calculation or deconvolution calculation according to an embodiment; and
[0043] Fig.29 A method for controlling an electronic device according to an embodiment is shown. DETAILED DESCRIPTION
[0044] The present disclosure will be described in more detail with reference to the accompanying drawings. For the sake of clarity and conciseness, detailed descriptions of known functions and / or configurations will be omitted.
[0045] The terms used in this specification and claims are general terms identified in view of the functions of the embodiments of the present disclosure. However, these terms may vary according to the intentions of technicians in the relevant fields, legal or technical interpretations, the emergence of new technologies, etc. Some terms may be arbitrarily defined in this article. Unless there is a specific definition of a term, the term may be interpreted based on the overall content and technical common sense of technicians in the relevant fields.
[0046] Expressions such as “have”, “may have”, “include”, “may include”, etc. indicate the existence of a corresponding number, function, operation or part, and do not exclude the existence of additional features.
[0047] Expressions such as “at least one of A or B” and “at least one of A and B” should be understood to mean “A”, “B” or “A and B”.
[0048] As used herein, terms such as “first” and “second” may identify corresponding components regardless of importance or order, and are used to distinguish a component from another component without limiting the components.
[0049] In addition, the description that one element (e.g., a first element) is “coupled” / “coupled to” or “connected to” another element (e.g., a second element) operably or communicatively should be interpreted to include: the first element is directly coupled to the second element; and the first element is coupled to the second element via a third element.
[0050] Unless otherwise specified, singular expressions include plural expressions. It should be understood that terms such as "including" or "consisting of" are used here to specify the presence of features, numbers, steps, operations, elements, components or combinations thereof, and do not exclude the presence or possibility of adding one or more other features, numbers, steps, operations, elements, components or combinations thereof.
[0051] Terms such as "module", "unit" and "part" are used to refer to elements that perform at least one function or operation and can be implemented as hardware or software or a combination of hardware and software. Except when each of a plurality of "modules", "units", "parts" etc. must be implemented in separate hardware, the components may be integrated in at least one module or chip and implemented in at least one processor.
[0052] In the following description, “user” may refer to a person using an electronic device or an artificial intelligence electronic device using an electronic device.
[0053] The calculations in the present disclosure may be performed by a machine learning based recognition system, and deep learning recognition may be described as an example of a classification system based on a series of machine learning algorithms based on neural networks.
[0054] The deep learning-based recognition system may include at least one classifier that may correspond to one or more processors. The processor may be implemented as an array of multiple logic gates and may be implemented as a combination of a general-purpose microprocessor and a memory, wherein a program that can be executed in the microprocessor is stored in the memory.
[0055] The classifier can be implemented as a neural network-based classifier, a support vector machine (SVM), an Adaboost classifier, a Bayesian classifier, a perceptron classifier, etc. The classifier will be described for an embodiment implemented as a classifier based on a convolutional neural network (CNN). A neural network-based classifier is a computational model implemented to simulate the computing power of a biological system by using a large number of artificial neurons connected by connecting lines, and human cognitive functions or learning processes can be performed by connecting lines with connection strengths (weights). However, the classifier of the present disclosure is not limited thereto, and can be implemented with various classifiers.
[0056] A general neural network may include an input layer, a hidden layer, and an output layer, and the hidden layer may be composed of one or more layers as needed. In this case, a back propagation algorithm may be used as an algorithm for training the neural network.
[0057] If some data is input to the input layer of the neural network, the classifier can train the neural network so that the output data of the input learning data is output to the output layer of the neural network. When feature information extracted from a captured image is input, the neural network can be used to classify the pattern of the feature information into any one of various categories, and the classification result can be output.
[0058] The processor can use a deep learning-based recognition system as a classification system based on a series of machine learning algorithms based on neural networks.
[0059] Figure 1 is a block diagram illustrating an electronic device according to an embodiment.
[0060] Reference Figure 1 , the electronic device 100 may include a memory 110 and a processor 120 .
[0061] The electronic device 100 may be a computer or a server including hardware for performing convolution computing operations.
[0062] The memory 110 may be implemented as an internal memory such as a read-only memory (ROM) such as an electrically erasable programmable read-only memory (EEPROM) and a random access memory (RAM) or a memory separate from the processor 120. In this case, the memory 110 may be implemented as at least one of a memory embedded in the electronic device 100 or a memory detachable from the electronic device 100 according to the use of data storage. For example, data for driving the electronic device 100 may be stored in a memory embedded in the electronic device 100, and data for upgrading the electronic device 100 may be stored in a memory detachable from the electronic device 100.
[0063] The memory 110 embedded in the electronic device 100 may be implemented as at least one of a volatile memory such as a dynamic random access memory (DRAM), a static random access memory (SRAM), a synchronous dynamic random access memory (SDRAM) or a nonvolatile memory such as a one-time programmable ROM (OTPROM), a programmable ROM (PROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a mask ROM, a flash ROM, a flash memory such as a NAND flash memory or a NOR flash memory, a hard disk drive (HDD), or a solid state drive (SSD). In the case where the memory is detachably mounted to the electronic device 100, the memory may be implemented as a memory card such as a compact flash (CF), a secure digital (SD), a micro secure digital (micro-SD), a mini secure digital (mini-SD), an extreme digital (xD), or a multimedia card (MMC) and an external memory such as a universal serial bus (USB) memory connectable to a USB port.
[0064] The processor 120 may perform overall control operations of the electronic device 100 .
[0065] The processor 120 may be implemented with at least one of a digital signal processor (DSP), a microprocessor and a time controller (TCON), a central processing unit (CPU), a microcontroller unit (MCU), a microprocessing unit, a controller, an application processor (AP), a communication processor (CP), and an advanced reduced instruction set computing (RISC) machine (ARM) processor, or may be defined as a corresponding term. The processor 120 may be implemented in a system on chip (SoC) type or a large scale integration (LSI) type with a built-in processing algorithm or in a field programmable gate array (FPGA) type. The processor 120 may perform various functions by executing computer executable instructions stored in the memory 110.
[0066] The processor 120 may include at least one of a control module 121 , a convolution calculation module 122 , an address generation module 123 , an accumulation calculation module 124 , and a buffer 125 .
[0067] The control module 121 may obtain calculation information and transmit the calculation information to the convolution calculation module 122 or the address generation module 123 .
[0068] The calculation information may include at least one of a calculation type, input data, filter data, or a stride.
[0069] The calculation type may refer to a convolution calculation or a deconvolution calculation (or a transposed convolution calculation).
[0070] Input data may refer to target data on which calculations are performed.
[0071] The filter data may indicate kernel data, and may be data for changing the size of input data. In addition, the filter data may include at least one weight.
[0072] The stride may refer to the movement interval used when performing the convolution calculation.
[0073] The convolution calculation module 122 may perform convolution calculation based on the received calculation information. The convolution calculation module 122 may perform all convolution calculations to obtain a calculation result value. The convolution calculation module 122 may perform one convolution calculation among multiple convolution calculations to obtain an intermediate value, and send the obtained intermediate value to the accumulation calculation module 124.
[0074] The address generation module 123 may generate and manage the address value of the memory or buffer. The convolution calculation result value or the deconvolution calculation result value may be stored in the address value of a specific memory, and the intermediate value obtained before the final calculation result value may be stored in the buffer. The address generation module 123 may generate and manage the address value of the memory storing the intermediate value (or calculation result value) or the address value of the buffer.
[0075] The accumulation calculation module 124 can perform the accumulation calculation by adding the new value to the existing value. Specifically, the accumulation calculation module 124 can accumulate the value already stored in the buffer 125 and the newly received value in the convolution calculation module 122, and send the accumulated new value to the buffer 125.
[0076] The buffer 125 may be configured to temporarily store data. The buffer 125 may temporarily store the partial sum (or cumulative sum) before the calculation result is completed, and may send the stored partial sum to the cumulative calculation module 124 or the memory 110 as needed.
[0077] The processor 120 may be connected to the memory 110 to control the electronic device 100. The overall control operation of the processor 120 may be performed by the control module 121. In the following description, for convenience of description, the processor 120 may be described instead of the control module 121.
[0078] When input data for deconvolution calculation is input, the processor 120 can obtain calculation information, obtain the size of output data based on the obtained calculation information, use the address generation module 123 to obtain multiple memory address values corresponding to the size of the output data, use the convolution calculation module 122 to perform convolution calculation based on the calculation information to obtain an intermediate value in the convolution calculation process, use the address generation module 123 to obtain a memory address value corresponding to the obtained intermediate value among multiple memory address values, store the obtained intermediate value in the memory address value corresponding to the intermediate value, use the accumulation calculation module 124 to accumulate at least one intermediate value based on the memory address value corresponding to the intermediate value, and obtain the deconvolution calculation value of the input data based on the at least one accumulated intermediate value.
[0079] The processor 120 may receive input data of the deconvolution calculation to obtain calculation information of the deconvolution calculation, wherein the calculation information may include data required to perform the deconvolution calculation. The calculation information may include at least one of the size of the input data, the size of the filter data used for the deconvolution calculation, the size of the stride for the convolution calculation, and the calculation order information corresponding to the first intermediate value.
[0080] The processor 120 may obtain the size of the output data based on the obtained calculation information. Accordingly, when the calculation information changes, the size of the output data may be changed. The size of the input data and the size of the output data may refer to the size of the matrix forming the input data and the output data (such as 3*3 or 4*4).
[0081] The processor 120 may obtain a memory address value corresponding to the size of the output data obtained by using the address generation module 123. The processor 120 may first obtain a memory address value to store the output data before obtaining a specific value of the output data. The obtained memory address value may be plural.
[0082] The deconvolution calculation may be the sum of the intermediate result values of multiple convolution calculations. When the deconvolution calculation is divided, it may be composed of multiple convolution calculations. Accordingly, the processor 120 may perform multiple convolution calculations to obtain the deconvolution calculation value and obtain the intermediate value in the multiple convolution calculation processes. The processor 120 may obtain one intermediate value in one convolution calculation.
[0083] The processor 120 may obtain a memory address value corresponding to the obtained intermediate value from the plurality of obtained memory address values based on the calculation information. Since the calculation information includes data required for the calculation, the processor 120 is able to identify which memory address value the obtained intermediate value corresponds to based on the calculation order. The processor 120 may store the obtained intermediate value in the memory address value corresponding to the obtained intermediate value.
[0084] The processor 120 may use the accumulation calculation module 124 to perform an accumulation operation. Specifically, the accumulation operation may indicate a process of adding the intermediate values. Based on the calculation information, the processor 120 may perform multiple convolution calculation operations and obtain multiple intermediate values. There may be intermediate values with the same memory address value corresponding to each intermediate value. If the memory address value is the same as the corresponding intermediate value, the intermediate value and the new intermediate value may be accumulated. If there is an intermediate value with the same address value, the accumulation operation may not be performed in the corresponding memory address value.
[0085] When all accumulation operations are completed, the processor 120 may obtain a final deconvolution calculation value based on the value stored in the memory address.
[0086] The processor 120 may use the address generation module 123 to obtain the first memory address value or the second memory address value based on the calculation information.
[0087] The first memory address value may correspond to a first intermediate value obtained during the first convolution calculation process. The processor 120 may obtain a first memory address value corresponding to the obtained first intermediate value from among a plurality of memory address values, wherein the first memory address value is obtained based on calculation order information corresponding to the first intermediate value. The processor 120 may store the first intermediate value in the memory 110 based on the first memory address value.
[0088] The second memory address value may correspond to the second intermediate value obtained during the second convolution calculation process. The processor 120 may obtain a second memory address value corresponding to the obtained second intermediate value from among the plurality of memory address values, wherein the second memory address value is obtained based on calculation order information corresponding to the second intermediate value. The processor 120 may store the second intermediate value in the memory 110 based on the second memory address value.
[0089] If the second memory address value corresponding to the second intermediate value obtained during the second convolution calculation is the same as the first memory address value, the processor 120 can use the accumulation calculation module 124 to accumulate the first intermediate value and the second intermediate value to obtain a partial sum, and store the obtained partial sum in the memory 110 based on the first memory address value.
[0090] If the second memory address value corresponding to the second intermediate value obtained during the second convolution calculation is different from the first memory address value, the processor 120 may store the second intermediate value in the second memory address value. If the second intermediate value is stored in the second memory address value, the first intermediate value and the second intermediate value may not be accumulated.
[0091] The processor 120 may sequentially perform convolution calculations on the input data using a plurality of values (weights) included in the horizontal lines of the filter data for deconvolution calculation to obtain a plurality of intermediate values. Rather than using only the filter data in the horizontal axis, the processor 120 may preferentially use the filter data corresponding to the horizontal axis between the horizontal axis and the vertical axis. The filter data may be in the form of a matrix including a plurality of rows (horizontal lines) and a plurality of columns (vertical lines). The processor 120 may perform convolution calculations for rows in the weights stored in the form of a matrix.
[0092] The filter data may be composed of a matrix and may include data arranged based on a horizontal axis (row) and a vertical axis (column). For example, the filter data may be 3*3 in size, including w0(1,1), w1(1,2), w2(1,3) in the first row (row1), w3(2,1), w4(2,2), w5(2,3) in the second row (row2), and w6(3,1), w7(3,2), w8(3,3) in the third row (row3). The processor 120 may perform convolution calculations sequentially for the horizontal axis (row) in the order of w0(1,1), w1(1,2), and w2(1,3). That is, the processor 120 may first use the filter data corresponding to the first row of the filter data, and then use the filter data corresponding to the second and third rows.
[0093] The convolution calculation is performed for the horizontal axis in order to efficiently manage the partial sum. If the convolution calculation is performed for the horizontal axis, the final partial sum corresponding to the memory address value can be obtained by the minimum calculation. Therefore, if the final partial sum is confirmed, the buffer can no longer be used, thereby improving the memory management efficiency. Figure 8 A detailed description thereof is provided in .
[0094] The processor 120 may use the accumulation calculation module 124 to accumulate at least some of the plurality of intermediate values, and may calculate (or obtain) the number of accumulation iterations based on the calculation information.
[0095] The processor 120 may store the intermediate value in the internal buffer 125 based on the address value of the internal buffer 125 corresponding to the intermediate value, identify whether there are additional intermediate values to be accumulated with the intermediate value, and if it is identified that there are no additional intermediate values, store the intermediate value stored in the internal buffer 125 in the memory 110.
[0096] The processor 120 may determine the size of the internal buffer in which the intermediate values are to be stored based on the calculation information. The size of the internal buffer may be fixed. However, for efficient management of the memory, the processor 120 may determine the size of the internal buffer based on the calculation information. Specifically, the larger the size of the input data, the larger the size of the internal buffer. Fig.19 and Fig. 20 Provides a detailed description of buffers, etc.
[0097] The obtained deconvolution calculation value can be obtained without using a deconvolution calculation module. Through the above description, it has been described that the deconvolution calculation is composed of multiple convolution calculation operations. Accordingly, the electronic device 100 can obtain the deconvolution calculation value using only a conventional convolution calculation module. The electronic device 100 may not include hardware used only for deconvolution calculations, and may perform deconvolution calculations using only a conventional convolution calculation module 122. Therefore, compared to an electronic device including hardware for convolution calculations and deconvolution calculations, the electronic device 100 may require less space for computing hardware. Therefore, since the electronic device 100 requires less hardware space than other electronic devices, space efficiency can be maximized.
[0098] Figure 2 A convolution calculation operation according to an embodiment is shown.
[0099] Reference Figure 2 , the electronic device 100 may perform convolution calculation on the input data 210 of size 5*5 and the filter data 220 of size 3*3. The input data 210 may have values from c0 to c24, and the filter data 220 may have values from w0 to w8. Figure 2 In the example, it is assumed that the stride is 1. The electronic device 100 may obtain the final output data 230 of the convolution calculation based on the input data 210, the filter data 220 and the stride. The size of the final output data of the convolution calculation may be 3*3.
[0100] Figure 3 Shows Figure 2 Part of the convolution operation.
[0101] Reference Figure 3 , in order to execute Figure 2 In the convolution calculation of the embodiment, the electronic device 100 can perform a convolution calculation operation on the partial data 210 - 0 of the input data 210 and the filter data 220 to obtain the partial data 230 - 0 of the output data 230 .
[0102] Figure 4 Shows Figure 2 Part of the convolution operation.
[0103] Reference Figure 4 ,exist Figure 3 After the calculation of , the electronic device 100 can move the stride 1 to perform the next convolution calculation operation. Specifically, the electronic device 100 can perform a convolution calculation operation on the partial data 210-1 of the input data 210 and the filter data 220 to obtain the partial data 230-1 of the output data 230.
[0104] Figure 5 Shows Figure 2 Part of the convolution operation.
[0105] Reference Figure 5 ,exist Figure 4 After the calculation, the electronic device 100 can move the stride 1 to perform multiple convolution calculation operations, and the last calculation operation in the multiple calculation operations is shown. Specifically, the electronic device 100 can perform a convolution calculation operation using the partial data 210-8 of the input data 210 and the filter data 220 to obtain the partial data 230-8 of the output data 230.
[0106] The electronic device 100 may eventually obtain Figure 5 The output data 230, and Figure 5 The output data 230 may be the final result value of the convolution calculation.
[0107] Reference Figure 2 , Figure 3 , Figure 4 and Figure 5 , the stride may be 1, but output data of different sizes may be generated according to the size of the stride. The input data 210 and the filter data 220 used in the convolution calculation operation are only examples, and data of various sizes may be applied.
[0108] Figure 2 , Figure 3 , Figure 4 and Figure 5 The input data, filter data, and output data shown may refer to input data of a convolution calculation, filter data of a convolution calculation, and output data of a convolution calculation.
[0109] Figure 6 Convolution calculations and deconvolution calculations according to an embodiment are shown.
[0110] Reference Figure 6 , the convolution calculation 200 may refer to obtaining the output data 203 based on the convolution calculation input data 201 and the filter data 202. The deconvolution calculation 600 may refer to performing the inverse process of the convolution calculation. Specifically, the deconvolution calculation 600 may refer to obtaining the convolution calculation input data 201 based on the convolution calculation output data 203 and the convolution calculation filter data 202.
[0111] In describing the deconvolution calculation 600, the output data 203 of the convolution calculation may indicate the input data of the deconvolution calculation, and the input data 201 of the convolution calculation may indicate the output data of the deconvolution calculation.
[0112] The deconvolution calculation 600 may be a transposed convolution calculation.
[0113] Figure 7 A deconvolution computation operation according to an embodiment is shown.
[0114] Reference Figure 7 , the electronic device 100 may perform deconvolution calculation 600 (or translational convolution calculation). Return to reference Figure 2 , the convolution calculation may be a process of obtaining the output data 230 of the convolution calculation based on the input data 210 of the convolution calculation, the filter data 220 of the convolution calculation, and the stride 1. The deconvolution calculation may be a process of obtaining the input data 210 of the convolution calculation based on the output data 230 of the convolution calculation, the filter data 220 of the convolution calculation, and the stride 1.
[0115] The output data 230 of the convolution calculation may be the input data 610 of the deconvolution calculation, the filter data 220 of the convolution calculation may be the filter data 620 of the deconvolution calculation, and the input data 210 of the convolution calculation may be the output data 630 of the deconvolution calculation.
[0116] Figure 7 , Figure 8 , Fig. 9 , Fig.10 , Figure 11 and Fig.12 The input data and output data may refer to the corresponding data of the deconvolution calculation.
[0117] Reference Figure 7 , the electronic device 100 can perform a deconvolution calculation based on the input data 610 of 3*3 size, the filter data 620 of 3*3 size and the stride 1 to obtain the output data 630 of 5*5.
[0118] Figure 8 Shows Figure 7 Part of the deconvolution operation.
[0119] Reference Figure 8 , the deconvolution calculation can divide the filter data 620 to perform a calculation operation. The electronic device 100 can use the convolution calculation to obtain a result value of the deconvolution calculation.
[0120] For example, the filter data 620 may be a 3*3 matrix and may have nine weights w0 to w8. The weights w0 to w8 included in the filter data 620 may be divided based on rows. The electronic device 100 may classify the weights (w0, w1, w2) corresponding to the first row of the filter data 620 into a first group 620-1, the weights (w3, w4, w5) corresponding to the second row of the filter data 620 into a second group 620-2, and the weights (w6, w7, w8) corresponding to the third row of the filter data 620 into a third group 620-3. The electronic device 100 may perform a convolution calculation of the input data 610 and the classified weight groups 620-1, 620-2, 620-3 to obtain a final result value of the deconvolution calculation.
[0121] The convolution calculation processing of the first group 620-1 of input data 610 and filter data 620 can be described as a first stage 600-1, the convolution calculation processing of the second group 620-2 of input data 610 and filter data 620 can be described as a second stage 600-2, and the convolution calculation processing of the third group 620-3 of input data 610 and filter data 620 can be described as a third stage 600-3.
[0122] Reference Figure 8 For the sake of brevity, only i0 610-0 of the input data 610 will be described.
[0123] The electronic device 100 may perform a convolution calculation using i0 610-0 and the first group 620-1 in the first stage 600-1 to obtain a plurality of intermediate values 630-1-0. The electronic device 100 may perform a convolution calculation using i0 610-0 and the second group 620-2 in the second stage 600-2 to obtain a plurality of intermediate values 630-2-0. The electronic device 100 may perform a convolution calculation using i0 610-0 and the third group 620-3 in the third stage 600-3 to obtain a plurality of intermediate values 630-3-0.
[0124] Reference Figure 8 The description of the first stage 600-1, the second stage 600-2 and the third stage 600-3 is for describing the calculation process of i0610-0 and will be described later. Fig. 9 , Fig.10 , Fig.11a and Fig.11b The overall computational processing according to the deconvolution calculation at each stage is described in .
[0125] The electronic device 100 can identify the stage based on the row of the matrix of the filter data 620. The stage is classified based on the row to efficiently perform data calculation. Therefore, the calculation of all data is the same, but if the calculation is performed preferentially on the row, the number of values waiting in the buffer for cumulative calculation can be reduced. If the number of values waiting in the buffer is reduced, efficient memory operations are available, and the electronic device 100 can reduce processing time and capacity.
[0126] Fig. 9 Shows Figure 7 Part of the deconvolution operation.
[0127] Reference Fig. 9 , the first stage 600 - 1 can be classified into nine steps 600 - 1 - 0 to 600 - 1 - 8 based on the input data 610 .
[0128] In the zeroth step 600-1-0 (stage 1-0) of the first stage 600-1, the electronic device 100 may perform a convolution calculation between i0 610-0 and the first group 620-1 of the filter data 620 to obtain a plurality of intermediate values 630-1-0. The plurality of intermediate values 630-1-0 may include three intermediate values i0w0, i0w1, and i0w2, and may be stored in a matrix 905 in a predetermined order.
[0129] In the first step 600-1-1 (stage 1-1) of the first stage 600-1, the electronic device 100 may perform a convolution calculation between i1 610-1 and the first group 620-1 of the filter data 620 to obtain a plurality of intermediate values 630-1-1. The plurality of intermediate values 630-1-1 may include three intermediate values i1w0, i1w1, and i1w2. The electronic device 100 may accumulate and store the plurality of intermediate values 630-1-1 by moving a stride 1 in the right direction (row direction) from the matrix 905 storing the plurality of intermediate values 630-1-0. The result of accumulating the plurality of intermediate values 630-1-1 may be a matrix 910.
[0130] In the second step 600-1-2 (stage 1-2) of the first stage 600-1, the electronic device 100 may perform a convolution calculation between i2 610-2 and the first group 620-1 of the filter data 620 to obtain a plurality of intermediate values 630-1-2. The plurality of intermediate values 630-1-2 may include three intermediate values i2w0, i2w1, and i2w2. In addition, the electronic device 100 may newly accumulate and store the plurality of intermediate values 630-1-2 by moving the stride 1 in the right direction from the matrix 910 to which the plurality of intermediate values 630-1-1 are accumulated. The result of accumulating the plurality of intermediate values 630-1-2 may be a matrix 915.
[0131] The third step (600-1-3) to the seventh step (600-1-7) are repeated, and the description thereof will be omitted.
[0132] In the eighth step 600-1-8 (stage 1-8) of the first stage 600-1, the electronic device 100 may perform a convolution calculation between i8 610-8 and the first group 620-1 of the filter data 620 to obtain a plurality of intermediate values 630-1-8. The plurality of intermediate values 630-1-8 may include three intermediate values i8w0, i8w1, and i8w2. The electronic device 100 may accumulate and store the plurality of intermediate values 630-1-8 by moving a stride of 1 in the right direction from the matrix in which the plurality of intermediate values i7w0, i7w1, and i7w2 are accumulated. The result of accumulating the plurality of intermediate values 630-1-8 may be a matrix 920.
[0133] Fig.10 Shows Figure 7 Part of the deconvolution operation.
[0134] Reference Fig.10 , the second stage 600 - 2 can be classified into nine steps 600 - 2 - 0 to 600 - 2 - 8 based on the input data 610 .
[0135] In the first step 600-2-0 (stage 2-0) of the second stage 600-2, the electronic device 100 may perform a convolution calculation between i0 610-0 and the second group 620-2 of the filter data 620 to obtain a plurality of intermediate values 630-2-0 that may include three intermediate values i0w3, i0w4, and i0w5. The plurality of intermediate values 630-2-0 may be additionally accumulated in the final accumulation matrix 920 of the first stage 600-1. The plurality of intermediate values 630-2-0 may be accumulated by moving in a downward direction (column direction) for the positions where the plurality of intermediate values 630-1-0 are stored in the matrix 905. The result of accumulating the plurality of intermediate values 630-2-0 may be a matrix 1005.
[0136] The first step (600-2-1) to the seventh step (600-1-7) repeat the first step, and redundant description will be omitted.
[0137] In the eighth step 600-2-8 (stage 2-8) of the second stage 600-2, the electronic device 100 may perform a convolution calculation between i8 610-8 and the second group 620-2 of the filter data 620 to obtain a plurality of intermediate values 630-2-8. The plurality of intermediate values 630-2-8 may include three intermediate values i8w3, i8w4, and i8w5. The electronic device 100 may accumulate and store the plurality of intermediate values 630-2-8 by moving a stride of 1 in the right direction from the matrix storing the plurality of intermediate values i7w3, i7w4, and i7w5. The result of accumulating the plurality of intermediate values 630-2-8 may be a matrix 1010.
[0138] Fig.11a and 11b It is shown Figure 7 Schematic diagram of partial operations of the deconvolution calculation operation.
[0139] Reference Fig.11a and Fig.11b , based on the input data 610 Figure 8 The third stage 600-3 is classified into nine steps 600-3-0 to 600-3-8.
[0140] In the zeroth step 600-3-0 (stage 3-0) of the third stage 600-3, the electronic device 100 may perform a convolution calculation between i0 610-0 and the third group 620-3 of the filter data 620 to obtain a plurality of intermediate values 630-3-0. The plurality of intermediate values 630-3-0 may include three intermediate values i0w6, i0w7, and i0w8, and may be additionally accumulated in the final accumulation matrix 1010 of the second stage 600-2. The plurality of intermediate values 630-3-0 may be accumulated by moving a step 1 in a downward direction (column direction) for the positions where the plurality of intermediate values 630-2-0 are stored in the matrix 1005. The result of accumulating the plurality of intermediate values 630-3-0 may be a matrix 1105.
[0141] The first to seventh steps overlap with the zeroth step, and descriptions of the first to seventh steps will be omitted.
[0142] In the eighth step 600-3-8 (stage 3-8) of the third stage 600-3, the electronic device 100 may perform a convolution calculation between i8 610-8 and the third group 620-3 of the filter data 620 to obtain a plurality of intermediate values 630-3-8 including three intermediate values i8w6, i8w7, and i8w8. The electronic device 100 may accumulate and store the plurality of intermediate values 630-3-8 by moving in the right direction with a step of 1 in the matrix storing the plurality of intermediate values i7w6, i7w7, i7w8. The result of accumulating the plurality of intermediate values 630-3-8 may be a matrix 1110.
[0143] If all stages of the filter data 620 divided into a total of three groups are completed, the electronic device 100 may obtain output data 630 as a result value of the deconvolution calculation. The output data 630 may include information about the matrix 1110.
[0144] like Fig. 9 , Fig.10 , Fig.11a and Fig.11b As shown, only convolution calculation and accumulation calculation are used when obtaining the result value of deconvolution calculation. Therefore, the electronic device 100 can only use the convolution calculation module to obtain the result value of deconvolution calculation.
[0145] Despite Fig. 9 , Fig.10 , Fig.11a and Fig.11b The size of the filter data described in is described as 1*3, but the minimum unit calculation processing can be performed in units of 1*1 filter data. That is, the electronic device 100 can obtain 1*1 intermediate values by using a 1*1 filter (or kernel) when performing a convolution calculation, and the obtained 1*1 intermediate values can be stored in the memory address corresponding to each intermediate value. The memory address corresponding to each intermediate value may vary according to the order of the convolution calculation.
[0146] Fig.12 It is shown that the deconvolution calculation operation is performed based on the size of the stride according to an embodiment.
[0147] Reference Fig.12 , when performing the deconvolution calculation 1200, the size of the input data 1210 is 3*3, the size of the filter data 1220 is 3*3, and the stride is 2.
[0148] The electronic device 100 may obtain output data 1230 of size 7*7 based on the input data 1210, the filter data 1220, and the stride 2. The electronic device 100 may remove edge data 1235 from the obtained output data 1230 to obtain only partial data 1240 as a final result value. In order to convert input data of size 3×3 into output data of size 6×6, the electronic device 100 may remove edge data 1235 from the output data 1230. The edge data 1235 may refer to the outermost data in the matrix corresponding to the output data 1230, including o0, o1, o2, o3, o4, o5, o6, o13, o20, o27, o34, o41, o48, o47, o46, o45, o44, o43, o42, o35, o28, o21, o14, and o7.
[0149] If the stride is greater than or equal to 2, the electronic device 100 can remove the portion corresponding to the edge data of the result value of the deconvolution calculation. In addition, the electronic device 100 can obtain the remaining data from which the portion corresponding to the edge data has been removed as the final result value in the result value of the deconvolution calculation. The reason why the electronic device 100 removes the edge data when the stride is greater than or equal to 2 is that if the stride is 1, the need to perform the removal operation is minimal. If the stride is 1, when the removal operation is performed, the size of the input data and the size of the output data can be the same. Therefore, since the upscaling of the deconvolution calculation can be eliminated, the electronic device 100 can remove the edge data only when the stride is greater than or equal to 2.
[0150] Fig.13 A method for controlling an electronic device according to a first embodiment is shown.
[0151] Reference Fig.13 , the electronic device 100 may obtain calculation information in step S1305. The calculation information may refer to various information required for calculation. For example, the calculation information may be at least one of the calculation type (convolution calculation or deconvolution calculation), input data, filter data (or kernel data), or stride.
[0152] In step S1310, the electronic device 100 may obtain the size of the output data based on the obtained calculation information. The output data may have different sizes based on the calculation information. In step S1315, the electronic device 100 may obtain a memory address corresponding to the size of the output data based on the obtained size of the output data. Specifically, the electronic device 100 may specify a memory space where the output data will be stored. For example, if the output data is a 5*5 matrix, the electronic device 100 may specify a storage space to store a 5*5 matrix. The memory space may be specified as a memory address value. That is, the electronic device 100 may obtain a memory address value where a 5*5 matrix will be stored. The electronic device 100 may exchange information with the memory 110 to obtain a memory address value. The following will refer to Fig.15 Provide a detailed description.
[0153] The electronic device 100 may perform one of the plurality of convolution calculations after obtaining the memory address. The plurality of convolution calculations may be performed sequentially by the electronic device 100 in a predetermined order. In step S1320, the electronic device 100 may perform a convolution calculation to obtain an intermediate value as a result of a convolution calculation. A convolution calculation may be a calculation of a 1*1 core unit (or a 1*1 channel). An intermediate value as a calculation result value of a 1*1 core unit may be obtained by performing a 1*1 core unit calculation.
[0154] In step S1325, the electronic device 100 may obtain a memory address value corresponding to the obtained intermediate value. Since each of the plurality of convolution calculations has a predetermined order, the electronic device 100 is able to obtain a memory address value corresponding to the intermediate value as a result of the convolution calculation based on the predetermined order. The electronic device 100 may obtain the result of the convolution calculation only after performing the calculation operation, but may obtain information about which memory address value the result of the convolution calculation will be stored in before performing the calculation operation.
[0155] In step S1330, the electronic device 100 may accumulate the obtained intermediate value in the memory address value corresponding to the obtained intermediate value to obtain a partial sum. The intermediate value may indicate a calculation result, and the partial sum may indicate the result of the sum of multiple intermediate values. That is, multiple intermediate values may be accumulated and stored in the same memory address value. The value of accumulating the intermediate values in a repeated manner is described as a partial sum, but even if the intermediate values are not additionally accumulated, the obtained intermediate value may also be described as a partial sum.
[0156] Will refer to Fig.16 , Fig.17 , Fig.18 and Fig.19 Describe the specific operation.
[0157] In step S1335, after obtaining the intermediate value, the electronic device 100 may determine whether all of the multiple convolution calculations have been performed. When the electronic device 100 determines that there is a convolution calculation to be performed additionally and the convolution calculation is not performed, the next convolution calculation may be performed according to a predetermined order. If the electronic device 100 determines in the operation of S1335 that all of the multiple convolution calculations have been performed, then in step S1340, the electronic device 100 may obtain the partial sum stored in the memory address value as the result of the deconvolution calculation.
[0158] Fig.14 A method for controlling an electronic device according to a second embodiment is shown.
[0159] Reference Fig.14 The electronic device 100 may include a memory 110 and a processor 120 , wherein the processor 120 may include a control module 121 , a convolution calculation module 122 , an address generation module 123 , an accumulation calculation module 124 and a buffer 125 .
[0160] In step S1405, the control module 121 may obtain calculation information. The calculation information may include at least one of a calculation type, input data information, filter data information, and stride information. The input data information may include size information of the input data and the input data information, and the filter data information may include size information of the filter data. The stride information may also include size information of the stride. In step S1410, the control module 121 may determine whether the calculation command is a deconvolution operation based on the type information of the calculation command included in the calculation information.
[0161] In step S1411, when the calculation command is a convolution calculation rather than a deconvolution calculation, the control module 121 may send the calculation information to the convolution calculation module 122. In step S1412, the convolution calculation module 122 may perform the convolution calculation based on the received information and obtain the result value of the convolution calculation. In step S1413, the convolution calculation module 122 may send the obtained result value of the convolution calculation to the memory 110. The convolution calculation module 122 has been described as sending the calculation result value to the memory 110, but as an alternative, the convolution calculation module 122 may send the calculation result value to the memory 110 through the buffer 125. In step S1414, the memory 110 may receive and store the calculation result value.
[0162] If the calculation command is a deconvolution calculation, then at step S1420, the control module 121 may send the calculation information to the address generation module 123. At step S1421, the address generation module 123 may obtain the size of the output data based on the calculation information. At step S1422, the address generation module 123 may obtain the memory address value corresponding to the output data based on the obtained size of the output data. Fig.15 In step S1423, the address generation module 123 may send the obtained memory address value to the memory 110.
[0163] When the calculation command is a deconvolution calculation, in step S1430, the control module 121 may send the calculation information to the convolution calculation module 122. In step S1435, the convolution calculation module 122 may perform the convolution calculation of the 1*1 core unit based on the received calculation information to obtain the intermediate result value (or intermediate value) of the convolution calculation. The convolution calculation of the 1*1 core unit may apply the filter data corresponding to the 1*1 channel. The calculation of the minimum unit in the separate convolution operation of the kernel (having a function similar to the filter) may be performed using the filter data of 1*1 size. The intermediate value obtained during the calculation of the minimum unit may be set to 1*1, and the intermediate value of 1*1 may be stored in the memory address corresponding to the intermediate value of 1*1. A description of the process for storing the intermediate value to the memory address will be provided in steps S1440 to S1465.
[0164] In step S1440, the convolution calculation module 122 may send the obtained intermediate value to the accumulation calculation module. In addition, in step S1445, the accumulation calculation module 124 may receive the intermediate value from the convolution calculation module 122, accumulate the received intermediate value with the existing partial sum, and determine whether there is an additional partial sum. The existing partial sum may indicate an intermediate value that has been obtained or an intermediate value obtained by the accumulation calculation module 124. The additional partial sum may indicate a new partial sum obtained by accumulating a new intermediate value that will be accumulated by additionally performing the convolution calculation. In step S1450, the accumulation calculation module 124 may send the partial sum to the buffer 125. The buffer 125 is capable of storing the partial sum received from the accumulation calculation module 124. In step S1455, the buffer 125 may send the partial sum to the memory 110. In step S1460, the memory 110 may store the partial sum as a result of the deconvolution calculation in the memory 110. Fig.16 Specific operations related to accumulation are described in .
[0165] Fig.15 A method for controlling an electronic device according to a third embodiment is shown.
[0166] Reference Fig.15, in step S1505, the control module 121 may obtain the calculation information. In step S1510, the control module 121 may send the obtained calculation information to the address generation module 123. In step S1515, the address generation module 123 may obtain the size information of the output data based on the received calculation information. In step S1520, the address generation module 123 may send the obtained size information of the output data to the memory 110. Specifically, step S1520 may refer to: specifying a memory area where the output data will be stored, and sending a request to the memory 110 to return a memory address value corresponding to the specified area. In step S1525, the memory 110 may specify a storage area where the output data will be stored based on the received output data size information, and generate a memory address value corresponding to the specified storage area. In step S1530, the memory 110 may send the generated memory address value to the address generation module 123. In step S1535, the address generation module 123 may obtain a memory address value corresponding to the size of the obtained output data.
[0167] Fig.16 A method for controlling an electronic device according to a fourth embodiment is shown.
[0168] Reference Fig.16 , assuming execution Fig.15 In step S1505 to step S1535 executed in the above embodiment. In step S1605, the address generation module 123 may obtain a memory address value corresponding to the size of the obtained output data. Step S1605 may be performed with Fig.15 Corresponds to S1535.
[0169] After executing step S1605, in step S1610, the address generation module 123 may send the obtained memory address value to the convolution calculation module 122. In step S1615, the control module 121 may send the calculation information to the convolution calculation module 122. The convolution calculation module 122 may determine the number of times to perform the convolution operation based on the received calculation information. In step S1620, the convolution calculation module 122 may perform multiple convolution calculations according to the preset sequence information. The intermediate result value obtained in one convolution calculation is called an intermediate value. The convolution calculation module 122 may perform one of the multiple convolution calculations to obtain an intermediate value. The convolution calculation module 122 may obtain the intermediate value and the memory address value corresponding to the intermediate value based on the received memory address value and the preset sequence information. In step S1625, the address generation module 123 may send the obtained intermediate value and the memory address value corresponding to the obtained intermediate value to the accumulation calculation module 124.
[0170] In step S1630, the cumulative calculation module 124 may determine whether there is a partial sum stored in the memory address value corresponding to the intermediate value. Since one or more intermediate values may be stored in the same memory address value, the cumulative calculation module 124 may update the partial sum. Accordingly, when the obtained intermediate value is stored in the memory address value corresponding to the intermediate value, the cumulative calculation module 124 may determine whether the existing partial sum is stored in the corresponding memory address value. The value obtained by the convolution calculation module 122 may be referred to as an intermediate value, and the value stored in the buffer 125 or the memory 110 may be referred to as a partial sum. Depending on the situation, the intermediate value initially stored in the memory address value without performing the cumulative calculation may be referred to as a partial sum.
[0171] If there is no pre-stored partial sum in the memory address value corresponding to the intermediate value, then at step S1635, the accumulation calculation module 124 may send the memory address value corresponding to the obtained intermediate value and the obtained intermediate value to the buffer 125. At step S1640, the buffer 125 may store the memory address value corresponding to the obtained intermediate value and the obtained intermediate value.
[0172] If there is a pre-stored partial sum in the memory address value corresponding to the intermediate value, then in step S1645, the cumulative calculation module 124 may request the memory address value corresponding to the obtained intermediate value from the buffer 125. In step S1650, the buffer 125 may send the partial sum pre-stored in the obtained memory address value to the cumulative calculation module 124. In step S1655, the cumulative calculation module 124 may sum the obtained intermediate value with the received partial sum to obtain a new partial sum. In step S1660, the cumulative calculation module 124 may send the new partial sum to the buffer 125. In step S1665, the buffer 125 may store the new partial sum in the memory address value corresponding to the new partial sum. The memory address value corresponding to the existing partial sum and the memory address value corresponding to the new partial sum may be the same.
[0173] After performing steps S1620 and S1665, the convolution calculation module 122 may identify a memory address value corresponding to the intermediate value obtained in step S1620, and determine in step S1670 whether there is an additional convolution calculation result to be stored in the identified memory address value. As described above, multiple intermediate values may be stored in one memory address value so that the partial sum may be updated. The convolution calculation module 122 may determine whether there is an intermediate value to be additionally accumulated in the memory address value.
[0174] If it is determined that there is an intermediate value to be additionally accumulated in the address value, the convolution calculation module 122 may perform the next convolution calculation based on a preset order to obtain the intermediate value.
[0175] If it is determined that there is no intermediate value to be additionally accumulated to the address value, the convolution calculation module 122 may send a control signal for finalizing the partial sum stored in the buffer to the accumulation calculation module 124 in step S1671. The accumulation calculation module 124 may finalize the partial sum stored in the buffer 125 in step S1675.
[0176] The accumulation calculation module 124 may request the buffer 125 to transmit the memory address value corresponding to the determined partial sum and the determined partial sum to the memory 110. The buffer 125 may transmit the memory address value corresponding to the determined partial sum and the determined partial sum to the memory 110. The memory 110 may store the finally determined partial sum in the memory address value corresponding to the accumulated partial sum.
[0177] Fig.17 A method for controlling an electronic device according to a fifth embodiment is shown.
[0178] Reference Fig.17 , assuming execution Fig.15 Steps S1505 to S1535 and Fig.16 In step S1605 to step S1690. In step S1705, the address generation module 123 may obtain a memory address value corresponding to the size of the obtained output data, wherein step S1705 may be Fig.15 Corresponding to step S1535.
[0179] In step S1710, the address generation module 123 may send the obtained memory address value to the convolution calculation module 122. In step S1715, the control module 121 may send the calculation information to the convolution calculation module 122.
[0180] The convolution calculation module 122 may determine the number of times to perform the convolution calculation based on the received calculation information. In step S1720, the convolution calculation module 122 may perform multiple convolution calculations according to the preset sequence information, wherein step S1720 may be performed with Fig.16 In step S1725, the convolution calculation module 122 may send the memory address value corresponding to the intermediate value and the intermediate value to the accumulation calculation module 124. In step S1730, the accumulation calculation module 124 may perform an accumulation calculation operation. In step S1735, the accumulation calculation module 124 may send the partial sum obtained in the accumulation calculation operation to the memory 110. In step S1740, the memory 110 may store the partial sum in the memory address value corresponding to the partial sum. Fig.17 Steps S1730, S1735, and S1740 described in Fig.16 Therefore, in Fig.17 Its detailed description will be omitted.
[0181] After executing step S1720, at step S1745, the convolution calculation module 122 may determine whether all of the multiple convolution calculations have been performed. If all of the multiple convolution calculations have not been performed, the convolution calculation module 122 may perform the next convolution calculation based on a preset order. If it is determined that all of the multiple convolution calculations have been performed, at step S1750, the convolution calculation module 122 may obtain the portion stored in the memory address value and the final result value as the deconvolution calculation.
[0182] Fig.18 An operation of assigning a result value of a deconvolution calculation operation to a memory address is shown.
[0183] Reference Fig.18 , the electronic device 100 can perform deconvolution calculation 600 based on input data 610, filter data 620 and stride 1. The above calculation operation is similar to Figure 7 The calculation operation described in is the same, so its repeated description will be omitted.
[0184] The electronic device 100 can obtain 5*5 output data based on the input data 610 of 3*3 size, the filter data 620 of 3*3 size, and the stride 1. The input data 610 of 3*3 size, the filter data 620 of 3*3 size, and the stride 1 may be included in the calculation information. The electronic device 100 may obtain information that the size of the output data is 5*5 based on the calculation information. The size of the output data may vary according to the size of the input data, the size of the filter data, and the stride included in the calculation information, and the information may have been stored in the electronic device 100.
[0185] When the size of the output data is 5*5, the electronic device 100 can obtain a memory address value corresponding to the size of the output data. The output data 630 can be in the form of a 5*5 matrix from c0 to c24. The values of c0 to c24 included in the output data 630 can be stored in a memory address value corresponding to the value of the output data 630 itself. The memory address value corresponding to the output data 630 can be directly calculated by the electronic device 100 or obtained by the memory 110, and can correspond to the matrix 1805.
[0186] The memory address values corresponding to the output data 630 may include a total of 25 address values from #m-00 to #m-24. For example, c0 included in the output data may be stored in #m-00, and c24 may be stored in #m-24.
[0187] Fig.19 An operation of assigning intermediate values obtained from a deconvolution calculation operation to a buffer address and a memory address according to the first embodiment is shown.
[0188] Reference Fig.19 , the address value of the buffer 125 and the address value of the memory 110 will be used to describe Fig. 9 The operation of step zero 600-1-0 in the first stage 600-1 of the process in Fig. 9 The detailed operations are described in , so redundant descriptions will be omitted.
[0189] Reference Fig.19 , assuming that the address value of the buffer is 3, and the address values of the buffer are #b-00 (125-0), #b-01 (125-1), #b-02 (125-2). The following describes the first address value 125-0 of the buffer, the second address value 125-1 of the buffer, and the third address value 125-2 of the buffer.
[0190] The electronic device 100 may obtain i0w0, i0w1, i0w2 as a plurality of intermediate values 630-1-0 in the zeroth step 600-1-0 of the first stage 600-1. Fig.18 As described in , each intermediate value may obtain a memory address value corresponding to the intermediate value. The value i0w0 may correspond to a first address value (#m-00, 110-1-00) of the memory, i0w1 may correspond to a second address value (#m-01, 110-1-01) of the memory, and i0w2 may correspond to a third address value (#m-02, 110-1-02) of the memory.
[0191] In one embodiment, i0w0, i0w1, and i0w2 as a plurality of intermediate values may be implemented in a form stored in the memory 110. However, in Fig.19 In the following, other embodiments of storing in the memory 110 through the buffer 125 will be described.
[0192] The electronic device 100 may store the obtained intermediate values i0w0, i0w1, i0w2 in order in the buffer 125 based on the order of obtaining. For example, the electronic device 100 may store i0w0 in the first address value 125-0 of the buffer, i0w1 in the second address value 125-1 of the buffer, and i0w2 in the third address value 125-2 of the buffer. Alternatively, the electronic device 100 may store the intermediate values together with the address value of the memory corresponding to the intermediate values in the address value of each buffer. For example, the intermediate value i0w0 and the address value #m-00 corresponding to the intermediate value i0w0 may be stored in the first address value 125-0 of the buffer, the intermediate value i0w1 and the address value #m-01 of the memory corresponding to i0w1 may be stored in the second address value 125-1 of the buffer, and the intermediate value i0w2 and the address value #m-02 of the memory corresponding to the intermediate value i0w2 may be stored in the third address value 125-2 of the buffer.
[0193] After the zeroth step 600 - 1 - 0 of the first stage 600 - 1 is completed, the storage format in the buffer 125 may be as shown in Table 1905 .
[0194] The electronic device 100 may determine whether there is an intermediate value to be additionally accumulated based on the memory address value corresponding to each intermediate value, which may be related to Fig.16 The electronic device 100 may identify the memory address value (#m-00) corresponding to the obtained intermediate value (i0w0), and may determine whether there is an intermediate value to be additionally accumulated in the memory address value (#m-00) corresponding to the obtained intermediate value (i0w0). The electronic device 100 may determine whether there is an additional intermediate value based on the size of the output data and the execution order of the convolution calculation.
[0195] Referring to the matrix 1110 corresponding to the output data of FIG. 11 , the intermediate value corresponding to the memory address value (#m-00) may be only i0w0. Therefore, the electronic device 100 may determine that there is no intermediate value to be added to the memory address value (#m-00) corresponding to the obtained intermediate value (i0w0). If there is no intermediate value to be additionally accumulated in the memory address value (#m-00) corresponding to the obtained intermediate value (i0w0), the electronic device 100 may send the intermediate value (i0w0) (or a partial sum) stored in the first address value 125-0 of the buffer to the memory 110. The memory 110 may store the received intermediate value i0w0 in the memory address value #m-00 corresponding to the intermediate value i0w0.
[0196] Referring to the matrix 1110 corresponding to the output data of FIG. 11 , the intermediate value i0w1 corresponding to the memory address value #m-01 may have an intermediate value i1w0 to be stored additionally. Therefore, the electronic device 100 may determine that there is an intermediate value to be added to the memory address value #m-01 corresponding to the obtained intermediate value i0w1. When there is an intermediate value to be further accumulated in the memory address value #m-01 corresponding to the obtained intermediate value i0w1, the electronic device 100 may continuously store the intermediate value i0w1 stored in the second address value 125-1 of the buffer in the second address value 125-1 of the buffer without sending the intermediate value i0w1 stored in the second address value 125-1 of the buffer to the memory.
[0197] The electronic device 100 may continuously store the intermediate value i0w2 stored in the third address value 125-2 of the buffer to the third address value 125-2 of the buffer without directly transmitting to the memory 110. The detailed description of the intermediate value i0w2 is the same as that of the intermediate value i0w1, and thus a repeated description will be omitted.
[0198] Referring to table 1910 , the electronic device 100 may transmit only the intermediate value i0w0 to the memory 110 after the zeroth step 600 - 1 - 0 of the first stage 600 - 1 , and store the intermediate value i0w0 in the address value #m-00 of the memory corresponding to the intermediate value i0w0 .
[0199] After sending the intermediate value i0w0 to the memory 110 (or after receiving a control signal indicating that the memory 110 has stored the intermediate value i0w0), the electronic device 100 may delete the intermediate value i0w0 stored in the first address value 125-0 of the buffer and the address value #m-00 of the memory corresponding to the intermediate value i0w0 from the first address value 125-0 of the buffer, because the intermediate value i0w0 is not accumulated additionally, it is not necessary to store the value in the buffer 125. After deleting the intermediate value i0w0 and the address value #m-00 of the memory corresponding to the intermediate value i0w0, the method may correspond to the table 1915.
[0200] Fig. 20 An operation of assigning intermediate values obtained from a deconvolution calculation operation to a buffer address and a memory address according to the second embodiment is shown.
[0201] Reference Fig. 20 , the address value of the buffer 115 and the address value of the memory 110 will be used to describe Fig. 9 The operation of the first step 600-1-1 (stage 1-1) of the first stage 600-1 during the process described in Fig. 9 The detailed operation is described in , so redundant description will be omitted. Fig. 20The operations described in Fig.19 The operations described in are continuous, so repeated descriptions are omitted.
[0202] The electronic device 100 may obtain i1w0, i1w1, i1w2 as a plurality of intermediate values 630-1-1 in the first step 600-1-1 of the first stage 600-1. Fig.18 As described in , each intermediate value may obtain a memory address value corresponding to the intermediate value. The value i1w0 may correspond to the second address value (#m-01, 110-1-01) of the memory, i1w1 may correspond to the third address value (#m-02, 110-1-02) of the memory, and i1w2 may correspond to the fourth address value (#m-03, 110-1-03) of the memory.
[0203] The electronic device 100 may store the obtained intermediate values i1w0, i1w1, i1w2 in the buffer 125, and may determine at which address in the buffer 125 the obtained intermediate values i1w0, i1w1, i1w2 are to be stored. Specifically, the electronic device 100 may determine whether information about a memory address value corresponding to the obtained intermediate value has been stored in the buffer 125. Fig.19 , since the intermediate value is not stored in the buffer 125, it is not determined whether the information about the memory address value corresponding to the intermediate value has been stored in the buffer 125. However, if the intermediate value has been accumulated in the buffer 125, the electronic device 100 may determine whether the memory address value corresponding to the newly obtained intermediate value is stored in the buffer. When the memory address value corresponding to the newly obtained intermediate value is stored in the buffer, the electronic device 100 may accumulate the new intermediate value in the buffer storing the corresponding memory address value.
[0204] The electronic device 100 may obtain the intermediate value i1w0 and the memory address value #m-01 corresponding to the intermediate value i1w0, and may determine whether the information of the memory address value #m-01 is stored in the buffer 125. If the information about the memory address value #m-01 exists in the buffer 125, the electronic device 100 may identify the address value #b-01, 125-1 of the buffer storing the memory address value #m-01, and may sum the existing partial sum i0w1 and the new intermediate value i0w1 to the identified address value #b-01, 125-1. The electronic device 100 is able to store the summed new partial sum i0w1+i1w0 in the identified address value #b-01, 125-1 of the buffer.
[0205] Similarly, the electronic device 100 can obtain the intermediate value i1w1 and the memory address value #m-02 corresponding to the intermediate value i1w1. The electronic device 100 can store the summed new partial sum i0w2+i1w1 in the address value #b-02, 125-2 of the identified buffer. Redundant description will be omitted.
[0206] The electronic device 100 can obtain the intermediate value i1w2 and the memory address value #m-03 corresponding to the intermediate value i1w2. Since the memory address value #m-03 corresponding to the intermediate value i1w2 is not stored in the buffer, the electronic device 100 can store the intermediate value i1w2 and the memory address value #m-03 corresponding to the intermediate value i1w2 in the address value of the new buffer or the address value #b-00, 125-0 of the empty buffer.
[0207] Table 2005 shows the final form in which the intermediate values i1w0 , i1w1 , i1w2 are stored in the buffer 125 .
[0208] like Fig.19 As shown, the electronic device 100 may determine whether there is an additional partial sum based on the memory address value corresponding to the intermediate value stored in the buffer, which may be related to Fig.16 Corresponding to step S1670.
[0209] refer to Fig.11b , the partial sum (i0w1+i1w0) corresponding to the memory address value (#m-01) does not have an intermediate value to be additionally accumulated. Therefore, the electronic device 100 may determine that there is no intermediate value to be additionally accumulated in the memory address value #m-01 corresponding to the obtained partial sum i0w1+i1w0. In this case, the electronic device 100 may send the partial sum i0w1+i1w0 stored in the second address value #b-01, 125-1 of the buffer to the memory 110. The memory 110 may store the received partial sum i0w1+i1w0 in the memory address value #m-01 corresponding to the partial sum i0w1+i1w0.
[0210] refer to Fig.11b, the partial sum i0w2+i1w1 corresponding to the memory address value #m-02 may have an intermediate value i2w0 to be stored additionally. Therefore, the electronic device 100 may determine that there is an intermediate value to be additionally accumulated to the memory address value #m-02 corresponding to the obtained partial sum i0w2+i1w1. In this case, the electronic device 100 may not send the partial sum i0w2+i1w1 stored in the third address value #b-02, 125-2 of the buffer to the memory, but may continue to store in the third address value #b-02, 125-2.
[0211] The electronic device 100 may directly and continuously store the intermediate value i1w2 stored in the first address value 125-0 of the buffer to the first address value 125-0 of the buffer without directly sending the intermediate value i1w2 to the memory 110. The detailed description of the intermediate value i1w2 is the same as that of the partial sum i0w2+i1w1, and therefore, the redundant description will be omitted.
[0212] Referring to table 2010, electronic device 100 may send only partial sum i0w1+i1w0 to memory 110 after first step 600-1-1 of first stage 600-1, and memory 110 may store partial sum i0w1+i1w0 in address value #m-01 of the memory corresponding to partial sum i0w1+i1w0.
[0213] After sending the partial sum i0w1+i1w0 to the memory 110 (after receiving a control signal for the memory to store the partial sum i0w1+i1w0), the electronic device 100 may delete the partial sum i0w1+i1w0 stored in the second address value #b-01, 125-1 of the buffer and the memory address value #m-01 corresponding to the partial sum i0w1+i1w0 from the second address value #b-01, 125-1 of the buffer. Since there is no intermediate value to be additionally accumulated, the memory address value #m-01 corresponding to the partial sum i0w1+i1w0 does not need to be stored in the buffer.
[0214] Reference Fig.19 and Fig. 20 , describes that there are three address values of the buffer, but the address value of the buffer may vary depending on the example.
[0215] Specifically, the number of address values of the buffer may vary depending on the size of the output data. Since the number of convolution calculations may be determined based on the size of the output data, the electronic device 100 may obtain the minimum number of buffers or the minimum number of address values of the buffer based on the size of the output data. For example, the minimum buffer in the 5*5 output data may be 10.
[0216] Fig.21 A hardware diagram according to an embodiment is shown.
[0217] exist Fig.21 In the embodiment, the electronic device 100 may include a dynamic random access memory (DRAM) 2105 , a control processor 2110 , and a computing module 2115 .
[0218] The electronic device 100 may process convolution calculations and deconvolution calculations in a MAC processing array (MPA). Therefore, dedicated hardware for each calculation may not be required. Convolution calculations and deconvolution calculations may be processed in the MAC processing array, and nonlinear calculations (e.g., rectified linear unit (ReLU) and element-wise sum (ESum)) may be processed in a post-processing array (PPA) using the results processed in the MAC processing array.
[0219] The electronic device 100 can read the value stored in the DRAM (external memory) through direct memory access (DMA). The data read through DMA can be the input data of the first layer of the deep neural network, the input data of the middle layer, and the weight value.
[0220] The electronic device 100 may read input data, intermediate result data (intermediate values), filter data from the DRAM, and store the input data, intermediate result data (intermediate values), and filter data in a global buffer which is an internal memory (on-chip memory).
[0221] The electronic device 100 may transfer data stored in the global buffer received through DMA to the MAC processing array and the post-processing array using the input buffer and the weight buffer.
[0222] The electronic device 100 can send the data of the calculation operation completed in the MAC processing array and the post-processing array to the output buffer, and the output buffer can store the corresponding data in the global buffer. The data stored in the global buffer can be sent to the DRAM through DMA.
[0223] The control processor 2110 may control the calculation module 2115 based on information required for the calculation (input / output activation size, kernel size, stride, calculation type, etc.) and information related to the DMA operation (data address, size, etc.).
[0224] Fig. 22 Input-activated movement is shown according to an embodiment.
[0225] Fig.23 The movement of weight values according to an embodiment is shown.
[0226] Fig.24The operation of a MAC processing array according to an embodiment is shown.
[0227] exist Fig. 22 In the embodiment, the MAC processing array 2215 is configured as a one-dimensional matrix of MAC processing units for processing MAC calculations for convolution calculations and deconvolution calculations.
[0228] In each MPU, input activation 2205 data may be input from an input buffer 2210, and weight data may be input from a weight buffer. Data sent from the input buffer may be shared with each MPU.
[0229] exist Fig.23 In the example, the weight data 2305 from the weight buffer 2310 may be a different value for each MPU.
[0230] Fig.23 The input activations and weights input to an MPU are shown in . The dimensions of the input activations and weights can be equal. For example, if the size of the input activation (x, y, z) is 100×100×64, then 1×1×64 data can be input to the MPA 2315 a total of 100 times.
[0231] exist Fig.24 In FIG. 2 , assuming that one output is generated in one MPU and n MPUs comprise array 2405, a total of N outputs can be generated at the MPA. Each MPU uses a different weight so that each output can correspond to an output channel in the output activation.
[0232] Fig.25 The unit calculation operation of the MPU according to the first embodiment is shown.
[0233] exist Fig.25 In one MPU, input activation 2505 may be input from an input buffer in the form of 1×1×IN_CH, and a kernel (filter) 2510 may also receive the same size of 1×1×IN_CH from a weight buffer. Two pieces of input data may be multiplied with data corresponding to each other's positions, and the electronic device 100 may obtain output data 2515.
[0234] Fig.26 A unit calculation operation of the MPU according to the second embodiment is shown.
[0235] exist Fig.26In the MPU, as many multiplication results as the number of IN_CH 2605 can be generated, and all multiplication results can be input to the adder tree 2610 inside the MPU. In the adder tree 2610, one output can be generated by adding all multiplication results of the input IN_CH 2605. The main calculations (multiplication and addition) of the MPU can be completed by the above processing. In the case of convolution calculation, the calculation can be 1×1×IN_CH*1×1×IN_CH=1×1×1. The output of the calculation can be stored in the storage space accumulator (ACC) 2615 within the MPA. In the ACC 2615 capable of storing multiple results, the intermediate result value partial sum can be stored, and the accumulation number and accumulation position can be determined according to the calculation type, core size and stride. If the intermediate result value is stored by the previous calculation, the electronic device 100 can store the partial sum calculated by adding the newly obtained intermediate result value. When the final output is stored in the ACC 2615 through several accumulations, all the final outputs stored in the ACC 2615 may be transferred to the post-processing array, and the electronic device 100 may perform a nonlinear operation.
[0236] Fig. 27 The operation of a post-processing array according to an embodiment is shown.
[0237] exist Fig. 27 In the embodiment of the present invention, the post-processing array calculation unit may be part of a processing nonlinear layer that is executed after the MAC calculation. Since the partial sum is calculated in the MAC processing array, the result outputs of the convolution calculation and the deconvolution can be performed jointly by the MAC processing array. The nonlinear calculation may be performed in the post-processing array calculation unit, and may representatively include, for example, ReLU 2705, ESum 2710, and pooling 2715. Similar to the MPA, there may be a post-processing unit (PPU) for calculating the output corresponding to one output channel, and the corresponding units may be arranged in parallel to form an array shape. Accordingly, the array shapes of the MPA and the PPA may be the same. For example, N outputs may be output in parallel at an MPA including N MPUs, and the PPA may also generate N outputs by receiving N inputs. The electronic device 100 may send the final output after the post-processing to the output buffer. The output buffer may issue a write request for the input output to the address of the global buffer set by the control processor. Pooling 2715 may indicate that the size of the convolution layer is adjusted to obtain a new layer.
[0238] Fig.28 Mathematical equations used in convolution calculation or deconvolution calculation are shown.
[0239] Equation 2805 can be used to obtain the intermediate value of the convolution calculation. O^* can be an intermediate result calculated by the MAC calculation of 1×1×IN_CH, I can be the input activation, and W can be the weight (filter data). F_W can be the horizontal size of the filter, and F_H can be the height of the filter. The part corresponding to [][] can be coordinate information. For example, [x+i][y+j], [x][y], and [F_W-i][F_H-j] can refer to coordinate information.
[0240] Equation 2810 can be used to obtain intermediate values of the convolution calculation when the stride is 2 or greater. The electronic device 100 can perform the calculation and store it in the ACC only when the coordinates of O^*[(x+i) / (STR_X)][(y+j) / (STR_Y)] are integers. Here, STR_X can indicate the stride in the x-axis direction, and STR_Y can indicate the stride in the y-axis direction. If the coordinates of O^*[(x+i) / (STR_X)][(y+j) / (STR_Y)] are not integers, the electronic device 100 may not perform the calculation operation and the operation stored in the ACC. As a result, the electronic device 100 can obtain the final output by repetition and accumulation as many times as the filter size.
[0241] Equation 2815 can be used in a deconvolution operation. Specifically, an intermediate result value (O^*[x+STR_X+i][y+STR_Y+j]) can be obtained by multiplying the input activation with the weight.
[0242] Fig.29 A method for controlling an electronic device according to an embodiment is shown.
[0243] Reference Fig.29 , the control method of the electronic device includes: in step S2905, based on the deconvolution calculation input data (input data used in the deconvolution calculation or input data for calculating the deconvolution) being input, calculation information is obtained; in step S2910, based on the obtained calculation information, the size of the output data is obtained; in step S2915, a plurality of memory address values corresponding to the size of the output data are obtained using the address generation module 123; in step S2920, an intermediate value in the convolution calculation process is obtained by performing a convolution calculation based on the calculation information; in step S2925, a memory address value corresponding to the obtained intermediate value among the plurality of obtained memory address values is obtained using the address generation module 123; in step S2930, the obtained intermediate value is stored in the memory address value corresponding to the intermediate value; in step S2935, at least one intermediate value is accumulated based on the memory address value corresponding to the intermediate value using the accumulation calculation module 124; and in step S2940, a deconvolution calculation value for the input data is obtained based on the accumulated at least one intermediate value.
[0244] Storing the obtained intermediate value in a memory address value corresponding to the intermediate value in step S2930 may include: storing the first intermediate value in the memory 110 based on a first memory address value corresponding to the first intermediate value obtained in the convolution calculation process; obtaining a partial sum by accumulating the first intermediate value and the second intermediate value using the accumulation calculation module 124 based on a second memory address value corresponding to the second intermediate value obtained in the convolution calculation process being the same as the first memory address value; and storing the obtained partial sum in the memory 110 based on the first memory address value.
[0245] The calculation information may include the size of the input data, the size of the filter data used in the deconvolution calculation, the size of the stride used for the convolution calculation, and at least one of the calculation order information corresponding to the first intermediate value, and in step S2925, the memory address value corresponding to the obtained intermediate value can be obtained by using the address generation module 123 to obtain the first memory address value based on the calculation information.
[0246] Obtaining the memory address value corresponding to the obtained intermediate value in step S2925 may include: obtaining a first memory address value corresponding to the obtained first intermediate value among the obtained multiple memory address values based on calculation order information corresponding to the first intermediate value.
[0247] Storing the obtained intermediate value to a memory address value corresponding to the intermediate value in step S2930 may include storing the second intermediate value in a second memory address value based on the second memory address value corresponding to the second intermediate value obtained in the convolution calculation process being different from the first memory address value.
[0248] Obtaining the intermediate values in step S2920 may include obtaining a plurality of intermediate values by sequentially performing convolution calculations on the input data using a plurality of weights included in a horizontal axis of the filter data used in the deconvolution calculation.
[0249] Accumulating at least one intermediate value at step S2935 may include accumulating at least a portion of the plurality of intermediate values using the accumulation calculation module 124 , and the amount of accumulation may be determined based on the calculation information.
[0250] Storing the obtained intermediate value to a memory address value corresponding to the intermediate value in step S2930 may include storing the intermediate value in the internal buffer 125 based on the address value of the internal buffer 125 corresponding to the intermediate value, and identifying whether there are additional intermediate values to be accumulated and calculated with the intermediate value, and if it is identified that there are no additional intermediate values, storing the intermediate value stored in the internal buffer 125 in the memory 110.
[0251] The control method may further include determining a size of an internal buffer in which the intermediate values are to be stored based on the calculation information.
[0252] The deconvolution calculation value can be obtained without using the deconvolution calculation module.
[0253] like Fig.29 The method of controlling an electronic device shown can be used in a Figure 1 The method may be executed in an electronic device having a configuration of and may be executed in an electronic device including other configurations.
[0254] The method according to the above-described embodiment may be implemented as an application format installable in an existing electronic device, and may be implemented as a software upgrade or a hardware upgrade for the existing electronic device.
[0255] The above-mentioned embodiments may be performed by an embedded server provided in the electronic device or an external server of at least one electronic device and the display apparatus.
[0256] Embodiments may be implemented in software including instructions stored on a machine-readable storage medium that is readable by a machine (e.g., a computer). A device (including an electronic device such as electronic device A) may call instructions from a storage medium and execute the called instructions. When the instructions are executed by a processor, the processor may perform functions corresponding to the instructions directly or by using other components under the control of the processor. The instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-temporary storage medium (this statement indicates that the storage medium is tangible and does not distinguish between a case where data is semi-permanently stored in the storage medium and a case where data is temporarily stored in the storage medium).
[0257] The method according to the above embodiment may be included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be published online in the form of a machine-readable storage medium (e.g., a compact disc read-only memory (CD-ROM)) or directly published online, or through an application store (e.g., PlayStore). TM ) online or directly publish the computer program product online. In the case of online publishing, at least part of the computer program product can be at least temporarily stored or temporarily generated in a manufacturer's server, an application store's server, or a machine-readable storage medium (such as a memory of a forwarding server).
[0258] Each of the above-mentioned elements (e.g., modules or programs) may include a single entity or multiple entities. At least one element or operation from the above-mentioned corresponding element may be omitted, or at least one other element or operation may be added. Alternatively or additionally, multiple components may be combined to form a single entity. In this case, the integrated entity may perform a function in at least one function of each element of the multiple elements in a manner identical or similar to that performed by the corresponding element in the multiple elements before integration. The module, program module or operation performed by other elements according to the embodiment may be performed continuously, in parallel, repeatedly or heuristically, or at least some operations may be performed according to different orders, may be omitted, or other operations may be added thereto.
[0259] The foregoing embodiments and advantages are merely examples and are not to be construed as limiting the present disclosure. The present disclosure may be easily applied to other types of devices. The description of the embodiments is intended to be illustrative rather than limiting the scope of the claims, and many alternatives, modifications, and variations will be apparent to those skilled in the art.
[0260] While the present disclosure has been particularly shown and described with reference to certain embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the scope of the subject matter defined by the appended claims and their equivalents.
Claims
1. An electronic device, include: Memory; and a processor connected to the memory, wherein the processor is configured to control the electronic device, Wherein, the processor is configured to: Input data based on the deconvolution calculation is input to obtain calculation information; Obtaining the size of output data based on the obtained calculation information; Using an address generation module to obtain a plurality of memory address values corresponding to the size of the output data; Using a convolution calculation module to perform a convolution calculation based on the calculation information to obtain an intermediate value in the convolution calculation process; Using an address generation module, obtain a memory address value corresponding to the obtained intermediate value from among a plurality of obtained memory address values; storing the obtained intermediate value in a memory address value corresponding to the intermediate value; accumulating at least one intermediate value based on a memory address value corresponding to the intermediate value using an accumulation calculation module; and A deconvolution calculation value for the input data is obtained based on the accumulated at least one intermediate value.
2. The electronic device according to claim 1, in, The processor is also configured to: storing the first intermediate value in a memory based on a first memory address value corresponding to the first intermediate value obtained in the convolution calculation process, and Based on the fact that the second memory address value corresponding to the second intermediate value obtained in the convolution calculation process is the same as the first memory address value, a partial sum is obtained by accumulating the first intermediate value and the second intermediate value using an accumulation calculation module, and the obtained partial sum is stored in the memory based on the first memory address value.
3. The electronic device according to claim 2, in, The calculation information includes at least one of a size of input data, a size of filter data used in deconvolution calculation, a size of a stride used in convolution calculation, and calculation order information corresponding to the first intermediate value, and The processor is further configured to: use an address generation module to obtain a first memory address value based on the calculation information.
4. The electronic device according to claim 3, in, The processor is further configured to obtain, based on the calculation order information corresponding to the first intermediate value, a first memory address value corresponding to the obtained first intermediate value from among the plurality of obtained memory address values.
5. The electronic device according to claim 2, in, The processor is further configured to: store the second intermediate value in the second memory address value based on that the second memory address value corresponding to the second intermediate value obtained in the convolution calculation process is different from the first memory address value.
6. The electronic device according to claim 1, in, The processor is further configured to obtain a plurality of intermediate values by sequentially performing convolution calculations of the input data using a plurality of weights included in a horizontal line of filter data used in the deconvolution calculations.
7. The electronic device according to claim 6, in, The processor is further configured to: use the accumulation calculation module to accumulate at least a portion of the plurality of intermediate values, and Wherein, the amount of at least a portion of the accumulation of the plurality of intermediate values is determined based on the calculation information.
8. The electronic device according to claim 1, in, The processor is also configured to: storing the intermediate value in an internal buffer based on an address value of the internal buffer corresponding to the intermediate value, identifying whether there are additional intermediate values to be calculated additively with the intermediate value, and Based on identifying that there are no additional intermediate values, the intermediate values are stored in an internal buffer in memory.
9. The electronic device according to claim 8, in, The processor is further configured to determine, based on the calculation information, a size of an internal buffer in which the intermediate values are to be stored.
10. The electronic device according to claim 1, in, The deconvolution calculation value is obtained without using a deconvolution calculation module.
11. A method for controlling an electronic device, the method include: Input data based on the deconvolution calculation is input to obtain calculation information; Obtaining the size of output data based on the obtained calculation information; Using an address generation module to obtain a plurality of memory address values corresponding to the size of the output data; Using a convolution calculation module to perform a convolution calculation based on the calculation information to obtain an intermediate value in the convolution calculation process; Using an address generation module, obtain a memory address value corresponding to the obtained intermediate value from among a plurality of obtained memory address values; storing the obtained intermediate value in a memory address value corresponding to the intermediate value; Accumulate at least one intermediate value based on a memory address value corresponding to the intermediate value using an accumulation calculation module; as well as A deconvolution calculation value for the input data is obtained based on the accumulated at least one intermediate value.
12. The method according to claim 11, in, The step of storing the obtained intermediate value in the corresponding memory address value comprises: storing the first intermediate value in a memory based on a first memory address value corresponding to the first intermediate value obtained in the convolution calculation process, Based on the fact that the second memory address value corresponding to the second intermediate value obtained in the convolution calculation process is the same as the first memory address value, a partial sum is obtained by accumulating the first intermediate value and the second intermediate value using an accumulation calculation module, and the obtained partial sum is stored in the memory based on the first memory address value.
13. The method according to claim 12, in, The calculation information includes at least one of a size of input data, a size of filter data used in deconvolution calculation, a size of a stride used in convolution calculation, and calculation order information corresponding to the first intermediate value, and The step of obtaining a memory address value corresponding to the obtained intermediate value includes: using an address generation module to obtain a first memory address value based on the calculation information.
14. The method according to claim 13, in, The step of obtaining a memory address value corresponding to the obtained intermediate value includes: obtaining a first memory address value corresponding to the obtained first intermediate value among a plurality of obtained memory address values based on the calculation order information corresponding to the first intermediate value.
15. The method according to claim 12, in, The step of storing the obtained intermediate value in a memory address value corresponding to the intermediate value includes storing the second intermediate value in a second memory address value based on that a second memory address value corresponding to the second intermediate value obtained in the convolution calculation process is different from the first memory address value.
