Approximation algorithm based convolution operation apparatus by using unified multiple constant multiplication circuit and design method of convolution operation apparatus

Abstract

A convolution operation device according to the first aspect of the present invention comprises an input unit for receiving input data, a constant generation circuit for generating a plurality of constants and including one or more nodes corresponding to each of the plurality of constants, a memory capable of storing computer-executable instructions, and a processor for performing a convolution operation based on the input data and the plurality of constants by executing the instructions. In this case, the plurality of constants are determined by approximating each reference constant with a similar value of each reference constant that is less than or equal to a predetermined threshold value among a plurality of predetermined reference constants, and the plurality of reference constants may be values ​​corresponding to a plurality of weights determined through the learning of a neural network.

Description

Technology Field

[0001] The present invention relates to a convolution operation device using an approximation algorithm-based UMCM circuit and a method for designing the same. Background Technology

[0002] The UMCM circuit is a circuit that applies a multiplexer circuit to perform multiplication operations by applying different constant sets depending on the input, given multiple constant sets of size n. The UMCM circuit does not use a multiplier and consists of an adder, a subtractor, and a multiplexer.

[0003] In a general convolution operation device, the operation unit of a DNN (deep neural network) accelerator in an ASIC (application-specific integrated circuit) or FPGA (application-specific integrated circuit) is configured through a PE (processing element), and weights are loaded from DRAM to perform multiplication operations.

[0004] This conventional method of computation has inefficiency in terms of design area and power consumption because access to DRAM is essential during computation and a memory controller is required, even when accelerating small networks such as edge devices or CMOS sensors.

[0005] When applying a UMCM circuit to a convolution unit, the UMCM circuit must be generated by considering the order of constants between multiple sets of constants. Consequently, as the number of sets of constants increases, the complexity of the circuit generation algorithm in considering the sequence can increase. Additionally, the need for a larger number of adders, subtractors, and multiplexers may lead to problems such as increased circuit complexity, design area, and power consumption. Prior art literature

[0006] Japanese Patent Publication No. 2010-134697 (Published June 17, 2010) The problem to be solved

[0007] The problem that the present invention aims to solve is to provide a convolution operation device using an approximation algorithm-based UMCM circuit that is efficient in terms of design area and power, and a method for designing the same.

[0008] However, the problems that the present invention aims to solve are not limited to those mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art to which the present invention pertains from the description below. means of solving the problem

[0009] A convolution operation device according to the first aspect of the present invention comprises an input unit for receiving input data, a constant generation circuit for generating a plurality of constants and including one or more nodes corresponding to each of the plurality of constants, a memory capable of storing computer-executable instructions, and a processor for performing a convolution operation based on the input data and the plurality of constants by executing the instructions. In this case, the plurality of constants are determined by approximating each reference constant with a similar value of each reference constant that is less than or equal to a predetermined threshold value among a plurality of predetermined reference constants, and the plurality of reference constants may be values ​​corresponding to a plurality of weights determined through the learning of a neural network.

[0010] The constant generation circuit may include at least one of an adder, a subtractor, and a shift circuit. In this case, each constant corresponding to one or more nodes may have been calculated based on the constant generation circuit.

[0011] The above constant generation circuit may include a multiplexer that supplies an input value or a result calculated from the input value through an arithmetic element to the next arithmetic element according to an applied signal.

[0012] The plurality of constants may be clustered into one or more sets of constants having a predetermined number of constants generated from the plurality of constants. In this case, the processor may perform the convolution operation by sequentially matching each set of constants to the input data according to a signal applied to the multiplexer.

[0013] Each constant in the above one or more constant sets may have a position index expressed in two dimensions assigned to it. In this case, the position index may include the position index of each constant in the above one or more constant sets, a value corresponding to at least one node corresponding to each constant, the position index of at least one node corresponding to each constant, a value corresponding to a previous node used to calculate each constant, and the position index of the previous node.

[0014] The above one or more sets of constants may be generated by clustering the plurality of constants in correspondence with the output size of the constant generation circuit.

[0015] For a node located after the multiplexer output terminal among the above one or more nodes, a first position index assigned corresponding to each constant included in the first constant set and a second position index assigned corresponding to each constant included in the second constant set may be generated to be identical.

[0016] Among the above one or more nodes, the position index of at least one node corresponding to constants having the same index in each of the above constant sets is the same, and the values ​​corresponding to the at least one node may be different from each other.

[0017] The above-mentioned previous node is a node located prior to the multiplexer input, and the position indices of the previous nodes used to calculate constants having the same index in each set of constants and the values ​​corresponding to the previous nodes may be generated to be different from each other.

[0018] The above one or more nodes may be generated based on an algorithm that determines the plurality of constants using the least amount of adders and subtractors included in the constant generation circuit.

[0019] A method for designing a convolution operation device according to another embodiment of the second aspect of the present invention comprises the steps of receiving a plurality of constants, clustering into one or more sets of constants having a predetermined number of constants generated from the plurality of constants, and generating a constant generation circuit comprising one or more nodes corresponding to each of the plurality of constants. In this case, the plurality of constants may be determined by approximating each reference constant with a similar value of each reference constant that is less than or equal to a predetermined threshold value, among a predetermined plurality of reference constants. Additionally, the plurality of reference constants may be values ​​corresponding to a predetermined plurality of weights determined through the learning of a neural network.

[0020] A computer-readable recording medium storing a computer-executable instruction according to a third aspect of the present invention, wherein the computer-executable instruction comprises, when executed by a processor, a step of receiving a plurality of constants, a step of clustering into one or more sets of constants having a predetermined number of constants generated from the plurality of constants, and a step of generating a constant generation circuit including one or more nodes corresponding to each of the plurality of constants, wherein the plurality of constants are determined by approximating each reference constant with a similar value of each reference constant that is less than or equal to a predetermined threshold, and wherein the plurality of reference constants are values ​​corresponding to a plurality of weights determined through the learning of a neural network.

[0021] A computer program stored in a computer-readable recording medium according to a fourth aspect of the present invention, wherein the computer program, when executed by a processor, comprises the steps of receiving a plurality of constants, clustering into one or more sets of constants having a predetermined number of constants generated from the plurality of constants, and generating a constant generation circuit comprising one or more nodes corresponding to each of the plurality of constants, wherein the plurality of constants are determined by approximating each reference constant with a similar value of each reference constant that is less than or equal to a predetermined threshold value, and wherein the plurality of reference constants are values ​​corresponding to a plurality of weights determined through the learning of a neural network. The computer program includes instructions for the processor to perform a method. Effects of the invention

[0022] According to the present invention, by applying an approximation algorithm, a convolution operation device using a UMCM circuit that is efficient in terms of design area and power, and a method for designing the same can be provided.

[0023] The effects obtainable from the present invention are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art to which the present disclosure belongs from the description below. Brief explanation of the drawing

[0024] FIG. 1 is a block diagram illustrating an exemplary convolution operation device according to a first aspect of the present invention. FIG. 2 is a block diagram illustrating an exemplary computing device capable of designing a convolution operation device according to the first aspect of the present invention. FIG. 3 is a flowchart exemplarily showing a method for designing a convolution operation device according to the second aspect. Figure 4 is a block diagram exemplifying the function of a convolution operation program. Figure 5 is a block diagram exemplifying the functions of a convolution operation unit design program. Figure 6 is an example diagram showing a UMCM circuit. Figure 7 is an example diagram illustrating a convolution operation method using a UMCM circuit. Figure 8 is an example diagram illustrating a method for designing a convolution operation device using a UMCM circuit. FIG. 9 is an exemplary diagram showing a convolution operation device designed according to the present invention. FIG. 10 is an example diagram showing a method of generating nodes of a constant generation circuit by applying an approximation algorithm according to the present invention. Specific details for implementing the invention

[0025] The advantages and features of the present invention and the methods for achieving them will become clear by referring to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below but can be implemented in various different forms. These embodiments are provided merely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention, and the present invention is defined only by the scope of the claims.

[0026] In describing the embodiments of the present invention, specific descriptions of known functions or configurations will be omitted if it is determined that such detailed descriptions could unnecessarily obscure the essence of the invention. Furthermore, the terms described below are defined in consideration of their functions in the embodiments of the present invention, and these definitions may vary depending on the intentions or practices of the user or operator. Therefore, such definitions should be based on the content throughout this specification.

[0027] The terms used in this specification will be briefly explained, and the invention will be described in detail.

[0028] The terms used in this specification have been selected to be as widely used as possible, taking into account the functions of the present invention; however, these terms may vary depending on the intent of those skilled in the art, case law, the emergence of new technologies, etc. Additionally, in specific cases, terms have been arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the relevant description of the invention. Therefore, the terms used in this invention should be defined not merely by their names, but based on their meanings and the overall content of the invention.

[0029] When a part of a specification is described as 'comprising' a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0030] Additionally, the term "part" as used in the specification refers to software or hardware components, such as FPGAs or ASICs, and the "part" performs certain roles. However, the meaning of "part" is not limited to software or hardware. The "part" may be configured to reside in an addressable storage medium or configured to run one or more processors. Thus, by example, the "part" includes components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functions provided within the components and "parts" may be combined into a smaller number of components and "parts" or further separated into additional components and "parts."

[0031] Below, embodiments of the present invention are described in detail with reference to the attached drawings so that those skilled in the art can easily implement the present invention.

[0032] FIG. 1 is a block diagram illustrating an exemplary convolution operation device according to a first aspect of the present invention.

[0033] As shown in FIG. 1, the convolution operation device (100) may include an input unit (110), an output unit (120), a processor (130), a memory (140), a constant generation circuit (160), and a communication unit (170).

[0034] For convenience of explanation, the following description describes an example in which the convolution operation device (100) includes an input unit (110), an output unit (120), a processor (130), a memory (140), a constant generation circuit (160), and a communication unit (170), but is not limited thereto. That is, each unit configuration can interact with the convolution operation device (100) from outside the convolution operation device (100).

[0035] The input unit (110) may be a hardware device capable of directly receiving commands, information, etc. used to control the convolution operation device (100) through a user interface (e.g., keyboard, mouse input, etc.).

[0036] In one embodiment, the input unit (110) can receive information required for a convolution operation from a user. Specifically, the user can input data required for a convolution operation, such as input data and constants, through the input unit (110).

[0037] The input data may include a feature map, activation, and operand data.

[0038] Constants may refer to data that undergoes convolution operations with the input data, such as weights, but are not limited thereto.

[0039] The output unit (120) can provide information including information required for convolution operations and the results of convolution operations to the user as visual information through an interface or display device.

[0040] The processor (130) can control the overall operation of the convolution operation device (100) to carry out the present invention.

[0041] The processor (130) can load the convolution operation program (150) and information necessary for the execution of the convolution operation program (150) from memory (140) in order to execute the convolution operation program (150).

[0042] The processor (130) can control the storage of data received from an external device through the communication unit (170) in the memory (140). Additionally, the processor (130) can control the transmission of information including information required for a convolution operation and the result of a convolution operation to an external device through the communication unit (170).

[0043] The processor (130) may refer to a processing device such as a microprocessor, a central processing unit (CPU), a graphic processing unit (GPU), a processor core, a multiprocessor, an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or a micro controller unit (MCU), but is not limited to the embodiments described above.

[0044] The memory (140) can store information necessary for the execution of the convolution operation program (150) and the convolution operation program (150). Additionally, the memory (140) can store processing results by the processor (130).

[0045] The convolution operation program (150) may mean software that includes instructions programmed to perform the method according to the present invention.

[0046] The memory (140) can store information including information required for a convolution operation and the result of a convolution operation. Additionally, the memory (140) can store information received from an external device through the communication unit (170).

[0047] Memory (140) may refer to a computer-readable recording medium such as a magnetic medium such as a hard disk, floppy disk, and magnetic tape, an optical recording medium such as a CD-ROM and DVD, a magneto-optical medium such as a floptical disk, and a hardware device specifically configured to store and execute program instructions such as a flash memory, but is not limited to the embodiments described above.

[0048] The constant generation circuit (160) may include at least one operation element among an adder, a subtractor, and a shift circuit.

[0049] The constant generation circuit (160) can generate multiple constants required for convolution operations using an arithmetic element.

[0050] The constant generation circuit (160) may include one or more nodes corresponding to each of the plurality of constants.

[0051] The constant generation circuit (160) may include a multiplexer that supplies an input value or a calculation result by an operation element for an input value to the next operation element according to an applied signal.

[0052] The communication unit (170) may be a wireless communication module capable of performing wireless communication by adopting a communication method such as CDMA, GSM, W-CDMA, TD-SCDMA, WiBro, LTE, EPC, 5G, wireless LAN, Wi-Fi, Bluetooth, Zigbee, WFD (Wi-Fi Direct), UWB (Ultra Wide Band), infrared communication (IrDA; infrared data association), BLE (Bluetooth Low Energy) or NFC (Near Field Communication), but is not limited to the above-described embodiment.

[0053] In addition, information input and output through the input unit (110) and output unit (120), information stored in the memory (140), and information transmitted and received through the communication unit (170) include all information related to the present invention and are not limited to the embodiments described above.

[0054] The function or operation of the convolution operation program (150) will be examined in detail through FIG. 4.

[0055] FIG. 2 is a block diagram illustrating an exemplary computing device capable of designing a convolution operation device according to the first aspect of the present invention.

[0056] As shown in FIG. 2, the convolution operation device design device (200) may include an input unit (210), an output unit (220), a processor (230), a memory (240), and a communication unit (260).

[0057] Among the input unit (210), output unit (220), processor (230), memory (240), and communication unit (260) shown in FIG. 2, the parts identical to the input unit (110), output unit (120), processor (130), memory (140), and communication unit (170) of FIG. 1 will be omitted from the description.

[0058] For convenience of explanation, the following description uses an example in which the convolution operation device design device (200) includes an input unit (210), an output unit (220), a processor (230), a memory (240), and a communication unit (260), but is not limited thereto. That is, each unit configuration can interact with the convolution operation device design device (200) from outside the convolution operation device design device (200).

[0059] A convolution operation device design device (200) can perform a convolution operation device design method comprising the steps of receiving a plurality of constants, clustering into one or more constant sets having a predetermined number of constants generated from the plurality of constants, and generating a constant generation circuit including one or more nodes corresponding to each of the plurality of constants. At this time, the plurality of constants may be determined by approximating each reference constant with a similar value of each reference constant that is less than or equal to a predetermined threshold value among a plurality of predetermined reference constants. Additionally, the plurality of reference constants may be values ​​corresponding to a plurality of weights determined through the learning of a neural network.

[0060] The input unit (210) may be a hardware device capable of directly receiving commands, information, etc. used to control the convolution operation device design device (200) through a user interface (e.g., keyboard, mouse input, etc.).

[0061] In one embodiment, the input unit (210) can receive information necessary for designing a convolution operation device from a user. Specifically, the user can input data through the input unit (210), including input data necessary for convolution operation, data such as constants, components necessary for a constant generation circuit (160), information related to an operation element, and information related to an approximation algorithm.

[0062] The output unit (220) can provide information including information necessary for designing a convolution operation device, information related to an approximation algorithm, and the results of designing a convolution operation device to the user as visual information through an interface or display device.

[0063] The processor (230) can control the overall operation of the convolution operation device design device (200) to carry out the present invention.

[0064] The processor (230) can load the convolutional arithmetic unit design program (250) and information necessary for the execution of the convolutional arithmetic unit design program (250) from memory (240) in order to execute the convolutional arithmetic unit design program (250).

[0065] The processor (230) can control the storage of data received from an external device through the communication unit (260) in the memory (240). Additionally, the processor (230) can control the transmission of information including information required for the design of a convolution operation device, information related to an approximation algorithm, and the design results of the convolution operation device to an external device through the communication unit (260).

[0066] The memory (240) can store information necessary for the execution of the convolutional computing device design program (250) and the convolutional computing device design program (250). Additionally, the memory (240) can store processing results by the processor (230).

[0067] The convolution operation device design program (250) may mean software that includes instructions programmed to perform the method according to the present invention.

[0068] The memory (240) can store information including information required for designing a convolution operation device, information related to approximation algorithms, and the results of the convolution operation device design. Additionally, the memory (240) can store information received from an external device through the communication unit (260).

[0069] In addition, information input and output through the input unit (210) and output unit (220), information stored in the memory (240), and information transmitted and received through the communication unit (260) include all information related to the present invention and are not limited to the embodiments described above.

[0070] The function or operation of the convolution operation device design program (250) will be examined in detail through FIG. 5.

[0071] FIG. 3 is a flowchart exemplarily showing a method for designing a convolution operation device according to the second aspect.

[0072] As shown in FIG. 3, a method for designing a convolution operation device according to a second aspect of the present invention comprises the steps of receiving a plurality of constants (S300), clustering into one or more sets of constants having a predetermined number of constants generated from the plurality of constants (S310), and generating a constant generation circuit including one or more nodes corresponding to each of the plurality of constants (S320). At this time, the plurality of constants may be determined by approximating each reference constant with a similar value of each reference constant that is less than or equal to a predetermined threshold value among a plurality of predetermined reference constants. Additionally, the plurality of reference constants may be values ​​corresponding to a plurality of weights determined through the learning of a neural network.

[0073] Multiple constants may refer to data that undergoes convolution operations with the input data, such as weights, but are not limited thereto.

[0074] A constant generation circuit is a common design for accelerating convolution layers, consisting of a structure that constructs a multiplier array and then adds values ​​through an adder-tree block; the multiplier array can refer to a multiplier block composed of different parameters that share the same input.

[0075] That is, the constant generation circuit may refer to a multiplier block for producing different output data corresponding to output channels by performing multiplication operations on input data and multiple constants and adding the results of each multiplication operation.

[0076] The output size of a constant generation circuit can refer to the number of output data outputs at once through the constant generation circuit.

[0077] A constant set includes some of a plurality of constants, and the number of constants included in the constant set can correspond to the output size of the constant generation circuit.

[0078] A multiplexer can refer to a circuit that performs convolution operations by sequentially matching each set of constants to input data according to the applied signal.

[0079] Accordingly, if one or more constant generation circuits are grouped according to the size of the multiplexer included in the constant generation circuit, the convolution operation device can perform different operations by different constant generation circuits depending on the signal applied to the multiplexer.

[0080] According to the above operation, a convolution operation unit can be generated based on one or more grouped constant generation circuits and one or more sets of constants.

[0081] Figure 4 is a block diagram exemplifying the function of a convolution operation program.

[0082] As shown in FIG. 4, the convolution operation program (150) may include an input unit (410), a constant generation unit (420), a multiplexer control unit (430), and a convolution operation execution unit (440). The input unit (410), the constant generation unit (420), the multiplexer control unit (430), and the convolution operation execution unit (440) are exemplary divisions of the functions of the convolution operation program (150) and are not limited thereto.

[0083] According to the embodiment, the functions of the input unit (410), constant generation unit (420), multiplexer control unit (430), and convolution operation execution unit (440) can be merged / separated and can be implemented as a series of instructions included in a single program.

[0084] The input unit (410), constant generation unit (420), multiplexer control unit (430), and convolution operation execution unit (440) may be implemented by a processor (130) and may refer to a data processing device embedded in hardware having a physically structured circuit to perform a function expressed by a code or instruction included in a convolution operation program (150) stored in memory (140).

[0085] The input unit (410) can receive input data required for convolution operations.

[0086] The input data may include a feature map, activation, and operand data.

[0087] The constant generation unit (420) can generate multiple constants required for convolution operations.

[0088] Constants may refer to data that undergoes convolution operations with the input data, such as weights, but are not limited thereto.

[0089] A plurality of constants may be determined by approximating each reference constant with a similar value of each reference constant that is less than or equal to a predetermined threshold among a plurality of reference constants. The method of approximating with similar values ​​will be described in detail in FIG. 10.

[0090] Multiple reference constants may be values ​​corresponding to multiple weights that have been determined through the learning of the neural network.

[0091] A plurality of constants may be clustered into one or more sets of constants having a predetermined number of constants. In this case, the predetermined number may be the same as the number of output channels of the convolution operation device.

[0092] One or more sets of constants may be generated by clustering multiple constants in correspondence with the output size of the constant generation circuit.

[0093] Multiple constants may be values ​​corresponding to multiple weight values ​​that have been determined through the learning of a neural network. In this case, the set of one or more constants may have constants corresponding to a predetermined number of weight values.

[0094] Each constant in one or more sets of constants may have a position index expressed in two dimensions assigned to it.

[0095] The location index may include the location index of each constant within one or more sets of constants, a value corresponding to at least one node corresponding to each constant, the location index of at least one node corresponding to each constant, a value corresponding to a previous node used to calculate each constant, and the location index of the previous node.

[0096] Specifically, l nth constant set Regarding, constant The constant that is the previous node and constants If calculated from, constant The position index corresponding to is It can appear as follows. Accordingly, constant set The position index for is a constant set It may include a value corresponding to each constant included in, a position index corresponding to each constant, a previous constant used to calculate each constant, and a position index of the previous constant. In this case, in the constant generation circuit, each constant may be represented as a node corresponding to each constant.

[0097] constant set The position index is It can be represented as follows. Constant set When there is, A circuit satisfying [condition] can be generated as a UMCM circuit. That is, for a single constant generation circuit, a constant generation circuit can be generated by applying a predetermined algorithm so that all sets of constants can be generated.

[0098] At this time, one or more nodes may be generated based on an algorithm that determines multiple constants using the least amount of adders and subtractors included in the arithmetic element.

[0099] Among one or more nodes included in the constant generation circuit, a node located after the multiplexer output terminal may be generated such that a first position index assigned corresponding to each constant included in the first constant set and a second position index assigned corresponding to each constant included in the second constant set are identical.

[0100] Among one or more nodes, the position index of at least one node corresponding to constants having the same index in each constant set may be the same, and the value corresponding to at least one node may be different. That is, the position index of at least one node corresponding to constants having the same index in each constant set is the same, but as different constant sets are applied by a multiplexer, the value corresponding to at least one node corresponding to different constant sets may be different.

[0101] The previous node used to calculate each constant is a node located before the multiplexer input, and the position indices of the previous nodes used to calculate constants having the same index in each set of constants and the values ​​corresponding to the previous nodes may be generated to be different from each other.

[0102] One or more nodes may be generated based on an algorithm that determines multiple constants using the fewest number of adders and subtractors included in the constant generation circuit. In this case, the multiple constants may be determined by approximating each reference constant with a similar value of each reference constant that is less than or equal to a predetermined threshold among multiple reference constants. Therefore, when determining one or more nodes corresponding to each constant included in the multiple constants using adders and subtractors, the constant generation circuit can be designed using the fewest number of adders and subtractors by applying an algorithm that approximates each reference constant with a similar value of each reference constant that is less than or equal to a predetermined threshold.

[0103] The multiplexer control unit (430) can control the multiplexer to supply an input value or a calculation result by a calculation element for an input value to the next calculation element according to the applied signal.

[0104] The operation element may include at least one of an adder, a subtractor, and a shift circuit.

[0105] Each constant corresponding to one or more nodes may have already been calculated by the constant generation unit (420).

[0106] The convolution operation execution unit (440) can perform a convolution operation based on input data and a plurality of constants.

[0107] The convolution operation execution unit (440) can perform convolution operations by sequentially matching each set of constants to the input data according to the signal applied to the multiplexer.

[0108] Figure 5 is a block diagram exemplifying the functions of a convolution operation unit design program.

[0109] As shown in FIG. 5, the convolution operation device design program (250) may include an input unit (510), a constant clustering unit (520), an approximation algorithm application unit (530), and a constant generation circuit generation unit (540). The input unit (510), the constant clustering unit (520), the approximation algorithm application unit (530), and the constant generation circuit generation unit (540) are exemplary divisions of the functions of the convolution operation device generation program (250) and are not limited thereto.

[0110] According to the embodiment, the functions of the input unit (510), the constant clustering unit (520), the approximation algorithm application unit (530), and the constant generation circuit generation unit (540) can be merged / separated and can be implemented as a series of instructions included in at least one program.

[0111] The input unit (510), constant clustering unit (520), approximation algorithm application unit (530), and constant generation circuit generation unit (540) may be implemented by a processor (230) and may refer to a data processing device embedded in hardware having a physically structured circuit to perform a function expressed by a code or instruction included in a convolution operation device design program (250) stored in memory (240).

[0112] The input unit (510) can receive multiple constants.

[0113] Constants may refer to data that undergoes convolution operations with the input data, such as weights, but are not limited thereto.

[0114] The constant clustering unit (520) can cluster a plurality of constants in correspondence with the output size of one or more constant generation circuits that calculate a constant using at least one of addition, subtraction, and shift operations.

[0115] The constant clustering unit (520) can generate one or more sets of constants having a predetermined number of constants from a plurality of constants.

[0116] The constant clustering unit (520) can assign one or more sets of constants to one or more constant generation circuits, respectively.

[0117] The constant clustering unit (520) can group one or more constant generation circuits corresponding to the size of the multiplexer included in the constant generation circuit.

[0118] The approximation algorithm application unit (530) can apply an approximation algorithm when determining one or more nodes corresponding to each of the multiple constants included in the constant generation circuit.

[0119] If the approximation algorithm is not applied, adders, subtractors, and shift operators must be placed within the constant generation circuit so that the same value as each constant included in multiple constants is produced.

[0120] In contrast, when an approximation algorithm is applied, adders, subtractors, and shift operators can be arranged within the constant generation circuit so that a similar value between each reference constant corresponding to each constant included in multiple constants and each reference constant below a predetermined threshold is calculated. Accordingly, the design area of ​​the constant generation circuit can be reduced by determining one or more nodes corresponding to multiple constants using fewer computational elements.

[0121] A constant generation circuit generation unit (540) can generate a plurality of constants and generate a constant generation circuit including one or more nodes corresponding to each of the plurality of constants.

[0122] Figure 6 is an example diagram showing a UMCM circuit.

[0123] A UMCM circuit can refer to a circuit that applies a multiplexer circuit to perform multiplication operations by applying different constant sets depending on the input when multiple constant sets of size n are given. A UMCM circuit can be composed of an adder, a subtractor, and a multiplexer without using a multiplier.

[0124] Specifically, a constant set containing n constants , a set of multiple constants including this Given input data x and a constant i from which a set of constants can be selected, there is a corresponding set of constants. For all constants included in, the result of performing a multiplication operation with the input data x can be obtained.

[0125] Figure 7 is an example diagram illustrating a convolution operation method using a UMCM circuit.

[0126] The output feature map cube COХHOХWO can be obtained by multiplying the input feature map cube CIХHIХWI and the weight tensors of CoХCiХKyХKx.

[0127] In this case, if the batch size is 1, it can be performed as a 6-dimensional nested loop operation.

[0128] As shown in lines 6-7 of FIG. 7, a single input feature map can be multiplied with different CO weights. Accordingly, when the size of each constant generation circuit is n, CO weights are assigned to CO / n constant generation blocks, and the entire weight tensor can be assigned to a total of CoХCiХKyХKx / n constant generation circuits.

[0129] When the number of constant sets to be assigned to one UMCM circuit is G, a total of CoХCiХKyХKx / (nХG) UMCM circuits may be required.

[0130] Figure 8 is an example diagram illustrating a method for designing a convolution operation device using a UMCM circuit.

[0131] The method for designing a convolution operation device according to the present invention can be specifically carried out in the following order.

[0132] 1. Input the weights of the pre-trained model

[0133] 2. Cluster each weight according to n, the size of the constant generation circuit

[0134] 3. Grouping constant generation circuits to create UMCM circuits

[0135] 4. Generate UMCM circuits for each UMCM group

[0136] 5. Convert UMCM circuit to Verilog code

[0137] FIG. 9 is an exemplary diagram showing a convolution operation device designed according to the present invention.

[0138] Figure 9 (a) shows that x is input as input data, and weights t1, t2, t3, and t4 are corresponding to it, and a convolution operation is performed.

[0139] FIG. 9 (b) shows a multiple constant multiplication (MCM) circuit in which a convolution operation is performed with weights t1, t2, t3, t4 generated through a constant generation circuit comprising an arithmetic element composed of an adder, a subtractor, and a shift circuit for input data x. The MCM circuit can perform multiplication operations on sets of constants through only addition, subtraction, and bit shifting via resource sharing. In this case, the set of constants may be {17, 9, 31, 15}.

[0140] In FIG. 9 (b), the node corresponding to t1 may be 17x, the node corresponding to t2 may be 9x, the node corresponding to t3 may be 31x, and the node corresponding to t4 may be 15x. At this time, the node corresponding to t2, 9x, may be used to calculate the node corresponding to t1, 17x. Also, the node corresponding to t4, 15x, may be used to calculate the node corresponding to t3, 31x.

[0141] In Fig. 9 (b), four multiplication operations can be replaced with four adders. This enables efficient multiplication operations for a fixed set of constants.

[0142] Figure 9 (c) shows a UMCM (unified multiple constant multiplication) circuit with a multiplexer applied to an MCM circuit. While the MCM circuit has a one-to-one mapping between the constants included in the constant set and the output, the UMCM circuit can be configured as a circuit that integrates multiple constant sets through a multiplexer.

[0143] Specifically, unlike the MCM circuit of FIG. 9(b), which required four nodes to generate t1, t2, t3, and t4, the UMCM circuit of FIG. 9(c) can generate t1, t2, t3, and t4 using two nodes by using a multiplexer. In this case, two sets of constants {17,9} and {31,15} can be applied depending on the signal applied to the multiplexer. Accordingly, the design area can be efficiently reduced.

[0144] FIG. 9 (d) shows a UMCM circuit to which an approximation algorithm is applied according to the present invention.

[0145] Since the input set of constants is {17, 9, 31, 15}, in order to perform multiplication operations between the input data x and each constant included in the set of constants, {17, 9} and {31, 15} must be output through the constant generation circuit. Therefore, as shown in FIG. 9 (c), additional operation elements may be required.

[0146] In this case, by applying an approximation algorithm to approximate the constant set {17, 9} to {18, 9} and {31, 15} to {30, 15}, the number of computational elements can be reduced, allowing for efficient computation.

[0147] FIG. 10 is an exemplary diagram showing a method of generating nodes of a constant generation circuit by applying an approximation algorithm according to the present invention.

[0148] FIG. 10 includes three nodes corresponding to {15, 9, 1} included in a constant generation circuit. The case of generating a node corresponding to {35} using an operation element from, and including 3 nodes corresponding to {29, 15, 1} We will explain the case assuming that a node corresponding to {63} is generated using an operation element from.

[0149] In the top of FIG. 10 (a), if the approximation algorithm is not applied, {30} can be generated by performing a shift operation once on {15}, {4} can be generated by performing a shift operation twice on {1}, {5} can be generated by performing an addition operation on {4} and {1}, and {35} can be generated by performing an addition operation on {30} and {5}. In this case, two shift operators and two adders may be required.

[0150] At the bottom of FIG. 10 (a), if the approximation algorithm is not applied, {58} can be generated by performing a shift operation once on {29}, {5} by performing a shift operation twice on {1}, and {63} can be generated by performing an addition operation on {58} and {5}. In this case, two shift operators and one adder may be required.

[0151] At the top of FIG. 10 (b), if the approximation algorithm is not applied, {36} can be generated by performing a shift operation twice on {9}, and {35} can be generated by performing a subtraction operation on {36} and {1}. In this case, one shift operator and one subtractor may be required. At this time, at the top of FIG. 10 (c), if the approximation algorithm is applied and the preset threshold is 1, node generation can be terminated by performing a shift operation twice on {9} to generate {36} and approximating {35} to {36}. In this case, only one shift operator may be required.

[0152] At the bottom of FIG. 10 (b), when the approximation algorithm is applied and the preset threshold is 1, {60} can be generated by performing a shift operation twice on {15}, {2} can be generated by performing a shift operation once on {1}, and {62} can be generated by performing an addition operation on {60} and {2}. At this time, node generation can be terminated by approximating {63} to {62}. In this case, two shift operators and one adder may be required.

[0153] At the bottom of Fig. 10 (c), when the approximation algorithm is applied and the preset threshold is 4, the node generation can be terminated by performing a shift operation twice on {15} to generate {60} and approximating {63} to {60}. In this case, only one shift operator may be required.

[0154] In this way, when generating a UMCM circuit by applying an approximation algorithm, a constant generation circuit that efficiently calculates multiple constants using fewer computational elements can be generated.

[0155] As described above, according to the present invention, by applying an approximation algorithm, a convolution operation device using a UMCM circuit that is efficient in terms of design area and power, and a method for designing the same can be provided.

[0156] The embodiments of the present invention described above may be implemented through various means. For example, the embodiments of the present invention may be implemented by hardware, firmware, software, or a combination thereof.

[0157] Combinations of each block of the block diagram attached to the present invention and each step of the flowchart may be performed by computer program instructions. Since these computer program instructions may be loaded into an encoding processor of a general-purpose computer, a special-purpose computer, or other programmable data processing equipment, the instructions performed through the encoding processor of the computer or other programmable data processing equipment create means for performing the functions described in each block of the block diagram or each step of the flowchart. Since these computer program instructions may also be stored in computer-available or computer-readable memory that can be directed toward the computer or other programmable data processing equipment to implement the function in a specific way, the instructions stored in computer-available or computer-readable memory may also produce a manufactured item containing instruction means for performing the function described in each block of the block diagram or each step of the flowchart. Since computer program instructions can be loaded onto a computer or other programmable data processing equipment, instructions that execute a computer or other programmable data processing equipment by performing a series of operation steps on the computer or other programmable data processing equipment to create a process executed by the computer can also provide steps for executing the functions described in each block of the block diagram and each step of the flowchart.

[0158] Additionally, each block or each step may represent a module, segment, or part of code containing one or more executable instructions for executing a specified logical function(s). In some embodiments, the functions mentioned in the blocks or steps may occur out of order. For example, two blocks or steps depicted consecutively may actually be performed substantially simultaneously, or the blocks or steps may be performed in reverse order according to the corresponding function.

[0159] The above description is merely an illustrative explanation of the technical concept of the present invention, and those skilled in the art to which the present invention pertains will be able to make various modifications and variations within the scope of the essential quality of the present invention. Accordingly, the embodiments disclosed in the present invention are intended to explain, not limit, the technical concept of the present invention, and the scope of the technical concept of the present invention is not limited by such embodiments. The scope of protection of the present invention shall be interpreted by the claims below, and all technical concepts within the equivalent scope shall be interpreted as being included within the scope of rights of the present invention. Explanation of the symbols

[0160] 100: Convolutional Unit 150: Convolution operation program 200: Convolutional Operation Unit Design Unit 250: Convolutional Unit Design Program

Claims

Claim 1 A convolution operation device comprising: an input unit for receiving input data; a constant generation circuit for generating a plurality of constants and including one or more nodes corresponding to each of the plurality of constants; a memory capable of storing computer-executable instructions; and a processor for performing a convolution operation based on the input data and the plurality of constants by executing the instructions, wherein the plurality of constants are determined by approximating each reference constant with a similar value of each reference constant that is less than or equal to a predetermined threshold value, and the plurality of reference constants are values ​​corresponding to a plurality of weights determined through neural network learning. Claim 2 A convolution operation device according to claim 1, wherein the constant generation circuit comprises at least one of an adder, a subtractor, and a shift circuit, and each constant corresponding to one or more nodes is calculated based on the constant generation circuit. Claim 3 In claim 1, the constant generation circuit comprises a multiplexer that supplies an input value or a result of the input value being computed through an arithmetic element to the next arithmetic element according to an applied signal, convolutional computing device. Claim 4 In paragraph 3, the plurality of constants are clustered into one or more sets of constants having a predetermined number of constants generated from the plurality of constants, and the processor is a convolution operation device that performs the convolution operation by sequentially matching each set of constants to the input data according to a signal applied to the multiplexer. Claim 5 A convolution operation device according to claim 4, wherein each constant in one or more constant sets is assigned a position index expressed in two dimensions, and the position index includes a position index of each constant in one or more constant sets, a value corresponding to at least one node corresponding to each constant, a position index of at least one node corresponding to each constant, a value corresponding to a previous node used to calculate each constant, and a position index of the previous node. Claim 6 A convolution operation device according to claim 5, wherein the one or more sets of constants are generated by clustering the plurality of constants in correspondence with the output size of the constant generation circuit. Claim 7 A convolution operation device according to claim 5, wherein for a node located after the multiplexer output terminal among the one or more nodes, a first position index assigned corresponding to each constant included in a first constant set and a second position index assigned corresponding to each constant included in a second constant set are generated such that they are identical. Claim 8 A convolution operation device according to claim 5, wherein the position index of at least one node corresponding to constants having the same index in each constant set among the one or more nodes is the same, and the values ​​corresponding to the at least one node are different from each other. Claim 9 A convolution operation device according to claim 5, wherein the previous node is a node located prior to the multiplexer input terminal, and the position index of the previous nodes used to calculate constants having the same index in each constant set and the value corresponding to the previous nodes are generated to be different from each other. Claim 10 A convolution operation device according to claim 1, wherein the one or more nodes are generated based on an algorithm that determines the plurality of constants using the least amount of adders and subtractors included in the constant generation circuit. Claim 11 A method for designing a convolution operation device performed by a computing device, comprising: receiving a plurality of constants; clustering into one or more sets of constants having a predetermined number of constants generated from the plurality of constants; and generating a constant generation circuit including one or more nodes corresponding to each of the plurality of constants, wherein the plurality of constants are determined by approximating each reference constant with a similar value of each reference constant that is less than or equal to a predetermined threshold value, and the plurality of reference constants are values ​​corresponding to a plurality of weights determined through the learning of a neural network. Claim 12 A computer-readable recording medium storing computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, include the steps of: receiving a plurality of constants; clustering into one or more sets of constants having a predetermined number of constants generated from the plurality of constants; and generating a constant generation circuit that generates the plurality of constants and includes one or more nodes corresponding to each of the plurality of constants, wherein the plurality of constants are determined by approximating each reference constant with a similar value of each reference constant that is less than or equal to a predetermined threshold value, and wherein the plurality of reference constants are values ​​corresponding to a plurality of weights determined through the learning of a neural network. Claim 13 A computer program stored on a computer-readable recording medium, wherein the computer program, when executed by a processor, comprises: a step of receiving a plurality of constants; a step of clustering into one or more sets of constants having a predetermined number of constants generated from the plurality of constants; and a step of generating a constant generation circuit comprising one or more nodes corresponding to each of the plurality of constants, wherein the plurality of constants are determined by approximating each reference constant with a similar value of each reference constant that is less than or equal to a predetermined threshold value, and wherein the plurality of reference constants are values ​​corresponding to a plurality of weights determined through learning of a neural network.

Images