Convolution operation device using umcm circuit and design method therefor

Abstract

A convolution operation device, according to a first aspect of the present invention, comprises: an input unit that receives input data; a constant generation circuit that generates a plurality of constants and includes at least one node corresponding to each of the plurality of constants, and a multiplexer that, according to an applied signal, supplies either an input value or the result of computation performed on the input value by a computation element to a subsequent computation element; a memory that can store computer-executable instructions; and a processor that, by executing the instructions, performs a convolution operation on the basis of the input data and the plurality of constants.

Description

Convolutional operation device using UMCM circuit and its design method

[0001] The present invention relates to a convolution operation device using a UMCM circuit and a design method thereof.

[0002] For reference, this application claims priority to Korean Patent Application No. 10-2024-0020355, filed on February 13, 2024. The entire contents of that application, which serves as the basis for this priority claim, are incorporated herein by reference.

[0003] A UMCM (unified multiple constant multiplication) circuit is a circuit that can perform a multiplication operation by applying different constant sets depending on the input when multiple constant sets, each with a size of n, are given by applying a multiplexer circuit. A UMCM circuit does not use a multiplier, but rather consists of an adder, a subtracter, and a multiplexer.

[0004] In a typical convolution operation device, the operation part of a deep neural network (DNN) accelerator in an ASIC (application-specific integrated circuit) or FPGA (application-specific integrated circuit) is configured through a processing element (PE), and the weights are loaded from DRAM to perform a multiplication operation.

[0005] This conventional computing method is inefficient in terms of design area and power because it requires access to DRAM during computing and a memory controller is required for this even when accelerating small networks such as edge devices or CMOS sensors.

[0006] The problem to be solved by the present invention is to provide a convolution operation device using a UMCM circuit that is efficient in terms of design area and power, and a design method thereof.

[0007] However, the problems to be solved by the present invention are not limited to those mentioned above, and other problems to be solved that are not mentioned can be clearly understood by a person having ordinary skill in the art to which the present invention pertains from the description below.

[0008] A convolution operation device according to a first aspect of the present invention comprises an input unit for receiving input data, a constant generation circuit for generating a plurality of constants, and including one or more nodes corresponding to each of the plurality of constants, and a multiplexer for supplying an input value or an operation result by an operation element for the input value to a next operation element according to an applied signal, a memory capable of storing computer-executable instructions, and a processor for performing a convolution operation based on the input input data and the plurality of constants by executing the instructions.

[0009] The above-mentioned operational element may include at least one of an adder, a subtracter, and a shift circuit. In this case, each constant corresponding to one or more nodes may be pre-calculated based on the constant generation circuit.

[0010] The above plurality of constants may be clustered into one or more constant sets each having a predetermined number of constants generated from the plurality of constants. In this case, the processor may sequentially correspond each constant set to the input data according to a signal applied to the multiplexer to perform the convolution operation.

[0011] The above predetermined number may be equal to the number of output channels of the above computing device.

[0012] The above-described plurality of constants may be values ​​corresponding to a plurality of weight values ​​determined through neural network learning. In this case, the one or more sets of constants may have constants corresponding to the predetermined number of weight values.

[0013] Each constant in the one or more constant sets may be assigned a position index expressed in two dimensions. At this time, the position index may include a position index of each constant in the one or more constant sets, a value corresponding to at least one node corresponding to each constant, a position index of at least one node corresponding to each constant, a value corresponding to a previous node used to calculate each constant, and a position index of the previous node.

[0014] The above one or more sets of constants may be generated by clustering the plurality of constants corresponding to the output size of the constant generation circuit.

[0015] For a node located after the multiplexer output terminal among the one or more nodes, a first position index designated corresponding to each constant included in the first constant set and a second position index designated corresponding to each constant included in the second constant set may be generated to be the same.

[0016] The position index of at least one node corresponding to constants having the same index in each constant set among the one or more nodes is the same, and the values ​​corresponding to the at least one node may be different from each other.

[0017] The above previous node is a node located before the multiplexer input terminal, and the position index of the previous nodes used to produce constants having the same index in each constant set and the values ​​corresponding to the previous nodes may be generated to be different from each other.

[0018] The one or more nodes may be generated based on an algorithm that determines the plurality of constants using the least number of adders and subtractors included in the operation elements.

[0019] A method for designing a convolution operation device according to a second aspect of the present invention includes a step of receiving a plurality of constants as input, a step of clustering the plurality of constants in correspondence with an output size of one or more constant generation blocks that produce constants using at least one of addition, subtraction, and shift operations, a step of generating one or more constant sets having a predetermined number of constants from the plurality of constants, a step of assigning the one or more constant sets to the one or more constant generation blocks, a step of grouping the one or more constant generation blocks in correspondence with a size of a multiplexer included in the constant generation block, and a step of generating a convolution operation device based on the grouped one or more constant generation blocks and the one or more constant sets.

[0020] A method for designing a convolution operation device according to another embodiment of the second aspect of the present invention includes a step of receiving a plurality of constants as input, and a step of generating a constant generation circuit that generates the plurality of constants and includes at least one node corresponding to each of the plurality of constants and a multiplexer that supplies an input value or an operation result by an operation element for the input value to a next operation element according to an applied signal.

[0021] A computer-readable recording medium storing computer-executable instructions according to a third aspect of the present invention, wherein the computer-executable instructions, when executed by a processor, cause the processor to perform a method including the steps of receiving a plurality of constants, generating the plurality of constants, and generating a constant generation circuit including at least one node corresponding to each of the plurality of constants and a multiplexer that supplies an input value or a result of an operation by an operation element for the input value to a next operation element according to an applied signal.

[0022] A computer program stored in a computer-readable recording medium according to a fourth aspect of the present invention, wherein the computer program, when executed by a processor, includes instructions for causing the processor to perform a method including the steps of receiving a plurality of constants, generating the plurality of constants, and generating a constant generation circuit including at least one node corresponding to each of the plurality of constants and a multiplexer that supplies an input value or a result of an operation by an operation element for the input value to a next operation element according to an applied signal.

[0023] According to the present invention, unlike conventional computing devices, access to DRAM is not necessarily required during computing, and a memory controller is not required accordingly, so that a convolution computing device that is efficient in terms of design area and power can be provided.

[0024] The effects that can be obtained from the present invention are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by a person having ordinary skill in the art to which the present disclosure pertains from the description below.

[0025] FIG. 1 is a block diagram exemplarily showing a convolution operation device according to the first aspect of the present invention.

[0026] FIG. 2 is a block diagram exemplarily showing a computing device capable of designing a convolution operation device according to the first aspect of the present invention.

[0027] Fig. 3 is a flowchart exemplarily showing a method for designing a convolution operation device according to the second aspect.

[0028] Figure 4 is a block diagram exemplifying the function of a convolution operation program.

[0029] Figure 5 is a block diagram exemplarily showing the functions of a convolution operation device design program.

[0030] Figure 6 is an example diagram showing a UMCM circuit.

[0031] Figure 7 is an exemplary diagram showing a convolution operation method according to the present invention.

[0032] Figure 8 is an exemplary diagram showing a method for designing a convolution operation device according to the present invention.

[0033] Fig. 9 is an exemplary diagram showing a convolution operation device designed according to the present invention.

[0034] The advantages and features of the present invention, and the methods for achieving them, will become clearer with reference to the embodiments described in detail below together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below and may be implemented in various different forms. These embodiments are provided solely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention, and the present invention is defined solely by the scope of the claims.

[0035] When describing embodiments of the present invention, detailed descriptions of known functions or configurations will be omitted if they are deemed to unnecessarily obscure the gist of the invention. Furthermore, the terms described below are defined in light of their functions in the embodiments of the present invention and may vary depending on the intent or custom of the user or operator. Therefore, their definitions should be based on the overall content of this specification.

[0036] The terms used in this specification will be briefly explained, and the present invention will be described in detail.

[0037] The terms used in this specification have been selected from widely used, current terms, taking into account the functions of the present invention. However, these terms may vary depending on the intentions of those skilled in the art, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, in which case their meanings will be described in detail in the relevant description of the invention. Therefore, the terms used in this invention should not be defined simply as names, but rather based on their inherent meanings and the overall content of the present invention.

[0038] When a part of a specification is said to 'include' a component, this does not mean that it excludes other components, but rather that it may include other components, unless otherwise stated.

[0039] Also, the term 'part' used in the specification means a software or hardware component such as an FPGA or ASIC, and the 'part' performs certain functions. However, the 'part' is not limited to software or hardware. The 'part' may be configured to reside on an addressable storage medium or may be configured to play one or more processors. Thus, as an example, the 'part' includes components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functionality provided within the components and 'parts' may be combined into a smaller number of components and 'parts' or further separated into additional components and 'parts'.

[0040] Below, with reference to the attached drawings, an embodiment of the present invention is described in detail so that a person having ordinary skill in the art to which the present invention pertains can easily practice it.

[0041] FIG. 1 is a block diagram exemplarily showing a convolution operation device according to the first aspect of the present invention.

[0042] As shown in FIG. 1, the convolution operation device (100) may include an input unit (110), an output unit (120), a processor (130), a memory (140), a constant generation circuit (160), and a communication unit (170).

[0043] Hereinafter, for the convenience of explanation, the convolution operation device (100) is described as an example in which an input unit (110), an output unit (120), a processor (130), a memory (140), a constant generation circuit (160), and a communication unit (170) are included, but the present invention is not limited thereto. That is, each unit configuration can interact with the convolution operation device (100) from outside the convolution operation device (100).

[0044] The input unit (110) may be a hardware device that can directly input commands, information, etc. used to control the convolution operation device (100) through a user interface (e.g., keyboard, mouse input, etc.).

[0045] In one embodiment, the input unit (110) can receive information required for a convolution operation from a user. Specifically, the user can input data required for the convolution operation, such as input data and constants, through the input unit (110).

[0046] Input data may include feature maps, activations, and operand data.

[0047] A constant may refer to data on which a convolution operation is performed, such as a weight, but is not limited thereto.

[0048] The output unit (120) can provide information including information required for the convolution operation and the result of the convolution operation to the user as visual information through an interface or display device.

[0049] The processor (130) can control the overall operation of the convolution operation device (100) to perform the present invention.

[0050] The processor (130) can load the convolution operation program (150) and information necessary for executing the convolution operation program (150) from the memory (140) to execute the convolution operation program (150).

[0051] The processor (130) can control to store data received from an external device through the communication unit (170) in the memory (140). In addition, the processor (130) can control to transmit information including information required for a convolution operation and a result of the convolution operation to the external device through the communication unit (170).

[0052] The processor (130) may refer to a processing device such as a microprocessor, a central processing unit (CPU), a graphic processing unit (GPU), a processor core, a multiprocessor, an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or a micro controller unit (MCU), but is not limited to the above-described embodiment.

[0053] The memory (140) can store a convolution operation program (150) and information required for executing the convolution operation program (150). In addition, the memory (140) can also store processing results by the processor (130).

[0054] The convolution operation program (150) may mean software including commands programmed to perform the method according to the present invention.

[0055] The memory (140) can store information required for convolution operations and information including the results of the convolution operations. In addition, the memory (140) can store information received from an external device via the communication unit (170).

[0056] Memory (140) may refer to a computer-readable recording medium, such as a hard disk, a magnetic media such as a floppy disk and a magnetic tape, an optical media such as a CD-ROM or a DVD, a magneto-optical media such as a floptical disk, a hardware device specifically configured to store and execute program instructions such as a flash memory, but is not limited to the above-described embodiment.

[0057] The constant generation circuit (160) may include at least one operational element among an adder, a subtracter, and a shift circuit.

[0058] The constant generation circuit (160) can generate multiple constants required for convolution operation using arithmetic elements.

[0059] The constant generation circuit (160) may include one or more nodes corresponding to each of a plurality of constants.

[0060] The constant generation circuit (160) may include a multiplexer that supplies an input value or an operation result by an operation element for an input value to the next operation element according to an applied signal.

[0061] The communication unit (170) may be a wireless communication module capable of performing wireless communication by adopting a communication method such as CDMA, GSM, W-CDMA, TD-SCDMA, WiBro, LTE, EPC, 5G, wireless LAN, Wi-Fi, Bluetooth, Zigbee, WFD (Wi-Fi Direct), UWB (Ultra Wide Band), infrared communication (IrDA; infrared data association), BLE (Bluetooth Low Energy), or NFC (Near Field Communication), but is not limited to the above-described embodiment.

[0062] In addition, information input and output through the input unit (110) and output unit (120), information stored in the memory (140), and information transmitted and received through the communication unit (170) include all information related to the present invention, and are not limited to the above-described embodiment.

[0063] The function or operation of the convolution operation program (150) will be examined in detail with reference to FIG. 4.

[0064] FIG. 2 is a block diagram exemplarily showing a computing device capable of designing a convolution operation device according to the first aspect of the present invention.

[0065] As shown in FIG. 2, the convolution operation device design device (200) may include an input unit (210), an output unit (220), a processor (230), a memory (240), and a communication unit (260).

[0066] Among the input unit (210), output unit (220), processor (230), memory (240), and communication unit (260) shown in Fig. 2, descriptions of the same parts as the input unit (110), output unit (120), processor (130), memory (140), and communication unit (170) of Fig. 1 will be omitted.

[0067] Hereinafter, for the convenience of explanation, the convolution operation device design device (200) is described as an example in which an input unit (210), an output unit (220), a processor (230), a memory (240), and a communication unit (260) are included, but the present invention is not limited thereto. That is, each unit configuration can interact with the convolution operation device design device (200) from outside the convolution operation device design device (200).

[0068] A convolution operation device design device (200) can perform a convolution operation device design method including a step of receiving a plurality of constants as input, a step of clustering the plurality of constants in correspondence with an output size of one or more constant generation blocks that produce constants using at least one of addition, subtraction, and shift operations, a step of generating one or more constant sets having a predetermined number of constants from the plurality of constants, a step of assigning the one or more constant sets to the one or more constant generation blocks, a step of grouping the one or more constant generation blocks in correspondence with a size of a multiplexer included in the constant generation block, and a step of generating a convolution operation device based on the grouped one or more constant generation blocks and the one or more constant sets.

[0069] In addition, the convolution operation device design device (200) can perform a convolution operation device design method including a step of receiving a plurality of constants as input, and a step of generating a constant generation circuit that generates the plurality of constants and includes one or more nodes corresponding to each of the plurality of constants and a multiplexer that supplies an input value or an operation result by an operation element for the input value to a next operation element according to an applied signal.

[0070] The input unit (210) may be a hardware device that can directly input commands, information, etc. used to control the convolution operation device design device (200) through a user interface (e.g., keyboard, mouse input, etc.).

[0071] In one embodiment, the input unit (210) may receive information from a user required for designing a convolution operation device. Specifically, the user may input data including input data required for a convolution operation, data such as constants, components required for a constant generation circuit (160), and information related to operation elements through the input unit (210).

[0072] The output unit (220) can provide information necessary for designing a convolution operation device and information including the design result of the convolution operation device to the user as visual information through an interface or display device.

[0073] The processor (230) can control the overall operation of the convolution operation device design device (200) to perform the present invention.

[0074] The processor (230) can load the convolution operation device design program (250) and information necessary for executing the convolution operation device design program (250) from the memory (240) to execute the convolution operation device design program (250).

[0075] The processor (230) can control to store data received from an external device through the communication unit (260) in the memory (240). In addition, the processor (230) can control to transmit information including information required for designing a convolution operation device and the result of designing a convolution operation device to an external device through the communication unit (260).

[0076] The memory (240) can store a convolution operation device design program (250) and information necessary for executing the convolution operation device design program (250). In addition, the memory (240) can also store processing results by the processor (230).

[0077] The convolution operation device design program (250) may mean software including commands programmed to perform the method according to the present invention.

[0078] The memory (240) can store information necessary for designing a convolution operation device and information including the design results of the convolution operation device. In addition, the memory (240) can store information received from an external device via the communication unit (260).

[0079] In addition, information input and output through the input unit (210) and output unit (220), information stored in the memory (240), and information transmitted and received through the communication unit (260) include all information related to the present invention, and are not limited to the above-described embodiment.

[0080] The function or operation of the convolution operation device design program (250) will be examined in detail with reference to FIG. 5.

[0081] Fig. 3 is a flowchart exemplarily showing a method for designing a convolution operation device according to the second aspect.

[0082] As shown in FIG. 3, a method for designing a convolution operation device according to a second aspect of the present invention includes a step of receiving a plurality of constants (S300), a step of clustering the plurality of constants in correspondence with an output size of one or more constant generation blocks that produce constants using at least one of addition, subtraction, and shift operations (S310), a step of generating one or more constant sets having a predetermined number of constants from the plurality of constants (S320), a step of assigning the one or more constant sets to the one or more constant generation blocks, respectively (S330), a step of grouping the one or more constant generation blocks in correspondence with a size of a multiplexer included in the constant generation block (S340), and a step of generating a convolution operation device based on the grouped one or more constant generation blocks and the one or more constant sets (S350).

[0083] Multiple constants may refer to data on which a convolution operation is performed, such as weights, with input data, but are not limited thereto.

[0084] A constant generation block may refer to a constant generation circuit.

[0085] The constant generation block is one of the common designs for accelerating the convolution layer. It is a structure that constructs a multiplier array and then adds values ​​through an adder-tree block. The multiplier array can mean a multiplier block composed of different parameters that share the same input.

[0086] That is, the constant generation block may mean a multiplier block that performs a multiplication operation on input data and multiple constants and adds the results of each multiplication operation to produce different output data corresponding to the output channel.

[0087] The output size of a constant generation block can mean the number of output data output at one time through the constant generation block.

[0088] A constant set includes some of multiple constants, and the number of constants included in the constant set can correspond to the output size of the constant generation block.

[0089] A multiplexer may refer to a circuit that performs a convolution operation by sequentially matching each set of constants to input data according to the applied signal.

[0090] Accordingly, if one or more constant generation blocks are grouped in accordance with the size of the multiplexer included in the constant generation block, the convolution operation device can perform different operations by different constant generation blocks depending on the signal applied to the multiplexer.

[0091] According to the above-described operation, a convolution operation unit can be generated based on one or more grouped constant generation blocks and one or more constant sets.

[0092] In addition, a method for designing a convolution operation device according to another embodiment of the second aspect of the present invention may include a step of receiving a plurality of constants as input, and a step of generating a constant generation circuit that generates the plurality of constants and includes one or more nodes corresponding to each of the plurality of constants and a multiplexer that supplies an input value or an operation result by an operation element for the input value to a next operation element according to an applied signal (not shown).

[0093] Figure 4 is a block diagram exemplifying the function of a convolution operation program.

[0094] As shown in Fig. 4, the convolution operation program (150) may include an input unit (410), a constant generation unit (420), a multiplexer control unit (430), and a convolution operation execution unit (440). The input unit (410), the constant generation unit (420), the multiplexer control unit (430), and the convolution operation execution unit (440) are exemplary divisions of the functions of the convolution operation program (150), and are not limited thereto.

[0095] According to the embodiment, the functions of the input unit (410), constant generation unit (420), multiplexer control unit (430), and convolution operation execution unit (440) can be merged / separated and implemented as a series of commands included in one program.

[0096] The input unit (410), constant generation unit (420), multiplexer control unit (430), and convolution operation execution unit (440) may be implemented by a processor (130), and may mean a data processing device built into hardware having a physically structured circuit to perform a function expressed by a code or command included in a convolution operation program (150) stored in a memory (140).

[0097] The input unit (410) can receive input data required for convolution operation.

[0098] Input data may include feature maps, activations, and operand data.

[0099] The constant generation unit (420) can generate multiple constants required for the convolution operation.

[0100] A constant may refer to data on which a convolution operation is performed, such as a weight, but is not limited thereto.

[0101] The plurality of constants may be clustered into one or more sets of constants each having a predetermined number of constants. In this case, the predetermined number may be equal to the number of output channels of the convolution operation device.

[0102] One or more sets of constants may be generated by clustering multiple constants corresponding to the output size of the constant generation circuit.

[0103] The plurality of constants may be values ​​corresponding to multiple weight values ​​determined through neural network learning. In this case, the one or more sets of constants may have constants corresponding to the predetermined number of weight values.

[0104] Each constant in a set of one or more constants may be assigned a position index expressed in two dimensions.

[0105] The position index may include a position index of each constant in one or more sets of constants, a value corresponding to at least one node corresponding to each constant, a position index of at least one node corresponding to each constant, a value corresponding to a previous node used to produce each constant, and a position index of the previous node.

[0106] Specifically, the lth set of constants About, constant is the previous node, constant and constant If derived from, constant The corresponding location index is can appear as follows. Accordingly, the constant set The position index for is a constant set It may include a value corresponding to each constant included in, a position index corresponding to each constant, a previous constant used to produce each constant, and a position index of the previous constant. At this time, in the constant generation circuit, each constant may be expressed as a node corresponding to each constant.

[0107] constant set The location index of It can be expressed as a set of constants , When there is, A circuit satisfying this can be generated as a UMCM circuit. That is, for one constant generation circuit, a constant generation circuit can be generated by applying a predetermined algorithm so that all constant sets can be generated.

[0108] At this time, one or more nodes may be generated based on an algorithm that determines multiple constants using the fewest number of adders and subtractors included in the computational elements.

[0109] Among one or more nodes included in the constant generation circuit, a node located after the multiplexer output terminal may be generated such that a first position index designated corresponding to each constant included in the first constant set and a second position index designated corresponding to each constant included in the second constant set are the same.

[0110] In each constant set, among one or more nodes, the position indices of at least one node corresponding to constants having the same index are the same, and the values ​​corresponding to the at least one node may be different from each other. That is, the position indices of at least one node corresponding to constants having the same index in each constant set are the same, but since different constant sets are applied by the multiplexer, the values ​​corresponding to at least one node corresponding to different constant sets may be different from each other.

[0111] The previous node used to produce each constant is a node located before the multiplexer input terminal, and the location indexes of the previous nodes used to produce constants having the same index in each constant set and the values ​​corresponding to the previous nodes may be generated to be different from each other.

[0112] The multiplexer control unit (430) can control the multiplexer to supply an input value or an operation result by an operation element for an input value to the next operation element according to an applied signal.

[0113] The operational element may include at least one of an adder, a subtracter, and a shift circuit.

[0114] Each constant corresponding to one or more nodes may be pre-generated by the constant generation unit (420).

[0115] The convolution operation performing unit (440) can perform a convolution operation based on input data and multiple constants.

[0116] The convolution operation performing unit (440) can perform a convolution operation by sequentially matching each constant set to input data according to a signal applied to the multiplexer.

[0117] Figure 5 is a block diagram exemplarily showing the functions of a convolution operation device design program.

[0118] As shown in FIG. 5, the convolution operation device design program (250) may include an input unit (510), a constant clustering unit (520), a constant set generation unit (530), a constant set allocation unit (540), a constant generation block grouping unit (550), and a convolution operation device generation unit (560). The input unit (510), the constant clustering unit (520), the constant set generation unit (530), the constant set allocation unit (540), the constant generation block grouping unit (550), and the convolution operation device generation unit (560) are exemplary divisions of the functions of the convolution operation device design program (250), and are not limited thereto.

[0119] According to an embodiment, the functions of the input unit (510), constant clustering unit (520), constant set generation unit (530), constant set allocation unit (540), constant generation block grouping unit (550), and convolution operation unit generation unit (560) can be merged / separated and implemented as a series of commands included in at least one program.

[0120] The input unit (510), constant clustering unit (520), constant set generation unit (530), constant set allocation unit (540), constant generation block grouping unit (550), and convolution operation unit generation unit (560) may be implemented by a processor (230), and may mean a data processing device built into hardware having a physically structured circuit to perform a function expressed by a code or command included in a convolution operation unit design program (250) stored in a memory (240).

[0121] The input unit (510) can receive multiple constants.

[0122] A constant may refer to data on which a convolution operation is performed, such as a weight, but is not limited thereto.

[0123] The constant clustering unit (520) can cluster a plurality of constants corresponding to the output size of one or more constant generation blocks that produce constants using at least one of addition, subtraction, and shift operations.

[0124] The constant set generation unit (530) can generate one or more constant sets having a predetermined number of constants from a plurality of constants.

[0125] The constant set allocation unit (540) can allocate one or more constant sets to one or more constant generation blocks.

[0126] The constant generation block grouping unit (550) can group one or more constant generation blocks in response to the size of the multiplexer included in the constant generation block.

[0127] The convolution operation device generation unit (560) can generate a convolution operation device based on one or more grouped constant generation blocks and one or more constant sets.

[0128] Figure 6 is an example diagram showing a UMCM circuit.

[0129] A UMCM circuit can be defined as a circuit that can perform a multiplication operation by applying different constant sets depending on the input when multiple constant sets, each of which has a size of n, are given by applying a multiplexer circuit. A UMCM circuit can be composed of an adder, a subtracter, and a multiplexer without using a multiplier.

[0130] Specifically, a set of constants containing n constants , a set of multiple constants containing it When there is an input data x and a set of constants that can be selected i, then the corresponding set of constants For all constants included in , the result of the multiplication operation with the input data x can be obtained.

[0131] Figure 7 is an exemplary diagram showing a convolution operation method according to the present invention.

[0132] The output feature map cube COХHOХWO can be obtained by multiplying the input feature map cube of CIХHIХWI and the weight tensors of CoХCiХKyХKx.

[0133] At this time, if the batch size is 1, it can be performed as an operation of a 6-dimensional nested loop.

[0134] As shown in lines 6-7 of Fig. 7, a single input feature map can be multiplied by CO different weights. Accordingly, when the size of each constant generation block is n, CO weights are assigned to CO / n constant generation blocks, and the entire weight tensor can be assigned to a total of CoХCiХKyХKx / n constant generation blocks.

[0135] When the number of constant sets to be assigned to one UMCM circuit is G, a total of CoХCiХKyХKx / (nХG) UMCM circuits may be required.

[0136] Figure 8 is an exemplary diagram showing a method for designing a convolution operation device according to the present invention.

[0137] The method for designing a convolution operation device according to the present invention can be specifically performed in the following order.

[0138] 1. Enter the weights of the pre-trained model

[0139] 2. Cluster each weight according to the size of the constant generation block, n.

[0140] 3. Group the constant generation blocks to create a UMCM circuit.

[0141] 4. Create a UMCM circuit for each UMCM group.

[0142] 5. Converting UMCM circuits into Verilog code

[0143] Fig. 9 is an exemplary diagram showing a convolution operation device designed according to the present invention.

[0144] Figure 9 (a) shows that x is input as input data, and weights t1, t2, t3, and t4 are corresponding to it, and a convolution operation is performed.

[0145] Fig. 9 (b) shows an MCM (multiple constant multiplication) circuit that performs a convolution operation on weights t1, t2, t3, and t4 generated through a constant generation circuit including arithmetic elements composed of an adder, a subtracter, and a shift circuit for input data x. The MCM circuit can perform a multiplication operation on constant sets only through addition, subtraction, and bit shift through resource sharing. In this case, the constant set can be {17, 9, 31, 15}.

[0146] In Fig. 9 (b), the node corresponding to t1 may be 17x, the node corresponding to t2 may be 9x, the node corresponding to t3 may be 31x, and the node corresponding to t4 may be 15x. In this case, the node corresponding to t2, 9x, may be used to produce the node corresponding to t1, 17x. In addition, the node corresponding to t4, 15x, may be used to produce the node corresponding to t3, 31x.

[0147] In Figure 9 (b), four multiplication operations can be replaced with four adders. This enables efficient multiplication operations for a fixed set of constants.

[0148] Fig. 9 (c) illustrates a UMCM circuit with a multiplexer applied to an MCM circuit according to the present invention. While an MCM circuit has a one-to-one mapping between constants included in a constant set and outputs, a UMCM circuit can be configured as a circuit that integrates multiple constant sets through a multiplexer.

[0149] Specifically, unlike the MCM circuit of Fig. 9(b) that required four nodes to generate t1, t2, t3, and t4, the UMCM circuit of Fig. 9(c) can generate t1, t2, t3, and t4 using two nodes by using a multiplexer. In this case, two sets of constants {17,9} and {31,15} can be applied depending on the signal applied to the multiplexer. Accordingly, the design area can be efficiently reduced.

[0150] In addition, rather than performing the convolution operation by loading the weights t1, t2, t3, and t4 from DRAM, the convolution operation can be performed efficiently in terms of design area and power by fixing the weights to the circuit in advance.

[0151] As described above, according to the present invention, unlike conventional computing devices, access to DRAM is not necessarily required during computing, and a memory controller is not required accordingly, so that a convolution computing device that is efficient in terms of design area and power can be provided.

[0152] The embodiments of the present invention described above may be implemented through various means. For example, the embodiments of the present invention may be implemented using hardware, firmware, software, or a combination thereof.

[0153] The combination of each block of the block diagram and each step of the flowchart attached to the present invention may be performed by computer program instructions. These computer program instructions may be installed in an encoding processor of a general-purpose computer, a special-purpose computer, or other programmable data processing equipment, so that the instructions executed by the encoding processor of the computer or other programmable data processing equipment create a means for performing the functions described in each block of the block diagram or each step of the flowchart. These computer program instructions may also be stored in a computer-available or computer-readable memory that can direct a computer or other programmable data processing equipment to implement the functions in a specific manner, so that the instructions stored in the computer-available or computer-readable memory can also produce an article of manufacture that includes an instruction means for performing the functions described in each block of the block diagram or each step of the flowchart. Since the computer program instructions can also be installed on a computer or other programmable data processing device, a series of operational steps are performed on the computer or other programmable data processing device to create a computer-executable process, and the instructions that cause the computer or other programmable data processing device to perform the steps for executing the functions described in each block of the block diagram and each step of the flowchart can also provide steps for executing the functions described in each block of the block diagram and each step of the flowchart.

[0154] Additionally, each block or step may represent a module, segment, or portion of code that includes one or more executable instructions for performing a specific logical function(s). In some embodiments, the functions mentioned in the blocks or steps may occur out of order. For example, two blocks or steps depicted in succession may actually be performed substantially simultaneously, or the blocks or steps may sometimes be performed in reverse order depending on the corresponding function.

[0155] The above description is merely an illustrative illustration of the technical idea of ​​the present invention, and those skilled in the art will appreciate that various modifications and variations can be made without departing from the essential quality of the present invention. Therefore, the embodiments disclosed in the present invention are intended to illustrate, rather than limit, the technical idea of ​​the present invention, and the scope of the technical idea of ​​the present invention is not limited by these embodiments. The scope of protection of the present invention should be interpreted by the following claims, and all technical ideas within a scope equivalent thereto should be interpreted as being included in the scope of the rights of the present invention.

Claims

1. Input section for receiving input data; A constant generation circuit that generates a plurality of constants and includes one or more nodes corresponding to each of the plurality of constants and a multiplexer that supplies an input value or an operation result by an operation element for the input value to the next operation element according to an applied signal; A memory capable of storing computer-executable instructions; and By executing the above command, A processor comprising a processor that performs a convolution operation based on the input data and the plurality of constants. Convolution operation unit.

2. In paragraph 1, The above operating elements are, Containing at least one of an adder, a subtracter and a shift circuit, Each constant corresponding to one or more of the above nodes is, Based on the above constant generation circuit, Convolution operation unit.

3. In paragraph 1, The above plural constants are, Clustered into one or more sets of constants having a predetermined number of constants generated from the above plurality of constants, The above processor, The convolution operation is performed by sequentially matching each set of constants to the input data according to the signal applied to the multiplexer. Convolution operation unit.

4. In paragraph 3, The above specified number is, The number of output channels of the above computing device is the same, Convolution operation unit.

5. In paragraph 3, The above multiple constants are values ​​corresponding to multiple weight values ​​determined through learning of the neural network, wherein said one or more sets of constants have constants corresponding to said predetermined number of weight values, Convolution operation unit.

6. In paragraph 5, For each constant in the set of one or more constants above, A position index expressed in two dimensions is specified, The above location index is, A set of one or more constants, each constant having a position index, a value corresponding to at least one node corresponding to each constant, a position index of at least one node corresponding to each constant, a value corresponding to a previous node used to calculate each constant, and a position index of the previous node. Convolution operation unit.

7. In paragraph 6, One or more sets of constants are provided above, Generated by clustering the plurality of constants corresponding to the output size of the constant generation circuit. Convolution operation unit.

8. In paragraph 6, For a node located after the multiplexer output terminal among the above one or more nodes, The first position index specified corresponding to each constant included in the first constant set and the second position index specified corresponding to each constant included in the second constant set are generated to be the same. Convolution operation unit.

9. In paragraph 6, The position index of at least one node corresponding to constants having the same index in each constant set among the one or more nodes is the same, and the values ​​corresponding to the at least one node are different from each other. Convolution operation unit.

10. In paragraph 6, The above previous node is a node located before the multiplexer input terminal, and the location index of the previous nodes used to produce constants having the same index in each constant set and the values ​​corresponding to the previous nodes are generated to be different from each other. Convolution operation unit.

11. In paragraph 1, One or more of the above nodes, It is generated based on an algorithm that determines the plurality of constants by using the adders and subtractors included in the above operation elements as little as possible. Convolution operation unit.

12. A method for designing a convolution operation device performed by a computing device, Step of receiving multiple constants; A step of clustering the plurality of constants corresponding to the output size of one or more constant generation blocks that produce constants using at least one of addition, subtraction, and shift operations; A step of generating one or more sets of constants having a predetermined number of constants from the plurality of constants; A step of assigning each of the one or more sets of constants to each of the one or more constant generation blocks; A step of grouping one or more constant generation blocks corresponding to the size of a multiplexer included in the constant generation block; and A step of generating a convolution operation unit based on the grouped one or more constant generation blocks and the one or more constant sets, A method for designing a convolution operation unit.

13. A method for designing a convolution operator performed by a computing device, A step of receiving multiple constants; and A step of generating a constant generation circuit that generates the plurality of constants, and includes one or more nodes corresponding to each of the plurality of constants and a multiplexer that supplies an input value or an operation result by an operation element for the input value to a next operation element according to an applied signal. A method for designing a convolution operator.

14. In paragraph 13, The above operating elements are, Containing at least one of an adder, a subtracter and a shift circuit, Each constant corresponding to one or more of the above nodes is, Based on the above constant generation circuit, A method for designing a convolution operator.

15. In paragraph 13, The above plural constants are, Clustered into one or more sets of constants having a predetermined number of constants generated from the above plurality of constants, A convolution operation is performed by sequentially corresponding each set of constants to the input data according to the signal applied to the multiplexer. A method for designing a convolution operator.

16. In paragraph 15, The above specified number is, The number of output channels of the above computing device is the same, A method for designing a convolution operator.

17. In paragraph 15, The above multiple constants are values ​​corresponding to multiple weight values ​​determined through learning of the neural network, wherein said one or more sets of constants have constants corresponding to said predetermined number of weight values, A method for designing a convolution operator.

18. In paragraph 17, For each constant in the set of one or more constants above, A position index expressed in two dimensions is specified, The above location index is, A set of one or more constants, each constant having a position index, a value corresponding to at least one node corresponding to each constant, a position index of at least one node corresponding to each constant, a value corresponding to a previous node used to calculate each constant, and a position index of the previous node. A method for designing a convolution operator.

19. In paragraph 18, One or more sets of constants are provided above, Generated by clustering the plurality of constants corresponding to the output size of the constant generation circuit. A method for designing a convolution operator.

20. A computer-readable recording medium storing computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, A step of receiving multiple constants; and A method comprising the step of generating a constant generation circuit that generates the plurality of constants, and includes one or more nodes corresponding to each of the plurality of constants and a multiplexer that supplies an input value or an operation result by an operation element for the input value to a next operation element according to an applied signal, wherein the processor performs the method. Computer-readable recording medium.

Images