Convolution operation apparatus using approximate algorithm-based umcm circuit and design method thereof

Abstract

A convolution operation apparatus according to a first aspect of the present invention comprises: an input unit that receives input data as an input; a constant generation circuit that generates a plurality of constants and includes one or more nodes corresponding to each of the plurality of constants; a memory capable of storing computer-executable instructions; and a processor that performs a convolution operation on the basis of the input data that has been input and the plurality of constants, by executing the instructions. Here, the plurality of constants are determined by approximating a plurality of predetermined reference constants to a similar value between each reference constant and each reference constant less than or equal to a predetermined threshold, and the plurality of reference constants may be values corresponding to a plurality of weights that are predetermined through training of a neural network.

Description

Convolutional operation device using UMCM circuit based on approximation algorithm and its design method

[0001] The present invention relates to a convolution operation device using an approximate algorithm-based UMCM circuit and a design method thereof.

[0002] For reference, this application claims priority to Korean Patent Application No. 10-2024-0020444, filed on February 13, 2024. The entire contents of that application, which serves as the basis for this priority claim, are incorporated herein by reference.

[0003] A UMCM (unified multiple constant multiplication) circuit is a circuit that can perform a multiplication operation by applying different constant sets depending on the input when multiple constant sets, each with a size of n, are given by applying a multiplexer circuit. A UMCM circuit does not use a multiplier, but rather consists of an adder, a subtracter, and a multiplexer.

[0004] In a typical convolution operation device, the operation part of a deep neural network (DNN) accelerator in an ASIC (application-specific integrated circuit) or FPGA (application-specific integrated circuit) is configured through a processing element (PE), and the weights are loaded from DRAM to perform a multiplication operation.

[0005] This conventional computing method is inefficient in terms of design area and power because it requires access to DRAM during computing and a memory controller is required for this even when accelerating small networks such as edge devices or CMOS sensors.

[0006] When applying UMCM circuits to convolutional processing units, the UMCM circuit must be generated by considering the order of the constants within each set for multiple constant sets. Consequently, as the number of constant sets increases, the complexity of the circuit generation algorithm can increase due to the need for sequence considerations. Furthermore, the need for more adders, subtractors, and multiplexers can lead to increased circuit complexity, design area, and power consumption.

[0007] The problem to be solved by the present invention is to provide a convolution operation device using an UMCM circuit based on an approximate algorithm that is efficient in terms of design area and power, and a design method thereof.

[0008] However, the problems to be solved by the present invention are not limited to those mentioned above, and other problems to be solved that are not mentioned can be clearly understood by a person having ordinary skill in the art to which the present invention pertains from the description below.

[0009] A convolution operation device according to a first aspect of the present invention includes an input unit for receiving input data, a constant generation circuit for generating a plurality of constants and including one or more nodes corresponding to each of the plurality of constants, a memory capable of storing computer-executable instructions, and a processor for performing a convolution operation based on the input input data and the plurality of constants by executing the instructions. In this case, the plurality of constants are determined by approximating a similar value of each reference constant that is lower than a predetermined threshold value among a plurality of predetermined reference constants, and the plurality of reference constants may be values ​​corresponding to a plurality of weights determined in advance through learning of a neural network.

[0010] The above constant generation circuit may include at least one of an adder, a subtracter, and a shift circuit. In this case, each constant corresponding to one or more nodes may be pre-calculated based on the constant generation circuit.

[0011] The above constant generation circuit may include a multiplexer that supplies an input value or a result of the input value being calculated through an operation element to the next operation element according to an applied signal.

[0012] The above plurality of constants may be clustered into one or more constant sets each having a predetermined number of constants generated from the plurality of constants. In this case, the processor may sequentially correspond each constant set to the input data according to a signal applied to the multiplexer to perform the convolution operation.

[0013] Each constant in the one or more constant sets may be assigned a position index expressed in two dimensions. At this time, the position index may include a position index of each constant in the one or more constant sets, a value corresponding to at least one node corresponding to each constant, a position index of at least one node corresponding to each constant, a value corresponding to a previous node used to calculate each constant, and a position index of the previous node.

[0014] The above one or more sets of constants may be generated by clustering the plurality of constants corresponding to the output size of the constant generation circuit.

[0015] For a node located after the multiplexer output terminal among the one or more nodes, a first position index designated corresponding to each constant included in the first constant set and a second position index designated corresponding to each constant included in the second constant set may be generated to be the same.

[0016] The position index of at least one node corresponding to constants having the same index in each constant set among the one or more nodes is the same, and the values ​​corresponding to the at least one node may be different from each other.

[0017] The above previous node is a node located before the multiplexer input terminal, and the position index of the previous nodes used to produce constants having the same index in each constant set and the values ​​corresponding to the previous nodes may be generated to be different from each other.

[0018] The one or more nodes may be generated based on an algorithm that determines the plurality of constants using the fewest number of adders and subtractors included in the constant generation circuit.

[0019] According to another embodiment of the second aspect of the present invention, a method for designing a convolution operation device includes the steps of receiving a plurality of constants as input, clustering the plurality of constants into one or more constant sets having a predetermined number of constants generated from the plurality of constants, and generating a constant generation circuit that generates the plurality of constants and includes one or more nodes corresponding to each of the plurality of constants. In this case, the plurality of constants may be determined by approximating a similar value of each reference constant that is lower than a predetermined threshold value among a plurality of predetermined reference constants. In addition, the plurality of reference constants may be values ​​corresponding to a plurality of weights that are predetermined through learning of a neural network.

[0020] A computer-readable recording medium storing computer-executable instructions according to a third aspect of the present invention, wherein the computer-executable instructions, when executed by a processor, cause the processor to perform a method including the steps of: receiving a plurality of constants; clustering the plurality of constants into one or more constant sets having a predetermined number of constants generated from the plurality of constants; and generating a constant generation circuit that generates the plurality of constants and includes one or more nodes corresponding to each of the plurality of constants, wherein the plurality of constants are determined by approximating a similar value of each of the plurality of reference constants that is less than or equal to a predetermined threshold value among a plurality of predetermined reference constants, and wherein the plurality of reference constants are values ​​corresponding to a plurality of weights that are predetermined through learning of a neural network.

[0021] A computer program stored in a computer-readable recording medium according to a fourth aspect of the present invention, wherein the computer program, when executed by a processor, comprises the steps of: receiving a plurality of constants; clustering the plurality of constants into one or more constant sets having a predetermined number of constants generated from the plurality of constants; and generating a constant generation circuit that generates the plurality of constants and includes one or more nodes corresponding to each of the plurality of constants, wherein the plurality of constants are determined by approximating a similar value of each of the plurality of reference constants that is less than or equal to a predetermined threshold value among a plurality of predetermined reference constants, and wherein the plurality of reference constants are values ​​corresponding to a plurality of weights that are determined in advance through learning of a neural network.

[0022] According to the present invention, a convolution operation device and a design method thereof using a UMCM circuit that are efficient in terms of design area and power by applying an approximation algorithm can be provided.

[0023] The effects that can be obtained from the present invention are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by a person having ordinary skill in the art to which the present disclosure pertains from the description below.

[0024] FIG. 1 is a block diagram exemplarily showing a convolution operation device according to the first aspect of the present invention.

[0025] FIG. 2 is a block diagram exemplarily showing a computing device capable of designing a convolution operation device according to the first aspect of the present invention.

[0026] Fig. 3 is a flowchart exemplarily showing a method for designing a convolution operation device according to the second aspect.

[0027] Figure 4 is a block diagram exemplifying the function of a convolution operation program.

[0028] Figure 5 is a block diagram exemplarily showing the functions of a convolution operation device design program.

[0029] Figure 6 is an example diagram showing a UMCM circuit.

[0030] Figure 7 is an example diagram showing a convolution operation method using a UMCM circuit.

[0031] Fig. 8 is an exemplary diagram showing a method for designing a convolution operation device using a UMCM circuit.

[0032] Fig. 9 is an exemplary diagram showing a convolution operation device designed according to the present invention.

[0033] FIG. 10 is an exemplary diagram showing a method for generating a node of a constant generation circuit by applying an approximation algorithm according to the present invention.

[0034] The advantages and features of the present invention, and the methods for achieving them, will become clearer with reference to the embodiments described in detail below together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below and may be implemented in various different forms. These embodiments are provided solely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention, and the present invention is defined solely by the scope of the claims.

[0035] When describing embodiments of the present invention, detailed descriptions of known functions or configurations will be omitted if they are deemed to unnecessarily obscure the gist of the invention. Furthermore, the terms described below are defined in light of their functions in the embodiments of the present invention and may vary depending on the intent or custom of the user or operator. Therefore, their definitions should be based on the overall content of this specification.

[0036] The terms used in this specification will be briefly explained, and the present invention will be described in detail.

[0037] The terms used in this specification have been selected from widely used, current terms, taking into account the functions of the present invention. However, these terms may vary depending on the intentions of those skilled in the art, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, in which case their meanings will be described in detail in the relevant description of the invention. Therefore, the terms used in this invention should not be defined simply as names, but rather based on their inherent meanings and the overall content of the present invention.

[0038] When a part of a specification is said to 'include' a component, this does not mean that it excludes other components, but rather that it may include other components, unless otherwise stated.

[0039] Also, the term 'part' used in the specification means a software or hardware component such as an FPGA or ASIC, and the 'part' performs certain functions. However, the 'part' is not limited to software or hardware. The 'part' may be configured to reside on an addressable storage medium or may be configured to play one or more processors. Thus, as an example, the 'part' includes components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functionality provided within the components and 'parts' may be combined into a smaller number of components and 'parts' or further separated into additional components and 'parts'.

[0040] Below, with reference to the attached drawings, an embodiment of the present invention is described in detail so that a person having ordinary skill in the art to which the present invention pertains can easily practice it.

[0041] FIG. 1 is a block diagram exemplarily showing a convolution operation device according to the first aspect of the present invention.

[0042] As shown in FIG. 1, the convolution operation device (100) may include an input unit (110), an output unit (120), a processor (130), a memory (140), a constant generation circuit (160), and a communication unit (170).

[0043] Hereinafter, for the convenience of explanation, the convolution operation device (100) is described as including an input unit (110), an output unit (120), a processor (130), a memory (140), a constant generation circuit (160), and a communication unit (170), but is not limited thereto. That is, each unit configuration can interact with the convolution operation device (100) from outside the convolution operation device (100).

[0044] The input unit (110) may be a hardware device that can directly input commands, information, etc. used to control the convolution operation device (100) through a user interface (e.g., keyboard, mouse input, etc.).

[0045] In one embodiment, the input unit (110) can receive information required for a convolution operation from a user. Specifically, the user can input data required for the convolution operation, such as input data and constants, through the input unit (110).

[0046] Input data may include feature maps, activations, and operand data.

[0047] A constant may refer to data on which a convolution operation is performed, such as a weight, but is not limited thereto.

[0048] The output unit (120) can provide information including information required for the convolution operation and the result of the convolution operation to the user as visual information through an interface or display device.

[0049] The processor (130) can control the overall operation of the convolution operation device (100) to perform the present invention.

[0050] The processor (130) can load the convolution operation program (150) and information necessary for executing the convolution operation program (150) from the memory (140) to execute the convolution operation program (150).

[0051] The processor (130) can control to store data received from an external device through the communication unit (170) in the memory (140). In addition, the processor (130) can control to transmit information including information required for a convolution operation and a result of the convolution operation to the external device through the communication unit (170).

[0052] The processor (130) may refer to a processing device such as a microprocessor, a central processing unit (CPU), a graphic processing unit (GPU), a processor core, a multiprocessor, an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or a micro controller unit (MCU), but is not limited to the above-described embodiment.

[0053] The memory (140) can store a convolution operation program (150) and information required for executing the convolution operation program (150). In addition, the memory (140) can also store processing results by the processor (130).

[0054] The convolution operation program (150) may mean software including commands programmed to perform the method according to the present invention.

[0055] The memory (140) can store information required for convolution operations and information including the results of the convolution operations. In addition, the memory (140) can store information received from an external device via the communication unit (170).

[0056] Memory (140) may refer to a computer-readable recording medium, such as a hard disk, a magnetic media such as a floppy disk and a magnetic tape, an optical media such as a CD-ROM or a DVD, a magneto-optical media such as a floptical disk, a hardware device specifically configured to store and execute program instructions such as a flash memory, but is not limited to the above-described embodiment.

[0057] The constant generation circuit (160) may include at least one operational element among an adder, a subtracter, and a shift circuit.

[0058] The constant generation circuit (160) can generate multiple constants required for convolution operation using arithmetic elements.

[0059] The constant generation circuit (160) may include one or more nodes corresponding to each of a plurality of constants.

[0060] The constant generation circuit (160) may include a multiplexer that supplies an input value or an operation result by an operation element for an input value to the next operation element according to an applied signal.

[0061] The communication unit (170) may be a wireless communication module capable of performing wireless communication by adopting a communication method such as CDMA, GSM, W-CDMA, TD-SCDMA, WiBro, LTE, EPC, 5G, wireless LAN, Wi-Fi, Bluetooth, Zigbee, WFD (Wi-Fi Direct), UWB (Ultra Wide Band), infrared communication (IrDA; infrared data association), BLE (Bluetooth Low Energy), or NFC (Near Field Communication), but is not limited to the above-described embodiment.

[0062] In addition, information input and output through the input unit (110) and output unit (120), information stored in the memory (140), and information transmitted and received through the communication unit (170) include all information related to the present invention, and are not limited to the above-described embodiment.

[0063] The function or operation of the convolution operation program (150) will be examined in detail with reference to FIG. 4.

[0064] FIG. 2 is a block diagram exemplarily showing a computing device capable of designing a convolution operation device according to the first aspect of the present invention.

[0065] As shown in FIG. 2, the convolution operation device design device (200) may include an input unit (210), an output unit (220), a processor (230), a memory (240), and a communication unit (260).

[0066] Among the input unit (210), output unit (220), processor (230), memory (240), and communication unit (260) shown in Fig. 2, descriptions of the same parts as the input unit (110), output unit (120), processor (130), memory (140), and communication unit (170) of Fig. 1 will be omitted.

[0067] Hereinafter, for the convenience of explanation, the convolution operation device design device (200) is described as an example in which an input unit (210), an output unit (220), a processor (230), a memory (240), and a communication unit (260) are included, but the present invention is not limited thereto. That is, each unit configuration can interact with the convolution operation device design device (200) from outside the convolution operation device design device (200).

[0068] A convolution operation device design device (200) may perform a convolution operation device design method including a step of receiving a plurality of constants, a step of clustering the plurality of constants into one or more constant sets having a predetermined number of constants generated from the plurality of constants, and a step of generating a constant generation circuit that generates the plurality of constants and includes one or more nodes corresponding to each of the plurality of constants. In this case, the plurality of constants may be determined by approximating a similar value of each reference constant that is lower than a predetermined threshold value among a plurality of predetermined reference constants. In addition, the plurality of reference constants may be values ​​corresponding to a plurality of weights that are predetermined through learning of a neural network.

[0069] The input unit (210) may be a hardware device that can directly input commands, information, etc. used to control the convolution operation device design device (200) through a user interface (e.g., keyboard, mouse input, etc.).

[0070] In one embodiment, the input unit (210) may receive information from a user required for designing a convolution operation device. Specifically, the user may input data including input data required for a convolution operation, data such as constants, components required for a constant generation circuit (160), information related to operation elements, and information related to an approximation algorithm through the input unit (210).

[0071] The output unit (220) can provide information including information required for designing a convolution operation device, information related to an approximation algorithm, and a convolution operation device design result to a user as visual information through an interface or display device.

[0072] The processor (230) can control the overall operation of the convolution operation device design device (200) to perform the present invention.

[0073] The processor (230) can load the convolution operation device design program (250) and information necessary for executing the convolution operation device design program (250) from the memory (240) to execute the convolution operation device design program (250).

[0074] The processor (230) can control to store data received from an external device through the communication unit (260) in the memory (240). In addition, the processor (230) can control to transmit information including information necessary for designing a convolution operation device, information related to an approximation algorithm, and a convolution operation device design result to an external device through the communication unit (260).

[0075] The memory (240) can store a convolution operation device design program (250) and information necessary for executing the convolution operation device design program (250). In addition, the memory (240) can also store processing results by the processor (230).

[0076] The convolution operation device design program (250) may mean software including commands programmed to perform the method according to the present invention.

[0077] The memory (240) can store information including information required for designing a convolution operation device, information related to an approximation algorithm, and the results of designing a convolution operation device. In addition, the memory (240) can store information received from an external device via a communication unit (260).

[0078] In addition, information input and output through the input unit (210) and output unit (220), information stored in the memory (240), and information transmitted and received through the communication unit (260) include all information related to the present invention, and are not limited to the above-described embodiment.

[0079] The function or operation of the convolution operation device design program (250) will be examined in detail with reference to FIG. 5.

[0080] Fig. 3 is a flowchart exemplarily showing a method for designing a convolution operation device according to the second aspect.

[0081] As shown in FIG. 3, a method for designing a convolution operation device according to a second aspect of the present invention includes a step of receiving a plurality of constants (S300), a step of clustering the plurality of constants into one or more constant sets having a predetermined number of constants generated from the plurality of constants (S310), and a step of generating a constant generation circuit that generates the plurality of constants and includes one or more nodes corresponding to each of the plurality of constants (S320). At this time, the plurality of constants may be determined by approximating a similar value of each reference constant that is lower than a predetermined threshold value of each reference constant among a plurality of predetermined reference constants. In addition, the plurality of reference constants may be values ​​corresponding to a plurality of weights that are predetermined through learning of a neural network.

[0082] Multiple constants may refer to data on which a convolution operation is performed, such as weights, but are not limited thereto.

[0083] The constant generation circuit is one of the common designs for accelerating the convolution layer. It is a structure that configures a multiplier array and then adds values ​​through an adder-tree block. The multiplier array can mean a multiplier block composed of different parameters that share the same input.

[0084] That is, the constant generation circuit may mean a multiplier block that performs a multiplication operation on input data and multiple constants and adds the results of each multiplication operation to produce different output data corresponding to the output channel.

[0085] The output size of a constant generation circuit can mean the number of output data output at one time through the constant generation circuit.

[0086] The constant set includes some of a plurality of constants, and the number of constants included in the constant set can correspond to the output size of the constant generation circuit.

[0087] A multiplexer may refer to a circuit that performs a convolution operation by sequentially matching each set of constants to input data according to the applied signal.

[0088] Accordingly, if one or more constant generation circuits are grouped in accordance with the size of the multiplexer included in the constant generation circuit, the convolution operation device can perform different operations by different constant generation circuits depending on the signal applied to the multiplexer.

[0089] According to the above-described operation, a convolution operation device can be generated based on one or more grouped constant generation circuits and one or more constant sets.

[0090] Figure 4 is a block diagram exemplifying the function of a convolution operation program.

[0091] As shown in Fig. 4, the convolution operation program (150) may include an input unit (410), a constant generation unit (420), a multiplexer control unit (430), and a convolution operation execution unit (440). The input unit (410), the constant generation unit (420), the multiplexer control unit (430), and the convolution operation execution unit (440) are exemplary divisions of the functions of the convolution operation program (150), and are not limited thereto.

[0092] According to the embodiment, the functions of the input unit (410), constant generation unit (420), multiplexer control unit (430), and convolution operation execution unit (440) can be merged / separated and implemented as a series of commands included in one program.

[0093] The input unit (410), constant generation unit (420), multiplexer control unit (430), and convolution operation execution unit (440) may be implemented by a processor (130), and may mean a data processing device built into hardware having a physically structured circuit to perform a function expressed by a code or command included in a convolution operation program (150) stored in a memory (140).

[0094] The input unit (410) can receive input data required for convolution operation.

[0095] Input data may include feature maps, activations, and operand data.

[0096] The constant generation unit (420) can generate multiple constants required for the convolution operation.

[0097] A constant may refer to data on which a convolution operation is performed, such as a weight, but is not limited thereto.

[0098] The plurality of constants may be determined by approximating each reference constant with a similar value that is lower than or equal to a predetermined threshold value among a plurality of predetermined reference constants. The method of approximating with similar values ​​is described in detail in Fig. 10.

[0099] Multiple reference constants may be values ​​corresponding to multiple weights determined through learning of the neural network.

[0100] The plurality of constants may be clustered into one or more sets of constants each having a predetermined number of constants. In this case, the predetermined number may be equal to the number of output channels of the convolution operation device.

[0101] One or more sets of constants may be generated by clustering multiple constants corresponding to the output size of the constant generation circuit.

[0102] The plurality of constants may be values ​​corresponding to multiple weight values ​​determined through neural network learning. In this case, the one or more sets of constants may have constants corresponding to the predetermined number of weight values.

[0103] Each constant in a set of one or more constants may be assigned a position index expressed in two dimensions.

[0104] The position index may include a position index of each constant in one or more sets of constants, a value corresponding to at least one node corresponding to each constant, a position index of at least one node corresponding to each constant, a value corresponding to a previous node used to produce each constant, and a position index of the previous node.

[0105] Specifically, the lth set of constants About, constant is the previous node, constant and constant If derived from, constant The corresponding location index is can appear as follows. Accordingly, the constant set The position index for is a constant set It may include a value corresponding to each constant included in, a position index corresponding to each constant, a previous constant used to produce each constant, and a position index of the previous constant. At this time, in the constant generation circuit, each constant may be expressed as a node corresponding to each constant.

[0106] constant set The location index of It can be expressed as a set of constants , When there is, A circuit satisfying this can be generated as a UMCM circuit. That is, for one constant generation circuit, a constant generation circuit can be generated by applying a predetermined algorithm so that all constant sets can be generated.

[0107] At this time, one or more nodes may be generated based on an algorithm that determines multiple constants using the fewest number of adders and subtractors included in the computational elements.

[0108] Among one or more nodes included in the constant generation circuit, a node located after the multiplexer output terminal may be generated such that a first position index designated corresponding to each constant included in the first constant set and a second position index designated corresponding to each constant included in the second constant set are the same.

[0109] In each constant set, among one or more nodes, the position indices of at least one node corresponding to constants having the same index are the same, and the values ​​corresponding to the at least one node may be different from each other. That is, the position indices of at least one node corresponding to constants having the same index in each constant set are the same, but since different constant sets are applied by the multiplexer, the values ​​corresponding to at least one node corresponding to different constant sets may be different from each other.

[0110] The previous node used to produce each constant is a node located before the multiplexer input terminal, and the location indexes of the previous nodes used to produce constants having the same index in each constant set and the values ​​corresponding to the previous nodes may be generated to be different from each other.

[0111] One or more nodes may be generated based on an algorithm that determines a plurality of constants using the fewest number of adders and subtractors included in the constant generation circuit. In this case, the plurality of constants may be determined by approximating each reference constant with a similar value of each reference constant that is less than or equal to a predetermined threshold among a plurality of predetermined reference constants. Therefore, when determining one or more nodes corresponding to each constant included in the plurality of constants using adders and subtractors, the constant generation circuit can be designed using the fewest number of adders and subtractors by applying an algorithm that approximates each reference constant with a similar value of each reference constant that is less than or equal to a predetermined threshold.

[0112] The multiplexer control unit (430) can control the multiplexer to supply an input value or an operation result by an operation element for an input value to the next operation element according to an applied signal.

[0113] The operational element may include at least one of an adder, a subtracter, and a shift circuit.

[0114] Each constant corresponding to one or more nodes may be pre-generated by the constant generation unit (420).

[0115] The convolution operation performing unit (440) can perform a convolution operation based on input data and multiple constants.

[0116] The convolution operation performing unit (440) can perform a convolution operation by sequentially matching each constant set to input data according to a signal applied to the multiplexer.

[0117] Figure 5 is a block diagram exemplarily showing the functions of a convolution operation device design program.

[0118] As shown in Fig. 5, the convolution operation device design program (250) may include an input unit (510), a constant clustering unit (520), an approximation algorithm application unit (530), and a constant generation circuit generation unit (540). The input unit (510), the constant clustering unit (520), the approximation algorithm application unit (530), and the constant generation circuit generation unit (540) are exemplary divisions of the functions of the convolution operation device creation program (250), and are not limited thereto.

[0119] According to an embodiment, the functions of the input unit (510), constant clustering unit (520), approximation algorithm application unit (530), and constant generation circuit generation unit (540) can be merged / separated, and implemented as a series of commands included in at least one program.

[0120] The input unit (510), constant clustering unit (520), approximation algorithm application unit (530), and constant generation circuit generation unit (540) may be implemented by a processor (230), and may mean a data processing device built into hardware having a physically structured circuit to perform a function expressed by a code or command included in a convolution operation device design program (250) stored in a memory (240).

[0121] The input unit (510) can receive multiple constants.

[0122] A constant may refer to data on which a convolution operation is performed, such as a weight, but is not limited thereto.

[0123] The constant clustering unit (520) can cluster a plurality of constants corresponding to the output size of one or more constant generation circuits that produce constants using at least one of addition, subtraction, and shift operations.

[0124] The constant clustering unit (520) can generate one or more constant sets having a predetermined number of constants from a plurality of constants.

[0125] The constant clustering unit (520) can assign one or more constant sets to one or more constant generation circuits.

[0126] The constant clustering unit (520) can group one or more constant generation circuits corresponding to the size of the multiplexer included in the constant generation circuit.

[0127] The approximation algorithm application unit (530) can apply an approximation algorithm when determining one or more nodes corresponding to each of a plurality of constants included in the constant generation circuit.

[0128] If an approximation algorithm is not applied, adders, subtractors, and shift operators must be placed within the constant generation circuit to produce the same value as each constant contained in the multiple constants.

[0129] In contrast, when an approximation algorithm is applied, adders, subtractors, and shift operators within the constant generation circuit can be arranged so that each reference constant corresponding to each constant included in the multiple constants is produced with a similar value of each reference constant that is less than or equal to a predetermined threshold. Accordingly, the design area of ​​the constant generation circuit can be reduced by determining one or more nodes corresponding to the multiple constants using fewer computational elements.

[0130] A constant generation circuit generation unit (540) can generate a constant generation circuit that generates a plurality of constants and includes one or more nodes corresponding to each of the plurality of constants.

[0131] Figure 6 is an example diagram showing a UMCM circuit.

[0132] A UMCM circuit can be defined as a circuit that can perform a multiplication operation by applying different constant sets depending on the input when multiple constant sets, each of which has a size of n, are given by applying a multiplexer circuit. A UMCM circuit can be composed of an adder, a subtracter, and a multiplexer without using a multiplier.

[0133] Specifically, a set of constants containing n constants , a set of multiple constants containing it When there is an input data x and a set of constants that can be selected i, then the corresponding set of constants For all constants included in , the result of the multiplication operation with the input data x can be obtained.

[0134] Figure 7 is an example diagram showing a convolution operation method using a UMCM circuit.

[0135] The output feature map cube COХHOХWO can be obtained by multiplying the input feature map cube of CIХHIХWI and the weight tensors of CoХCiХKyХKx.

[0136] At this time, if the batch size is 1, it can be performed as an operation of a 6-dimensional nested loop.

[0137] As shown in lines 6-7 of Fig. 7, a single input feature map can be multiplied by CO different weights. Accordingly, when the size of each constant generation circuit is n, CO weights are assigned to CO / n constant generation blocks, and the entire weight tensor can be assigned to a total of CoХCiХKyХKx / n constant generation circuits.

[0138] When the number of constant sets to be assigned to one UMCM circuit is G, a total of CoХCiХKyХKx / (nХG) UMCM circuits may be required.

[0139] Fig. 8 is an exemplary diagram showing a method for designing a convolution operation device using a UMCM circuit.

[0140] The method for designing a convolution operation device according to the present invention can be specifically performed in the following order.

[0141] 1. Enter the weights of the pre-trained model

[0142] 2. Cluster each weight according to the size of the constant generation circuit, n.

[0143] 3. Grouping constant generation circuits to create UMCM circuits

[0144] 4. Create a UMCM circuit for each UMCM group.

[0145] 5. Converting UMCM circuits into Verilog code

[0146] Fig. 9 is an exemplary diagram showing a convolution operation device designed according to the present invention.

[0147] Figure 9 (a) shows that x is input as input data, and weights t1, t2, t3, and t4 are associated with it, and a convolution operation is performed.

[0148] Fig. 9 (b) shows an MCM (multiple constant multiplication) circuit that performs a convolution operation on weights t1, t2, t3, and t4 generated through a constant generation circuit including arithmetic elements composed of an adder, a subtracter, and a shift circuit for input data x. The MCM circuit can perform a multiplication operation on constant sets only through addition, subtraction, and bit shift through resource sharing. In this case, the constant set can be {17, 9, 31, 15}.

[0149] In Fig. 9 (b), the node corresponding to t1 may be 17x, the node corresponding to t2 may be 9x, the node corresponding to t3 may be 31x, and the node corresponding to t4 may be 15x. In this case, the node corresponding to t2, 9x, may be used to produce the node corresponding to t1, 17x. In addition, the node corresponding to t4, 15x, may be used to produce the node corresponding to t3, 31x.

[0150] In Figure 9 (b), four multiplication operations can be replaced with four adders. This enables efficient multiplication operations for a fixed set of constants.

[0151] Figure 9 (c) illustrates a UMCM circuit with a multiplexer applied to the MCM circuit. While the MCM circuit has a one-to-one mapping between constants included in a constant set and outputs, the UMCM circuit can configure multiple constant sets into an integrated circuit through a multiplexer.

[0152] Specifically, unlike the MCM circuit of Fig. 9(b) that required four nodes to generate t1, t2, t3, and t4, the UMCM circuit of Fig. 9(c) can generate t1, t2, t3, and t4 using two nodes by using a multiplexer. In this case, two sets of constants {17,9} and {31,15} can be applied depending on the signal applied to the multiplexer. Accordingly, the design area can be efficiently reduced.

[0153] Fig. 9 (d) shows a UMCM circuit to which an approximation algorithm is applied according to the present invention.

[0154] Since the input constant set is {17, 9, 31, 15}, in order to perform a multiplication operation between the input data x and each constant included in the constant set, {17, 9} and {31, 15} must be output through the constant generation circuit. Therefore, an additional operation element may be required, as shown in Fig. 9 (c).

[0155] In this case, by applying an approximation algorithm to approximate the constant set {17, 9} to {18, 9} and {31, 15} to {30, 15}, the number of computational elements can be reduced and the computation can be performed efficiently.

[0156] FIG. 10 is an exemplary diagram showing a method for generating a node of a constant generation circuit by applying an approximation algorithm according to the present invention.

[0157] Figure 10 includes three nodes corresponding to {15, 9, 1} included in the constant generation circuit. In the case of generating a node corresponding to {35} using an operation element from and including three nodes corresponding to {29, 15, 1}, Let us explain by assuming that a node corresponding to {63} is created using an operation element from .

[0158] In the upper part of Fig. 10 (a), when the approximation algorithm is not applied, {30} can be generated by performing one shift operation on {15}, {4} can be generated by performing two shift operations on {1}, {5} can be generated by performing an addition operation on {4} and {1}, and {35} can be generated by performing an addition operation on {30} and {5}. In this case, two shift operators and two adders may be required.

[0159] In the bottom of Fig. 10 (a), when the approximation algorithm is not applied, {58} can be generated by performing one shift operation on {29}, {5} can be generated by performing two shift operations on {1}, and {63} can be generated by performing an addition operation on {58} and {5}. In this case, two shift operators and one adder may be required.

[0160] In the upper part of Fig. 10 (b), when the approximation algorithm is not applied, {36} can be generated by performing a shift operation twice on {9}, and {35} can be generated by performing a subtraction operation on {36} and {1}. In this case, one shift operator and one subtractor may be required. At this time, in the upper part of Fig. 10 (c), when the approximation algorithm is applied and the preset threshold is 1, {36} can be generated by performing a shift operation twice on {9}, and node generation can be terminated by approximating {35} to {36}. In this case, only one shift operator may be required.

[0161] In the lower part of Fig. 10 (b), when the approximation algorithm is applied and the preset threshold is 1, {60} can be generated by performing a shift operation twice on {15}, {2} can be generated by performing a shift operation once on {1}, and {62} can be generated by performing an addition operation on {60} and {2}. At this time, node generation can be terminated by approximating {63} to {62}. In this case, two shift operators and one adder may be required.

[0162] In the bottom of Fig. 10 (c), when the approximation algorithm is applied and the preset threshold is 4, the node generation can be terminated by performing the shift operation twice on {15} to generate {60} and approximating {63} to {60}. In this case, only one shift operator may be required.

[0163] In this way, when generating a UMCM circuit by applying an approximation algorithm, a constant generation circuit that efficiently produces multiple constants using fewer computational elements can be generated.

[0164] As described above, according to the present invention, a convolution operation device and a design method thereof using a UMCM circuit that are efficient in terms of design area and power can be provided by applying an approximation algorithm.

[0165] The embodiments of the present invention described above may be implemented through various means. For example, the embodiments of the present invention may be implemented using hardware, firmware, software, or a combination thereof.

[0166] The combination of each block of the block diagram and each step of the flowchart attached to the present invention may be performed by computer program instructions. These computer program instructions may be installed in an encoding processor of a general-purpose computer, a special-purpose computer, or other programmable data processing equipment, so that the instructions executed by the encoding processor of the computer or other programmable data processing equipment create a means for performing the functions described in each block of the block diagram or each step of the flowchart. These computer program instructions may also be stored in a computer-available or computer-readable memory that can direct a computer or other programmable data processing equipment to implement the functions in a specific manner, so that the instructions stored in the computer-available or computer-readable memory can also produce an article of manufacture that includes an instruction means for performing the functions described in each block of the block diagram or each step of the flowchart. Since the computer program instructions can also be installed on a computer or other programmable data processing device, a series of operational steps are performed on the computer or other programmable data processing device to create a computer-executable process, and the instructions that cause the computer or other programmable data processing device to perform the steps for executing the functions described in each block of the block diagram and each step of the flowchart can also provide steps for executing the functions described in each block of the block diagram and each step of the flowchart.

[0167] Additionally, each block or step may represent a module, segment, or portion of code that includes one or more executable instructions for performing a specific logical function(s). In some embodiments, the functions mentioned in the blocks or steps may occur out of order. For example, two blocks or steps depicted in succession may actually be performed substantially simultaneously, or the blocks or steps may sometimes be performed in reverse order depending on the corresponding function.

[0168] The above description is merely an illustrative illustration of the technical idea of ​​the present invention, and those skilled in the art will appreciate that various modifications and variations can be made without departing from the essential quality of the present invention. Therefore, the embodiments disclosed in the present invention are intended to illustrate, rather than limit, the technical idea of ​​the present invention, and the scope of the technical idea of ​​the present invention is not limited by these embodiments. The scope of protection of the present invention should be interpreted by the following claims, and all technical ideas within a scope equivalent thereto should be interpreted as being included in the scope of the rights of the present invention.

Claims

1. Input section for receiving input data; A constant generation circuit that generates a plurality of constants and includes one or more nodes corresponding to each of the plurality of constants; A memory capable of storing computer-executable instructions; and By executing the above command, A processor that performs a convolution operation based on the input data and the plurality of constants, The above plural constants are, In a plurality of predetermined reference constants, it is determined by approximating each reference constant with a similar value of each reference constant that is lower than a predetermined threshold. The above multiple reference constants are values ​​corresponding to multiple weights determined through learning of the neural network. Convolution operation unit.

2. In paragraph 1, The above constant generation circuit is, Containing at least one of an adder, a subtracter and a shift circuit, Each constant corresponding to one or more of the above nodes is, Based on the above constant generation circuit, Convolution operation unit.

3. In paragraph 1, The above constant generation circuit is, A multiplexer that supplies an input value or a result of the input value being calculated through an operation element to the next operation element according to an applied signal. Convolution operation unit.

4. In paragraph 3, The above plural constants are, Clustered into one or more sets of constants having a predetermined number of constants generated from the above plurality of constants, The above processor, The convolution operation is performed by sequentially matching each set of constants to the input data according to the signal applied to the multiplexer. Convolution operation unit.

5. In paragraph 4, For each constant in the set of one or more constants above, A position index expressed in two dimensions is specified, The above location index is, A set of one or more constants, each constant having a position index, a value corresponding to at least one node corresponding to each constant, a position index of at least one node corresponding to each constant, a value corresponding to a previous node used to calculate each constant, and a position index of the previous node. Convolution operation unit.

6. In paragraph 5, The above set of one or more constants is, Generated by clustering the plurality of constants corresponding to the output size of the constant generation circuit. Convolution operation unit.

7. In paragraph 5, For a node located after the multiplexer output terminal among the above one or more nodes, The first position index specified corresponding to each constant included in the first constant set and the second position index specified corresponding to each constant included in the second constant set are generated to be the same. Convolution operation unit.

8. In paragraph 5, The position index of at least one node corresponding to constants having the same index in each constant set among the one or more nodes is the same, and the values ​​corresponding to the at least one node are different from each other. Convolution operation unit.

9. In paragraph 5, The above previous node is a node located before the multiplexer input terminal, and the location index of the previous nodes used to produce constants having the same index in each constant set and the values ​​corresponding to the previous nodes are generated to be different from each other. Convolution operation unit.

10. In paragraph 1, One or more of the above nodes, It is generated based on an algorithm that determines the plurality of constants by using the adders and subtractors included in the constant generation circuit as little as possible. Convolution operation unit.

11. A method for designing a convolution operation device performed by a computing device, Step of receiving multiple constants; A step of clustering into one or more sets of constants having a predetermined number of constants generated from the plurality of constants; and A step of generating a plurality of constants and generating a constant generating circuit including one or more nodes corresponding to each of the plurality of constants, The above plural constants are, In a plurality of predetermined reference constants, it is determined by approximating each reference constant with a similar value of each reference constant that is lower than a predetermined threshold. The above multiple reference constants are values ​​corresponding to multiple weights determined through learning of the neural network. A method for designing a convolution operation unit.

12. In paragraph 11, The above constant generation circuit is, Containing at least one of an adder, a subtracter and a shift circuit, Each constant corresponding to one or more of the above nodes is, Based on the above constant generation circuit, A method for designing a convolution operation unit.

13. In paragraph 11, The above constant generation circuit is, A multiplexer that supplies an input value or a result of the input value being calculated through an operation element to the next operation element according to an applied signal. A method for designing a convolution operation unit.

14. In paragraph 13, The above plural constants are, Clustered into one or more sets of constants having a predetermined number of constants generated from the above plurality of constants, A convolution operation is performed by sequentially corresponding each set of constants to the input data according to the signal applied to the multiplexer. A method for designing a convolution operation unit.

15. In paragraph 14, For each constant in the set of one or more constants above, A position index expressed in two dimensions is specified, The above location index is, A set of one or more constants, each constant having a position index, a value corresponding to at least one node corresponding to each constant, a position index of at least one node corresponding to each constant, a value corresponding to a previous node used to calculate each constant, and a position index of the previous node. A method for designing a convolution operation unit.

16. In paragraph 15, The above set of one or more constants is, Generated by clustering the plurality of constants corresponding to the output size of the constant generation circuit. A method for designing a convolution operation unit.

17. In paragraph 15, For a node located after the multiplexer output terminal among the above one or more nodes, The first position index specified corresponding to each constant included in the first constant set and the second position index specified corresponding to each constant included in the second constant set are generated to be the same. A method for designing a convolution operation unit.

18. In paragraph 15, The position index of at least one node corresponding to constants having the same index in each constant set among the one or more nodes is the same, and the values ​​corresponding to the at least one node are different from each other. A method for designing a convolution operation unit.

19. In paragraph 11, One or more of the above nodes, It is generated based on an algorithm that determines the plurality of constants by using the adders and subtractors included in the constant generation circuit as little as possible. A method for designing a convolution operation unit.

20. A computer-readable recording medium storing computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, Step of receiving multiple constants; A step of clustering into one or more sets of constants having a predetermined number of constants generated from the plurality of constants; and A step of generating a plurality of constants and generating a constant generating circuit including one or more nodes corresponding to each of the plurality of constants, The above plural constants are, In a plurality of predetermined reference constants, it is determined by approximating each reference constant with a similar value of each reference constant that is lower than a predetermined threshold. The method of causing the processor to perform the above method, wherein the plurality of reference constants are values ​​corresponding to a plurality of weights determined through learning of the neural network. Computer-readable recording medium.

Images