Method and system for performing dilated convolution operation in neural network

Abstract

A system and a method for performing convolution operations using variable expansion rates in a neural network are provided. A computing system may determine the size of one or more decomposed kernels based on at least one of the size of a first kernel, the basic structure of memory associated with the computing system, and an expansion rate. Additionally, the computing system may determine the address of one or more blocks of an input image corresponding to each of the one or more decomposed kernels based on the expansion rate and one or more parameters associated with the size of memory and the input image. Additionally, the computing system may retrieve one or more blocks of an input image and one or more decomposed kernels from memory based on the address. Additionally, the computing system may determine an output image based on the convolution operation of each of the one or more decomposed kernels and one or more blocks of the input image.

Description

Technology Field

[0001] The present disclosure relates to a system and a method for performing extended convolution operations. Background Technology

[0002] Dilated convolution is used in deep learning neural networks to perform scene segmentation and molecular depth estimation on input images or input videos. Dilated convolution can provide more accurate results compared to un-dilated convolution.

[0003] Convolution techniques involve calculating an inner product between one or more kernel values ​​and one or more input image values. The inner product can be calculated over the width and height of the input image. In the case of expansion convolution techniques, one or more expansion kernel values ​​are obtained by inserting zeros between one or more kernel values ​​in the x and y directions. The number of zeros inserted to obtain the expansion kernel is based on the expansion rate. The convolution operation between the expansion kernel and the input image involves a multiplication operation between one or more values ​​of the input image and the zero-inserted expansion kernel.

[0004] Recently, there is a demand for technology to efficiently perform extended convolution. The problem to be solved

[0005] The invention provides a system and a method for performing extended convolution operations. Additionally, the invention provides a computer-readable recording medium storing a program for executing the above method on a computer. The technical problems to be solved are not limited to those described above, and other technical problems may exist. means of solving the problem

[0006] As a technical means for achieving the technical problem described above, a first aspect of the present disclosure may provide a method for performing a convolution operation using a variable expansion rate in a neural network, comprising: receiving a size and expansion rate of a first kernel by a computing system; determining a size of one or more decomposed kernels by the computing system based on at least one of the size of the first kernel, a basic structure of memory associated with the computing system, and the expansion rate; determining an address for one or more blocks of an input image corresponding to each of the one or more decomposed kernels by the computing system based on the expansion rate and one or more parameters associated with the size of the memory and the input image by the computing system; fetching the one or more blocks of the input image and the one or more decomposed kernels from the memory by the computing system based on the address; and determining an output image by the computing system based on a convolution operation of each of the one or more decomposed kernels and the one or more blocks of the input image.

[0007] A second aspect of the present disclosure provides a computing system for performing convolution operations using a variable expansion rate in a neural network, comprising: a processor; and a memory coupled to the processor in a communicable manner; wherein the processor receives the size and expansion rate of a first kernel by executing processor instructions stored in the memory, determines the size of one or more decomposed kernels based on at least one of the size of the first kernel, the basic structure of the memory associated with the computing system, and the expansion rate, determines an address for one or more blocks of an input image corresponding to each of the one or more decomposed kernels based on the expansion rate and one or more parameters associated with the size of the memory and the input image, retrieves the one or more blocks of the input image and the one or more decomposed kernels from the memory based on the address, and determines an output image based on a convolution operation of each of the one or more decomposed kernels and the one or more blocks of the input image.

[0008] A third aspect of the present disclosure may provide a computer-readable recording medium having a program for executing the method of the first aspect on a computer. Brief explanation of the drawing

[0009] FIGS. 1a to 1d are exemplary drawings for explaining a method of performing a convolution operation according to one embodiment. FIG. 2a is a block diagram of a computing system according to one embodiment. FIG. 2b is an exemplary drawing for illustrating one or more blocks of an input image according to one embodiment. FIG. 3 is a flowchart illustrating a method of performing convolution operations using a variable expansion rate in a neural network according to one embodiment. FIGS. 4a to 4i are drawings for explaining an example of performing a convolution operation using a variable expansion rate according to one embodiment. FIG. 5 is an exemplary block diagram of a computer system according to one embodiment. Specific details for implementing the invention

[0010] Phrases such as "in some embodiments" or "in one embodiment" appearing in various places in this specification do not necessarily refer to the same embodiment.

[0011] Some embodiments of the present disclosure may be represented by functional block configurations and various processing steps. Some or all of these functional blocks may be implemented by various numbers of hardware and / or software configurations that execute specific functions. For example, the functional blocks of the present disclosure may be implemented by one or more microprocessors or by circuit configurations for a specific function. Additionally, for example, the functional blocks of the present disclosure may be implemented in various programming or scripting languages. The functional blocks may be implemented as algorithms executed on one or more processors. Furthermore, the present disclosure may employ prior art for electronic configuration, signal processing, and / or data processing, etc. Terms such as “mechanism,” “element,” “means,” and “configuration” may be used broadly and are not limited to mechanical and physical configurations.

[0012] Furthermore, the connecting lines or connecting members between the components depicted in the drawings are merely illustrative of functional connections and / or physical or circuit connections. In the actual device, connections between components may be represented by various alternative or added functional connections, physical connections, or circuit connections.

[0013] The present disclosure will be described in detail below with reference to the attached drawings.

[0014] FIGS. 1a to 1d are exemplary drawings for explaining a method of performing a convolution operation according to one embodiment.

[0015] The present disclosure relates to a method for performing convolution operations using a variable scaling rate in a neural network. A convolution operation is an operation for extracting features from an input image to perform one or more image processing tasks, such as object recognition or scene segmentation. The convolution operation includes calculating an inner product between one or more kernel values ​​and a plurality of pixel values ​​of the input image. Additionally, the inner product is calculated over the width and height of the input image.

[0016] For example, as illustrated in FIG. 1a, a convolution operation can be performed between an input image (101) having a plurality of pixel values ​​(104) and a first kernel (102) having one or more kernel values ​​(105).

[0017] The input image (101) is represented as a matrix having 6 rows and 6 columns. The first kernel (102) is represented as a matrix having 3 rows and 3 columns. As illustrated in FIG. 1a, an output image (103) can be obtained as a result of performing a convolution operation between the input image (101) and the first kernel (102) when the stride is "1".

[0018] The value corresponding to the first row and first column (i.e., -5) and the value corresponding to the first row and second column (i.e., -4) of the output image (103) obtained as a result of performing convolution between the input image (101) and the first kernel (102) are calculated as shown in FIG. 1b.

[0019] In addition, the number of rows and columns of the output image (103) can be calculated as shown in the following mathematical formulas 1 and 2.

[0020]

[0021]

[0022] In the above mathematical formulas 1 and 2, the stride represents the number of pixels to skip across the width and height to calculate the inner product of the input image (101). For example, a stride of "2" indicates shifting 2 rows and 2 columns across the width and height while calculating the inner product of the input image (101).

[0023] As the value of the stride increases, the number of inner products calculated over the width and height of the input image (101) decreases, so the size of the output image (103) decreases. For example, referring to FIG. 1c, the result of performing a convolution operation between the input image (101) and the first kernel (102) when the stride is "2" is shown.

[0024] In one embodiment, if the number of rows and columns of the output image (103) is equal to the number of rows and columns of the input image (101), one or more rows and one or more columns of the input image (101) may be padded with "0". For example, referring to FIG. 1d, two rows and two columns of the input image (101) are padded with "0".

[0025] FIG. 2a is a block diagram of a computing system (200) according to one embodiment.

[0026] Referring to FIG. 2a, the computing system (200) may include a processor (203) as a central processing unit and a memory (202) that stores instructions executable by the processor (203).

[0027] The processor (203) may include at least one data processor that executes program components for executing a user or system-generated request. Memory (202) may be communicably connected to the processor (203).

[0028] The computing system (200) further includes an I / O (input / output) interface (201). The I / O interface (201) may be connected to a processor (203) capable of communicating input signals or output signals. In one embodiment, the computing system (200) may receive an input image (101), a first kernel (102), and an extension rate through the I / O interface (201).

[0029] In one embodiment, the computing system (200) may include data (204) and a module (209) as illustrated in FIG. 2a. For example, the data (204) and the module (209) may be stored in a memory (202) configured in the computing system (200). For example, the data (204) may include input and output data (205), extended data (206), address data (207), and other data (208). The data (204) will be described in detail below.

[0030] In one embodiment, the input and output data (205) includes at least one of a plurality of pixel values ​​(104) of an input image (101), one or more kernel values ​​(105) associated with a first kernel (102), and a plurality of pixel values ​​(104) of an output image (103). Additionally, the input and output data (205) includes at least one of the size of the input image (101), the size of the output image (103), and the size of the first kernel (102). The size represents the number of rows and columns included in the matrix of the input image (101), the first kernel (102), and the output image (103).

[0031] For example, as shown in FIG. 1a, the size of the input image (101) having multiple pixel values ​​(104) is 6x6, the size of the first kernel (102) having one or more kernel values ​​(105) is 3x3, and the size of the output image (103) having multiple pixel values ​​(104) is 4x4.

[0032] In one embodiment, the expansion data (206) includes at least one of an expansion rate, one or more kernel values ​​(105) associated with the second kernel, and the size of the second kernel. The size represents the number of rows and columns included in the matrix of the second kernel.

[0033] The expansion rate represents the number of one or more "0s" to be inserted between one or more kernel values ​​(105) of the first kernel (102) to create a second kernel. One or more "0s" are inserted in the x-direction and y-direction of the first kernel (102). For example, if the expansion rate is "3", two "0s" are inserted between one or more kernel values ​​(105) in the x-direction and y-direction of the first kernel (102). The number of one or more "0s" inserted between one or more kernel values ​​(105) is one less than the expansion rate.

[0034] In one embodiment, the address data (207) includes at least one of an address for one or more blocks of an input image (101), an address for one or more kernel values ​​(105) associated with a first kernel (102), and an address for a kernel value (105) associated with a second kernel.

[0035] FIG. 2b is an exemplary drawing for illustrating one or more blocks of an input image according to one embodiment.

[0036] One or more pixel values ​​among the multiple pixel values ​​(104) of the input image (101) can be grouped to form one or more blocks. For example, referring to FIG. 2b, one or more pixel values ​​among the multiple pixel values ​​(104) of the input image (101) shown in FIG. 1a are grouped into one or more blocks (215) (i.e., four blocks (215)).

[0037] Additionally, the address of one or more blocks (215) is expressed using at least one of binary, hexadecimal, etc. For example, the address of the first block of the input image (101) expressed in binary is indicated as "0100110110" and the address of the first block of the input image (101) expressed in hexadecimal is indicated as "1D295F".

[0038] In one embodiment, other data (208) may include the size of the basic structure of the memory (202), the address of one or more decomposed kernels, and maximum value information associated with counters (e.g., cell number X, cell number Y, cell number Z, extended offset X and extended offset Y, etc.).

[0039] In one embodiment, data (204) may be stored in memory (202) in the form of various data structures. Additionally, data (204) may be configured using a data model such as a relational data model or a hierarchical data model. Other data (208) may store data including temporary data and temporary files generated by a module (209) to perform various functions of the computing system (200). Additionally, in one embodiment, memory (202) may include at least one of Static Random-Access Memory (SRAM), Dynamic Random-Access Memory (DRAM), a global buffer, etc.

[0040] In one embodiment, data (204) stored in memory (202) may be processed by a module (209) of the computing system (200). The module (209) may be stored within memory (202), may be connected to communicate with the processor (203), or may exist outside memory (202) as illustrated in FIG. 2a. In this disclosure, the term module (209) may refer to an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), an electronic circuit, a processor (203) (shared, private, or group), and memory (202) executing one or more software or firmware programs, combinational logic circuits, and / or other suitable components providing the described functions. In another embodiment, the module (209) may be implemented using at least one of an ASIC and an FPGA.

[0041] In one embodiment, the module (209) may include, for example, a size determination module (210), an address generation module (211), a read (or read) write module (212), a convolution module (213), and other modules (214). A person skilled in the art will understand that the above-described module (209) may be represented as a single module or a combination of different modules.

[0042] In one embodiment, the size determination module (210) can determine the size of one or more decomposed kernels. The size of one or more decomposed kernels can be determined based on at least one of the size of the first kernel (102), the basic structure of the memory (202) associated with the computing system (200), and the expansion rate.

[0043] The basic structure of the memory (202) may represent the number of multiple pixel values ​​(104) stored in each of one or more blocks (215) of the memory (202). The multiple pixel values ​​(104) associated with the first block among the one or more blocks (215) can be read from the memory (202) through one read cycle.

[0044] Multiple pixel values ​​(104) are associated with one or more rows and one or more columns of an input image (101). For example, the basic structure of one or more blocks (215) shown in FIG. 2b is 3x3. A read operation associated with the first block of FIG. 2b retrieves a group of nine pixel values ​​(104) associated with the first three rows and the first three columns of the input image (101) shown in FIG. 1a.

[0045] In another embodiment, for an 8x4 basic structure with a first kernel (102) having a size of 3x3 and an expansion rate of "6", the size of one or more decomposed kernels is 3x7.

[0046] In one embodiment, the address generation module (211) can generate the address of one or more blocks (215) of the input image (101).

[0047] The address of one or more blocks (215) can be determined using one or more parameters including at least one of the base address of the first block among one or more blocks (215) in memory (202), column stride, cell number X, row stride, cell number Y, channel stride, cell number Z, extended column stride, extended offset X, extended row stride, and extended offset Y. X may represent the block (215) number in the horizontal direction of the input image (101), Y may represent the block (215) number in the vertical direction of the input image (101), and Z may represent the channel-specific block (215) number in the input image (101).

[0048] The address of one or more blocks (215) can be determined through the following mathematical formulas 3 to 8.

[0049]

[0050]

[0051]

[0052]

[0053]

[0054]

[0055] In the above mathematical formula 8, the base address represents the address of the pixel value of the first row and first column of the input image (10). Cell number X, cell number Y, cell number Z, extended offset X, and extended offset Y represent counters.

[0056] In one embodiment, the read / write module (212) can retrieve one or more blocks (215) of an input image (101) from memory (202) based on the address of one or more blocks (215) generated by the address generation module (211). Retrieving one or more blocks (215) of an input image (101) from the read / write module (212) includes retrieving a plurality of pixel values ​​(104) corresponding to each of the one or more blocks (215) of the input image (101).

[0057] Additionally, the read / write module (212) can retrieve one or more decomposed kernels from memory (202). Additionally, the read / write module (212) can write one or more pixel values ​​(104) of the output image (103) to memory (202).

[0058] In one embodiment, the convolution module (213) may perform a convolution operation between one or more blocks (215) of an input image (101) and one or more decomposed kernels to determine an output image (103).

[0059] The convolution module (213) can determine an output image (103) by generating one or more matrices by multiplying one or more kernel values ​​(105) of each of one or more decomposed kernels by a plurality of pixel values ​​(104) corresponding to each of one or more blocks (215) of the input image (101). Additionally, the output image (103) can be determined by adding one or more matrices.

[0060] In one embodiment, the other module (214) may determine a second kernel based on the first kernel (102) and the expansion rate, determine the address of one or more decomposed kernels, determine at least one of a column stride, a row stride, and a channel stride, and determine values ​​associated with cell number X, cell number Y, cell number Z, expansion offset X, expansion offset Y, etc.

[0061] FIG. 3 is a flowchart illustrating a method of performing convolution operations using a variable expansion rate in a neural network according to one embodiment.

[0062] FIGS. 4a to 4i are drawings for explaining an example of performing a convolution operation using a variable expansion rate according to one embodiment.

[0063] Below, each step of FIG. 3 is described in detail with reference to FIG. 4a to 4i.

[0064] Referring to FIG. 3, in step 301, the computing system (200) may receive a first kernel (102) and an expansion rate. For example, the size of the first kernel (102) may be "3x3" and the expansion rate may be "8".

[0065] In one embodiment, the first kernel (102) represents an un-dilated kernel. The un-dilated kernel may include one or more rows and one or more columns of one or more kernel values ​​(105). The size of the first kernel (102) may be 2x2, 3x3, 4x4, etc.

[0066] Additionally, the expansion rate indicates the number of one or more "0s" to be inserted between one or more kernel values ​​(105) of the first kernel (102) to generate the second kernel. One or more "0s" are inserted in the x-direction and y-direction of the first kernel (102). The number of one or more "0s" inserted between one or more kernel values ​​(105) is one less than the expansion rate. For example, assuming the first kernel (102) has a size of 3x3, the gray-shaded portion shown in FIG. 4a may represent one or more kernel values ​​(105) of the first kernel (102).

[0067] For example, as the first kernel (102) is expanded with an expansion rate of "2", a second kernel (401) with a size of 5x5 is created. One or more "0"s inserted during the process of creating the second kernel (401) are displayed in white shading.

[0068] Alternatively, as the first kernel (102) is expanded according to an expansion rate of "5", a second kernel (401) having a size of 11x11 is generated. The gray shading of the second kernel (401) represents one or more kernel values ​​(105), and the white shading represents "0" inserted between one or more kernel values ​​(105).

[0069] In step 302, the computing system (200) can determine the size of one or more decomposed kernels based on at least one of the size of the first kernel, the basic structure of the memory associated with the computing system, and the expansion rate.

[0070] In one embodiment, the basic structure of the memory (202) represents a count of a plurality of pixel values ​​(104) read from the memory (202) during one read cycle. Additionally, the size of one or more blocks (215) of the input image (101) stored in the memory (202) is based on the basic structure of the memory (202).

[0071] For example, if the basic structure is 4x8, 32 pixel values ​​(104) are read from memory during a read cycle. The 32 pixel values ​​(104) can be stored in one row or one column in memory (202). Additionally, the 32 pixel values ​​(104) can correspond to multiple pixel values ​​in the input image (101). The 32 pixel values ​​(104) in the input image (101) can correspond to 4 rows and 8 columns.

[0072] For example, FIG. 4b shows an input image (101) of size 20x24. Multiple pixel values ​​(104) are grouped into 15 blocks (215) of size 4x8, where each block contains 32 pixel values ​​(104). The 32 pixel values ​​(104) corresponding to each block are stored in one row of memory (202). A computing system (200) can read the 32 pixel values ​​(104) stored in one row of memory (202) in one read cycle.

[0073] In one embodiment, one or more blocks (215) of an input image (101) stored in memory (202) may be based on at least one of the basic structure and expansion rate of memory (202). For example, if the basic structure of memory (202) is 4x8 and the expansion rate is "6", a plurality of pixel values ​​(104) of the input image (101) are divided into shards as shown in FIG. 4c. Each shard containing 32 pixel values ​​(104) is stored in memory (202) as one row or one column.

[0074] In one embodiment, for a basic structure of size 4x8, the size of one or more decomposed kernels corresponding to a second kernel (401) having a variable expansion rate is shown in the table (402) of FIG. 4d.

[0075] "n" listed in the table (402) of FIG. 4d is a natural number. The value of "n" is selected to obtain the received expansion rate. Based on the value of "n" and the expansion rate, the size corresponding to one or more decomposed kernels is determined from the table (402). For example, if the expansion rate is 14, the value of "n" is selected as "1", and the size of one or more decomposed kernels becomes 3x7. In other examples, if the expansion rate is 10, 18, 26, 34, etc., the size of one or more decomposed kernels becomes 3x5. A person skilled in the art will understand that for one or more basic structures of memory (202) and one or more expansion rates, the table (402) is determined as in FIG. 4d.

[0076] In step 303, the computing system (200) can determine the address of one or more blocks of the input image corresponding to each of one or more decomposed kernels based on one or more parameters associated with the expansion rate and the size of the memory and the input image.

[0077] In one embodiment, an address for one or more blocks may be determined using one or more parameters including at least one of the base address of a first block among one or more blocks in memory, a column stride, a cell number X, a row stride, a cell number Y, a channel stride, a cell number Z, an extended column stride, an extended offset X, an extended row stride, and an extended offset Y. X represents the horizontal block number of the input image, Y represents the vertical block number of the input image, and Z represents the block number per channel of the image.

[0078] The addresses of one or more blocks (215) can be generated through the above mathematical formulas 3 through 8. In one embodiment, the maximum value of the counter may be predetermined based on the expansion rate. For example, assuming the size of the basic structure of the memory (202) is 4x8, the expansion rate is 8, and the size of the input image (101) is 20x24, the addresses of one or more blocks (215) of the input image (101) can be determined as shown in FIG. 4e. The addresses determined for one or more blocks (215) are shown in gray shading in FIG. 4e.

[0079] One or more unshaded blocks (215) of the input image (101) are multiplied by one or more "0s" inserted into the second kernel (410) and do not contribute to the determination of the output image (103). Therefore, since one or more unshaded blocks (215) are not required to determine the output image (103), the address for one or more unshaded blocks (215) is not determined.

[0080] In step 304, the computing system (200) can retrieve one or more blocks (215) and one or more decomposed kernels from memory (202) based on an address.

[0081] In one embodiment, the step of retrieving one or more blocks (215) of an input image (101) includes the step of retrieving a plurality of pixel values ​​(104) corresponding to each of the one or more blocks (215) of the input image (101).

[0082] The addresses of one or more blocks (215) of an input image (101) are provided to memory (202) to retrieve multiple pixel values ​​(104). A computing system (200) can retrieve a subset of one or more blocks (215) of an input image (101) related to determining an output image (103) based on convolution.

[0083] In one embodiment, the step of retrieving one or more decomposed kernels from memory (202) may include the step of identifying one or more kernel values ​​(105) corresponding to one or more decomposed kernels of the second kernel (401), wherein the second kernel (401) may be generated based on the first kernel (102) and the expansion rate. Additionally, the step of retrieving one or more decomposed kernels from memory (202) may include the step of retrieving one or more kernel values ​​corresponding to one or more decomposed kernels based on the size of one or more decomposed kernels.

[0084] For example, assume a second kernel (401) having a size of 17x17 generated from a first kernel (102) having a size of 3x3 for an expansion rate of "8". A kernel (403) having a size of 1x1 is identified as shown in FIG. 4f. Or, one or more kernel values ​​(105) corresponding to one or more decomposed kernels (403) having a size of 1x5 corresponding to a second kernel (401) having a size of 25x25 are as shown in FIG. 4g.

[0085] Additionally, the address of one or more decomposed kernels (403) may be provided to memory (202) to retrieve one or more kernel values ​​(105) associated with each of the one or more decomposed kernels (403).

[0086] In step 305, the computing system (200) can determine an output image based on one or more decomposed kernels and convolution operations of one or more blocks of the input image.

[0087] In one embodiment, the step of determining the output image (103) may include the step of generating one or more matrices by multiplying one or more kernel values ​​(105) of each of one or more decomposed kernels (403) by a plurality of corresponding pixel values ​​(104). Additionally, the step of determining the output image (103) may include the step of adding one or more matrices.

[0088] For example, as illustrated in FIG. 4h, assume an input image (101) of size 20x24 and one or more decomposed kernels (403) of size 1x1. One or more blocks (215) of the input image (101) can be multiplied by the corresponding one or more decomposed kernels (403) to generate one or more matrices as illustrated in FIG. 4h. To determine the output image (103), the generated one or more matrices can be added.

[0089] In one embodiment, as illustrated in FIG. 4i, an input image (101) may be divided into shards (404) based on an expansion rate and stored in memory (202). When the input image (101) is divided into shards (404), the expansion rate decreases based on the number of shards (404) created. For example, if the input image (101) is divided into 4 shards, the expansion rate decreases from 6 to 3.

[0090] For each shard (404), one or more matrices may be generated by performing a convolution operation with each of one or more decomposed kernels (403). Additionally, one or more matrices may be treated as shards (404), and an output image (103) may be generated by combining one or more matrices.

[0091] In one embodiment, a method for performing convolution operations using a variable expansion rate in a neural network can divide a second kernel (401) into one or more decomposed kernels (403) and perform convolution operations between one or more blocks (215) of an input image (101) using each decomposed kernel (403).

[0092] Since one or more blocks (215) in which a convolution operation is performed with "0" in the second kernel (401) are not retrieved from memory (202), one or more blocks (215) to be retrieved from memory (202), i.e., multiple pixel values ​​(104) of the input image (101) are reduced.

[0093] The number of read cycles required to retrieve one or more blocks (215) from memory (202) is reduced. Additionally, the step of retrieving one or more decomposed kernels (403) includes the step of retrieving one or more partial kernel values ​​(105) from the second kernel (401), so the number of read cycles and one or more kernel values ​​(105) retrieved from memory (202) is reduced.

[0094] In addition, fewer pixel values ​​(104) and fewer kernel values ​​(105) are retrieved from memory (202), thereby reducing the number of multiplication and addition operations performed during the convolution process. In addition, as the number of multiplication and addition operations is reduced, the determination speed of the output image (103) can be increased.

[0095] FIG. 5 is an exemplary block diagram of a computer system (500) according to one embodiment.

[0096] In one embodiment, a computer system (500) may be used to implement a method for performing convolution operations using a variable expansion rate in a neural network. The computer system (500) may include a processor (502). The processor (502) may include at least one data processor for implementing program components for performing convolution operations using a variable expansion rate in a neural network. The processor (502) may include an integrated system (bus) controller, a memory management control unit, a floating-point unit, a graphics processing unit, a digital signal processing unit, etc.

[0097] The processor (502) may be configured to communicate with one or more I / O devices (not shown) through an I / O interface (501). The I / O interface (501) may use communication protocols / methods without limitation, such as audio, analog, digital, monaural, RCA, stereo, IEEE-1394, serial bus, USB (Universal Serial Bus), infrared, PS / 2, BNC, coaxial, component, composite, digital visual interface (DVI), high definition multimedia interface (HDMI), RF antenna, S-Video, VGA, IEEE 802.n / b / g / n / x, Bluetooth, cellular (e.g., Code Division Multiple Access (CDMA), High Speed ​​Packet Access (HSPA+), Global Mobile Communication System (GSM), LTE (long-term evolution), WiMax, etc.).

[0098] Using an I / O interface (501), a computer system (500) can communicate with one or more I / O devices. For example, an input device (510) may include an antenna, a keyboard, a mouse, a joystick, a (infrared) remote control, a camera, a card reader, a fax machine, a dongle, a biometric reader, a microphone, a touch screen, a touchpad, a trackball, a stylus, a scanner, a storage device, a transceiver, a video device / source, etc. An output device (511) may include a printer, a fax machine, a video display (e.g., a cathode ray tube (CRT), a liquid crystal display (LCD), a light-emitting diode (LED), a plasma display panel (PDP), an organic light-emitting diode display (OLED), etc.), an audio speaker, etc.

[0099] In one embodiment, the computer system (500) may be connected to a service operator via a communication network (509). A processor (502) may be configured to communicate with the communication network (509) via a network interface (503). The network interface (503) may communicate with the communication network (509).

[0100] The network interface (503) may use a direct connection, Ethernet (e.g., twisted pair 10 / 100 / 1000 Base T), Transmission Control Protocol / Internet Protocol (TCP / IP), Token Ring, IEEE 802.11a / b / g / n / x, etc., but is not limited thereto. The communication network (509) may include a direct interconnection, an e-commerce network, a P2P network, a LAN (local area network), a WAN (wide area network), a wireless network (e.g. using a wireless application protocol), the Internet, Wi-Fi, etc., but is not limited thereto. Using the network interface (503) and the communication network (509), the computer system (500) may communicate with one or more service operators.

[0101] In one embodiment, the processor (502) may be configured to communicate with memory (505) (e.g., RAM, ROM, etc. not shown in FIG. 5) through a storage interface (504). The storage interface (504) may be connected to memory (505), including but not limited to memory drives, removable disk drives, etc., by employing connection protocols such as SATA (Serial Advanced Technology Attachment), IDE (Integrated Drive Electronics), IEEE-1394, USB (Universal Serial Bus), Optical Channel, SCSI (Small Computer Systems Interface), etc. Memory drives may further include drums, magnetic disk drives, optical magnetic drives, optical drives, RAID (Redundant Array of Independent Discs), solid-state memory devices, solid-state drives, etc.

[0102] Memory (505) may store a collection of program or database components, including but not limited to user / application data (506), an operating system (507), a web server (508), etc. In one embodiment, a computer system (500) stores user / application data (506), such as data, variables, records, etc., as described in this disclosure. Such a database may be implemented as a fault-tolerant, relational, scalable, and secure database, such as Oracle or Sybase.

[0103] An operating system (507) can facilitate resource management and operation of a computer system (500). Examples of operating systems (507) may include APPLE® MACINTOSH® OS X®, UNIX®, UNIX-like system distributions (e.g., BERKELEY SOFTWARE DISTRIBUTION® (BSD), FREEBSD®, NETBSD®, OPENBSD, etc.), LINUX® distributions (e.g., RED HAT®, UBUNTU®, KUBUNTU®, etc.), IBM®OS / 2®, MICROSOFT® WINDOWS® (XP®, VISTA® / 7 / 8, 10, etc.), APPLE® IOS®, GOOGLE™ ANDROID™, BLACKBERRY® OS, etc.

[0104] In one embodiment, the computer system (500) may implement a web browser storage program component. The web browser may be a hypertext viewing application such as MICROSOFT® INTERNET EXPLORER®, GOOGLE™ CHROMETM, MOZILLA® FIREFOX®, APPLE® SAFARI®, etc. Secure web browsing may be provided using HTTPS (Secure Hypertext Transport Protocol), SSL (Secure Sockets Layer), TLS (Transport Layer Security), etc. The web browser may use functions such as AJAX, HTML, ADOBE® FLASH®, JAVASCRIPT®, JAVA®, API (Application Programming Interfaces), etc.

[0105] The computer system (500) may implement a mail server (not shown) storage program component. The mail server may be an internet mail server such as Microsoft Exchange. The mail server may use functions such as ASP (Active Server Pages), ACTIVEX®, ANSI® C++ / C#, MICROSOFT®, .NET, CGI SCRIPTS, JAVA®, JAVASCRIPT®, PERL®, PHP, PYTHON®, WEBOBJECTS®, etc. The mail server may use communication protocols such as IMAP (Internet Message Access Protocol), MAPI (Messaging Application Programming Interface), MICROSOFT® Exchange, POP (Post Office Protocol), SMTP (Simple Mail Transfer Protocol), etc. In one embodiment, the computer system (500) may implement a mail client storage program component (not shown). Mail clients can be mail viewing applications such as APPLE® MAIL, MICROSOFT® ENTOURAGE®, MICROSOFT® OUTLOOK®, MOZILLA® THUNDERBIRD®, etc.

[0106] The basic structure of the input image (101), expansion rate, and memory (202) can be received from remote devices (512).

[0107] The embodiments may also be implemented in the form of a recording medium containing computer-executable instructions, such as program modules executed by a computer. A computer-readable medium may be any available medium accessible by a computer and includes both volatile and non-volatile media, and both removable and inseparable media. Additionally, a computer-readable medium may include both computer storage media and communication media. A computer storage medium includes both volatile and non-volatile, removable and inseparable media implemented by any method or technique for storing information, such as computer-readable instructions, data structures, program modules, or other data. A communication medium typically includes computer-readable instructions, data structures, other data of modulated data signals such as program modules, or other transmission mechanisms, and includes any information transmission medium.

[0108] Additionally, in this specification, “part” may be a hardware component, such as a processor or circuit, and / or a software component executed by a hardware component, such as a processor.

[0109] The foregoing description of this specification is for illustrative purposes only, and those skilled in the art will understand that other specific forms can be easily modified without altering the technical spirit or essential features of the invention. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, each component described as a single unit may be implemented in a distributed manner, and components described as distributed may likewise be implemented in a combined form.

[0110] The scope of the present embodiment is defined by the claims set forth below rather than by the detailed description above, and should be interpreted to include all modifications or variations derived from the meaning and scope of the claims and the concept of equivalents.

Claims

Claim 1 A method for performing convolution operations using a variable dilation rate in a neural network comprises: receiving the size and dilation rate of a first kernel by a computing system; determining a second kernel by the computing system based on the first kernel and the dilation rate; dividing the second kernel into one or more decomposed kernels by the computing system; determining the size of the one or more decomposed kernels by the computing system based on at least one of the size of the first kernel, the baseline architecture of memory associated with the computing system, and the dilation rate; determining the address of one or more blocks of an input image corresponding to each of the one or more decomposed kernels by the computing system based on the dilation rate and one or more parameters associated with the size of the memory and the input image; fetching the one or more blocks of the input image and the one or more decomposed kernels from the memory by the computing system based on the address. A method comprising the step of determining an output image based on a convolution operation of each of the one or more decomposed kernels and the one or more blocks of the input image by the computing system. Claim 2 The method according to claim 1, wherein the step of determining an address for one or more blocks comprises: determining an address for one or more blocks using at least one parameter including a base address of a first block among one or more blocks in memory, a column stride, a cell number X, a row stride, a cell number Y, a channel stride, a cell number Z, an extended column stride, an extended offset X, an extended row stride, and an extended offset Y; wherein X represents a horizontal block number of the input image, Y represents a vertical block number of the input image, and Z represents a block number per channel of the image. Claim 3 A method according to claim 1, wherein the size of one or more blocks of the input image stored in the memory is determined based on the basic structure of the memory. Claim 4 A method according to claim 1, wherein the size of one or more blocks of the input image stored in the memory is determined based on at least one of the basic structure of the memory and the expansion rate. Claim 5 A method according to claim 1, wherein the step of retrieving one or more blocks of the input image from the memory comprises the step of retrieving a plurality of pixel values ​​of the input image corresponding to each of the one or more blocks of the input image. Claim 6 A method according to claim 1, wherein the step of retrieving one or more decomposed kernels from the memory comprises: a step of identifying one or more kernel values ​​corresponding to one or more decomposed kernels within the second kernel; and a step of retrieving one or more kernel values ​​corresponding to one or more decomposed kernels based on the size of the one or more decomposed kernels. Claim 7 A method according to claim 1, wherein the step of determining the output image comprises: generating one or more matrices by multiplying one or more kernel values ​​of each of the one or more decomposed kernels and a plurality of pixel values ​​corresponding to each of the one or more blocks of the input image; and determining the output image by adding the one or more matrices. Claim 8 A computing system for performing convolution operations using a variable expansion rate in a neural network, comprising: a processor; and a memory coupled to the processor so as to be communicable with the processor; wherein the processor receives the size and expansion rate of a first kernel by executing processor instructions stored in the memory, determines a second kernel based on the first kernel and the expansion rate, divides the second kernel into one or more decomposed kernels, determines the size of the one or more decomposed kernels based on at least one of the size of the first kernel, the basic structure of the memory associated with the computing system, and the expansion rate, determines an address for one or more blocks of an input image corresponding to each of the one or more decomposed kernels based on the expansion rate and one or more parameters associated with the size of the memory and the input image, retrieves the one or more blocks of the input image and the one or more decomposed kernels from the memory based on the address, and determines an output image based on a convolution operation between each of the one or more decomposed kernels and the one or more blocks of the input image. Claim 9 A computing system according to claim 8, wherein the processor determines an address for one or more blocks using at least one parameter including a base address of a first block among one or more blocks in memory, a column stride, a cell number X, a row stride, a cell number Y, a channel stride, a cell number Z, an extended column stride, an extended offset X, an extended row stride, and an extended offset Y, wherein X represents a horizontal block number of the input image, Y represents a vertical block number of the input image, and Z represents a block number per channel of the image. Claim 10 A computing system according to claim 8, wherein the processor determines the size of one or more blocks of the input image stored in the memory based on the basic structure of the memory. Claim 11 A computing system according to claim 8, wherein the processor determines the size of one or more blocks of the input image stored in the memory based on at least one of the basic structure of the memory and the expansion rate. Claim 12 In claim 8, the computing system wherein the processor retrieves a plurality of pixel values ​​of the input image corresponding to each of the one or more blocks of the input image. Claim 13 A computing system according to claim 8, wherein the processor identifies one or more kernel values ​​corresponding to one or more decomposed kernels within the second kernel, and retrieves one or more kernel values ​​corresponding to one or more decomposed kernels based on the size of the one or more decomposed kernels. Claim 14 A computing system according to claim 8, wherein the processor generates one or more matrices by multiplying one or more kernel values ​​of each of the one or more decomposed kernels and a plurality of pixel values ​​corresponding to each of the one or more blocks of the input image, and determines the output image by adding the one or more matrices. Claim 15 A computer-readable recording medium storing a program for executing the method of claim 1 on a computer.

Images