Information processing device, image processing unit, method and program

Abstract

An object of the present disclosure is to provide an information processing device that can reduce the circuit area. The information processing device (1) according to the present disclosure includes a plurality of memories (11A, 11B) configured to store the entire data of one column of input image data, a plurality of multiplexers (12A, 12B) configured to select one memory and thereby acquire a portion of the data of the one column from the selected memory, and a controller (13) configured to select one memory to store the portion of the data of the one column.

Description

[Technical field]

[0001] The present disclosure relates to an information processing device, an image processing unit, an information processing method, and a non-transitory computer-readable medium. [Background technology]

[0002] Real-time vision systems, such as robots, intelligent vehicles, surveillance, and inspection systems, are equipped with image processing systems (IP systems).

[0003] The hardware architecture of the image processing system is described in Non-Patent Document 1 (NPL1). The hardware architecture mentioned in Non-Patent Document 1 is an energy-efficient hardware architecture for an image processing system, which is proposed in Non-Patent Document 1 to perform feature extraction operations. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] “FIXYNN: EFFICIENT HARDWARE FOR MOBILE COMPUTER VISION VIA TRANSFER LEARNING” Paul N. Whatmough, Chuteng Zhou, Patrick Hansen, Shreyas Kolala Venkataramanaiah, Jae-sun Seo, Matthew Mattina, Proceedings of the 2nd SysML Conference, Palo Alto, CA, USA, 2019. Summary of the Invention [Problem to be solved by the invention]

[0005] A typical IP system has a line buffer upstream of the image processing logic (IPL). In general, a large size of the line buffer helps to improve throughput by parallel processing. However, the hardware architecture of the line buffer of the IP system in real-time applications requires a large silicon area, which is undesirable.

[0006] In Non-Patent Document 1, the hardware architecture of the line buffer occupies a large area, and thus there is a problem that the area efficiency is not good.

[0007] An object of the present disclosure is to provide an information processing device, an image processing unit, an information processing method, and a non-transitory computer-readable medium that can reduce a circuit area. [Means for solving the problem]

[0008] In a first example embodiment, an information processing device includes a plurality of memories configured to store data of one column of input image data as a whole, a plurality of multiplexers configured to select one of the memories to obtain a portion of the data of the one column from the selected memory, and a plurality of multiplexers configured to store the portion of the data of the one column. R A controller configured to select one of the memories.

[0009] In a second example embodiment, the image processing unit includes a plurality of memories configured to store data for one column of input image data in its entirety, a plurality of multiplexers configured to select one of the memories to retrieve a portion of the data for the one column from the selected memory, and a plurality of multiplexers configured to store the portion of the data for the one column. R A controller configured to select one of the memories.

[0010] In a third example embodiment, the information processing method includes storing data of one column of input image data across a plurality of memories, selecting one memory to obtain a portion of the data of the one column from the selected memory, and storing the portion of the data of the one column. R Selecting one memory.

[0011] A fourth example embodiment is a method for storing one column of input image data in a plurality of memories, selecting one memory, obtaining a portion of the one column of data from the selected memory, and storing the portion of the one column of data. R A non-transitory computer-readable medium that stores a program that causes a computer to select one of the memories. Effect of the Invention

[0012] According to the present disclosure, it is possible to provide an information processing device, an image processing unit, an information processing method, and a non-transitory computer-readable medium that can reduce a circuit area. [Brief description of the drawings]

[0013] [Figure 1] FIG. 1 is a diagram showing the configuration of an image processing system 100. As shown in FIG. [Diagram 2] FIG. 2 is a block diagram showing the configuration of the image processing unit 120. As shown in FIG. [Diagram 3] FIG. 3 is a diagram showing the configuration of the image pixel data 111. [Figure 4] FIG. 4 is a diagram showing the configuration of the line buffer 130. As shown in FIG. [Diagram 5] FIG. 5 is a diagram showing the configuration of the selector 132 and the shift register 133. [Figure 6] FIG. 6 is a block diagram showing the selector 132 and the line buffer controller 134. As shown in FIG. [Figure 7] FIG. 7 is a diagram showing the configuration of the line buffer 230. As shown in FIG. [Figure 8] FIG. 8 is a timing diagram for the phases of data flow in an SRAM memory line of line buffer 130. [Figure 9] FIG. 9 lists the state of the SRAM memory line 235 in each phase. [Figure 10]FIG. 10 is a list of the SRAM memory lines 235 selected by multiplexer 236 in each phase. [Figure 11] FIG. 11 is a diagram showing the configuration of the line buffer 330. As shown in FIG. [Figure 12] FIG. 12 is a block diagram showing the connection between the selector 332 and the line buffer controller 334. As shown in FIG. [Figure 13] FIG. 13 is a configuration diagram showing a configuration of the first embodiment of the present disclosure. [Figure 14] FIG. 14 is a configuration diagram showing a configuration of the second embodiment of the present disclosure. [Figure 15] FIG. 15 is a diagram showing the configuration of the line buffer 30. As shown in FIG. [Figure 16] FIG. 16 is a diagram showing the configuration of the selector 32 and the shift register 33. As shown in FIG. [Figure 17] FIG. 17 is a block diagram showing the selector 32 and the line buffer controller . [Figure 18] FIG. 18 lists the parameters of the reference line buffer and the line buffer 30. [Figure 19] FIG. 19 is a list of the reference line buffers and the sizes of the line buffer 30. [Figure 20] FIG. 20 is a diagram showing the configuration of the line buffer 40. As shown in FIG. [Figure 21] FIG. 21 is a timing diagram for the phases of data flow in an SRAM memory line of line buffer 40. [Figure 22] FIG. 22 lists the state of the SRAM memory line 45 in each phase. [Diagram 23] FIG. 23 is a list of the SRAM memory lines 45 selected by the multiplexer 46 in each phase. [Figure 24] FIG. 24 is a configuration diagram of an information processing device according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] (IP System Overview) Before describing embodiments according to the present disclosure, an overview of a reference IP system will be described with reference to FIGS.

[0015] FIG. 1 shows the configuration of an IP system 100. It is a block diagram of the IP system 100 that performs image processing tasks on a source image 110. The IP system 100 takes the source image 110 as input, performs one or more image processing operations on it, and produces a destination image 110. 12 Output.

[0016] An image processing system (hereinafter referred to as IP system) 100 has one or more image processing units 120 (IPUs), as shown in Figure 1. An IPU 120 takes image pixel data 111 as input and processes it. It then outputs the processed data to the next IPU 120. The image pixel data 111 may have been processed by a previous IPU 120, or it may be the source image 110 itself.

[0017] A block diagram of the IPU 120 is shown in FIG. 2. The IPU 120 includes an image processing logic (IPL) 121 and a line buffer 130. The IPL 121 can perform several types of image processing operations, including image modification operations, image feature extraction operations, and the like. The image modification operations may include operations such as color interpolation, image sharpening, and color correction on the image pixel data 111. The image feature extraction operations may include operations such as convolution, deconvolution, and the like on the image pixel data 111. When a sequence of image feature extraction operations is performed on the source image 110, the IP system 100 is also referred to as a convolutional neural network model.

[0018] The image pixel data 111 input to the first IPU 120 is the source image 110, and the image pixel data 111 input to the remaining IPUs 20 are the intermediate image pixel data. For image pixel data 111 of height H and width W in a particular IPU 120, the IPL 121 processes K*K pixel data at a time and slides across the image pixel data 111 with a stride S, where the kernel size K and stride size S are greater than or equal to 1 and less than or equal to H and W. Figure 3 shows the organization of the image pixel data 111. The image pixel data 111 is composed of H columns of pixels, where each pixel column has a width W. In Figure 3, K is equal to 3 and less than H.

[0019] The main task of line buffer 130 is to buffer image pixel data 111. While image processing operations are performed on image pixel data 111 by IPL 121, some or all of the pixel data is reused by IPL 121. To facilitate reuse of the pixel data, line buffer 130 buffers image pixel data 111.

[0020] 4, the line buffer 130 includes a buffer memory 131, a selector 132, a shift register 133, and a line buffer controller 134. The function of the buffer memory 131 of the line buffer 130 is to store the image pixel data 111. The buffer memory 131 may be composed of memory elements such as a static random-access memory (SRAM) and a shift register. The buffer memory 131 includes a plurality of buffer memory lines 135, numbered 1 to M. Each buffer memory line 135 may include an SRAM.

[0021] In a typical line buffer 130 configuration, one buffer memory line 135 stores one column of pixels of image pixel data 111. The column of pixels is written into the buffer memory using a write clock and read out using a read clock.

[0022] For a particular IPU 120, the size of each buffer memory line 135 in the line buffer 130 is equal to W in terms of pixel data. The number of buffer memory lines 135 is equal to the sum of K of the next IPL 121 and S of the IPL of the next IPU. In FIG. 4, there are M (=K+S) buffer memory lines 135. The total number of pixel data stored in the buffer memory 131 of the line buffer 130, also referred to as the total size of the buffer memory 131, is equal to the product of W and the sum of K and S, i.e., W*(K+S)(=W*M).

[0023] The problem with such a large capacity buffer memory is that it requires a large amount of silicon area on a chip. In general, to reduce the area occupied by the buffer memory 131, a single port SRAM memory is more desirable than a dual port SRAM memory.

[0024] The selector 132 has a number of multiplexers 136. The function of the multiplexer 136 is to select pixels of only one buffer memory line 135 from the pixels of the multiple buffer memory lines 135 based on the corresponding selection signal at that time. Therefore, (K+S)×1 is the size of the multiplexer 136 used in the line buffer 130. The number of multiplexers 136 in the line buffer 130 is equal to K. As shown in FIG. 3, all K multiplexers 136 receive inputs from all K+S (=M) buffer memory lines 135.

[0025] All K multiplexers 136 output only pixels of K buffer memory lines 135 (i.e., one buffer memory line per multiplexer). Therefore, the total number of pixels output from all multiplexers 136 is K pixels. The K pixels provided as outputs of multiplexers 136 are accumulated in shift register 133.

[0026] 5, shift register 133 stores the output pixels as K*K pixels. Image processing operations are performed by reading out K*K pixels from shift register 133 by IPL 121. The size of shift register 133 is equal to K*K in terms of pixel data.

[0027] 4, the line buffer controller 134 generates address line signals AL_1 to AL_M for all the buffer memory lines 135. The total number of address line signals generated by the line buffer controller 134 is equal to the number of the buffer memory lines 135, i.e., the sum (=M) of K and S. Furthermore, each address line signal indicates an address 0 to W-1.

[0028] 3, the line buffer controller 134 generates read signals RW_1 through RW_M for all the buffer memory lines 135. Each read signal determines the read or write state of each buffer memory line 135. The total number of read signals generated by the line buffer controller 134 is equal to the number of buffer memory lines 135, i.e., M.

[0029] In addition, the line buffer controller 134 generates selection signals (control signals) S for all multiplexers 136 in the line buffer 130. The total number of selection signals S generated by the line buffer controller 134 is equal to the number of multiplexers 136, i.e., K. More specifically, as shown in FIG. 6, the line buffer controller 134 transmits selection signals S_1 to S_K to each multiplexer 136.

[0030] As previously mentioned, the necessary signals required for the flow of image pixel data 111 from the input of line buffer 130, through buffer memory 131, selector 132, shift register 133 to the output of line buffer 130 are generated by line buffer controller 134.

[0031] The total time to read or write the entire buffer memory line is called one phase. The number of cycles in one phase is equal to the number of cycles required to read or write the entire buffer memory line. The total number of phases is equal to the number of buffer memory lines, i.e., K+S(=M).

[0032] The occurrence of the phases is in a round robin fashion. The value of the signal generated by the line buffer controller 134 during a particular phase will be the same when that particular phase is repeated. If the buffer memory line 135 has a high read signal during a particular cycle, the buffer memory line 135 is said to be in a read state, and if the read signal is low, the buffer memory line 135 is said to be in a write state.

[0033] First, the first column of image pixel data 111 is written to one of the buffer memory lines 135. Then, successive columns of image pixel data 111 are written in each phase to one of the buffer memory lines 135. As soon as K columns of image pixel data 111 are written to the buffer memory 131, the selector 132 outputs K pixels from each of the K buffer memory lines 135.

[0034] The number of cycles from the first pixel of image pixel data 111 written to buffer memory 131 to the first pixel read out of buffer memory 131 is called the latency of buffer memory 131. The smaller the number of latency cycles, the faster shift register 133 can accumulate K*K pixels and the faster IPL 121 can perform image processing operations. As mentioned above, the buffer memory used to store the image pixel data can be an SRAM-based buffer memory, a shift register-based buffer memory, etc.

[0035] A sample configuration example of a line buffer with K=3 and S=1 (i.e., M=4) is shown in Fig. 7. The configuration of the line buffer 230 includes a single-port SRAM memory-based buffer memory 231, a selector 232, a shift register 233 of size 3x3 in terms of pixel data, and a line buffer controller 234.

[0036] The buffer memory 231 includes four SRAM memory lines 235A to 235D. Furthermore, the selector 232 includes three 4x1 multiplexers 236A to 236C.

[0037] In the first three phases, the line buffer 230 in this example buffers three columns of image pixel data 211 in three SRAM memory lines 235A, 235B, and 235C. The number of cycles required to write three columns to three SRAM memory lines (i.e., one column to each SRAM memory line) is 3*W cycles, or three phases.

[0038] As shown in Figure 8, in the first phase, the entire first column of image pixel data 211 is written into SRAM-1 memory line 235A. Similarly, in the second and third phases, the second and third columns of image data are written into memory lines SRAM-2 235B and SRAM-3 235C, respectively.

[0039] In the third phase, after one cycle of writing one pixel in the third column of image pixel data 211 to SRAM-3 memory line 235C, all three pixels (one pixel from each of the three columns of image pixel data 211) required for multiplexer 236 to read are available in SRAM memory line 235. However, because a single-port SRAM memory based buffer memory is used and SRAM-3 memory line 235C is in a write state, the third phase multiplexer cannot initiate a read operation from SRAM-3 memory line 235C.

[0040] Starting from the next cycle, i.e., the fourth phase, after 3*W cycles, the three multiplexers output three pixels from three SRAM memory lines, i.e., SRAM-1 235A, SRAM-2 235B, and SRAM-3 235C memory lines in the line buffer 230. In the same phase, since S=1, the line buffer 230 buffers the next column of image pixel data 211 in SRAM-4 memory line 235D.

[0041] In all phases, any one SRAM memory line remains in a write state until the entire corresponding SRAM memory line is written. FIG. 9 shows the state of the SRAM memory lines 235 of the line buffer 230 in each phase. In FIG. 9, "R" means a read state and "W" means a write state. In the first phase, the SRAM_1 memory line 235A is in a write state and the SRAM memory lines 235B to 235D are in a read state. In the second phase, the SRAM_2 memory line 235B is in a write state and the SRAM memory lines 235A, 235C, and 235D are in a read state. In the third phase, the SRAM_3 memory line 235C is in a write state and the SRAM memory lines 235A, 235B, and 235D are in a read state. In the fourth phase, the SRAM_4 memory line 235D is in a write state and the SRAM memory lines 235A to 235C are in a read state.

[0042] In the write state, the read signal is generated by the line buffer controller 234 of the line buffer 230. The address line signal is sent from the first address to the last address of each SRAM memory line. The total number of phases is equal to 4. The number of multiplexers is equal to 3 as shown in FIG.

[0043] FIG. 10 shows the SRAM memory lines 235 selected by the multiplexers in successive phases. In FIG. 10, "MUX1" refers to multiplexer 236A, "MUX2" refers to multiplexer 236B, and "MUX3" refers to multiplexer 236C. In the first phase, multiplexer 236A selects SRAM_2 memory line 235B, multiplexer 236B selects SRAM_3 memory line 235C, and multiplexer 236C selects SRAM_4 memory line 235D. In the second phase, multiplexer 236A selects SRAM_3 memory line 235C, multiplexer 236B selects SRAM_4 memory line 235D, and multiplexer 236C selects SRAM_1 memory line 235A. In the third phase, multiplexer 236A selects SRAM_4 memory line 235D, multiplexer 236B selects SRAM_1 memory line 235A, and multiplexer 236C selects SRAM_2 memory line 235B. In the fourth phase, multiplexer 236A selects SRAM_1 memory line 235A, multiplexer 236B selects SRAM_2 memory line 235B, and multiplexer 236C selects SRAM_3 memory line 235C. In this manner, each multiplexer selects one of the SRAM memory lines in turn.

[0044] The shift register 233 takes three pixel data from the multiplexers as input. The shift register 233 stores 3x3 pixels and is read out by the IPL 121. With a stride of S=1, the IPL 121 reads out the 3x3 pixels from the shift register 233 successively. The stride S is the stride of the image pixel data, specifically the stride of the next layer. The next layer is the one connected to the shift register 233. That is, this layer is connected to the output of the multiplexers 236. In the line buffer 230 configuration, 3*W cycles is the latency time for one of the multiplexers to output pixel data from one of the SRAM line memories 235.

[0045] Figures 11 and 12 show another variation of the line buffer configuration used in the IPU. Unlike the line buffer 130 in Figure 4, the line buffer 330 in Figure 11 does not have a shift register. The IPL 321 reads the K*K pixel data directly from the buffer memory 331 of the line buffer 330 via a multiplexer, as shown in Figures 11 and 12.

[0046] The buffer memory 331 has a number of buffer memory lines 335A, 335B, 335C. The width of each buffer memory line in the line buffer 330 is equal to W / K in terms of pixel data. The number of buffer memory lines is indicated as "F" in FIG. 11 and is equal to the sum of S*K and K*K, i.e. (K*K+S*K).

[0047] The total number of pixel data stored in the buffer memory 331 of the line buffer 330, also referred to as the total size of the buffer memory 331, is equal to the product of W and the sum of K and S, i.e., W*(K+S). The size of the buffer memory of the line buffer 230 is the same as the size of the buffer memory of the line buffer 330.

[0048] The line buffer is one of the key logics in the IP system, and the line buffer 230 or 330 is used to buffer image pixel data in the IP system.

[0049] The first problem with this configuration is that the line buffer configuration used to buffer the intermediate image data of the IPU requires a large silicon area in the IP system. In other words, this configuration is not area-efficient. The cost of manufacturing a large silicon area is large, and such an area requires a large power budget. The reason for the first problem is that the buffer memory requires a very large area to store the full width of one column of image pixel data.

[0050] Specifically, suppose a line buffer is formed in one layer, which connects two adjacent layers in an Application Specific Integrated Circuit (ASIC), the kernel size of the next layer is K, the min(height, width) of the input frame of the next layer is equal to D, the stride of the next layer is equal to S, and the channel size of the previous layer is equal to C. In this case, in terms of pixels, the size of one SRAM line in the line buffer is D*C, and the total size of the SRAM lines in the line buffer is D*C*(K+S). Therefore, the area occupied by a line buffer of such size in the ASIC is large, and such an ASIC is costly.

[0051] The second problem with this configuration is that the IP system has high line buffer latency. The reason for the second problem is that every IPU must wait for the subsequent IPL to start processing until enough intermediate image pixel data is buffered in the line buffer.

[0052] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. These embodiments can be applied to systems and devices that deploy IP systems, such as robots, intelligent vehicles, surveillance, inspection systems, etc.

[0053] (Embodiment 1) First, the information processing device 1 according to the first embodiment will be described with reference to FIG.

[0054] Referring to FIG. 13 which is the first embodiment of the present disclosure, the information processing device 1 includes an IPU 10. The information processing device 1 is, for example, a computer or a machine. The IPU 10 is, for example, a chip circuit or an integrated circuit, and processes input image data. The IPU 10 has a plurality of memories 11A and 11B, a plurality of multiplexers 12A and 12B, and a controller 13. The input image data can be stored inside or outside the information processing device 1.

[0055] Memories 11A and 11B are configured to store data for one column of input image data as a whole. That is, each memory stores data for less than one column of the input image. In this embodiment, "data for one column of the input image" means that the length of the column may be either the width W of the input image data or the height H of the input image data.

[0056] For example, the sum of the sizes of memories 11A and 11B may be 1. The sizes of memories 11A and 11B may be the same. That is, each of memories 11A and 11B may store half of the data of one column of the input image. However, the sizes of memories 11A and 11B may not be equal. For example, memory 11A may store 2 / 3 of the data of one column of the input image, and memory 11B may store 1 / 3 of the data of one column of the input image.

[0057] The multiplexers 12A and 12B are configured to select one memory to obtain a portion of one column of data from the selected memory. The controller 13 selects a memory to store a portion of one column of data. R The controller 13 is configured to select one of the memories, for example, by sending a signal to the memories 11A and 11B to put them into a write state.

[0058] More specifically, the data of one column of the input image data is divided in half, i.e., into two equal data portions, and one data portion can be stored in each of the memories 11A and 11B under the control of the controller 13. However, the method of dividing the data of one column is not limited to this example. For example, the data of one column of the input image data may be divided into another portion, which is 2 / 3 of the data of the column and 1 / 3 of the data of the column.

[0059] Next, the effect of this embodiment will be described. As described above, in the reference IP system, there are K+S buffer memory lines, and the total size of the buffer memories is equal to W*(K+S), where K is the kernel size, S is the stride size, and W is the width of the image pixel data (i.e., the size of one buffer memory). However, in the first embodiment, the size of one memory is less than the width W or height H. For K, the number of memories needs to be increased, but this is not the case for S. In conclusion, the total size of the memories in the first embodiment can be made smaller than the total size of the memories in the reference IP system. Therefore, it is useful for reducing the circuit area of ​​the IPU 10.

[0060] For example, if the data of one column of the input image data is divided into two, i.e., into two equal data parts, and one data part is stored in each of the memories 11A and 11B, and the size of both the memories 11A and 11B is W / 2, then the total size of the memory is equal to (W / 2)*(2K+S). Compared with the total size of the buffer memory of the reference IP system, the total size of the memory can be reduced by D1. (1)D1=W*(K+S)-(W / 2)*(2K+S)=W*S / 2 In equation (1), the values ​​of W and S are always positive, so D1 is always positive.

[0061] In the first embodiment, there are two memories 11A and 11B that store one column of data of the input image data. However, there may be more than two memories (three, four, etc.) that store the above data. For example, IPU 10 may have three memories, and the total size of the three memories may be one column of data. Each of the three memories may store 1 / 3 of the data of one column of the input image. However, one memory may store half of the data of one column of the input image, and each of the other two memories may store 1 / 4 of the data of one column of the input image. The method of dividing the data of one column is not limited to these examples.

[0062] Also, provided that the above effect continues, the sum of the sizes of memories 11A and 11B may exceed one column of the input image data. For example, if the data of one column of the input image data is split in half, i.e., into two equal data parts, and one data part is stored in each of memories 11A and 11B, the size of one memory may be from W / 2 to (W / 2)*W*(K+S) / ((2K+S)*(W / 2)). The same is true for N memories, where N is a positive integer greater than 1.

[0063] The information processing device 1 may include a plurality of IPUs 10 described in the present embodiment. With this configuration, the information processing device 1 can perform complex image processing.

[0064] In the first embodiment, a single IPU 10 is equipped with a plurality of memories 11A and 11B, a plurality of multiplexers 12A and 12B, and a controller 13. However, these components may be equipped in several units or chips within the information processing device 1.

[0065] (Embodiment 2) A second embodiment of the present disclosure will be described below with reference to the accompanying drawings.

[0066] 14, the information processing device 2 includes an IPU 20 and is configured as an IP system. The IPU 20 is, for example, a chip circuit or an integrated circuit, and processes image pixel data 21. The IPU 20 takes in image pixel data 21 as input, processes it, and then outputs the processed data. The image pixel data 21 is either processed by a previous IPU or is the source image itself.

[0067] 14 shows a block diagram of IPU 20. IPU 20 includes IPL 22 and line buffer 30. IPL 22 may perform several types of image processing operations, including image modification operations, image feature extraction operations, etc. Image modification operations may include operations such as color interpolation, image sharpening, color correction, etc. on image pixel data 21. Image feature extraction operations may include operations such as convolution, deconvolution, etc. on image pixel data 21.

[0068] For image pixel data 21 of height H and width W of IPU 20, IPL 22 processes K*K pixel data at a time, sliding across image pixel data 21 with stride S, where kernel size K and stride size S are greater than or equal to 1 and less than or equal to H and W, and S is the stride of the next layer. The next layer is connected to shift register 33, i.e., this layer is connected to the outputs of multiplexers 36. Image pixel data 21 has the same configuration as image pixel data 21 shown in FIG. 3. Image pixel data 21 is made up of H columns of pixels, each column of pixels having a width of W.

[0069] The main task of the line buffer 30 is to buffer image pixel data 21. 1 While image processing operations are being performed on image pixel data 21, some or all of the pixel data is reused by IPL 22. To facilitate pixel data reuse, line buffer 30 buffers image pixel data 21.

[0070] 15 and 16, the line buffer 30 includes a buffer memory 31, a selector 32, a shift register 33, and a line buffer controller 34. Furthermore, the buffer memory 31 includes a plurality of buffer memory lines 35. The size of each buffer memory line 35 is the size of each buffer memory line 135 divided by N. The selector 32 includes a plurality of buffer memory lines 35. Image pixels are written to the buffer memory lines 35 using a write clock and read out using a read clock.

[0071] The buffer memory 31 stores the image pixel data 21. The size of the buffer memory 31 of the line buffer 30 depends not only on W, K, and S, but also on the coefficient of the partition N. The value of N is a positive integer and is always greater than 1. In a single buffer memory line 35 of the line buffer 30, the number of pixel data that can be stored is equal to the upper limit value of the ratio of W to N (i.e., W / N). Therefore, one column of the image pixel data 21 is stored in N buffer lines. For simplicity, in this embodiment 2, the channel size of the previous layer is equal to 1, but this channel size can be any channel size.

[0072] The K pixels provided as output from the buffer memory line 35 are stored by the shift register 33. The K*K pixels from the shift register 33 are read out by the IPL 22 for image processing operations. The size of the shift register 33 is equal to K*K in terms of pixel data.

[0073] The line buffer controller 34 generates address line signals and read signals for all buffer memory lines 35 in the line buffer 30, as shown in Fig. 15. The total number of address line signals AL_1 to AL_M generated by the line buffer controller 34 is equal to the number of buffer memory lines 35, i.e., M = N * K + S. Similarly, the total number of read signals RW_1 to RW_M generated by the line buffer controller 34 is equal to the number of buffer memory lines, i.e., M = N * K + S. Each read signal determines the read or write state of each buffer memory line 35.

[0074] Furthermore, the line buffer controller 34 generates a selection signal (control signal) for each buffer memory line 35. The total number of selection signals generated by the line buffer controller 34 is equal to the number of multiplexers 36, i.e., K. Furthermore, each address line signal indicates an address 0 to (W / N)-1 for storing one column of divided image data.

[0075] The function of the multiplexers 36 is to select pixels of only one buffer memory line at a time from the pixels of the multiple buffer memory lines 35 based on the corresponding selection signal. The size of each multiplexer is (N*K+S)x1. The number of multiplexers remains the same as in the reference example, i.e., it is equal to K.

[0076] All K multiplexers 36 have inputs from all N*K+S buffer memory lines. The K multiplexers 36 output pixels only from the K buffer memory lines 35 (i.e., one buffer memory line 35 per multiplexer 36). Thus, a total of K pixels are output from all multiplexers 36.

[0077] The necessary signals required for the flow of image pixel data 21 from the input of the line buffer 30 through the buffer memory 31, selector 32, shift register 33 to the output of the line buffer 30 are generated by a line buffer controller 34. The total time to read or write the entire buffer memory line 35 is called one phase. The number of phases in the line buffer 30 is equal to the number of buffer memory lines 35, i.e. M=N*K+S. The number of cycles in one phase is equal to the number of cycles required to read or write the entire buffer memory line, i.e. (W / N).

[0078] The occurrence of the phases is done in a round robin fashion, such that the value of the signal generated by line buffer controller 34 during a particular phase will be the same each time that particular phase is repeated.

[0079] The number of buffer memory lines 35 is equal to the sum of S of the IPL 22 of the previous IPU and the product of N and K of the next IPL 22 (i.e., N*K+S). Therefore, the total number of pixel data stored in the buffer memory 31 of the line buffer 30, also referred to as the total size of the buffer memory 31, is equal to the product of W / N and S+(N*K) (i.e., (W / N)*(N*K+S)).

[0080] When the size of the buffer memory 31 of the line buffer 30 and the size of the buffer memory 131 of the line buffer 130 are compared for pixel data, the size of the buffer memory can be reduced by DS. (2)DS=W*(K+S)-(W / N)*(N*K+S)=W*S*(1-(1 / N)) In equation (2), the values ​​of W and S are always positive, and the coefficient of partition N is always a positive integer greater than 1. Therefore, DS is always positive. Therefore, for all values ​​of N, S, K, and W, the size of buffer memory 31 of line buffer 30 is always smaller than the size of buffer memory 131 of line buffer 130.

[0081] Fig. 18 shows the relationship between the parameters of the buffer memory 31 of the line buffer 30 and the parameters of the buffer memory 131 of the reference line buffer 130. As shown in Fig. 18, K, W, and S of the buffer memory 131 correspond to N*K, W / N, and S of the buffer memory 31, respectively. The size of the buffer memory 131 is W*(K+S), which corresponds to the size of the buffer memory 31, (W / N)*(N*K+S).

[0082] FIG. 19 shows a comparison of the size of the buffer memory 31 of the line buffer 30 and the size of the buffer memory 131 of the reference line buffer 130 with some sample values. There are four examples in FIG. 19; (a) K=3, S=1, W=100, N=2, (b) K=3, S=1, W=100, N=4, (c) K=3, S=2, W=100, N=2, (d) K=5, S=1, W=100, N=2. As mentioned above, the size of the buffer memory 31 is calculated by (W / N)*(S+N*K), while the size of the buffer memory 131 is calculated by W*(K+S).

[0083] In (a), the size of the buffer memory 31 is 350, while the size of the buffer memory 131 is 400. The former size is reduced to 87.50% of the latter. In (b), the size of the buffer memory 31 is 325, while the size of the buffer memory 131 is 400. The former size is reduced to 81.25% of the latter. In (c), the size of the buffer memory 31 is 400, while the size of the buffer memory 131 is 500. The former size is reduced to 80.00% of the latter. In (d), the size of the buffer memory 31 is 550, while the size of the buffer memory 131 is 600. The former size is reduced to 91.67% of the latter. From these examples, it can be said that the larger the value of N, the smaller the size of the buffer memory 31 of the line buffer 30.

[0084] A sample example of a line buffer with K=3, S=1, and N=2 is shown in Figure 20. The configuration of the line buffer 40 includes a single-port SRAM memory-based buffer memory 41, a selector 42, a shift register 43 of size 3x3 for pixel data, and a line buffer controller 44. The buffer memory 41 includes SRAM memory lines 45A to 45G, and the selector 42 includes three 7x1 multiplexers 46A to 46C.

[0085] The line buffer 40 buffers three columns of image pixel data 21 across six SRAM memory lines in six phases, with each phase being W / 2 cycles. The number of cycles required to write three columns across six SRAM memory lines (i.e. one column is written across two SRAM memory lines) is 6*W / 2 cycles, or six phases.

[0086] 21 shows a timing diagram for the phases of data flow for each SRAM memory line 45 of the line buffer 40. In the first phase, the left half of the first column of image pixel data 21 is written to the SRAM-1 memory line 45A. In the second phase, the right half of the first column of image pixel data 21 is written to the SRAM-2 memory line 45B. In FIG. 21, "1L(w)" means the former state, and "1R(w)" means the latter state.

[0087] In the third phase, the left half of the second column of image pixel data 21 is written to SRAM-3 memory line 45C. In the fourth phase, the right half of the second column of image pixel data 21 is written to SRAM-4 memory line 45D. In Figure 21, "2L(w)" means the former state, and "2R(w)" means the latter state.

[0088] In the fifth phase, the left half of the third column of the image pixel data 21 is written to the SRAM-5 memory line 45E. "3L(w)" in FIG. 21 denotes this state. In the fifth phase, after one cycle of writing one pixel of the third column of the image pixel data 21, all three pixels (one from each of the three columns of the image pixel data 21) required for the multiplexer to read are available in the SRAM memory line. However, since a single-port SRAM memory based buffer memory is used and the SRAM-5 memory line 45E is in a write state, the fifth phase multiplexer cannot start a read operation from the SRAM-5 memory line 45E. From the first phase to the fifth phase, one of the SRAM memory lines is in an active mode and the other lines are in an inactive mode.

[0089] In the sixth phase, the right half of the third column of the image pixel data 21 is written to the SRAM-6 memory line 45F. "3R(w)" in FIG. 21 means this state. Furthermore, with the start of the sixth phase, the three SRAM memory lines 45A, 45C, and 45E are in the read state, and the three memory lines 45 simultaneously output three pixels. In FIG. 21, "1L(r)", "2L(r)", and "3L(r)" mean these states. Also, "1L(r)", "2L(r)", and "3L(r)" are the same left side in the divided data of one column. Therefore, the latency cycle of the SRAM memory lines of the line buffer 40 is 5*W / 2 cycles. In the sixth phase, the SRAM memory lines 45A, 45C, 45E, and 44F are in the active mode, and the others are in the inactive mode. In the sixth phase, the read and write operations of pixel data of the same column (the third column) of the image pixel data can be overlapped.

[0090] In the seventh phase, the left half of the fourth column of the image pixel data 21 is written to the SRAM-7 memory line 45G. "4L(w)" in FIG. 21 means this state. Furthermore, with the start of the seventh phase, the three SRAM memory lines 45B, 45D, and 45F are in the read state, and the three memory lines 45 simultaneously output three pixels. In FIG. 21, "1R(r)", "2R(r)", and "3R(r)" mean these states. Also, "1R(r)", "2R(r)", and "3R(r)" are the same right side in the divided one column of data. In the seventh phase, the memory lines 45B, 45D, 45F, and 45G are in the active mode, and the others are in the inactive mode. Therefore, the line buffer controller 34 generates a selection signal for the multiplexer 36 to obtain the divided one column of data in one phase and obtain the one column of data in two (N) phases.

[0091] In the next first phase, the right half of the fourth column of the image pixel data 21 is written to the SRAM-1 memory line 45A. "4R(w)" in FIG. 21 indicates this state. Furthermore, with the start of the first phase, three SRAM memory lines 45C, 45E, and 45G enter a read state, and the three memory lines 45 output three pixels. In FIG. 21, "2L(r)", "3L(r)", and "4L(r)" indicate these states. In the first phase, the read and write operations of pixel data in the same column (the fourth column) of the image pixel data can be overlapped.

[0092] In the next second phase, the SRAM-2 memory line 45B is in a write state. "5L(w)" in FIG. 21 indicates this state. Furthermore, with the start of the second phase, the three SRAM memory lines 45A, 45D, and 45F are in a read state, and the three memory lines 45 output three pixels. In FIG. 21, "2R(r)", "3R(r)", and "4R(r)" indicate these states.

[0093] As mentioned above, splitting the buffer memory lines 35 of the buffer memory 31 provides the line buffer controller 34 with the flexibility to overlap read and write operations of pixel data of the same column of the image pixel data 21. With the advantage of overlapping read and write operations, the total size of the buffer memory 31 required in the line buffer 30 is smaller than the total size of the buffer memory 131 required in the line buffer 130. Furthermore, another advantage of overlapping read and write operations of pixel data of the same column of the image pixel data 21 is that the number of cycles required for the multiplexer to wait to read the buffer memory lines is reduced.

[0094] FIG. 22 shows the state of the SRAM memory lines 45 in the line buffer 40 in each phase. In FIG. 22, "R" means a read state and "W" means a write state. In the first phase, the SRAM_1 memory line 45A is in a write state and the other SRAM memory lines are in a read state. In the second phase, the SRAM_2 memory line 45B is in a write state and the other SRAM memory lines are in a read state. In the third phase, the SRAM_3 memory line 45C is in a write state and the other SRAM memory lines are in a read state. In the fourth phase, the SRAM _4 Memory 45D In the fifth phase, the SRAM memory lines are in the write state and the other SRAM memory lines are in the read state. _5 Memory 45E In the sixth phase, the SRAM memory lines are in the write state and the other SRAM memory lines are in the read state. _6 Memory 45F In the seventh phase, the SRAM memory lines are in the write state and the other SRAM memory lines are in the read state. _7 Memory 45G is in a write state and the other SRAM memory lines are in a read state.

[0095] In the write state, the read signal is generated by the line buffer controller 44. The address line signal is sent from the first address to the last address of each SRAM memory line. The number of multiplexers 46 is 3, and the size of each multiplexer is 7x1, as shown in Figure 20.

[0096] FIG. 23 shows the SRAM memory lines 45 selected by the multiplexers 46 in successive phases. In FIG. 23, "MUX1" refers to the multiplexer 46A, "MUX2" refers to the multiplexer 46B, and "MUX3" refers to the multiplexer 46C. In the first phase, the multiplexer 46A selects the SRAM_3 memory line 45C, the multiplexer 46B selects the SRAM_5 memory line 45E, and the multiplexer 46C selects the SRAM_7 memory line 45G. In the second phase, the multiplexer 46A selects the SRAM_4 memory line 45D, the multiplexer 46B selects the SRAM_6 memory line 45F, and the multiplexer 46C selects the SRAM_1 memory line 45A. In the third phase, multiplexer 46A selects SRAM_5 memory line 45E, multiplexer 46B selects SRAM_7 memory line 45G, and multiplexer 46C selects SRAM_2 memory line 45B. In the fourth phase, multiplexer 46A selects SRAM_6 memory line 45F, multiplexer 46B selects SRAM_1 memory line 45A, and multiplexer 46C selects SRAM_3 memory line 45C. In the fifth phase, multiplexer 46A selects SRAM_7 memory line 45G, multiplexer 46B selects SRAM_2 memory line 45B, and multiplexer 46C selects SRAM_4 memory line 45D. In the sixth phase, multiplexer 46A selects SRAM_1 memory line 45A, multiplexer 46B selects SRAM_3 memory line 45C, and multiplexer 46C selects SRAM_5 memory line 45E. In the seventh phase, multiplexer 46A selects SRAM_2 memory line 45B, multiplexer 46B selects SRAM_4 memory line 45D, and multiplexer 46C selects SRAM_6 memory line 45F.

[0097] Thus, when the phase is incremented by one, multiplexer 46 selects the next SRAM memory line 45. Furthermore, in one phase, multiplexer 46B selects an SRAM memory line that is 2 (=N) memory lines away from the SRAM memory line selected by multiplexer 46A. Similarly, in one phase, multiplexer 46C selects an SRAM memory line that is 2 (=N) memory lines away from the SRAM memory line selected by multiplexer 46B.

[0098] It should be noted from Fig. 23 that the SRAM memory lines 45 selected by each multiplexer 46 of the line buffer 40 in successive phases are different from the SRAM memory lines selected by each multiplexer of the line buffer 230 shown in Fig. 10 in successive phases. As shown in Fig. 20, the shift register 43 inputs three image pixel data 21 from the SRAM memory lines 45. The shift register 43 accumulates 3x3 pixels, which are read out from the line buffer 40 by the IPL 22. With a stride of S=1, the IPL 22 successively reads out the 3x3 pixels from the shift register 43 to perform image processing operations.

[0099] Compared with the latency cycles of the reference line buffer 230, the latency cycles of the line buffer 40 are W / 2 cycles less. The improvement in latency cycles is observed in the line buffer 40. Since an IP system can be equipped with multiple IPUs 20, the overall performance improvement in terms of latency cycles is equal to the number of IPUs 20 multiplied by W / 2.

[0100] Comparing the latency cycles of the buffer memory 31 of the line buffer 30 and the latency cycles of the buffer memory 131 of the line buffer 130 for pixel data, it is possible to reduce the latency cycles by DL. (3)DL=W*K-(W*(K-1)+W / N)=W*(N-1) / N In equation (3), the value of W is always positive, and the coefficient of partition N is always a positive integer greater than 1. Therefore, DL is always positive. Therefore, it is proven that for all values ​​of N, K, and W, the latency cycle of the buffer memory 31 of the line buffer 30 is always smaller than the latency cycle of the buffer memory 131 of the line buffer 130 of the reference example.

[0101] The improvement in latency cycles can be seen in the line buffer 30. Since an IP system can have multiple IPUs 20, the overall performance improvement in terms of latency cycles is equal to the number of IPUs 20 multiplied by W*(N-1) / N.

[0102] Thus, the present disclosure provides not only an area efficient configuration, but also improved performance in terms of latency cycles.

[0103] As explained above, the channel size of the previous layer can be any channel size. Furthermore, the width W can also be replaced by the min(height, width) of the input frame of the next layer. Suppose a line buffer is formed in one layer, which connects two adjacent layers in an Application Specific Integrated Circuit (ASIC), the kernel size of the next layer is K, the min(height, width) of the input frame of the next layer is equal to D, the stride of the next layer is S, and the channel size of the previous layer is equal to C. In this case, for a pixel, the size of one SRAM line in the line buffer is (D / N)*C, and the total size of SRAM lines in the line buffer is (D / N)*C*(N*K+S). The effect of this example is similar to the one explained earlier. Furthermore, D can be replaced by either the height or width of the input image, which can be defined as the height or width of the input frame of the next layer.

[0104] Furthermore, under the condition that the above effects continue, the total size of the SRAM memory lines 45 may exceed W / 2. For example, in FIG. 20, the size of one SRAM memory line 45 may be from W / 2 to (W / 2)*W*(K+S) / ((2K+S)*W / 2).

[0105] (Embodiment 3) As described above, the configuration of the line buffer in the second embodiment includes a shift register 43 that accumulates the output from the multiplexer. However, this shift register is not an essential component. In the third embodiment, the line buffer has a buffer memory, a multiplexer, and a line buffer controller, but does not have a shift register.

[0106] The line buffer has a buffer memory line divided into N small parts. The width of the buffer memory line of the buffer memory is equal to W / (K*N) for pixel data. The number of the buffer memory lines is equal to the sum of S*K and K*K*N, i.e. (K*K*N+S*K). The total number of pixel data stored in the buffer memory of the line buffer, also called the total size of the buffer memory, is equal to the product of W / N and S+N*K, i.e. (W / N)*(S+N*K). The size of the buffer memory in the third embodiment and the size of the buffer memory in the second embodiment are the same for pixel data.

[0107] For pixel data, the reduction in the size of the buffer memory of the third embodiment compared to the size of the buffer memory 301 of the line buffer 330 is shown in equation (2). As mentioned above, the values ​​of W and S are always positive, and the coefficient of the partition N is always a positive integer greater than 1. The difference between the size of the buffer memory 301 of the line buffer 330 and the size of the buffer memory of the third embodiment is always positive. Therefore, it is proven that for all values ​​of N, S, K, and W, the size of the buffer memory of the third embodiment is always smaller than the size of the buffer memory 301 of the line buffer 330.

[0108] In general, compared to the latency cycle of the line buffer 330, the latency cycle of this embodiment is W*(N-1) / N cycles less as shown in formula (3). In the third embodiment, an improvement in the latency cycle is confirmed. Since an IP system can be equipped with multiple IPUs 20, the improvement in overall performance in terms of latency cycle is equal to the product of the number of IPUs 20 and W*(N-1) / N.

[0109] Next, a configuration example of the information processing device described in the above embodiments will be described below with reference to FIG.

[0110] 24 is a block diagram showing an example of the configuration of an information processing device 91. As shown in FIG.

[0111] The processor 92 executes the processes executed by the information processing device 91 described with reference to the sequence diagrams and flowcharts of the above embodiment in the memory 9 3 The processor 92 executes the program by loading and executing software (computer programs) from the processor 92. The processor 92 may be, for example, a microprocessor, a micro processing unit (MPU) or a central processing unit (CPU). The processor 92 may include multiple processors.

[0112] The memory 93 is configured by a combination of volatile memory and non-volatile memory. The memory 93 may include storage located remotely from the processor 92. In this case, the processor 92 may access the memory 93 via an I / O interface (not shown).

[0113] 24, the memory 93 is used to store a group of software modules. The processor 92 reads the group of software modules from the memory 93 and executes the read software modules to perform the processing performed by the information processing device described in the above embodiment.

[0114] As described above with reference to FIG. 24, each processor included in the information processing device of the above embodiment executes one or more programs including a group of instructions to cause a computer to execute the algorithm described above with reference to the drawings.

[0115] The information processing device 91 may also have a network interface. The network interface is used for communication with other network node devices constituting the communication system. The network interface may include, for example, a network interface card (NIC) conforming to the IEEE 802.3 series. The information processing device 91 may receive an input feature map or transmit an output feature map using the network interface.

[0116] In the above example, the program can be stored and provided to the computer by using any kind of non-transitory computer-readable medium. The non-transitory computer-readable medium includes any kind of tangible storage medium. Examples of the non-transitory computer-readable medium include magnetic storage media (such as floppy disks, magnetic tapes, hard disk drives, etc.), optical magnetic storage media (such as optical magnetic disks), compact disc read only memory (CD-ROM), compact disc recordable (CD-R), compact disc rewritable (CD-R / W), and semiconductor memory (such as mask ROM, programmable ROM (PROM), erasable PROM (EPROM), flash ROM, random access memory (RAM), etc.). The program may be provided to the computer by using any kind of temporary computer-readable medium. Examples of the temporary computer-readable medium include electric signals, optical signals, and electromagnetic waves. The temporary computer-readable medium can provide the program to the computer via a wired communication line (such as an electric wire, an optical fiber) or a wireless communication line.

[0117] A part or all of the above-described embodiments can be described as follows, but the present invention is not limited thereto. (Appendix 1) a plurality of memories configured to collectively store data for one column of the input image data; a plurality of multiplexers configured to select one of the memories to obtain a portion of the column of data from the selected memory; Store a part of the data in the previous column R a controller configured to select one of said memories; An information processing device comprising: (Appendix 2) The portion of the data in the one column is obtained by dividing the data in the one column by N, where N is a positive integer greater than 1. 2. An information processing device according to claim 1. (Appendix 3) The number of memories is (N*K)+S, where K is the number of multiplexers and S is the stride of the layers connected to the outputs of the multiplexers. 3. An information processing device according to claim 2. (Appendix 4) the memory is single-ported; 4. An information processing device according to claim 1. (Appendix 5) a shift register configured to store the output of the multiplexer; 5. An information processing device according to any one of claims 1 to 4. (Appendix 6) A read operation of a part of the data of the one column and a write operation of another part of the data of the one column are performed in the same phase in different memories. 6. An information processing device according to any one of claims 1 to 5. (Appendix 7) The controller generates a control signal to the multiplexer so as to acquire the divided one column of data in one phase and acquire the one column of data in N phases. 4. The information processing device according to claim 2 or 3. (Appendix 8) a plurality of memories configured to collectively store data for one column of the input image data; a plurality of multiplexers configured to select one of the memories to obtain a portion of the column of data from the selected memory; Store a part of the data in the previous column R a controller configured to select one of said memories; An image processing unit comprising: (Appendix 9) The portion of the data in the one column is obtained by dividing the data in the one column by N, where N is a positive integer greater than 1. 9. The image processing unit according to claim 8. (Appendix 10) One column of input image data is stored across multiple memories, selecting one of the memories to obtain a portion of the data of the one column from the selected memory; Store a part of the data in the previous column R selecting one of said memories; An image processing method comprising: (Appendix 11) One column of input image data is stored across multiple memories, selecting one of the memories to obtain a portion of the data of the one column from the selected memory; Store a part of the data in the previous column R selecting one of said memories; A non-transitory computer-readable medium that stores a program that causes a computer to execute the program.

[0118] The present disclosure is not limited to the above-described embodiments, and can be modified as appropriate without departing from the spirit and scope of the present disclosure. [Explanation of symbols]

[0119] 1, 2 Information processing device 10, 20 Image processing unit 11. Memory 12 Multiplexer 13 Controller 21 Image pixel data 22 Image Processing Logic 30, 40 line buffer 31, 41 Buffer memory 32, 42 selector 33, 43 Shift Register 34, 44 Line Buffer Controller 35 Buffer Memory Lines 45 SRAM memory lines 36, 46 Multiplexer

Claims

1. a plurality of memories configured to collectively store data for one column of the input image data; a plurality of multiplexers configured to select one of the memories to obtain a portion of the one column of data from the selected memory; a controller configured to select one of the memories to store a portion of the one column of data; The portion of the data of the one column is obtained by dividing the data of the one column by N, where N is a positive integer greater than 1, The number of memories is (N*K)+S, where K is the number of multiplexers and S is the stride of the layers connected to the outputs of the multiplexers. Information processing device.

2. A plurality of memories configured to store data for one column of input image data as a whole; a plurality of multiplexers configured to select one of the memories to obtain a portion of the one column of data from the selected memory; a controller configured to select one of the memories to store a portion of the one column of data; a shift register configured to store the output of the multiplexer; An information processing device comprising:

3. A plurality of memories configured to store data for one column of input image data as a whole; a plurality of multiplexers configured to select one of the memories to obtain a portion of the one column of data from the selected memory; a controller configured to select one of the memories to store a portion of the one column of data; A read operation of a part of the data of the one column and a write operation of another part of the data of the one column are performed in the same phase in different memories. Information processing device.

4. the memory is single-ported; The information processing device according to claim 1 .

5. A plurality of memories configured to store data for one column of input image data as a whole; a plurality of multiplexers configured to select one of the memories to obtain a portion of the one column of data from the selected memory; a controller configured to select one of the memories to store a portion of the one column of data; The portion of the data of the one column is obtained by dividing the data of the one column by N, where N is a positive integer greater than 1, The controller generates a control signal to the multiplexer so as to acquire the divided one column of data in one phase and acquire the one column of data in N phases. Information processing device.

6. a plurality of memories configured to collectively store data for one column of the input image data; a plurality of multiplexers configured to select one of the memories to obtain a portion of the one column of data from the selected memory; a controller configured to select one of the memories to store a portion of the one column of data; The portion of the data of the one column is obtained by dividing the data of the one column by N, where N is a positive integer greater than 1, The number of memories is (N*K)+S, where K is the number of multiplexers and S is the stride of the layers connected to the outputs of the multiplexers. Image processing unit.

7. A column of input image data is stored in a plurality of memories; selecting one of the memories to obtain a portion of the data of the one column from the selected memory; selecting one of the memories to store a portion of the one column of data; The portion of the data of the one column is obtained by dividing the data of the one column by N, where N is a positive integer greater than 1, The number of the memories is (N*K)+S, where K is the number of multiplexers that select one of the memories to obtain a portion of the data of one column from the selected memory, and S is the stride of the layer connected to the output of the multiplexers. Image processing methods.

8. storing data of one column of the input image data across multiple memories; selecting one of the memories to obtain a portion of the one column of data from the selected memory; and selecting one of the memories to store a portion of the one column of data; The portion of the data of the one column is obtained by dividing the data of the one column by N, where N is a positive integer greater than 1, The number of the memories is (N*K)+S, where K is the number of multiplexers that select one of the memories to obtain a portion of the data of one column from the selected memory, and S is the stride of the layer connected to the output of the multiplexers. A program that causes a computer to carry out the method.

Images