Information processing apparatus

JP2024165900APending Publication Date: 2024-11-28CANON KK
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
JP2023082484
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-18
Publication Date
2024-11-28

AI Technical Summary

Benefits of technology

【0010】 本発明によれば、必要なメモリ容量を抑えつつ積和演算回路の効率的な動作を可能とする技術を提供することができる。

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024165900000001_ABST
    Figure 2024165900000001_ABST
Patent Text Reader

Abstract

To allow the efficient operation of a product sum arithmetic circuit.SOLUTION: An information processing apparatus comprises: a parallel operation unit that has M product sum operation units that can operate in parallel; a coefficient memory that holds a plurality of weighting coefficients used in the parallel operation unit; and a coefficient transfer unit that transfers the weighting coefficients held in the coefficient memory to the parallel operation unit. The coefficient memory holds the plurality of weighting coefficients according to the order of output channels of the parallel operation unit and the word align of the coefficient memory. When the parallel operation unit executes N pieces of arithmetic processing in parallel, the coefficient transfer unit transmits at least N weighting coefficients to the parallel operation unit in a predetermined transfer format.SELECTED DRAWING: Figure 3
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to arithmetic processing using a neural network (NN). [Background technology]

[0002] Methods using hierarchical calculations, such as convolutional neural networks (CNN), have been attracting attention as a method for recognizing objects in photos and videos. It is known that recognition processing using CNNs and other such methods involves many product-sum calculations, which take up most of the processing time. Therefore, there is a demand for calculation devices that can efficiently perform product-sum calculations from the viewpoints of speed and power saving.

[0003] In addition, methods to store weight coefficients in less memory are being considered for recognition processing in devices such as mobile terminals and cameras that are easily constrained by memory capacity. For example, mixed-precision CNNs that process weight coefficients with appropriate bit precision for each calculation or storage method have been proposed.

[0004] Patent Document 1 proposes a method for facilitating reading of weight coefficients by arranging weight coefficients with different bit precision in the same address. Patent Document 2 proposes a method for dividing the memory for storing weight coefficients into two and storing them in different formats so as not to leave free space in the memory storage area. Non-Patent Document 1 proposes a method for improving the utilization efficiency of memory and arithmetic devices by arranging feature data and weight coefficients in the channel direction of an input feature image. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent No. 6823495 [Patent Document 2] Patent Publication No. 2021-9491 [Non-patent literature]

[0006] [Non-Patent Document 1] F. Shafiq et al., "Automated flow for compressing convolution neural networks for efficient edge-computation with FPGA", NIPS 2017, arXiv:1712.06272 Summary of the Invention [Problem to be solved by the invention]

[0007] However, in Patent Document 1, if the weighting coefficient has a bit width that does not match the word alignment of the memory, part of the storage area will be unused, which may lead to an increase in the required memory capacity. In addition, in Patent Document 2, since the weighting coefficients are stored in two memories in different formats, a circuit is required to calculate the address of each memory when reading out the weighting coefficient. In Non-Patent Document 1, a circuit is required for raster scanning of the input feature image and channel conversion.

[0008] The present invention has been made in view of the above problems, and has an object to provide a technique that enables efficient operation of a product-sum calculation circuit while suppressing the required memory capacity. [Means for solving the problem]

[0009] In order to solve the above-mentioned problems, an information processing device according to the present invention has the following arrangement. a parallel arithmetic unit having M (M is an integer of 2 or more) multiply-and-accumulate units operable in parallel; a coefficient memory for storing a plurality of weighting coefficients used in the parallel calculation unit; a coefficient transfer unit that transfers the weighting coefficients stored in the coefficient memory to the parallel calculation unit; Equipped with the coefficient memory holds the weighting coefficients in the order of the output channels of the parallel arithmetic unit and in accordance with a word alignment of the coefficient memory; When the parallel arithmetic unit executes N arithmetic processes in parallel (N is a positive integer equal to or less than M), the coefficient transfer unit transfers at least N weighting coefficients to the parallel arithmetic unit in a predetermined transfer format. Effect of the Invention

[0010] According to the present invention, it is possible to provide a technique that enables efficient operation of a product-sum calculation circuit while suppressing the required memory capacity. [Brief description of the drawings]

[0011] [Figure 1] 1 is a block diagram showing an example of the configuration of an information processing device; [Diagram 2] FIG. 2 is a diagram illustrating an example of the configuration of a network to be processed. [Diagram 3] 11 is a block diagram showing a configuration example of a convolution processing unit. FIG. [Figure 4] 13 is a flowchart for storing weighting coefficients. [Diagram 5] FIG. 13 is a diagram illustrating an example of a storage format of weighting coefficients. [Figure 6] FIG. 2 is a block diagram showing a configuration example of a MAC calculator group. [Figure 7] FIG. 2 is a block diagram showing an example of the configuration of a MAC calculator. [Figure 8] 1 is a flowchart of CNN processing. [Figure 9] FIG. 13 is a diagram illustrating a transfer format of a weighting coefficient (second embodiment). [Figure 10] FIG. 13 is a diagram illustrating an example of a storage format of weighting coefficients (second embodiment). [Figure 11] FIG. 13 is a block diagram showing an example of the configuration of a MAC calculator (second embodiment). [Figure 12] 13 is a flowchart of a CNN process (second embodiment). [Figure 13] 11 is a block diagram showing an example of the configuration of a bit width conversion unit; [Figure 14] FIG. 13 is a diagram illustrating a transfer format of a weighting coefficient (third embodiment). [Figure 15] 13 is a flowchart for storing weighting coefficients (third embodiment). [Figure 16] FIG. 13 is a diagram illustrating an example of a storage format of weighting coefficients (third embodiment). [Figure 17] FIG. 13 is a block diagram showing an example of the configuration of a MAC calculator (third embodiment). [Figure 18] FIG. 13 is a flowchart of CNN processing (third embodiment). [Figure 19] FIG. 13 is a diagram illustrating an example of the operation of a MAC calculator (third embodiment). DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0013] (First embodiment) As a first embodiment of an information processing device according to the present invention, an information processing device that performs arithmetic processing using a convolutional neural network (CNN) will be described below as an example.

[0014] <Device configuration> Fig. 1 is a block diagram showing a configuration example of an information processing device. Note that, although each unit shown in Fig. 1 is illustrated as being included in a single device, each unit may be configured to be distributed and included in multiple devices connected by communication paths.

[0015] The input unit 101 is a device for inputting instructions and data from a user, and includes a keyboard, a pointing device, buttons, etc. Alternatively, the display unit 104 and the input unit 101 may be the same device, as in a known touch screen device. In that case, input via the touch screen is treated as input to the input unit 101.

[0016] The data storage unit 102 is a section that stores image data. It is usually configured as a hard disk, a flexible disk, an optical disk, a flash memory, or the like. In addition to image data, the data storage unit 102 can also store programs and other data. Alternatively, a part of the RAM 108 described later may be used as the data storage unit 102. Alternatively, the data storage unit 102 may be configured virtually such that a storage device connected via a communication unit 103 described later is used via the communication unit 103.

[0017] The communication unit 103 is an I / F for communication between devices. The display unit 104 is a device for displaying images before and after image processing, or images such as a graphical user interface (GUI), and is generally a CRT or liquid crystal display. Alternatively, it may be an external display device connected by a cable or the like.

[0018] The image processing unit 105 is a processing unit that receives a command from the CPU 106, reads out the image data written in the data storage unit 102, adjusts the range of the pixel values, and writes the result back into the RAM 108.

[0019] CPU 106 is a device that controls the operation of the entire device. ROM 107 and RAM 108 provide CPU 106 with programs, data, work areas, and the like required for the processing. When a program required for the processing described below is stored in data storage unit 102 or in ROM 107, it is first loaded into RAM 108 and then executed. Alternatively, when the device receives a program via communication unit 103, it is first recorded in data storage unit 102 and then loaded into RAM 108, or it is directly loaded from communication unit 103 into RAM 108 and then executed.

[0020] The convolution processing unit 109 is a device that processes product-sum operations related to CNN. Upon receiving a command from the CPU 106, the convolution processing unit 109 performs CNN processing including product-sum operations on the results of image processing stored in the RAM 108, and writes the results to the RAM 108. The CPU 106 performs image processing or image recognition based on the results of the CNN processing, and records the results in the RAM 108.

[0021] <Network configuration> 2 is a diagram showing an example of the configuration of a network of the convolution processing unit 109. The network structure is characterized by information on each layer (connection relationships between layers, size and number of channels of feature images, bit width of feature data, size of filters, bit width of weighting coefficients, etc.). CNN200 shows a CNN with four layers (layers 1 to 4). Each layer includes 32 channels of feature images and a filter composed of multiple weighting coefficients.

[0022] The calculation contents in layers 1 to 4 will be explained. In layer 1, a multiply-and-accumulate result O(n) is calculated using a feature image I(m) of multiple channels in layer 201 and a filter C(m,n) based on formula (1). In addition, a feature image 202 of multiple channels in layer 202 in layer 2 is generated by performing processes such as activation processing and pooling.

[0023]

number

[0024] Similarly, in layers 2 to 4, the feature images and filters are used to obtain product-sum calculation results, and feature images for multiple channels are generated by performing activation processing, pooling, etc. In this way, the generation of feature images for each layer is repeated to obtain the processing results of CNN.

[0025] The variables in formula (1) are explained below. Variable m is the number of feature image I in the previous layer, and variable M is the number of channels of the feature image in the previous layer. Variable n is the number of feature image O in the current layer, variables i and j are the coordinates of the feature data of feature image I or O, variables x and y are the coordinates of the weighting coefficient of filter C, and variables X and Y are the size of the filter.

[0026] <Configuration of the convolution processing unit> 3 is a block diagram showing an example of the configuration of the convolution processing unit 109. The convolution processing unit 109 is made up of a feature data memory 301, a feature data transfer unit 302, a weighting coefficient memory 303, a weighting coefficient transfer unit 304, a parallel calculation unit 305, and a control unit 306.

[0027] A feature data memory 301 temporarily holds input feature images I(m) of multiple channels. A feature data transfer unit 302 reads out multiple pieces of feature data to be used in a product-sum operation executed by a parallel calculation unit 305 from the feature data memory 301 and transmits them to the parallel calculation unit 305.

[0028] The weighting coefficient memory 303 temporarily holds a plurality of weighting coefficients C(m,n). The weighting coefficient memory 303 is composed of, for example, an SRAM. One layer contains X×Y×M×N weighting coefficients, and the total amount for the entire network is the sum of these for each layer. The network to be processed is not limited to the structure shown in FIG. 2, and may be a network having more layers and feature images, in which case it is necessary to store even more weighting coefficients. By storing the weighting coefficients according to the flowchart shown in FIG. 4, which will be described later, the weighting coefficients can be efficiently stored in the memory.

[0029] The weighting coefficient transfer unit 304 reads out from the weighting coefficient memory 303 a number of weighting coefficients to be used in the product-sum calculation executed by the parallel calculation unit 305, and transmits them to the parallel calculation unit 305. The parallel calculation unit 305 executes the product-sum calculation, activation processing, and pooling processing using the received number of feature data and weighting coefficients. The configuration shown in Fig. 6, which will be described later, makes it possible to efficiently execute the product-sum calculation in parallel and suppress an increase in power consumption.

[0030] The control unit 306 holds network structure information and controls the operation of the convolution processing unit 109 .

[0031] <Weighting coefficient storage format> We will explain the format in which the weight coefficients of the convolutional network are stored in the weight coefficient memory 303. Here, we will explain the issues when storing coefficients using an example in which the word length (number of bits in 1 word) (WD bits) of the weight coefficient memory 303 is 32 bits, the filter size is 3×3, and the bit width of the weight coefficient is 8 bits (bit width CA).

[0032] If the nine weighting coefficients contained in a 3 x 3 filter are stored in order for each pixel, four weighting coefficients will be stored in the first and second words, and one in the third word. If the next filter is stored subsequently, three weighting coefficients will be stored in the free space in the third word, four in the fourth word, and two in the fifth word. In this case, because the bit positions used to store the necessary weighting coefficients differ between the first and second filters, a circuit is required to switch the bit positions from which the weighting coefficients are extracted.

[0033] As an alternative storage method, if the second filter is stored from the fourth word so as to align the bit position of the weight coefficient, a 24-bit unused area will be created in the third word. Even if three weight coefficients are arranged in each word, an 8-bit unused area will be created in each word. The creation of unused areas in this way leads to an increase in memory capacity.

[0034] 4 is a flowchart for storing weighting coefficients in the first embodiment. In the first embodiment, a case where the bit width of the weighting coefficients is all the same will be described as an example.

[0035] In S401, the control unit 306 divides the number of channels of the output feature image of the layer to be processed into groups for each number of weighting coefficients that fit into one word of the weighting coefficient memory 303. When one word is 32 bits and the weighting coefficient is 8 bits, the groups are divided into groups for every 4 channels.

[0036] In S402, the control unit 306 starts a loop to store each divided group. Initially, the first group created in S401 is the four channels with n being 0 to 3.

[0037] In S403, the control unit 306 starts a loop to store the number of channels of the input feature image of the layer to be processed. In the example of Fig. 2, the number of channels of the input feature image is 32, so the loop is performed 32 times.

[0038] In S404, the control unit 306 starts a loop to store the number of pixels equal to the filter size. If the filter size is 3×3, the loop is repeated nine times.

[0039] In S405, the control unit 306 stores weighting coefficients for the output channels in the group. Here, weighting coefficients for four channels are stored to match the number of channels in the group. This data fits into one word.

[0040] In S406, the control unit 306 determines whether the loop has been repeated the number of times equal to the filter size. If all pixels of the filter have been stored, the process proceeds to S407. If not, the process returns to S404 to start storing the next pixel.

[0041] In S407, the control unit 306 determines whether the loop has been repeated for the number of channels of the input feature image. If the loop has been repeated for all channels of the input feature image of the layer to be processed, the process proceeds to S408. If not, the process returns to S403 and starts storing the input feature image of the next channel.

[0042] In S408, the control unit 306 determines whether the loop has been repeated for the number of divided groups. If storage of all divided groups has been completed, storage of weighting coefficients ends. If not, the process returns to S402 and storage of the next group begins. Through the above process, the multiple weighting coefficients are stored in the order of the output channels of the parallel calculation unit 305 and in accordance with the word alignment of the weighting coefficient memory 303.

[0043] 5 is a diagram showing an example of a storage format of weighting coefficients in the first embodiment, that is, showing the structure of weighting coefficients stored in the coefficient memory 303 according to the above steps.

[0044] The word stores weighting coefficients of the output feature image for the number of channels included in the group stored in accordance with S405. The first word 501 stores weighting coefficients for the filter coordinates (0,0) corresponding to channels 0 to 3 (total of 4 channels) of the output feature image.

[0045] Weighting coefficients for 9 pixels corresponding to a filter size of 3 × 3 are stored in an area 502 from the 1st to 9th words by the loop of S404. Weighting coefficients for 32 channels of the input feature image are stored in an area 503 from the 1st to 288th words by the loop of S403.

[0046] If the number of channels of the output feature image included in the layer to be processed is four or more, the second group of channels of the output feature image separated in S401 is stored in a similar format from the 289th word.

[0047] As described above, the weighting coefficients are arranged in each word in the order of the channel direction of the output feature image. This allows weighting coefficients to be stored without creating unused space in the word regardless of the size of the filter, and also makes it easy to identify the bit position of the weighting coefficient according to the channel number.

[0048] Although the word length of the coefficient memory 303 is exemplified as 32 bits, the number of bits is not limited to this, and for example, the number of bits per word may be further increased to shorten the time required for storing the weight coefficients.

[0049] The weighting coefficients may be learned in advance and stored in the data storage unit 102 or the ROM 107 according to the flowchart of FIG. 4, and may be stored in the weighting coefficient memory 303 when the convolution process shown in FIG. 8 is executed.

[0050] <Configuration of the parallel calculation unit 303> 6 is a block diagram showing a configuration example of a MAC calculator group which is parallel calculation unit 303. Parallel calculation unit 303 includes a MAC calculator group 600 including M (M is an integer equal to or greater than 2) MAC calculators which can operate in parallel. MAC (Multiply and ACcumulation) means a multiply-and-accumulate operation.

[0051] The MAC calculator group 600 is a circuit that processes in parallel feature data for one pixel of multiple channels of an output feature image, and includes a weighting coefficient separation unit 601 and multiple MAC calculators 602 to 605. The MAC calculator group 600 includes MAC calculators equal in number to the number of channels of the output feature image to be processed in parallel, and copies and inputs the feature data transferred from the feature data transfer unit 302 to each MAC calculator. Since the common feature data copied by each MAC calculator can be used, it is possible to reduce the amount of bandwidth used for transferring input data.

[0052] The weighting coefficient separation unit 601 separates the weighting coefficients for the multiple channels transferred from the weighting coefficient transfer unit 304 for each channel and transmits them to each MAC calculator. Specifically, when the first word 501 of the weighting coefficient memory 303 is transferred, the weighting coefficient C 0,0 (0,0) is sent to the MAC calculator 602. Similarly, the weighting coefficient C 0,0 (0,1) to the MAC calculator 603, and the weighting coefficient C 0,0 (0,2) is input to the MAC calculator 604, and the weighting coefficient C 0,0 (0,3) is sent to the MAC calculator 605 .

[0053] Since the bit position for each channel in the word corresponds to each MAC calculator, the weighting coefficient separator 601 only needs to connect the input weighting coefficient bit to the corresponding MAC calculator. In other words, there is no need for a circuit to switch the bit extraction position.

[0054] As shown in FIG. 5, the weight coefficient separation unit 601 can distribute the weight coefficients of each channel included therein to each MAC arithmetic unit by receiving the weight coefficients arranged in the order of the output feature images for each word. Therefore, the weight coefficient transfer unit 304 may simply read and transmit the weight coefficients from the head address of the target filter in the weight coefficient memory 303 incrementally, and can be configured with a simple circuit.

[0055] Note that the number of channels of the weight coefficients included in a word of the weight coefficient memory 303 is not necessarily the same as the number of MAC arithmetic units included in the MAC arithmetic unit group 600. For example, the weight coefficient transfer unit 304 may reduce the number of weight coefficients to be transmitted according to the number of MAC arithmetic units included in the MAC arithmetic unit group 600 to reduce the scale of the transmission path and the arithmetic units.

[0056] FIG. 7 is a block diagram showing a configuration example of the MAC arithmetic unit in the first embodiment. The MAC arithmetic unit includes a feature data holding unit 701, a weight coefficient holding unit 702, a product-sum arithmetic unit 703, and an activation / pooling processing unit 704. The weight coefficient holding unit 702 holds the weight coefficient C x,y (m,n), and the feature data holding unit 701 holds the feature image I(m). The product-sum arithmetic unit 703 calculates the convolution processing result from the weight coefficient and the feature data based on Equation (1) and stores it in the accumulation register 705. The activation / pooling processing unit 704 calculates the activation / pooling processing result based on the convolution processing result.

[0057] <Operation of arithmetic processing using CNN (CNN processing)> FIG. 8 is a flowchart of the CNN processing. S801 to S815 are control operations of a CPU or a sequencer included in the control unit 306. Each step of the flowchart will be described based on the configuration of the convolution processing unit 109.

[0058] In S801, the control unit 306 reads out feature data of an input feature image, weighting coefficients of a filter, and network structure information from the RAM 108. Then, the control unit 306 stores the read feature data in the feature data memory 301, the weighting coefficients in the weighting coefficient memory 303, and the network structure in the control unit 306.

[0059] In S802, the control unit 306 starts a loop of layers, starting with layer 1.

[0060] In S803, the control unit 306 groups the output feature data in the layer for each number of channels parallelized by the MAC calculator group 600. Since the MAC calculator group 600 of this embodiment includes four MAC calculators, the data is grouped for each four channels.

[0061] In S804, the control unit 306 starts a loop for each group divided in S803. In this embodiment, since the groups are divided into four channels, first, the 0th to 3rd channels of the output feature image are processed.

[0062] In S805, the control unit 306 starts a loop of pixel positions. The process is repeated for the entire image so as to output an output feature image for each pixel. If the parallel calculation unit 305 is configured with multiple MAC calculation unit groups 600, the loop is repeated while processing the pixels of the output feature image in parallel for the number of MAC calculation unit groups 600.

[0063] In S806, the control unit 306 performs initialization by setting the convolution processing results held in the accumulation registers 705 of each MAC calculator to zero.

[0064] In S807, the control unit 306 starts a loop of the input feature image, and processes the input feature data in the layer in order, starting with the input feature data of the first channel.

[0065] In S808, the control unit 306 controls the feature data transfer unit 302 to read the input feature image to be processed from the feature data memory 301, and transfers it to the parallel calculation unit 305. If the filter size is 3×3, first, the feature data transmission unit 301 transmits the feature data contained in the coordinates (0,0) to (2,2). In parallel, the control unit 306 controls the coefficient transfer unit 304 to read the weighting coefficient for the input feature image to be processed from the weighting coefficient memory 303, and transfers it to the parallel calculation unit 305. When the parallel calculation unit 305 executes N calculation processes in parallel (N is a positive integer equal to or less than M), at least N weighting coefficients are transferred to the parallel calculation unit 305.

[0066] When the filter size shown in FIG. 5 is 3×3 and the input feature image of the first channel is to be processed and the output feature images of the first to fourth channels are to be processed, the 1st to 36th words 503 are read and transmitted. The transmitted feature data is copied to each MAC calculator in the MAC calculator group 600 as described above and is held in each feature data holding unit 701. The transmitted weighting coefficients of the multiple channels are distributed to the MAC calculators for each channel and are held in each weighting coefficient holding unit 702. In a configuration including multiple MAC calculator groups 600 in order to parallelize the processing in the spatial direction of the feature image, feature data corresponding to the output coordinates is transmitted to each MAC calculator group. As an example, in the case of 2 parallelism, feature data corresponding to the coordinates where i in formula (1) is an even number is assigned to the MAC calculator group 0, and feature data corresponding to the coordinates where i is an odd number is assigned to the MAC calculator group 1.

[0067] In S809, the control unit 306 transmits a control signal for executing the convolution process to the product-sum calculation unit 703 of each MAC calculator. The product-sum calculation unit 703 of each MAC calculator calculates the result of the convolution process based on the formula (1) using the input feature data stored in the feature data storage unit 701 and the weighting coefficients stored in the weighting coefficient storage unit 702.

[0068] In S810, the control unit 306 determines whether the loop of the input feature image has ended. If the processing of all the input feature data has ended, the process proceeds to S811. If not, the process returns to S807 and starts processing the next input feature image.

[0069] In S811, the control unit 306 transmits a control signal for executing the activation and pooling process to the activation and pooling processing unit 704 of each MAC calculation unit. The activation and pooling processing unit 704 of each MAC calculation unit performs the activation process based on the convolution process result stored in the product-sum calculation unit 703. Here, the activation process result is calculated by the following formula (2).

[0070]

number

[0071] f(x) is the activation function, and x is the input data. In this example, the activation function is realized using ReLU (Rectified Linear Unit), but it is not limited to ReLU and may be realized using other nonlinear functions or quantization functions. In addition, the activation / pooling processing unit 704 performs pooling processing based on the result of activation processing according to the layer information, and adjusts the size of the output feature image as necessary.

[0072] In S812, the control unit 306 stores the activation pooling process result in the feature data memory 301, and handles it as a feature image of the next layer.

[0073] In S813, the control unit 306 determines whether the loop for pixel positions has ended. If the processing for all pixels has ended, the process proceeds to S814. If not, the process returns to S805 to process the next pixel position.

[0074] In S814, the control unit 306 determines whether the loop for each group of output feature images has ended. If all output feature image groups have been processed, the process proceeds to S815. If not, the process returns to S804 and starts processing the next output feature image group.

[0075] In S815, the control unit 306 determines whether the layer loop has ended. If all layers have been processed, the CNN processing ends. If not, the process returns to S802, the processing target layer is changed, and processing of the next layer begins.

[0076] As described above, according to the first embodiment, the weighting coefficients are stored in the channel direction of the output feature image. This allows the weighting coefficients to be arranged without free space in the weighting coefficient memory 303, and the memory can be used efficiently. In addition, the parallel calculation unit 305 includes a group of MAC calculators that process feature data in parallel in the channel direction of the output feature image, and each MAC calculator includes an accumulation register. Therefore, calculations can be performed efficiently in accordance with the storage format of the weighting coefficients. Furthermore, since the configuration allows expansion in the spatial direction of the feature image without impairing the efficiency of calculation execution, it is also applicable to a CNN configuration that requires high-speed processing.

[0077] Second embodiment In the second embodiment, a configuration in which weighting coefficients of different bit widths are used will be described, and a description of the same configuration as in the first embodiment will be omitted.

[0078] <Network configuration> The network configuration is the same as that of the first embodiment (FIG. 2), except that the bit width of the weighting coefficients in only layer 1 is 1 bit, and the bit width of the weighting coefficients in the other layers is 8 bits.

[0079] <Configuration of the convolution processing unit> The configuration of the convolution processing unit 109 is the same as that of the first embodiment (FIG. 3). However, the weighting coefficient transfer unit 304 converts the weighting coefficient into the transfer format shown in FIG. 9 according to the bit width and transmits it to the parallel calculation unit 305. As for the transfer format, the number of weighting coefficients transmitted to the parallel calculation unit 305 is the same as the number of MAC calculators included in the MAC calculation unit group regardless of the bit width, and the arrangement of the weighting coefficients differs depending on the bit width.

[0080] Fig. 9 is a diagram showing a predetermined transfer format of the weighting coefficient in the second embodiment. Fig. 9(A) shows a transfer format when the weighting coefficient is 8 bits (bit width CA). When the MAC calculator group 600 has four MAC calculators, four 8-bit weighting coefficients are transmitted. On the other hand, Fig. 9(B) shows a transfer format when the weighting coefficient is 1 bit (bit width CB), and when the MAC calculator group 600 has four MAC calculators, a 1-bit weighting coefficient is arranged in the lowest bit of every 8 bits and transmitted. In this way, when the number of MAC calculators is M, the transfer format is configured to include M fixed-length fields regardless of the bit width of the weighting coefficient.

[0081] The weighting coefficients are converted in this way and transmitted to parallel calculation unit 305. This allows weighting coefficient separator 601 of MAC calculation unit group 600 to distribute the weighting coefficients for each channel to each MAC calculation unit, regardless of the bit width of the weighting coefficients.

[0082] <Storage behavior and format of weighting coefficients> The flowchart for storing weighting coefficients is the same as that of the first embodiment (FIG. 4). The 8-bit weighting coefficients of layers 2 to 4 are stored in weighting coefficient memory 303 in the same storage format as that of the first embodiment (FIG. 5). The storage format of the 1-bit (bit width CB) weighting coefficient of layer 1 will be described with reference to FIG. 10.

[0083] Fig. 10 is a diagram showing an example of a storage format of weighting coefficients in the second embodiment, that is, the structure of a 1-bit (CB bit) weighting coefficient of layer 1 stored in weighting coefficient memory 303 according to the flowchart of Fig. 4 is shown.

[0084] When the word length of the weighting coefficient memory 303 is 32 bits and the weighting coefficient is 1 bit, in S401, the channels of the output feature image are grouped into 32 channels. In addition, in S405, the weighting coefficients for 32 channels are stored in one word. The first word 1001 is C 0,0 (0,0)~C 0,0(0, 31) is stored. Regardless of the bit width, if the filter size is 3×3, 9 pixels are stored in the area 1002 of the 1st to 9th words, and 32 channels of the input feature image are stored in the area 1003 of the 1st to 288th words.

[0085] In this way, within the word of the weight coefficient memory 303, regardless of the bit width of the weight coefficient (1 or 8 bits), they are stored in order along the channel direction of the output feature image. That is, each word area holds either WD / CA 8-bit-width weight coefficients or WD / CB 1-bit-width weight coefficients. As a result, even when dealing with weight coefficients of multiple bit widths, they can be stored without creating unused areas within the word of the weight coefficient memory 303. Also, the pixel position of each weight coefficient can be configured to be independent of the bit width of the weight coefficient.

[0086] <Configuration of the parallel operation unit 303> FIG. 11 is a block diagram showing a configuration example of the MAC arithmetic unit in the second embodiment. The MAC arithmetic unit in the second embodiment has a configuration in which a bit width conversion unit 1105 is added to the configuration of the first embodiment (FIG. 7).

[0087] The bit width conversion unit 1105 converts the weight coefficient to 8 bits according to the weight coefficient of the processing target specified by the control unit 306 and transmits it to the product-sum operation unit 1103. This enables the product-sum operation unit 1103 to process 1-bit and 8-bit weight coefficients with the same arithmetic unit, and suppresses an increase in circuit scale due to implementing arithmetic units with different bit widths.

[0088] <Arithmetic processing operation using CNN> FIG. 12 is a flowchart of the CNN processing in the second embodiment. The flowchart in the second embodiment has a configuration in which S1203 and S1210 are added to the first embodiment (FIG. 8). Hereinafter, steps different from the first embodiment will be described.

[0089] In S1203, the control unit 306 sets the bit width of the weighting coefficient in the bit width conversion unit 1105 according to the retained network structure information. In this example, the bit width of the weighting coefficient of the same layer is the same, but a different bit width may be set for each output feature image. In addition, when the output feature image is divided into a plurality of groups, it is also possible to improve processing efficiency by setting the bit width of the weighting coefficient for each group and performing parallel processing on a group basis.

[0090] In S1209, the control unit 306 controls the feature data transfer unit 302 to read the input feature image to be processed from the feature data memory 301, and transfers the input feature image to be processed to the parallel calculation unit 305. In parallel with this, the control unit 306 controls the coefficient transfer unit 304 to read the weighting coefficient for the input feature image to be processed from the weighting coefficient memory 303, converts the weighting coefficient into the transfer format shown in FIG.

[0091] In S1210, the control unit 306 transmits a control signal for executing a bit width conversion process to the bit width conversion unit 1105. The bit width conversion unit 1105 converts the weighting coefficient based on the stored weighting coefficient and information on the bit width of the weighting coefficient.

[0092] If the bit width of the weighting factor is not 1 bit (i.e., 8 bits), the weighting factor C x,y (m,n) is not converted. When the bit width of the weighting coefficient is 1 bit, the processing may be performed using the value "0" or "1" stored in the weighting coefficient memory 303 as is, but in that case, negative numbers cannot be displayed. In convolution processing with a bit width of 1 bit, if a weighting coefficient with a value of "0" or "1" is used, it is difficult to improve the recognition accuracy, so a weighting coefficient with a value of "-1" or "+1" is often used. In this embodiment, before performing the convolution processing, the bit width conversion unit 1105 converts the weighting coefficient C x,y (m,n) is converted using the following formula (3). That is, a weighting coefficient with a value of "0" is converted to "-1", and a weighting coefficient with a value of "1" becomes "+1".

[0093]

number

[0094] The bit width conversion unit 1105 expands the bit width of the weighting coefficient from 1 bit to 8 bits. When the value of the weighting coefficient is expressed in binary, the formula (3) becomes the following formula (4).

[0095]

number

[0096] Weighting factor C before conversion x,y (m,n) has a bit width of 1 bit and does not have a sign bit. The converted weighting coefficient C'x,y(m,n) has a bit width of 8 bits and has a sign bit. The weighting coefficient is expressed in "2's complement".

[0097] 13 is a block diagram showing an example of the configuration of the bit width conversion unit 1105. The bit width conversion unit 1105 expands the bit width to 8 bits based on a weighting coefficient bit width switching signal and a 1-bit weighting coefficient.

[0098] When the weighting coefficient before conversion is 1 bit, the circuit 1301 converts it into an 8-bit weighting coefficient according to the formula (4). As can be seen from the formula (4), the least significant bit of the 8-bit weighting coefficient after conversion is always "1". Furthermore, the remaining 7 bits all have values ​​that are inverted with respect to the 1-bit weighting coefficient before conversion. The result of processing by the circuit 1301 is then selected by the selection unit 1302. Unlike the normal bit expansion, the 1-bit weighting coefficient C x,y If (m,n) is "0", the value is converted, and if it is "1", the value is output as is. Therefore, the least significant bit is fixed at "1" without using the input signal.

[0099] On the other hand, if the weighting coefficient before conversion is 8 bits, the 8-bit weighting coefficient before conversion is directly selected by the selection unit 1302. In this way, the bit width conversion unit 1105 can simultaneously realize value conversion and bit width expansion with a simple circuit.

[0100] As described above, according to the second embodiment, even when weighting coefficients of different bit widths are mixed, memory can be used efficiently as in the first embodiment. Therefore, weighting coefficients of a small bit width (for example, 1 bit) are used in layers and nodes where the recognition accuracy is unlikely to decrease even if the bit width of the weighting coefficient is reduced. This makes it possible to reduce the memory capacity related to holding the weighting coefficients.

[0101] Third embodiment In the third embodiment, a configuration in which a single MAC calculator processes multiple calculations in parallel will be described. In the first and second embodiments, an example in which an input feature image is input to a single MAC calculator one channel at a time and processed has been described. In the third embodiment, multiple channels of an input feature image are also processed in parallel, thereby shortening the processing time and reducing power consumption. A description of the same configuration as in the first or second embodiment will be omitted.

[0102] <Network configuration> The CNN to be processed in the third embodiment has the same structure as in the first and second embodiments (FIG. 2). However, the following example will be described in which the bit widths of feature data and weighting coefficients are set for each layer as follows.

[0103] In layer 1, the bit width of the feature data is 2 bits, and the bit width of the weighting coefficient is 1 bit. In layer 2, the bit width of the feature data is 4 bits, and the bit width of the weighting coefficient is 1 bit. In layer 3, the bit width of the feature data is 8 bits, and the bit width of the weighting coefficient is 1 bit. In layer 4, the bit width of the feature data is 8 bits, and the bit width of the weighting coefficient is 8 bits.

[0104] <Configuration of the convolution processing unit> Fig. 14 is a diagram showing a transfer format of weighting coefficients in the third embodiment. The configuration of the convolution processing unit 109 is the same as that of the first embodiment (Fig. 3). The weighting coefficient transfer unit 304 converts the format according to the bit width of the weighting coefficients and the channel-wise parallelism (number of parallel processes) of the input feature image to be processed by the MAC calculator, and transmits the converted format to the parallel calculation unit 305. Specifically, as will be described with reference to Figs. 14(A) to (D), the number of weighting coefficients to be transmitted to the parallel calculation unit 305 is the number obtained by multiplying "the number of MAC calculators included in the MAC calculator group (four in this case)" by "the number of channels of the input feature image to be processed in parallel."

[0105] Fig. 14(A) shows the transfer format when the bit width of the weighting coefficient is 8 bits. In this case, four weighting coefficients are stored (i.e., the same as Fig. 9(A)). Fig. 14(B) to (D) respectively show the transfer format when the bit width of the weighting coefficient is 1 bit. However, Fig. 14(B) to (D) each have a different number of divisions (L) of the input feature image for parallel processing.

[0106] Fig. 14(B) shows a case where the input feature image, which is the processing data, is processed without being divided (the number of input channels is 1). When the bit width processed by one MAC calculator is 8 bits, the four weighting coefficients are stored in the lowest bit of every 8 bits (i.e., the same as Fig. 9(B)).

[0107] 14C shows a case where an input feature image is divided into two for processing (the number of input channels is two). When the bit width processed by one MAC calculator is 8 bits, the 8 bits are divided into two and the weight coefficient corresponding to the input feature image to be processed in parallel is stored in the lowest 1 bit of each of the 4 bits. When there are four MAC calculators in the MAC calculator group 600, four sets of these are arranged.

[0108] 14(D) shows a case where the input feature image is divided into four for processing (the number of input channels is 4). When the bit width processed by one MAC calculator is 8 bits, the 8 bits are divided into four and the weight coefficient corresponding to the input feature image to be processed in parallel is stored in the lowest 1 bit of each of the 2 bits. When there are four MAC calculators in the MAC calculator group 600, four sets of these are arranged.

[0109] By converting the format in this way and transferring the weighting coefficients to the parallel calculation unit 305, the weighting coefficient separation unit 601 can distribute the weighting coefficients for each input channel to each MAC calculation unit.

[0110] <Storage behavior and format of weighting coefficients> FIG. 15 is a flowchart showing a process for storing weighting coefficients in the third embodiment.

[0111] In S1501, the control unit 306 divides the channels of the output feature image of the layer to be processed into a plurality of output groups. In the third embodiment, the channels of the output feature image are grouped for each number of weighting coefficients that fit within a bit width obtained by dividing the "word length of the weighting coefficient memory 303" by the "number of channels (IN) of the input feature image to be processed in parallel by the MAC calculator."

[0112] If one word is 32 bits, the weighting coefficient is 1 bit, and the number of channels of the input feature image to be processed in parallel is 4, then the number of channels will be limited to 8 bits (32 bits divided by 4), so the group will be divided into groups of 8 channels.

[0113] In S1503, the control unit 306 groups the channels of the input feature image by the number of channels to be processed in parallel by one MAC calculator. When the MAC calculator processes two channels in parallel, the control unit 306 divides the channels into groups of two channels, and when the MAC calculator processes four channels in parallel, the control unit 306 divides the channels into groups of four channels.

[0114] In S1507, the control unit 306 stores weighting coefficients for the number of channels of the input feature image grouped in S1503. Initially, the four channels grouped in S1503 with m being 0 to 3 are targeted.

[0115] In S1508, the control unit 306 determines whether the loop has been repeated for the number of channels in the output group. If storage for the output group has been completed, the process proceeds to S1509. Furthermore, when storage for the output group has been completed, storage of one word of weighting coefficients is complete. In this example, since 1-bit weighting coefficients with m being 0 to 3 and n being 0 to 7 are stored, storage of 32 bits of data is completed. If not, the process returns to S1506 and storage of the next channel is started.

[0116] The structure of the weighting coefficients stored in the weighting coefficient memory 303 will be described with reference to the flowchart in Fig. 15. An 8-bit weighting coefficient is stored in the weighting coefficient memory 303 in the same storage format as in the first embodiment (Fig. 5). When the weighting coefficient is 1 bit and the bits of the input feature image are not divided (the number of channels to be processed in parallel is 1), it is stored in the weighting coefficient memory 303 in the same storage format as in the second embodiment (Fig. 10).

[0117] 16 is a diagram showing an example of a storage format of weighting coefficients in the third embodiment. In particular, examples of the storage format are shown for the cases where the number of channels for parallel processing of an input feature image is 2 and 4. Specifically, when the bit width of the weighting coefficient is C and the word length of the weighting coefficient memory 303 is WD bits, the weighting coefficient memory 303 holds a set of WD / (C×L) weighting coefficients for each word area.

[0118] FIG. 16(A) shows a structure in the case where the weighting coefficient is 1 bit and the number of channels for parallel processing of the input feature image is 2. When the word length of the weighting coefficient memory 303 is 32 bits, the weighting coefficient is 1 bit, and the number of channels for parallel processing is 2, in S1501, the channels of the output feature image are grouped into groups of 16 (=32 / 2) channels. In S1503, the channels of the input feature image are grouped into groups of 2 channels, which is the number of channels for parallel processing. In S1507, the weighting coefficient for every 2 channels of the input feature image is stored. By repeating the loop of S1506 to S1508 for 16 channels, a set of 16 weighting coefficients (C 0,0(0,0) to C 0,0 (1,15)) is stored.

[0119] Figure 16(B) shows the structure when the weight coefficient is 1 bit and the number of channels for parallel processing of the input feature image is 4. When the word length of the weight coefficient memory 303 is 32 bits, the weight coefficient is 1 bit, and the number of channels for parallel processing is 4, in S1501, the channels of the output feature image are grouped every 8 (= 32 / 4) channels. In S1503, the channels of the input feature image are grouped every 4 channels, which is the number of channels for parallel processing. In S1507, the weight coefficients for every 4 channels of the input feature image are stored. By repeating the loop of S1506 to S1508 for 8 channels, 8 sets of weight coefficients (C 0,0 (0,0) to C 0,0 (3,7)) is stored.

[0120] <Configuration of parallel arithmetic unit 303> Figure 17 is a block diagram showing a configuration example of the MAC arithmetic unit in the third embodiment. The MAC arithmetic unit in the third embodiment has a configuration in which a shift arithmetic unit 1706 and an addition unit 1707 are added to the configuration of the second embodiment (Figure 11).

[0121] The shift arithmetic unit 1706 performs a process of shifting (bit-shifting) the result of the product-sum operation 1703. The addition unit 1707 performs a process of adding together a plurality of product-sum operation results. Also, the product-sum operation unit 1703 has the configuration shown in Figure 19 described later and is provided with an accumulation register for each product-sum operation unit that divides and processes the input. Thereby, it becomes possible to perform parallel processing also in the channel direction of the input feature image. Also, operations can be executed efficiently in accordance with the storage format of the weight coefficients.

[0122] <Operation processing operation using CNN> Fig. 18 is a flowchart of the CNN process in the third embodiment. The flowchart in the third embodiment has a configuration in which S1804 and S1813 are added to the second embodiment (Fig. 12). The steps that differ from the second embodiment will be described below.

[0123] In S1804, the control unit 306 sets, in accordance with the layer information, shift parameters for the shift calculation unit 1706. In S1813, the shift calculation unit 1706 shifts the product-sum calculation result based on the shift parameters set in S1804.

[0124] <Channel-wise parallel processing of input feature images> Fig. 19 is a diagram showing an example of the operation of the MAC calculator in the third embodiment. As described above, in the third embodiment, input feature data and weighting coefficient bits are divided to process the input feature image in parallel in the channel direction as well. Fig. 19(A) and (B) show examples in which the feature data is 8 bits and 2 bits, respectively.

[0125] As shown in FIG. 19(A), when the feature data is 8 bits, the MAC calculator divides the feature data 1901 (decimal value "234"; binary value "11101010") into 2-bit pieces. The product-sum calculator 1703 calculates four product-sum calculation results using the four divided 2-bit data (decimal values ​​"2", "2", "2", "3") and a common weighting coefficient. The shift calculator 1706 shifts the four product-sum calculation results based on four shift parameters. The adder 1707 adds the four shifted product-sum calculation results to calculate one feature data (the product-sum calculation result of one 8-bit input feature data). In this way, the MAC calculator processes one 8-bit input feature data.

[0126] As shown in FIG. 19B, when the feature data is 2 bits, the MAC calculator divides the feature data 1902 (decimal value "234"; binary value "11101010") into 2-bit pieces. The product-sum calculator 1703 calculates the results of four product-sum calculations using the four divided 2-bit data (decimal values ​​"2", "2", "2", "3") and four weighting coefficients. The shift calculator 1706 shifts the four product-sum calculation results based on one shift parameter. Here, since the shift parameter is zero, the results before and after the shift calculation are the same. The adder 1707 adds the four shifted product-sum calculation results to calculate one feature data (the sum of the product-sum calculation results of the four 2-bit input feature data). In this way, the MAC calculator processes four 2-bit input feature data in parallel.

[0127] As described above, by bit-dividing the feature data and weighting coefficients and processing them, it becomes possible to handle a variety of bit widths and parallelism. At this time, as shown in input 1901 (8-bit width, 1 parallel) and input 1902 (2-bit width, 4 parallel), the value obtained by multiplying the bit width of one channel by the parallelism is made constant. This makes the transfer bandwidth of the input data constant, making it easy to predict the bandwidth that the convolution processing unit 109 will use for data transfer.

[0128] A more specific explanation will be given for the case where the number of input feature images is M and the filter size is 1 x 1. Since the filter size is 1 pixel and the values ​​of variables x and y are constants, Oi,j(n) is calculated using Ii,j(n). The product-sum calculation (Equation (1)) is simplified to the following Equation (5).

[0129]

number

[0130] When the filter size is greater than 1×1, the multiply-accumulate unit calculates the convolution result of the weighting coefficients and the input feature data, but when the filter size is 1×1, the multiply-accumulate unit calculates the product of I(m) and C(m,n).

[0131] It is assumed that the feature data to be processed has two types of bit widths, α bits and β bits (β>α). It is also assumed that there are P product-accumulation units and P product-accumulation units in the product-accumulation calculation unit 1703. α, β, and P satisfy the following conditions.

[0132]

number

[0133] When the bit width of the input feature data I'(β) is β bits, the output of the adder 1707 is given by the following formula (7).

[0134]

number

[0135] O(n) is the product-sum operation result of the nth output feature image, I(α),p(m) is the input data of the product-sum operation unit of α-bit data, Cp(m,n) is a weighting coefficient, and S(p) is a shift parameter. Variable m is the number of an α-bit input feature image group (1 group=P images) (processing number of the product-sum operation unit 1703), variable p is the number of the product-sum operation unit and shift operation unit, and variable n is the number of the output feature image. The shift operation is expressed as a power-of-two process.

[0136] The weighting coefficient Cp(m,n) is the weighting coefficient C'(m,n) corresponding to the m-th β-bit feature image. Since the weighting coefficient is common to the α-bit input feature image group, p can be omitted. The number of weighting coefficients supplied in parallel to the P multiply-add units is one, and the number of transfers is one.

[0137]

number

[0138] Here, the input feature data I'(β) is divided into P pieces of α-bit data I(α),p(m). The value of the shift parameter S(p) is calculated based on the number p of the multiply-accumulate unit and the bit width α of the divided data.

[0139]

number

[0140] The β-bit input feature data I'(β) is represented by P divided α-bit data I(α),p(m).

[0141]

number

[0142] By substituting the formulas (8) to (10) into the formula (7), the calculation formula for the output data O(n) is given by the formula (11).

[0143]

number

[0144] On the other hand, when the bit width of the input feature data I'(α) is α bits, the output of the adder 1707 is as shown in the following formula (12).

[0145]

number

[0146] O(n) is the product-sum operation result of the nth output feature image, I(α),p(m) is the input data of the product-sum operation unit of α-bit data, Cp(m,n) is a weighting coefficient, and S(p) is a shift parameter. Variable m is the number of an α-bit input feature image group (1 group=P images) (processing number of the product-sum operation unit 1703), variable p is the number of the product-sum operation unit and shift operation unit, and variable n is the number of the output feature image. The shift operation is expressed as a power-of-two process.

[0147] The weighting coefficient Cp(m,n) is the weighting coefficient C'((m-1)×P+p,n) corresponding to the (m-1)×P+p-th α-bit feature image. Since the weighting coefficient differs depending on the number p of the multiply-add calculation unit, the number of weighting coefficients supplied in parallel to the P multiply-add calculation units is P, and the number of transfers is P.

[0148]

number

[0149] The input feature data I'(α) becomes the input data I(α),p(m) of the multiply-add unit of α-bit data, and the value of the shift parameter S(p) is always 0.

[0150]

number

[0151] P pieces of α-bit input feature data I'(α) are input as is to the product-sum operation unit, but each of the P pieces of input data is feature data of a different feature image. The feature image number is expressed by the product-sum operation unit number p, the number P of product-sum operation units, and the processing number m of the product-sum operation unit 1703.

[0152]

number

[0153] By substituting the formulas (13) to (15) into the formula (12), the calculation formula for the output data O(n) is given by the formula (16).

[0154]

number

[0155] The value of the shift parameter S(p) and the number of weighting coefficients are changed. As described above, the feature data I'(α) with a bit width of α bits and the feature data I'(β) with a bit width of β bits can be processed by the same MAC calculator.

[0156] (Modification) In the first and second embodiments, a configuration for performing parallel processing in the channel direction of an output feature plane has been described, and in the third embodiment, a configuration for performing parallel processing also in the channel direction of an input feature plane has been described. As a modified example, a configuration may be adopted in which the operation mode of the convolution processing unit 109 can be dynamically changed. For example, the convolution processing unit 109 may be configured to include a register or the like for setting the operation mode, and the operation described in the first to third embodiments may be switched by changing the value of the register according to the configuration of the network.

[0157] The disclosure of this specification includes the following information processing device. (Item 1) a parallel arithmetic unit having M (M is an integer of 2 or more) multiply-and-accumulate units operable in parallel; a coefficient memory for storing a plurality of weighting coefficients used in the parallel calculation unit; a coefficient transfer unit that transfers the weighting coefficients stored in the coefficient memory to the parallel calculation unit; Equipped with the coefficient memory holds the weighting coefficients in the order of the output channels of the parallel arithmetic unit and in accordance with a word alignment of the coefficient memory; When the parallel arithmetic unit executes N arithmetic processes in parallel (N is a positive integer equal to or smaller than M), the coefficient transfer unit transfers at least N weighting coefficients to the parallel arithmetic unit in a predetermined transfer format. 23. An information processing apparatus comprising: (Item 2) the predetermined transfer format includes M fixed length fields; The coefficient transfer unit stores M weighting coefficients in M ​​fixed-length fields and transfers them to the parallel calculation unit. 2. The information processing device according to item 1, (Item 3) The parallel calculation unit further includes a separation unit that separates the M weighting coefficients included in the predetermined transfer format and transmits the separated weighting coefficients to the M product-sum calculation units. 3. The information processing device according to item 2. (Item 4) The coefficient memory holds a plurality of weighting coefficients including a first weighting coefficient of a bit width CA and a second weighting coefficient of a bit width CB; When the word length of the coefficient memory is WD bits, the coefficient memory holds either WD / CA of the first weighting coefficients or WD / CB of the second weighting coefficients for each word area. 4. The information processing device according to any one of items 1 to 3. (Item 5) The parallel computing unit is configured to be able to switch between a first mode in which processing is performed using the first weighting factor and a second mode in which processing is performed using the second weighting factor. 5. The information processing device according to item 4, (Item 6) Each of the M multiply-and-accumulate units divides processing data into L input channels (L is a positive integer) and processes the data in parallel; the coefficient memory holds weight coefficients in the order of output channels of the multiply-accumulate units corresponding to the input channels; The coefficient transfer unit transfers at least N×L weighting coefficients to the parallel calculation unit in the predetermined transfer format. 6. The information processing device according to any one of items 1 to 5. (Item 7) The coefficient memory holds a set of WD / (C×L) weight coefficients for each word area, where C is the bit width of the weight coefficients and WD is the word length of the coefficient memory. 7. The information processing device according to item 6, (Item 8) The parallel computing unit is configured to be able to switch the value of L. 8. The information processing device according to item 6 or 7. (Item 9) The parallel computing unit is a computing unit that configures a convolutional neural network (CNN). 9. The information processing device according to any one of items 1 to 8,

[0158] (Other Examples) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0159] The invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0160] 101 input unit; 102 data storage unit; 103 communication unit; 104 display unit; 105 image processing unit; 106 CPU; 107 ROM; 108 RAM; 109 convolution processing unit; 701 feature data storage unit; 702 weighting coefficient storage unit; 703 product-sum operation unit; 704 activation pooling processing unit

Claims

1. a parallel arithmetic unit having M (M is an integer of 2 or more) multiply-and-accumulate units operable in parallel; a coefficient memory for storing a plurality of weighting coefficients used in the parallel calculation unit; a coefficient transfer unit that transfers the weighting coefficients stored in the coefficient memory to the parallel calculation unit; Equipped with the coefficient memory holds the weighting coefficients in the order of the output channels of the parallel arithmetic unit and in accordance with a word alignment of the coefficient memory; When the parallel arithmetic unit executes N arithmetic processes in parallel (N is a positive integer equal to or smaller than M), the coefficient transfer unit transfers at least N weighting coefficients to the parallel arithmetic unit in a predetermined transfer format.

23. An information processing apparatus comprising:

2. the predetermined transfer format includes M fixed length fields; The coefficient transfer unit stores M weighting coefficients in M ​​fixed-length fields and transfers them to the parallel calculation unit.

2. The information processing apparatus according to claim 1,

3. The parallel calculation unit further includes a separation unit that separates the M weighting coefficients included in the predetermined transfer format and transmits the separated weighting coefficients to the M product-sum calculation units.

3. The information processing apparatus according to claim 2.

4. The coefficient memory holds a plurality of weighting coefficients including a first weighting coefficient of a bit width CA and a second weighting coefficient of a bit width CB; When the word length of the coefficient memory is WD bits, the coefficient memory holds either WD / CA of the first weighting coefficients or WD / CB of the second weighting coefficients for each word area.

2. The information processing apparatus according to claim 1,

5. The parallel computing unit is configured to be able to switch between a first mode in which processing is performed using the first weighting factor and a second mode in which processing is performed using the second weighting factor.

5. The information processing apparatus according to claim 4.

6. Each of the M product-sum operation units divides processing data into L input channels (L is a positive integer) and processes the data in parallel; the coefficient memory holds weight coefficients in the order of output channels of the multiply-accumulate operation units corresponding to the input channels; The coefficient transfer unit transfers at least N×L weighting coefficients to the parallel calculation unit in the predetermined transfer format.

2. The information processing apparatus according to claim 1,

7. The coefficient memory holds a set of WD / (C×L) weight coefficients for each word area, where C is the bit width of the weight coefficients and WD is the word length of the coefficient memory.

7. The information processing apparatus according to claim 6,

8. The parallel computing unit is configured to be able to switch the value of L.

7. The information processing apparatus according to claim 6,

9. The parallel computing unit is a computing unit that configures a convolutional neural network (CNN).

2. The information processing apparatus according to claim 1,

Citation Information

Patent Citations

  • Information processing device, information processing method, and program

    JP2021009491A

  • Information processing device and image recognition device

    JP6823495B2