Information processing device, information processing method

Abstract

This invention provides an information processing device and information processing method that more efficiently obtain the normalized result of a convolution operation. [Solution] The CNN processing unit of the information processing device, which functions as a convolutional neural network (CNN) accelerator, comprises a processing unit 203 that performs a convolution operation between feature data and coefficient data, a statistics calculation unit 206 that calculates statistics for the feature data, and a post-processing unit 207 that normalizes the result of the convolution operation based on the result of the convolution operation and the statistics.

Description

Technical Field

[0001] The present invention relates to convolution operation technology.

Background Art

[0002] Hierarchical operation methods represented by convolutional neural networks (hereinafter abbreviated as CNN) are widely used. For example, Non-Patent Document 1 discloses a computationally efficient baseline network structure for restoring image data (such as denoising and blur correction).

[0003] In the above-mentioned CNN, when learning convolution coefficients by deep learning technology, normalization of feature data by layer normalization is performed to accelerate and stabilize learning. In layer normalization, the mean and variance of feature data are used, and a technique such as that of Patent Document 1 is disclosed as a method for calculating the variance.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Non-Patent Documents

[0005]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] When performing inference using a model that repeatedly performs layer normalization and convolution operations, it was necessary to normalize the feature data that serves as input for layer normalization, use the normalized feature data as input for the convolution operation, and repeatedly obtain the convolution operation result. In this case, it was necessary to transfer the normalized feature data to a large-capacity storage device and then transfer the normalized feature data from the storage device to the convolution unit before performing the convolution operation, which consumed a lot of bandwidth. Alternatively, the normalized feature data was held in a dedicated memory and supplied to the convolution unit via this dedicated memory, which increased the capacity of the dedicated memory. This invention provides a technology for more efficiently obtaining the normalized result of the convolution operation result. [Means for solving the problem]

[0007] According to one aspect of the present invention, the invention comprises processing means for performing a convolution operation between feature data and coefficient data, and for calculating a statistical quantity of the feature data, and post-processing means for normalizing the result based on the result of the convolution operation and the statistical quantity. [Effects of the Invention]

[0008] According to the present invention, the normalization result of the convolution operation can be obtained more efficiently. [Brief explanation of the drawing]

[0009] [Figure 1] A block diagram showing an example of the hardware configuration of an information processing device. [Figure 2] A block diagram showing an example of the hardware configuration of the CNN processing unit 101. [Figure 3] Flowchart of the NAFNet model. [Figure 4] A block diagram showing an example configuration of the calculation unit 203 and the statistical calculation unit 206. [Figure 5] A diagram illustrating the transfer order of feature maps and feature data. [Figure 6] A block diagram showing an example configuration of the post-processing unit 207. [Figure 7] A block diagram showing an example of the hardware configuration of the CNN processing unit 101. [Figure 8] A diagram illustrating the processing of consecutive frame images. [Modes for carrying out the invention]

[0010] The embodiments will be described in detail below with reference to the attached drawings. Note that the following embodiments do not limit the invention to the claims. While the embodiments describe multiple features, not all of these features are essential to the invention, and the features may be combined in any way. Furthermore, in the attached drawings, the same or similar configurations are given the same reference numerals, and redundant descriptions are omitted.

[0011] [First Embodiment] In this embodiment, as an example of a hierarchical neural network, the NAFNet (Nonlinear Activation Free Network) model described in Non-Patent Literature 1 for repairing input image data is used. In this embodiment, a case in which inference is performed using the NAFNet model by an information processing device that functions as a CNN accelerator will be described. First, an example of the hardware configuration of the information processing device according to this embodiment will be described using the block diagram in Figure 1.

[0012] The CNN processing unit 101 performs inference using the NAFNet model. The image input unit 102 is either a still image capture unit that captures still images periodically or irregularly, or a video capture unit that captures moving images. When the image input unit 102 is a still image capture unit, it outputs the captured still image as the input image. When the image input unit 102 is a video capture unit, it outputs the image of each frame in the captured moving image as the input image.

[0013] The image input unit 102 includes, for example, an optical system such as a lens, a photoelectric conversion device such as a CCD (Charge-Coupled Devices) or CMOS (Complimentary Metal Oxide Semiconductor) sensor, a driver circuit for controlling the photoelectric conversion device, an A / D converter, and the like.

[0014] The CNN operation unit 101 performs inference on the input image output from the image input unit 102 using the NAFNet model, thereby executing image processing such as noise reduction and blur correction on the input image.

[0015] The CPU (Central Prosessing Unit) 103 executes various processes using the computer programs and data stored in the RAM (Random Access Memory) 105. Thereby, the CPU 103 controls the operation of the entire information processing device and executes or controls various processes described as the processes performed by the information processing device.

[0016] The ROM (Read Only Memory) 104 stores setting data of the information processing device, computer programs and data related to the startup of the information processing device, computer programs and data related to the basic operation of the information processing device, and the like. The ROM 104 also stores computer programs and data for causing the CPU 103 and the CNN operation unit 101 to execute or control various processes described as the processes performed by the information processing device.

[0017] The RAM 105 can be composed of, for example, a large-capacity DRAM (Dynamic Access Memory) or the like. The RAM 105 has an area for storing computer programs and data loaded from the ROM 104 and an area for storing the input image output from the image input unit 102. Furthermore, the RAM 105 has a work area used when the CPU 103 and the CNN operation unit 101 execute various processes. Thus, the RAM 105 can appropriately provide various areas.

[0018] The CNN arithmetic unit 101 executes the specified CNN arithmetic according to an instruction from the CPU 103, and stores intermediate feature data and an image as an output result generated in the CNN arithmetic in the RAM 105.

[0019] The user interface unit 106 is a user interface such as a keyboard or a mouse, and various instructions and information can be input to the information processing apparatus by a user's operation. Note that the user interface unit 106 may include a display device having a liquid crystal screen or a touch panel screen.

[0020] For example, the user selects a process to be executed by the CNN arithmetic unit 101 by operating the GUI (Graphical User Interface) displayed on the user interface unit 106 using the user interface unit 106.

[0021] The CNN arithmetic unit 101, the image input unit 102, the CPU 103, the ROM 104, the RAM 105, and the user interface unit 106 are all connected to the data bus 107. The data bus 107 is a data transfer path between devices.

[0022] Next, a hardware configuration example of the CNN arithmetic unit 101 will be described using the block diagram of FIG. 2. The I / F unit 201 functions as an interface for enabling data communication between the DMAC (Direct Memory Accsecc Controller) 202 and the control unit 208 and the CPU 103 via the data bus 107.

[0023] DMAC202 functions as a controller that manages data transfer between the CNN calculation unit 101 and RAM 105. The calculation unit 203 performs convolution operations by referring to the "coefficient data for each layer of the NAFNet model (coefficient data used in multiple layer calculations)" stored in buffer 204 and the "feature data (input image, or the calculation result of the previous layer for the layer to be processed)" stored in buffer 205.

[0024] Buffer 204 supplies coefficient data to the arithmetic unit 203 with low latency. Buffer 204 includes, for example, high-speed SRAM (Stereo RAM) or registers. Here, the coefficient data is a pre-combined set of coefficient data obtained by machine learning of convolution operations and coefficient data obtained by machine learning of layer normalization.

[0025] Buffer 205 stores the input image, the calculation results from the arithmetic unit 203, and the processing results from the post-processing unit 207, and, for example, has high-speed SRAM or registers, similar to buffer 204.

[0026] The statistics calculation unit 206 calculates statistics for the feature data held in the buffer 205. The post-processing unit 207 adjusts the output range of the calculation results from the arithmetic unit 203 by adding a bias value or multiplying it by a gamma value. The control unit 208 controls the operation of the entire CNN arithmetic unit 101. The control unit 208 is configured using a sequencer or a simple CPU to control the arithmetic unit 203.

[0027] In the calculation unit 203, assuming that the kernel (filter coefficient matrix) size for the convolution operation is columnSizexrowSize and the number of feature maps in the previous layer (the layer prior to the layer being processed) is C, one feature data is calculated by the convolution operation according to the following equation (1).

[0028]

number

[0029] input(x,y,c): 2D coordinates (x,y), reference pixel value at feature map position c. output(x,y,c): Calculation result at 2D coordinates (x,y) and feature map position c. weight(column,row,c): Coordinates (x+column, y+row), coefficient data at feature map position c (obtained by machine learning) bias(c): Addition value at feature map position c (obtained through machine learning) C: Number of feature maps in the previous layer columnSize: Horizontal size of the 2D convolution kernel. rowSize: Vertical size of the 2D convolution kernel Generally, in CNN computation, a feature map is calculated by repeatedly performing sum-of-accumulate operations while scanning multiple convolution kernels pixel by pixel in the input image according to equation (1) above. On the other hand, in the NAFNet model, the layer normalization in step S301 and the 1x1 convolution operation in step S302 are repeatedly performed as shown in Figure 3(a). In the layer normalization in step S301, the feature data is normalized by the operation shown in equation (2) below.

[0030]

number

[0031] input(x,y,c): 2D coordinates (x,y), reference pixel value at feature map position c. output(x,y,c): Calculation result at 2D coordinates (x,y) and feature map position c. μ(x,y): Mean value in 2D coordinates (x, y) σ(x,y) 2 : Variance in 2D coordinates (x, y) ε: A fixed value to prevent division by 0.

[0032] γ(c): Normalized variance at feature map position c (obtained by machine learning) β(c): Normalized mean value at feature map position c (obtained by machine learning) In the layer normalization step S301, the mean and variance of the population to be normalized (here, pixels with the same 2D coordinates and feature map positions 1 to C) are changed to a variance value γ and a mean value β determined by learning, thereby speeding up and stabilizing the learning process.

[0033] In contrast, in this embodiment, normalized feature data is obtained by performing processing according to the processing flow shown in Figure 3(b) instead of the processing flow shown in Figure 3(a). Note that the calculation of statistics in step S303 and the 1x1 convolution operation in step S304 can be performed on the same feature map.

[0034] An example configuration of the arithmetic unit 203 and the statistical calculation unit 206 will be explained using the block diagram in Figure 4. The arithmetic unit 203 has a multiplier and an adder, and in step S304 it performs the convolution operation of equation (1) above. The multiplier multiplies the feature data sequence supplied from buffer 205 and the coefficient data sequence supplied from buffer 204. As shown in Figure 5(a), spatially continuous feature data sequences 502 of the feature map 501 held in RAM 105 are transferred from RAM 105 to buffer 205 via DMAC 202, and supplied from buffer 205 to the arithmetic unit 203. Similarly, coefficient data sequences are transferred from RAM 105 to buffer 204 via DMAC 202, and the necessary coefficient data sequences are supplied from buffer 204 to the arithmetic unit 203. Figure 5(b) shows how the feature data sequence 502 is sequentially transferred to the arithmetic unit 203. If the convolution operation is a 1x1 filter coefficient operation, the coefficient data is supplied to the arithmetic unit 203 according to the position of the feature map (number c in the figure). The cumulative adder accumulates the multiplication results from the multiplier and outputs the result of the convolution operation after accumulating for each feature map.

[0035] The statistical calculation unit 206 has a squarer and an adder, and in step S303, it calculates two types of statistics related to the feature data. The first statistical quantity related to the feature data is the cumulative result of squaring the feature data. To obtain the cumulative result of squaring the feature data, the squarer calculates the squared value of the feature data, and the adder accumulates the squared values ​​for each feature map, thereby obtaining the cumulative result of the squared value at each coordinate (x, y) on the feature map. The second statistical quantity for the feature data is the cumulative result for the feature data. To obtain the cumulative result for the feature data, the adder accumulates the feature data at each coordinate (x, y) on the feature map for each feature map. These two types of statistics are used in the post-processing unit 207 to calculate the mean and variance of the feature data.

[0036] Next, an example configuration of the post-processing unit 207 will be explained using the block diagram in Figure 6. Figure 6 shows an example configuration for calculating the convolution operation result for one of the pixels processed in parallel in step S305. Other pixels are also processed in parallel, but since they can be processed with a similar configuration, the explanation will be omitted.

[0037] The cumulative result obtained by the statistical calculation unit 206 is divided by the number of feature maps to obtain the mean value of the feature data. Furthermore, the variance of the feature data can be obtained by dividing the cumulative result of the squared values ​​by the number of feature maps and subtracting the square of the mean value of the feature data. The convolution result obtained by the calculation unit 203 is obtained by subtracting the result obtained by multiplying the mean value of the feature data by the "merge weight, which will be described later." Additionally, a fixed value ε is added to the variance of the feature data, the square root is calculated, and the result is divided by this square root value, with the "merge bias, which will be described later," added. Through this process, a result equivalent to that obtained by sequentially performing the layer normalization in step S301 and the 1x1 convolution in step S302, as shown in Figure 3(a), can be obtained.

[0038] This result can be derived from equations (1) and (2) above. First, since the convolution operation shown in equation (1) is generalized in terms of filter size, to simplify the explanation, we rewrite it for a 1x1 filter size, resulting in the following equation (3).

[0039]

number

[0040] input(x,y,c): 2D coordinates (x,y), reference pixel value at feature map position c. output(x,y,c): Calculation result at 2D coordinates (x,y) and feature map position c. weight(c): Coefficient data at feature map position c (obtained by machine learning) bias(c): Addition value at feature map position c (obtained through machine learning) C: Number of feature maps in the previous layer Substituting the output of the layer normalization shown in equation (2) above into the input of the convolution operation shown in equation (3) above, we get the following equation

[0041]

number

[0042] And when this equation is rearranged, we get equation (4) as follows.

[0043]

number

[0044] Here, we define the following variables.

[0045]

number

[0046] Using equations (5), (6), and (7) above, equation (4) above can be expressed as equation (8) as shown below.

[0047]

number

[0048] Equation (5) above allows us to obtain weight' by pre-calculating the weight of the 1x1 convolution operation and the γ of layer normalization, which were originally obtained by machine learning, before performing inference. If we consider weight' as coefficient data, the first term of equation (8) becomes the 1x1 convolution operation. In addition, the merge weight, which is multiplied by the mean value of the feature data, and the merge bias, which is added to the whole, can also be pre-calculated by equations (6) and (7). The merge weight can be thought of as the sum of the weight' calculated by equation (5). The merge bias can be calculated by summing the product of the weight of the 1x1 convolution operation and the β of layer normalization, which were originally obtained by machine learning, and adding the bias value of the output channel. Furthermore, since the merge weight is a common value regardless of the feature map position, and the merge bias replaces the bias originally calculated by machine learning, the memory area for holding the coefficients does not increase.

[0049] Therefore, equation (8), which can obtain a result equivalent to the convolution operation after layer normalization, can be processed by the calculation unit 203, the statistical calculation unit 206, and the post-processing unit 207 described above.

[0050] Thus, in this embodiment, we demonstrated that while performing a convolution operation from feature data and coefficient data, statistical quantities of the same feature data are calculated in parallel, and the convolution operation result after normalization by layer normalization is obtained from the convolution operation result and the statistical quantities.

[0051] Traditionally, in processing flows where layer normalization is performed followed by convolution, the feature data after layer normalization had to be transferred to a large storage medium such as RAM. Furthermore, when performing the convolution operation, the normalized feature data had to be read again from the storage medium for processing, which increased the bandwidth and processing time between the storage medium and the processing unit. Alternatively, it is conceivable to store the feature data after layer normalization in a dedicated memory and use it in the processing unit, thus avoiding transfer to an external storage medium, but this increases the cost due to the provision of dedicated memory.

[0052] In contrast, in this embodiment, the part of the feature data reading process is combined into layer normalization and convolution operations, and the necessary operations for each can be performed in parallel with a single reading, thus enabling efficient processing.

[0053] Furthermore, the division by the square root of the variance used in equation (8) can be calculated quickly using methods such as fast inverse square root or one-dimensional interpolation using a lookup table.

[0054] In this embodiment, only the layer normalization and convolution operations of the NAFNet model to which this embodiment is applied are described, and the other layer processing operations are omitted from the description. When executing the entire NAFNet model, the calculation unit and post-processing unit described in this embodiment may be used to repeatedly perform the desired operations, or dedicated processing elements may be added to perform high-speed processing.

[0055] In this embodiment, an example of a convolution operation with a filter coefficient size of 1x1 has been described, but this embodiment can also be applied to convolution operations with filter coefficient sizes of 3x3 or 5x5. In that case, the amount of feature data transferred from buffer 205 to calculation unit 203 should be changed to match the filter coefficient size. In addition, the statistics calculation unit 206 can change the unit of normalization not only by layer normalization but also by changing the amount of feature data supplied from buffer 205 to match the unit of normalization. If the processing range of the feature data required by calculation unit 203 and the feature data required by statistics calculation unit 206 differs, this embodiment can be applied by transferring feature data that matches the wider range and processing only the necessary area in calculation unit 203 and statistics calculation unit 206.

[0056] [Second Embodiment] This embodiment describes an embodiment in which the data references for the convolution operation and layer normalization are different. In particular, it describes an example in which a convolution operation with a filter coefficient size of 1x1 and layer normalization on the entire feature data are efficiently performed when processing consecutive frame images such as video data. Configurations that are identical to the first embodiment will not be described.

[0057] An example of the hardware configuration of the CNN calculation unit 101 in this embodiment will be explained with reference to Figure 7. The difference from the hardware configuration example shown in Figure 2 is that a statistics buffer 209 exists between the statistics calculation unit 206 and the post-processing unit 207.

[0058] The statistics buffer 209 holds statistics data for each layer when processing consecutive frame images and supplies the statistics data to the post-processing unit 207.

[0059] In this embodiment, layer normalization is performed by normalizing the feature data using the following equation (9).

[0060] TIFF2026062467000009.tif20137…(9)

[0061] input(x,y,c): 2D coordinates (x,y), reference pixel value at feature map position c. output(x,y,c): Calculation result at 2D coordinates (x,y) and feature map position c. μ: Mean value of the entire feature data σ 2 : Variance of the entire feature data ε: Fixed value to prevent division by 0 γ(c): Normalized variance at feature map position c (obtained by machine learning) β(c): Normalized mean value at feature map position c (obtained by machine learning)

[0062] The difference from equation (2) is the mean μ and variance σ of the feature data. 2 The unit used to acquire the features differs, with equation (9) representing the entire feature data. Therefore, the 1x1 convolution operation shown in equation (3) and the reference range of the feature data are different. An example of how this works is explained below.

[0063] The convolution unit 203 performs a convolution operation on the feature data with feature map positions 1 to c supplied from the CNN feature buffer 205, similar to the first embodiment, and outputs the result to the post-processing unit 207. The statistics calculation unit 206 accumulates statistics for the feature data with feature map positions 1 to c supplied from the CNN feature buffer 205, and then continues accumulating statistics for the feature data for the next 2D coordinate. After the statistics have been calculated for the entire feature data, the statistics data is sent to the statistics buffer 209, where the statistics buffer 209 holds the statistics data. The statistics buffer 209 supplies the statistics data of the layer on which the convolution operation is being performed from the held statistics data to the post-processing unit 207. The post-processing unit 207 generates the 1x1 convolution result after layer normalization in the same manner as the first embodiment.

[0064] The processing of consecutive frame images will be explained using Figure 8. Figure 8 assumes a CNN processing that performs layer processing of 1x1 convolution and layer normalization (separation of statistical calculation and post-processing) for three layers, and this processing is performed on consecutive frame images. As mentioned above, the data reference ranges for the 1x1 convolution and statistical calculation are different in this embodiment, so it is not possible to calculate the 1x1 convolution result and the statistical calculation result simultaneously. On the other hand, the similarity of feature data to each frame image is high, and this tendency is even higher for feature data from frame images that are close in time. Also, since the statistical amount for the entire feature data is rounded down to the details of the feature data, using the statistical amount of a nearby frame does not affect the accuracy of the CNN calculation. Therefore, in processing consecutive frame images, post-processing of the 1x1 convolution result is performed by referring to the calculation results of the statistical amount calculated for past frames for each layer. As a result, in processing each frame image, the feature data is referenced only once, enabling efficient processing.

[0065] [Third Embodiment] In the first and second embodiments, cases were described in which computational processing such as convolution and layer normalization is performed using the CNN computation unit 101 as hardware. However, such computational processing may also be performed using a processor such as a CPU / GPU (Graphics Processing Unit) / DSP (Digital Signal Processing Unit). Alternatively, such computational processing may be performed by cooperative operation between hardware such as the CNN computation unit 101 and a processor.

[0066] Furthermore, in the first and second embodiments, the image input unit 102 was described as an imaging unit that captures still images or moving images. However, since the method for acquiring input images is not limited to imaging, the image input unit 102 does not have to be an imaging unit. For example, the image input unit 102 may be a device that reads input images from a storage device such as a hard disk drive, or a device that downloads input images from an external server device.

[0067] Furthermore, while the first and second embodiments described cases in which each functional unit shown in Figure 2 is implemented in hardware, some parts may be implemented in software (computer programs). For example, the arithmetic unit 203, the statistical calculation unit 206, and the post-processing unit 207 may be implemented in software. In this case, the control unit 208 and the CPU 103 execute this software to realize the functions of the corresponding functional units.

[0068] The numerical values, processing timing, processing order, processing entity, data (information) structure / acquisition method / destination / source / storage location, etc., used in the above embodiment are given as examples for the purpose of providing a concrete explanation, and are not intended to limit the scope to such examples.

[0069] Furthermore, some or all of the embodiments described above may be used in appropriate combinations. Alternatively, some or all of the embodiments described above may be used selectively.

[0070] (Other embodiments) The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0071] The inventions described herein include the following information processing devices, information processing methods, and computer programs. (Item 1) A processing means that performs a convolution operation between feature data and coefficient data, and calculates a statistical value of the feature data. Post-processing means for normalizing the result based on the result of the convolution operation and the statistical quantity. An information processing device characterized by comprising: (Item 2) The information processing apparatus according to item 1, characterized in that the processing means performs the convolution operation and the calculation of the statistical quantity in parallel with respect to the same feature data. (Item 3) The information processing apparatus according to item 1 or 2, characterized in that the post-processing means calculates the convolution result after layer normalization based on the result of the convolution operation and the statistical quantity. (Item 4) The information processing device according to any one of items 1 to 3, characterized in that the convolution operation is a convolution operation with filter coefficients of size 1 × 1. (Item 5) The information processing device according to any one of items 1 to 4, characterized in that the aforementioned statistics include the cumulative result of the feature data and the cumulative result of the square of the feature data. (Item 6) The information processing apparatus according to item 5, characterized in that the post-processing means calculates an average value from the cumulative result of the feature data, and calculates the variance value of the feature data from the cumulative result of the square of the feature data and the average value. (Item 7) The information processing device according to any one of items 1 to 6, characterized in that the coefficient data is coefficient data obtained by pre-combining coefficients obtained by machine learning of convolution operations and coefficients obtained by machine learning of layer normalization. (Item 8) The information processing apparatus according to item 3, characterized in that the post-processing means calculates the convolution operation result after layer normalization by using a value obtained by synthesizing the coefficients obtained by machine learning in advance. (Item 9) moreover, An information processing device according to any one of items 1 to 8, characterized by comprising a buffer for storing the aforementioned feature data and a buffer for storing the aforementioned coefficient data. (Item 10) The information processing apparatus according to item 1, characterized in that the post-processing means normalizes the convolution result based on statistical quantities obtained in past frame images and the convolution result obtained in the current frame image when processing consecutive frame images. (Item 11) An information processing method performed by an information processing device, The processing means of the information processing apparatus includes a processing step of performing a convolution operation between feature data and coefficient data, and calculating a statistical value of the feature data. The post-processing means of the information processing apparatus includes a post-processing step of normalizing the result based on the result of the convolution operation and the statistical quantity. An information processing method characterized by comprising: (Item 12) A computer program that causes a computer to function as one of the means of an information processing device described in any one of items 1 through 10.

[0072] The invention is not limited to the embodiments described above, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, claims are attached to disclose the scope of the invention. [Explanation of symbols]

[0073] 201: I / F section 202: DMAC 203: Calculation section 204: Buffer 205: Buffer 206: Statistical calculation section 207: Post-processing section 208: Control section 209: Statistical buffer

Claims

1. A processing means that performs a convolution operation between feature data and coefficient data, and calculates a statistical value of the feature data. Post-processing means for normalizing the result based on the result of the convolution operation and the statistical quantity. An information processing device characterized by comprising:

2. The information processing apparatus according to claim 1, characterized in that the processing means performs the convolution operation and the calculation of the statistical quantity in parallel with respect to the same feature data.

3. The information processing apparatus according to claim 1, characterized in that the post-processing means calculates the convolution result after layer normalization based on the result of the convolution operation and the statistical quantity.

4. The information processing apparatus according to claim 1, characterized in that the convolution operation is a convolution operation with filter coefficients of size 1 × 1.

5. The information processing apparatus according to claim 1, characterized in that the aforementioned statistics include the cumulative result of the feature data and the cumulative result of the square of the feature data.

6. The information processing apparatus according to claim 5, characterized in that the post-processing means calculates an average value from the cumulative result of the feature data, and calculates the variance value of the feature data from the cumulative result of the square of the feature data and the average value.

7. The information processing apparatus according to claim 1, characterized in that the coefficient data is coefficient data obtained by pre-combining coefficients obtained by machine learning of convolution operations and coefficients obtained by machine learning of layer normalization.

8. The information processing apparatus according to claim 3, characterized in that the post-processing means calculates the convolution operation result after layer normalization by using a value obtained by synthesizing the coefficients obtained by machine learning in advance.

9. moreover, The information processing apparatus according to claim 1, comprising a buffer for storing the feature data and a buffer for storing the coefficient data.

10. The information processing apparatus according to claim 1, wherein the post-processing means normalizes the convolution result based on statistical quantities obtained in past frame images and the convolution result obtained in the current frame image when processing consecutive frame images.

11. An information processing method performed by an information processing device, The processing means of the information processing apparatus includes a processing step of performing a convolution operation between feature data and coefficient data, and calculating a statistical value of the feature data. The post-processing means of the information processing apparatus includes a post-processing step of normalizing the result based on the result of the convolution operation and the statistical quantity. An information processing method characterized by comprising:

12. A computer program for causing a computer to function as one of the means of an information processing apparatus described in any one of claims 1 to 10.

Images