A neural network processor, a data denoising processing method and related products

Abstract

This invention provides a neural network processor, a data denoising method, and related products. The neural network processor includes a preprocessing module, a video denoising module, and a reverse post-processing module. The preprocessing module performs nonlinear processing on the original video stream data to obtain first video stream data. The video denoising module performs two-stage denoising processing to obtain second video stream data. The first-stage denoising processing is used to obtain a first output result for each frame of the first video stream data based on each frame image and the corresponding first output result of the previous frame image. The second-stage denoising processing is used to perform single-frame image denoising on the first output results to obtain a second output result for each frame image. The reverse post-processing module performs reverse post-processing on the second video stream data. The neural network processor, data denoising method, and related products provided by this invention reduce bandwidth and computing power consumption.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to a neural network processor, a data noise reduction processing method, and related products. Background Technology

[0002] With the application and popularization of artificial intelligence methods in various fields, AI Image Signal Processor (AI ISP) has emerged as a powerful tool to break through the bottlenecks of traditional ISP.

[0003] Among existing technologies, neural network-based video denoising methods offer the most significant image quality improvement in intelligent security systems. They leverage continuous multi-frame image information from a video stream, using data-driven autonomous learning to remove noise and preserve signal strength. Furthermore, neural network-based video denoising methods avoid motion blur and background blur issues common in traditional methods, effectively mitigating poor image quality at darker areas. However, the multi-frame image input and high-performance neural network architecture are limited by the bandwidth, buffering, and computing power resources of low-cost edge chips, making them difficult to deploy on the edge. Summary of the Invention

[0004] To address the problems in the prior art, embodiments of the present invention provide a neural network processor, a data noise reduction processing method, and related products, which can at least partially solve the problems existing in the prior art.

[0005] In a first aspect, the present invention proposes a neural network processor, comprising a preprocessing module, a video noise reduction processing module, and an inverse postprocessing module, wherein: The preprocessing module is used to perform non-linear processing on the original video stream data to obtain the first video stream data; The video noise reduction module is used to perform two-stage noise reduction processing on the first video stream data to obtain the second video stream data; wherein, the first-stage noise reduction processing is used to obtain the first output result of each frame of the first video stream data based on each frame of the first video stream data and the first output result of the corresponding previous frame of the first video stream data; the second-stage noise reduction processing is used to perform single-frame image noise reduction on the first output result of each frame of the first video stream data to obtain the second output result of each frame of the first video stream data, and the second output results of each frame of the second video stream data constitute the second video stream data. The reverse post-processing module is used to perform reverse post-processing on the second video stream data to obtain processed video stream data.

[0006] Furthermore, the video noise reduction module includes a first noise reduction unit, which is used to perform the first stage noise reduction processing on the first video stream data, including: Based on the first output result of the current frame image and the previous frame image of the current frame image, and the first denoising model, the motion estimation vector corresponding to the current frame image is obtained; wherein, the first denoising model is obtained in advance; The first output result of the current frame image is obtained by weighted fusing the first output result of the current frame image and the previous frame image based on the motion estimation vector corresponding to the current frame image.

[0007] Further, the first noise reduction model includes a first downsampling layer, a second downsampling layer, a first feature extraction layer, a second feature extraction layer, a third feature extraction layer, a first upsampling layer, a second upsampling layer, a feature connection layer, and a feature fusion layer. The output of the first downsampling layer is connected to the inputs of the second upsampling layer and the second feature extraction layer. The output of the second downsampling layer is connected to the input of the third feature extraction layer. The input of the feature connection layer is connected to the outputs of the first feature extraction layer, the second feature extraction layer, and the third feature extraction layer, respectively. The output of the feature connection layer is connected to the input of the feature fusion layer. Accordingly, the first noise reduction unit is specifically used for: The first output results of the current frame image and the previous frame image are input into the first downsampling layer, and the first downsampling result of the current frame image and the first downsampling result of the previous frame image are output; the first downsampling result of the current frame image and the first downsampling result of the previous frame image are input into the second downsampling layer, and the second downsampling result of the current frame image and the second downsampling result of the previous frame image are output. The first output results of the current frame image and the previous frame image are input into the first feature extraction layer to output the first extracted feature; the first downsampling result of the current frame image and the first downsampling result of the first output result of the previous frame image are input into the second feature extraction layer to output the second extracted feature; the second downsampling result of the current frame image and the second downsampling result of the first output result of the previous frame image are input into the third feature extraction layer to output the third extracted feature. The second extracted feature is input into the first upsampling layer, and the first upsampling feature is output; the third extracted feature is input into the second upsampling layer, and the second upsampling feature is output. The first extracted feature, the first upsampled feature, and the second upsampled feature are input into the feature connection layer, and cascaded features are output. The cascaded features are input into the feature fusion layer, and the motion estimation vector corresponding to the current frame image is output.

[0008] Furthermore, the video noise reduction module includes a second noise reduction unit, which is used to perform a second-stage noise reduction process on the result obtained from the first-stage noise reduction process, including: Based on the first output result of the current frame image and the motion mask corresponding to the first output result of the current frame image, the motion region denoising data of the first output result of the current frame image is obtained, and based on the first output result of the current frame image and the still mask corresponding to the first output result of the current frame image, the still region denoising data of the first output result of the current frame image is obtained; wherein, the motion mask and the still mask corresponding to the first output result of the current frame image are obtained in advance; Based on the motion region denoising data of the first output result of the current frame image and the second denoising model, the motion region output result corresponding to the first output result of the current frame image is obtained; and based on the static region denoising data of the first output result of the current frame image and the third denoising model, the static region output result corresponding to the first output result of the current frame image is obtained; wherein, the second denoising model and the third denoising model are obtained in advance; The motion region output result corresponding to the first output result of the current frame image and the stationary region output result corresponding to the first output result of the current frame image are fused to obtain the second output result corresponding to the current frame image.

[0009] Furthermore, the nonlinear processing includes logarithmic transformation.

[0010] Furthermore, the inverse post-processing includes an inverse logarithmic transformation.

[0011] In a second aspect, the present invention provides a data noise reduction processing method, applied to the neural network processor described in any of the above embodiments, comprising: The original video stream data is subjected to non-linear processing to obtain the first video stream data; The first video stream data is subjected to two-stage noise reduction processing to obtain the second video stream data; wherein, the first-stage noise reduction processing is used to obtain the first output result of each frame of the first video stream data based on each frame of the first video stream data and the first output result of the corresponding previous frame of the first video stream data; the second-stage noise reduction processing is used to perform single-frame image noise reduction on the first output result of each frame of the first video stream data to obtain the second output result of each frame of the first video stream data, and the second output results corresponding to each frame of the first video stream data constitute the second video stream data. The second video stream data is reverse-processed to obtain the processed video stream data.

[0012] Further, the step of obtaining the first output result of each frame of the first video stream data based on each frame of the first video stream data and the corresponding first output result of the previous frame includes: Based on the first output results of the current frame image and the previous frame image, and the first denoising model, the motion estimation vector corresponding to the current frame image is obtained; wherein, the first denoising model is obtained in advance; The first output result of the current frame image is obtained by weighted fusing the first output result of the current frame image and the previous frame image based on the motion estimation vector corresponding to the current frame image.

[0013] Further, the first noise reduction model includes a first downsampling layer, a second downsampling layer, a first feature extraction layer, a second feature extraction layer, a third feature extraction layer, a first upsampling layer, a second upsampling layer, a feature connection layer, and a feature fusion layer. The output of the first downsampling layer is connected to the inputs of the second upsampling layer and the second feature extraction layer. The output of the second downsampling layer is connected to the input of the third feature extraction layer. The input of the feature connection layer is connected to the outputs of the first feature extraction layer, the second feature extraction layer, and the third feature extraction layer, respectively. The output of the feature connection layer is connected to the input of the feature fusion layer. Correspondingly, obtaining the motion estimation vector corresponding to the current frame image based on the first output result of the current frame image and the previous frame image of the current frame image, and the first noise reduction model, includes: The first output results of the current frame image and the previous frame image are input into the first downsampling layer, and the first downsampling result of the current frame image and the first downsampling result of the previous frame image are output; the first downsampling result of the current frame image and the first downsampling result of the previous frame image are input into the second downsampling layer, and the second downsampling result of the current frame image and the second downsampling result of the previous frame image are output. The first output results of the current frame image and the previous frame image are input into the first feature extraction layer to output the first extracted feature; the first downsampling result of the current frame image and the first downsampling result of the first output result of the previous frame image are input into the second feature extraction layer to output the second extracted feature; the second downsampling result of the current frame image and the second downsampling result of the first output result of the previous frame image are input into the third feature extraction layer to output the third extracted feature. The second extracted feature is input into the first upsampling layer, and the first upsampling feature is output; the third extracted feature is input into the second upsampling layer, and the second upsampling feature is output. The first extracted feature, the first upsampled feature, and the second upsampled feature are input into the feature connection layer, and cascaded features are output. The cascaded features are input into the feature fusion layer, and the motion estimation vector corresponding to the current frame image is output.

[0014] Further, the step of performing single-frame image noise reduction on the first output result of each frame of the first video stream data to obtain the second output result of each frame includes: Based on the first output result of the current frame image and the motion mask corresponding to the first output result of the current frame image, the motion region denoising data of the first output result of the current frame image is obtained, and based on the first output result of the current frame image and the still mask corresponding to the first output result of the current frame image, the still region denoising data of the first output result of the current frame image is obtained; wherein, the motion mask and the still mask corresponding to the first output result of the current frame image are obtained in advance; Based on the motion region denoising data of the first output result of the current frame image and the second denoising model, the motion region output result corresponding to the first output result of the current frame image is obtained; and based on the static region denoising data of the first output result of the current frame image and the third denoising model, the static region output result corresponding to the first output result of the current frame image is obtained; wherein, the second denoising model and the third denoising model are obtained in advance; The motion region output result corresponding to the first output result of the current frame image and the stationary region output result corresponding to the first output result of the current frame image are fused to obtain the second output result corresponding to the current frame image.

[0015] Thirdly, the present invention provides an artificial intelligence image signal processor, including the neural network processor described in any of the above embodiments.

[0016] Fourthly, the present invention provides a chip including at least one artificial intelligence image signal processor as described in the above embodiments.

[0017] Fifthly, the present invention provides a board card including at least one chip as described in the above embodiments.

[0018] In a sixth aspect, the present invention provides an electronic device comprising at least one chip as described in the above embodiments or at least one board as described in the above embodiments.

[0019] The neural network processor, data denoising method, and related products provided in this invention include a preprocessing module, a video denoising module, and a reverse post-processing module. The preprocessing module performs nonlinear processing on the original video stream data to obtain first video stream data. The video denoising module performs two-stage denoising processing on the first video stream data to obtain second video stream data. The first-stage denoising processing is used to obtain a first output result for each frame of the first video stream data based on each frame image and the first output result of the corresponding previous frame image. The second-stage denoising processing is used to perform single-frame image denoising on the first output result of each frame image of the first video stream data to obtain a second output result for each frame image. The second output results of each frame image constitute the second video stream data. The reverse post-processing module performs reverse post-processing on the second video stream data to obtain processed video stream data. During the denoising process, the amount of data requiring buffering is reduced, thus reducing data throughput time and bandwidth and computing power consumption. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings: Figure 1 This is a schematic diagram of the structure of a neural network processor provided in an embodiment of the present invention.

[0021] Figure 2 This is a schematic diagram of the structure of the first noise reduction model provided in an embodiment of the present invention.

[0022] Figure 3 This is a schematic flowchart of a data noise reduction processing method provided in an embodiment of the present invention.

[0023] Figure 4 This is a schematic flowchart of a data noise reduction processing method provided in another embodiment of the present invention.

[0024] Figure 5 This is a schematic flowchart of a data noise reduction processing method provided in another embodiment of the present invention.

[0025] Figure 6 This is a schematic flowchart of a data noise reduction processing method provided in another embodiment of the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other. The acquisition, storage, use, and processing of data in the technical solutions of this application all comply with the relevant provisions of laws and regulations. The user information in the embodiments of this application is obtained through legal and compliant means, and the acquisition, storage, use, and processing of user information have been agreed upon by the customer.

[0027] To facilitate understanding of the technical solution provided in this application, the relevant content of the technical solution in this application will be explained below.

[0028] For existing technical solutions, when adapting to edge chips with limited resources, there are problems including but not limited to (1) the single-frame computing power requirement is too high, making it impossible to deploy on edge chips to meet the implementation frame rate; (2) there is a lot of buffered intermediate data, resulting in high bandwidth load and high time consumption of interactive data; (3) traditional ISPs are difficult to accurately estimate motion under extremely low illumination (light intensity), which easily produces ghosting and penetration. In response to the bandwidth and computing power consumption of edge deployment, this application proposes a neural network processor that can be deployed on edge chips. In the first stage of noise reduction processing, less data needs to be buffered, avoiding the high bandwidth consumption of open-source lightweight fast deep video denoising network (Fast Deep Video Denoising Network, abbreviated as FastDVDnet), which is more conducive to meeting the real-time operation of 4M resolution. The historical frame image information is applied in a loop to avoid the bandwidth consumption of multi-frame image noise reduction data, which is conducive to edge deployment.

[0029] Figure 1 This is a schematic diagram of the structure of a neural network processor provided in an embodiment of the present invention, as shown below. Figure 1 As shown, the neural network processor provided in this embodiment of the invention includes a preprocessing module 1, a video noise reduction processing module 2, and a reverse post-processing module 3, wherein: Preprocessing module 1 is used to perform non-linear processing on the raw video stream data to obtain the first video stream data; The video noise reduction module 2 is used to perform two-stage noise reduction processing on the first video stream data to obtain the second video stream data; wherein, the first-stage noise reduction processing is used to obtain the first output result of each frame of the first video stream data based on each frame of the first video stream data and the first output result of the corresponding previous frame of the first video stream data; the second-stage noise reduction processing is used to perform single-frame image noise reduction on the first output result of each frame of the first video stream data to obtain the second output result of each frame of the first video stream data, and the second output results of each frame of the second video stream data constitute the second video stream data. The reverse post-processing module 3 is used to perform reverse post-processing on the second video stream data to obtain the processed video stream data.

[0030] Specifically, the raw video stream data output by the image sensor is provided to the preprocessing module 1. The preprocessing module 1 performs nonlinear processing on the received raw video stream data to obtain the first video stream data. The nonlinear processing includes, but is not limited to, logarithmic transformation, and is selected as needed; this embodiment of the invention does not impose limitations. Nonlinear processing makes the noise variance of the processed raw video stream data approximately constant, resulting in a more uniform noise distribution; it also compresses the dynamic range of bright areas while simultaneously increasing the dynamic range of dark areas.

[0031] The video denoising module 2 performs two-stage denoising processing on the first video stream data to obtain the second video stream data. The two-stage denoising process includes a first-stage denoising process and a second-stage denoising process. The first-stage denoising process requires less data buffering, resulting in less bandwidth consumption compared to the high buffering requirements of FastDVDnet, making it more suitable for deploying neural network processors on edge chips. The second-stage denoising process is used to perform single-frame image denoising on the first output result obtained from the first-stage denoising process.

[0032] The first stage of noise reduction processing uses each frame of the first video stream data and the first output result of the corresponding previous frame to obtain the first output result of each frame of the first video stream data. The second stage of noise reduction processing performs single-frame image noise reduction on the first output result of each frame of the first video stream data to obtain the second output result of each frame.

[0033] The reverse post-processing module 3 performs reverse post-processing on the second video stream data to obtain the processed video stream data. Reverse post-processing is the inverse process of nonlinear processing. For example, if nonlinear processing uses logarithmic transformation, the corresponding reverse post-processing is inverse logarithmic transformation. The processed video stream data is the original video stream data after denoising.

[0034] The neural network processor provided in this embodiment of the invention includes a preprocessing module, a video denoising module, and a reverse post-processing module. The preprocessing module performs nonlinear processing on the original video stream data to obtain first video stream data. The video denoising module performs two-stage denoising processing on the first video stream data to obtain second video stream data. The first-stage denoising processing is used to obtain a first output result for each frame of the first video stream data based on each frame image and the first output result of the corresponding previous frame image. The second-stage denoising processing is used to perform single-frame image denoising on the first output result of each frame image of the first video stream data to obtain a second output result for each frame image. The second output results of each frame image constitute the second video stream data. The reverse post-processing module performs reverse post-processing on the second video stream data to obtain processed video stream data. During the denoising process, the amount of data requiring buffering is reduced, thus reducing data throughput time and bandwidth and computing power consumption.

[0035] Based on the above embodiments, the video noise reduction module further includes a first noise reduction unit, which is used to perform the first stage noise reduction processing on the first video stream data, including: Based on the first output result of the current frame image and the previous frame image of the current frame image, and the first denoising model, the motion estimation vector corresponding to the current frame image is obtained; wherein, the first denoising model is obtained in advance; The first output result of the current frame image is obtained by weighted fusing the first output result of the current frame image and the previous frame image based on the motion estimation vector corresponding to the current frame image.

[0036] Specifically, the first denoising unit processes the first video stream data sequentially. The first denoising unit acquires a single frame as the current frame and obtains the first output result of the previous frame. The first output result based on the current frame and the previous frame is input into the first denoising model, which outputs the motion estimation vector corresponding to the current frame. The first denoising model is pre-trained. It is understandable that for the first frame in the first video stream data, since there is no previous frame, the first output result of the previous frame can be set to empty.

[0037] The first noise reduction unit normalizes the motion estimation vector corresponding to the current frame image and uses it as the weight of the current frame image. It calculates 1 minus the motion estimation vector corresponding to the current frame image as the weight of the first output result of the previous frame image. Then, it calculates the product of the weight of the current frame image and the current frame image, and the product of the weight of the first output result of the previous frame image and the current frame image. Finally, it calculates the sum of these two products as the first output result of the current frame image. By weighting and fusing the first output results of the current frame image and the previous frame image using the motion estimation vector, it can avoid ghosting, motion blur, and blurring phenomena that are common in traditional video noise reduction methods.

[0038] In the embodiments of this invention, the only data that needs to be buffered is the first output result of each frame image and the motion estimation vector corresponding to each frame image. Compared with other open-source lightweight FastDVDnet, it reduces the internal buffering limitations of the edge neural network processor and the DDR buffering limitations, and also reduces the data throughput time, reserving more time for computation, which is more conducive to low cost and edge device deployment.

[0039] In some implementations, historical first video stream data and its first output can be collected as training data. The motion estimation vectors corresponding to each frame in the historical first video stream data can be used as labels to train a preset model, thus obtaining a first denoising model. The preset model and the first denoising model have the same model structure.

[0040] In some implementations, the normalized result of the motion estimation vector corresponding to the current frame image is 'a', and the current frame image is represented as X. i The first output result of the previous frame of the current frame is represented as Y. i-1 If the weight of the first output result of the previous frame of the current frame is 1-a, then the weight of the first output result Y of the current frame is... i =aX i +(1-a)Y i-1 .

[0041] In some implementations, the raw video stream data undergoes non-linear processing by preprocessing module 1 to obtain the first video stream data. The first video stream data is represented as X1, X2, X3...X n-1 X n , where X i This serves as the current frame image. Accordingly, the first output result can be represented as: Y1, Y2, Y3.........Y n-1 Y n , where Y i As the current frame image X iThe first output result.

[0042] The first noise reduction unit will use the current frame image X i and the current frame image X i The previous frame image X i-1 The first output result Y i-1 The input is fed into the first denoising model, and the output is the current frame image X. i The corresponding motion estimation vector Alpaha_Mvd. Current frame image X i The corresponding motion estimation vector Alpaha_Mvd, after normalization, is used as the current frame image X. i The corresponding weight 'a'. The first noise reduction unit calculates the first output result Y of the current frame image. i =aX i +(1-a)Y i-1 .

[0043] The first noise reduction unit performs the first stage of noise reduction processing throughout the entire time sequence of the first video stream data. Since the first output result of the previous frame is used when processing each frame, it can converge to a stable state using enough preceding information. This recursive convergence, for video stream signals submerged in noise under extremely low illumination, removes a large amount of noise while minimizing the loss of detail, motion blur, and penetration in moving areas, thus greatly improving the quality of black-light imaging.

[0044] Based on the above embodiments, further, such as Figure 2 As shown, the first noise reduction model includes a first downsampling layer, a second downsampling layer, a first feature extraction layer, a second feature extraction layer, a third feature extraction layer, a first upsampling layer, a second upsampling layer, a feature connection layer, and a feature fusion layer. The output of the first downsampling layer is connected to the input of the second upsampling layer and the input of the second feature extraction layer. The output of the second downsampling layer is connected to the input of the third feature extraction layer. The input of the feature connection layer is connected to the outputs of the first feature extraction layer, the second feature extraction layer, and the third feature extraction layer, respectively. The output of the feature connection layer is connected to the input of the feature fusion layer.

[0045] Specifically, the first denoising model employs a multi-scale network structure, performing differential calculations at different resolution levels and feeding them into the corresponding resolution feature extraction layers for feature abstraction. The abstracted features are then upsampled and fused to generate a motion estimation vector. The first downsampling layer performs a 2x nearest-neighbor downsampling scaling on the input data, and the second downsampling layer performs the same. The first, second, and third feature extraction layers extract features from the input data, and can employ a three-layer convolutional neural network (CNN). The first upsampling layer corresponds to the first downsampling layer and aligns the input data to its original resolution, i.e., the resolution of the input data for the first denoising model. The second upsampling layer corresponds to the second downsampling layer and aligns the input data to its original resolution, i.e., the resolution of the input data for the first denoising model. The feature connection layer performs step-by-step feature concatenation on the various input feature data. The feature fusion layer fuses the input data features and outputs a motion estimation vector; the feature fusion layer can employ a 3×3 convolutional kernel.

[0046] Based on the above embodiments, the first noise reduction unit is further specifically used for: The first output results of the current frame image and the previous frame image are input into the first downsampling layer, and the first downsampling result of the current frame image and the first downsampling result of the previous frame image are output; the first downsampling result of the current frame image and the first downsampling result of the previous frame image are input into the second downsampling layer, and the second downsampling result of the current frame image and the second downsampling result of the previous frame image are output. The first output results of the current frame image and the previous frame image are input into the first feature extraction layer to output the first extracted feature; the first downsampling result of the current frame image and the first downsampling result of the first output result of the previous frame image are input into the second feature extraction layer to output the second extracted feature; the second downsampling result of the current frame image and the second downsampling result of the first output result of the previous frame image are input into the third feature extraction layer to output the third extracted feature. The second extracted feature is input into the first upsampling layer, and the first upsampling feature is output; the third extracted feature is input into the second upsampling layer, and the second upsampling feature is output. The first extracted feature, the first upsampled feature, and the second upsampled feature are input into the feature connection layer, and cascaded features are output. The cascaded features are input into the feature fusion layer, and the motion estimation vector corresponding to the current frame image is output.

[0047] Specifically, the first output results of the current frame image and the previous frame image are input into the first downsampling layer. The first downsampling layer downsamples the current frame image to obtain the first downsampled result of the current frame image. The first downsampling layer downsamples the first output result of the previous frame image to obtain the first downsampled result of the first output result of the previous frame image.

[0048] The first downsampling result of the current frame image and the first downsampling result of the first output result of the previous frame image are input into the second downsampling layer. The second downsampling layer downsamples the first downsampling result of the current frame image to obtain the second downsampling result of the current frame image. The second downsampling layer downsamples the first downsampling result of the first output result of the previous frame image to obtain the second downsampling result of the first output result of the previous frame image.

[0049] The current frame image, the first downsampled result of the current frame image, and the second downsampled result of the first output result of the previous frame image are images of different scales and have different resolutions; the first output result of the previous frame image, the first downsampled result of the first output result of the previous frame image, and the second downsampled result of the first output result of the previous frame image are images of different scales and have different resolutions.

[0050] The first output of the current frame image and the previous frame image are concatenated along the channel dimension and input into the first feature extraction layer for feature extraction, outputting the first extracted feature. The first downsampled result of the current frame image and the first downsampled result of the first output of the previous frame image are concatenated along the channel dimension and input into the second feature extraction layer for feature extraction, outputting the second extracted feature. The second downsampled result of the current frame image and the second downsampled result of the first output of the previous frame image are concatenated along the channel dimension and input into the third feature extraction layer for feature extraction, outputting the third extracted feature.

[0051] The second extracted feature is input into the first upsampling layer, which upsamples the second extracted feature and outputs a first upsampled feature with the same resolution as the first extracted feature. The third extracted feature is input into the second upsampling layer, which upsamples the third extracted feature and outputs a second upsampled feature with the same resolution as the first extracted feature.

[0052] The first extracted feature, the first upsampled feature, and the second upsampled feature are input into the feature connection layer, and the feature connection layer concatenates the first extracted feature, the first upsampled feature, and the second upsampled feature to output a concatenated feature.

[0053] The cascaded features are input into the feature fusion layer for feature fusion, and the motion estimation vector corresponding to the current frame image is output.

[0054] This application utilizes multi-scale images to obtain motion estimation, and then aligns it to the original resolution design to learn more accurate motion vector information. The motion region values ​​are large enough that the subsequent weighted fusion will not produce ghosting, trailing, and penetration phenomena that are common in traditional video denoising results.

[0055] Based on the above embodiments, the video noise reduction module further includes a second noise reduction unit, which is used to perform a second stage noise reduction process on the result obtained from the first stage noise reduction process, including: Based on the first output result of the current frame image and the motion mask corresponding to the first output result of the current frame image, the motion region denoising data of the first output result of the current frame image is obtained, and based on the first output result of the current frame image and the still mask corresponding to the first output result of the current frame image, the still region denoising data of the first output result of the current frame image is obtained; wherein, the motion mask and the still mask corresponding to the first output result of the current frame image are obtained in advance; Based on the motion region denoising data of the first output result of the current frame image and the second denoising model, the motion region output result corresponding to the first output result of the current frame image is obtained; and based on the static region denoising data of the first output result of the current frame image and the third denoising model, the static region output result corresponding to the first output result of the current frame image is obtained; wherein, the second denoising model and the third denoising model are obtained in advance; The motion region output result corresponding to the first output result of the current frame image and the stationary region output result corresponding to the first output result of the current frame image are fused to obtain the second output result corresponding to the current frame image.

[0056] Specifically, the second denoising unit concatenates the first output of the current frame image with its corresponding motion mask along the channel dimension to obtain the motion region denoising data of the first output of the current frame image. It also concatenates the first output of the current frame image with its corresponding still mask along the channel dimension to obtain the still region denoising data of the first output of the current frame image. The motion mask and still mask corresponding to the first output of the current frame image are obtained in advance.

[0057] The motion region denoising data of the first output result of the current frame image is input into the second denoising model, which outputs the motion region output result corresponding to the first output result of the current frame image. The stationary region denoising data of the first output result of the current frame image is input into the third denoising model, which outputs the stationary region output result corresponding to the first output result of the current frame image. The neural network used in the second denoising model has a stronger denoising capability than the neural network used in the third denoising model. By distinguishing between motion regions and stationary regions and using different denoising models, the computationally limited denoising model can have targeted denoising intensity, that is, stronger denoising for motion regions and weaker denoising for stationary regions, thereby better preserving image details.

[0058] The motion region output result corresponding to the first output result of the current frame image and the stationary region output result corresponding to the first output result of the current frame image are fused together, and the motion region output result corresponding to the first output result of the current frame image and the stationary region output result corresponding to the first output result of the current frame image are merged to obtain the second output result corresponding to the current frame image.

[0059] In some implementations, a first preset number of first output results can be obtained as training data, and each of these first output results can be labeled with a mask label, including motion masks and still masks. The deep learning network model is trained using the first preset number of first output results and their corresponding mask labels to obtain a mask output model. By inputting the first output result of the current frame image into the mask output model, the motion mask and still mask corresponding to the first output result of the current frame image can be obtained.

[0060] In some embodiments, a second preset number of motion region denoising data of the first output results are obtained as training data, and the second preset number of motion region denoising data of the first output results are labeled. Based on the second preset number of motion region denoising data of the first output results and the corresponding labels, a deep learning network model is trained to obtain a second denoising model. Similarly, a third preset number of static region denoising data of the first output results are obtained as training data, and the third preset number of static region denoising data of the first output results are labeled. Based on the third preset number of static region denoising data of the first output results and the corresponding labels, a deep learning network model is trained to obtain a third denoising model.

[0061] In some embodiments, fusing the motion region output result corresponding to the first output result of the current frame image and the stationary region output result corresponding to the first output result of the current frame image includes: dividing the first output result of the current frame image into motion regions and stationary regions according to the motion mask; obtaining the pixel value at the corresponding position from the motion region output result corresponding to the first output result of the current frame image for the motion region; and obtaining the pixel value at the corresponding position from the stationary region output result corresponding to the first output result of the current frame image for the stationary region, thereby merging the motion region output result corresponding to the first output result of the current frame image and the stationary region output result corresponding to the first output result of the current frame image.

[0062] Building upon the above embodiments, the nonlinear processing further includes logarithmic transformation. Logarithmic transformation is a preprocessing operation that nonlinearly maps the intensity values ​​of input image pixels using the natural logarithm or a common logarithmic function to obtain processed pixel intensity values. In this application, nonlinear processing makes the noise variance of the processed original video stream data approximately constant, resulting in a more uniform noise distribution; it also compresses the dynamic range of bright areas while simultaneously increasing the dynamic range of dark areas.

[0063] Building upon the above embodiments, the inverse post-processing further includes an inverse logarithmic transform. The inverse logarithmic transform is the inverse operation of the logarithmic transform, capable of mapping the second video stream data, after two stages of noise reduction, back to the linear intensity domain.

[0064] Figure 3 This is a schematic flowchart of a data noise reduction processing method provided in an embodiment of the present invention, as shown below. Figure 3 As shown, the data noise reduction processing method provided by this embodiment of the invention, applied to the neural network processor described in any of the above embodiments, includes: S301. Perform non-linear processing on the original video stream data to obtain the first video stream data; Specifically, raw video stream data can be received from an image sensor, and then nonlinear processing can be performed on the raw video stream data to obtain the first video stream data. The nonlinear processing method can be selected according to actual needs, and this embodiment of the invention does not impose any limitations.

[0065] S302. Perform two-stage noise reduction processing on the first video stream data to obtain second video stream data; wherein, the first-stage noise reduction processing is used to obtain the first output result of each frame of the first video stream data based on each frame of the first video stream data and the first output result of the corresponding previous frame of the first video stream data; the second-stage noise reduction processing is used to perform single-frame image noise reduction on the first output result of each frame of the first video stream data to obtain the second output result of each frame of the first video stream data, and the second output results corresponding to each frame of the first video stream data constitute the second video stream data. Specifically, based on each frame of the first video stream data and its corresponding previous frame, a first-stage noise reduction process is performed on each frame to obtain a first output result for each frame of the first video stream data. Single-frame noise reduction is then performed on the first output result for each frame of the first video stream data to obtain a second output result for each frame, thus achieving a second-stage noise reduction process. Finally, a two-stage noise reduction process is performed on the first video stream data to obtain the second video stream data.

[0066] S303. Perform reverse post-processing on the second video stream data to obtain the processed video stream data.

[0067] Specifically, the second video stream data undergoes reverse post-processing to obtain processed video stream data. Reverse post-processing is the inverse process of non-linear processing, and the resulting video stream data is the original video stream data after denoising.

[0068] The data denoising processing method provided in this embodiment of the invention performs nonlinear processing on the original video stream data to obtain first video stream data; performs two-stage denoising processing on the first video stream data to obtain second video stream data; wherein, the first-stage denoising processing is used to obtain a first output result of each frame of the first video stream data based on each frame image of the first video stream data and the first output result of the corresponding previous frame image; the second-stage denoising processing is used to perform single-frame image denoising on the first output result of each frame image of the first video stream data to obtain a second output result of each frame image, and the second output results corresponding to each frame image constitute the second video stream data; the second video stream data is then subjected to inverse post-processing to obtain processed video stream data. In the denoising processing process, the amount of data that needs to be buffered is reduced, the data throughput time is reduced, and the denoising processing efficiency of the original video stream data is improved.

[0069] Figure 4This is a flowchart illustrating a data noise reduction processing method provided in another embodiment of the present invention, as shown below. Figure 4 As shown, based on the above embodiments, further, obtaining the first output result of each frame of the first video stream data based on each frame of the first video stream data and the corresponding first output result of the previous frame includes: S401. Based on the first output result of the current frame image and the previous frame image of the current frame image, and the first denoising model, obtain the motion estimation vector corresponding to the current frame image; wherein, the first denoising model is obtained in advance; Specifically, the first output results of the current frame image and the previous frame image are input into the first denoising model. After processing by the first denoising model, the motion estimation vector corresponding to the current frame image is output. Here, the first denoising model is obtained in advance; the current frame image is a frame image being processed from the first video stream data.

[0070] S402. Based on the motion estimation vector corresponding to the current frame image, the first output result of the current frame image and the previous frame image of the current frame image are weighted and fused to obtain the first output result of the current frame image.

[0071] Specifically, the motion estimation vector corresponding to the current frame image is normalized and used as the weight of the current frame image. The result of subtracting the motion estimation vector corresponding to the current frame image from 1 is used as the weight of the first output result of the previous frame image of the current frame image. Then, the product of the weight of the current frame image and the current frame image is calculated, and the product of the weight of the first output result of the previous frame image and the current frame image is calculated. Finally, the sum of the above two product results is calculated as the first output result of the current frame image.

[0072] By weighting and fusing the first output results of the current frame image and the previous frame image of the current frame image with motion estimation vectors, ghosting, trailing and penetration phenomena that are prone to occur in video streams of traditional video denoising results can be avoided.

[0073] Based on the above embodiments, the first noise reduction model further includes a first downsampling layer, a second downsampling layer, a first feature extraction layer, a second feature extraction layer, a third feature extraction layer, a first upsampling layer, a second upsampling layer, a feature connection layer, and a feature fusion layer. The output of the first downsampling layer is connected to the input of the second upsampling layer and the input of the second feature extraction layer. The output of the second downsampling layer is connected to the input of the third feature extraction layer. The input of the feature connection layer is connected to the outputs of the first feature extraction layer, the second feature extraction layer, and the third feature extraction layer, respectively. The output of the feature connection layer is connected to the input of the feature fusion layer.

[0074] Figure 5 This is a flowchart illustrating a data noise reduction processing method provided in another embodiment of the present invention, as shown below. Figure 5 As shown, obtaining the motion estimation vector corresponding to the current frame image based on the first output result of the current frame image and the previous frame image, as well as the first noise reduction model, includes: S501, Input the first output result of the current frame image and the previous frame image of the current frame image into the first downsampling layer, and output the first downsampling result of the current frame image and the first downsampling result of the first output result of the previous frame image of the current frame image; Input the first downsampling result of the current frame image and the first downsampling result of the first output result of the previous frame image of the current frame image into the second downsampling layer, and output the second downsampling result of the current frame image and the second downsampling result of the first output result of the previous frame image of the current frame image; S502, Input the first output results of the current frame image and the previous frame image of the current frame image into the first feature extraction layer, and output the first extracted feature; Input the first downsampling result of the current frame image and the first downsampling result of the first output result of the previous frame image of the current frame image into the second feature extraction layer, and output the second extracted feature; Input the second downsampling result of the current frame image and the second downsampling result of the first output result of the previous frame image of the current frame image into the third feature extraction layer, and output the third extracted feature; S503. Input the second extracted feature into the first upsampling layer and output the first upsampling feature; input the third extracted feature into the second upsampling layer and output the second upsampling feature; S504. Input the first extracted feature, the first upsampled feature, and the second upsampled feature into the feature connection layer, and output the cascaded feature; S505. Input the cascaded features into the feature fusion layer and output the motion estimation vector corresponding to the current frame image.

[0075] In this embodiment of the invention, the specific implementation process of obtaining the motion estimation vector corresponding to the current frame image based on the first output result of the current frame image and the previous frame image of the current frame image and the first denoising model can be referred to the detailed description of the embodiment of the first denoising unit above, and will not be repeated here.

[0076] Figure 6 This is a flowchart illustrating a data noise reduction processing method provided in another embodiment of the present invention, as shown below. Figure 6 As shown, based on the above embodiments, further, the step of performing single-frame image noise reduction on the first output result of each frame of the first video stream data to obtain the second output result of each frame includes: S601. Based on the first output result of the current frame image and the motion mask corresponding to the first output result of the current frame image, obtain the motion region denoising data of the first output result of the current frame image, and based on the first output result of the current frame image and the still mask corresponding to the first output result of the current frame image, obtain the still region denoising data of the first output result of the current frame image; wherein, the motion mask and the still mask corresponding to the first output result of the current frame image are obtained in advance. S602. Based on the motion region denoising data of the first output result of the current frame image and the second denoising model, obtain the motion region output result corresponding to the first output result of the current frame image; and based on the static region denoising data of the first output result of the current frame image and the third denoising model, obtain the static region output result corresponding to the first output result of the current frame image; wherein, the second denoising model and the third denoising model are obtained in advance; S603. The motion region output result corresponding to the first output result of the current frame image and the stationary region output result corresponding to the first output result of the current frame image are fused to obtain the second output result corresponding to the current frame image.

[0077] In this embodiment of the invention, the specific implementation process of performing single-frame image denoising on the first output result of each frame of the first video stream data to obtain the second output result of each frame can be referred to the detailed description of the embodiment of the second denoising unit described above, and will not be repeated here.

[0078] This invention provides an Artificial Intelligence Image Signal Processor (AI-ISP), which includes the neural network processor described in any of the above embodiments.

[0079] This invention provides a chip that includes at least one artificial intelligence image signal processor as described in the above embodiments.

[0080] This invention provides a board card that includes at least one chip as described in the above embodiments.

[0081] This invention provides an electronic device, including at least one chip or at least one board as described in the above embodiments.

[0082] The electronic devices mentioned include, but are not limited to, mobile devices, user terminals, personal digital assistants (PDAs), handheld devices, in-vehicle devices, drones, monitoring equipment, cameras, and other devices with AI application requirements.

[0083] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0084] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0085] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0086] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0087] In the description of this specification, the references to terms such as "an embodiment," "a specific embodiment," "some embodiments," "for example," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0088] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A neural network processor, characterized in that, It includes a preprocessing module, a video noise reduction module, and a reverse post-processing module, wherein: The preprocessing module is used to perform non-linear processing on the original video stream data to obtain the first video stream data; The video noise reduction module is used to perform two-stage noise reduction processing on the first video stream data to obtain the second video stream data; wherein, the first-stage noise reduction processing is used to obtain the first output result of each frame of the first video stream data based on each frame of the first video stream data and the first output result of the corresponding previous frame of the first video stream data; the second-stage noise reduction processing is used to perform single-frame image noise reduction on the first output result of each frame of the first video stream data to obtain the second output result of each frame of the first video stream data, and the second output results of each frame of the second video stream data constitute the second video stream data. The reverse post-processing module is used to perform reverse post-processing on the second video stream data to obtain processed video stream data.

2. The neural network processor according to claim 1, characterized in that, The video noise reduction module includes a first noise reduction unit, which is used to perform the first stage of noise reduction processing on the first video stream data, including: Based on the first output result of the current frame image and the previous frame image of the current frame image, and the first denoising model, the motion estimation vector corresponding to the current frame image is obtained; wherein, the first denoising model is obtained in advance; The first output result of the current frame image is obtained by weighted fusing the first output result of the current frame image and the previous frame image based on the motion estimation vector corresponding to the current frame image.

3. The neural network processor according to claim 2, characterized in that, The first noise reduction model includes a first downsampling layer, a second downsampling layer, a first feature extraction layer, a second feature extraction layer, a third feature extraction layer, a first upsampling layer, a second upsampling layer, a feature connection layer, and a feature fusion layer. The output of the first downsampling layer is connected to the inputs of the second upsampling layer and the second feature extraction layer. The output of the second downsampling layer is connected to the input of the third feature extraction layer. The input of the feature connection layer is connected to the outputs of the first feature extraction layer, the second feature extraction layer, and the third feature extraction layer, respectively. The output of the feature connection layer is connected to the input of the feature fusion layer. Accordingly, the first noise reduction unit is specifically used for: The first output results of the current frame image and the previous frame image are input into the first downsampling layer, and the first downsampling result of the current frame image and the first downsampling result of the previous frame image are output; the first downsampling result of the current frame image and the first downsampling result of the previous frame image are input into the second downsampling layer, and the second downsampling result of the current frame image and the second downsampling result of the previous frame image are output. The first output results of the current frame image and the previous frame image are input into the first feature extraction layer, and the first extracted features are output. The first downsampling result of the current frame image and the first downsampling result of the first output result of the previous frame image are input into the second feature extraction layer, and the second extracted feature is output. The second downsampling result of the current frame image and the second downsampling result of the first output result of the previous frame image are input into the third feature extraction layer, and the third extracted feature is output. The second extracted feature is input into the first upsampling layer, and the first upsampling feature is output. The third extracted feature is input into the second upsampling layer, and the second upsampling feature is output. The first extracted feature, the first upsampled feature, and the second upsampled feature are input into the feature connection layer, and cascaded features are output. The cascaded features are input into the feature fusion layer, and the motion estimation vector corresponding to the current frame image is output.

4. The neural network processor according to claim 1, characterized in that, The video noise reduction module includes a second noise reduction unit, which is used to perform a second-stage noise reduction process on the result obtained from the first-stage noise reduction process, including: Based on the first output result of the current frame image and the motion mask corresponding to the first output result of the current frame image, the motion region denoising data of the first output result of the current frame image is obtained, and based on the first output result of the current frame image and the still mask corresponding to the first output result of the current frame image, the still region denoising data of the first output result of the current frame image is obtained; wherein, the motion mask and the still mask corresponding to the first output result of the current frame image are obtained in advance; Based on the motion region denoising data of the first output result of the current frame image and the second denoising model, the motion region output result corresponding to the first output result of the current frame image is obtained; and based on the static region denoising data of the first output result of the current frame image and the third denoising model, the static region output result corresponding to the first output result of the current frame image is obtained; wherein, the second denoising model and the third denoising model are obtained in advance; The motion region output result corresponding to the first output result of the current frame image and the stationary region output result corresponding to the first output result of the current frame image are fused to obtain the second output result corresponding to the current frame image.

5. The neural network processor according to any one of claims 1 to 4, characterized in that, The nonlinear processing includes logarithmic transformation.

6. The neural network processor according to claim 5, characterized in that, The reverse post-processing includes inverse logarithmic transformation.

7. A data noise reduction processing method, characterized in that, The neural network processor according to any one of claims 1 to 6 comprises: The original video stream data is subjected to non-linear processing to obtain the first video stream data; The first video stream data is subjected to two-stage noise reduction processing to obtain the second video stream data; wherein, the first-stage noise reduction processing is used to obtain the first output result of each frame of the first video stream data based on each frame of the first video stream data and the first output result of the corresponding previous frame of the first video stream data; the second-stage noise reduction processing is used to perform single-frame image noise reduction on the first output result of each frame of the first video stream data to obtain the second output result of each frame of the first video stream data, and the second output results corresponding to each frame of the first video stream data constitute the second video stream data. The second video stream data is reverse-processed to obtain the processed video stream data.

8. The method according to claim 7, characterized in that, The first output result of each frame of the first video stream data, based on each frame of the first video stream data and the corresponding first output result of the previous frame, includes: Based on the first output results of the current frame image and the previous frame image, and the first denoising model, the motion estimation vector corresponding to the current frame image is obtained; wherein, the first denoising model is obtained in advance; The first output result of the current frame image is obtained by weighted fusing the first output result of the current frame image and the previous frame image based on the motion estimation vector corresponding to the current frame image.

9. The method according to claim 8, characterized in that, The first noise reduction model includes a first downsampling layer, a second downsampling layer, a first feature extraction layer, a second feature extraction layer, a third feature extraction layer, a first upsampling layer, a second upsampling layer, a feature connection layer, and a feature fusion layer. The output of the first downsampling layer is connected to the inputs of the second upsampling layer and the second feature extraction layer. The output of the second downsampling layer is connected to the input of the third feature extraction layer. The input of the feature connection layer is connected to the outputs of the first feature extraction layer, the second feature extraction layer, and the third feature extraction layer, respectively. The output of the feature connection layer is connected to the input of the feature fusion layer. Correspondingly, obtaining the motion estimation vector corresponding to the current frame image based on the first output result of the current frame image and the previous frame image of the current frame image, and the first noise reduction model, includes: The first output results of the current frame image and the previous frame image are input into the first downsampling layer, and the first downsampling result of the current frame image and the first downsampling result of the previous frame image are output; the first downsampling result of the current frame image and the first downsampling result of the previous frame image are input into the second downsampling layer, and the second downsampling result of the current frame image and the second downsampling result of the previous frame image are output. The first output results of the current frame image and the previous frame image are input into the first feature extraction layer to output the first extracted feature; the first downsampling result of the current frame image and the first downsampling result of the first output result of the previous frame image are input into the second feature extraction layer to output the second extracted feature; the second downsampling result of the current frame image and the second downsampling result of the first output result of the previous frame image are input into the third feature extraction layer to output the third extracted feature. The second extracted feature is input into the first upsampling layer, and the first upsampling feature is output; the third extracted feature is input into the second upsampling layer, and the second upsampling feature is output. The first extracted feature, the first upsampled feature, and the second upsampled feature are input into the feature connection layer, and cascaded features are output. The cascaded features are input into the feature fusion layer, and the motion estimation vector corresponding to the current frame image is output.

10. The method according to any one of claims 7 to 9, characterized in that, The step of performing single-frame image noise reduction on the first output result of each frame of the first video stream data to obtain the second output result of each frame includes: Based on the first output result of the current frame image and the motion mask corresponding to the first output result of the current frame image, the motion region denoising data of the first output result of the current frame image is obtained, and based on the first output result of the current frame image and the still mask corresponding to the first output result of the current frame image, the still region denoising data of the first output result of the current frame image is obtained; wherein, the motion mask and the still mask corresponding to the first output result of the current frame image are obtained in advance; Based on the motion region denoising data of the first output result of the current frame image and the second denoising model, the motion region output result corresponding to the first output result of the current frame image is obtained; and based on the static region denoising data of the first output result of the current frame image and the third denoising model, the static region output result corresponding to the first output result of the current frame image is obtained; wherein, the second denoising model and the third denoising model are obtained in advance; The motion region output result corresponding to the first output result of the current frame image and the stationary region output result corresponding to the first output result of the current frame image are fused to obtain the second output result corresponding to the current frame image.

11. An artificial intelligence image signal processor, characterized in that, Includes the neural network processor according to any one of claims 1 to 6.

12. A chip, characterized in that, It includes at least one artificial intelligence image signal processor as described in claim 11.

13. A circuit board, characterized in that, It includes at least one chip as described in claim 12.

14. An electronic device, characterized in that, It includes at least one chip as described in claim 12 or at least one board as described in claim 13.

Images