Video denoising method and video denoising apparatus

Abstract

The application relates to the technical field of image processing, in particular to a video denoising method and device, a computer readable storage medium and an electronic device, and solves the problems of high difficulty and poor denoising effect of microscopic video denoising. The video denoising method comprises the following steps: acquiring a video denoising model, wherein the video denoising model is obtained by training an initial network model based on a plurality of noise-increasing videos corresponding to a to-be-denoised video and the to-be-denoised video; and generating a noise-reducing video corresponding to the to-be-denoised video based on the to-be-denoised video by using the video denoising model, so that the initial network model can learn noise information of the to-be-denoised video in the training, the generated video denoising model can accurately denoise the to-be-denoised video, and therefore, the noise-reducing video corresponding to the to-be-denoised video generated by using the video denoising model is more accurate.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a video denoising method and device, and a computer readable storage medium and an electronic device. BACKGROUND

[0002] Microscopic video refers to a video generated by recording a dynamic experimental process observed under an optical microscope by using a camera and then processing the video by using related software. For example, a calcium imaging video is a kind of microscopic video. Fluorescent proteins are used to specifically label neuron cells, and fluorescent proteins that label calcium ions can emit different intensities of fluorescence according to the change in the concentration of calcium ions in the neuron cells. The change in the fluorescent signal can be recorded in the form of a video. The video that realizes imaging according to the change in the concentration of calcium ions can be simply referred to as a calcium imaging video.

[0003] In the process of producing a calcium imaging video, in order to reduce the phototoxicity to organisms, the number of photons participating in imaging is much smaller than the number of photons in natural imaging. Therefore, the signal-to-noise ratio of various microscopic videos including calcium imaging videos is much smaller than the signal-to-noise ratio of videos in natural imaging, resulting in high difficulty in microscopic video denoising and poor denoising effect. SUMMARY

[0004] Therefore, an embodiment of the present application provides a video denoising method and device, and a computer readable storage medium and an electronic device, which solve the problems of high difficulty in microscopic video denoising and poor denoising effect.

[0005] In a first aspect, an embodiment of the present application provides a video denoising method, which comprises: determining a video denoising model, wherein the video denoising model is obtained by training an initial network model based on a plurality of noise-increasing videos corresponding to a to-be-denoised video and the to-be-denoised video; and generating a noise-reduced video corresponding to the to-be-denoised video based on the to-be-denoised video by using the video denoising model.

[0006] In combination with the first aspect of the present application, in some embodiments, generating the noise-reduced video corresponding to the to-be-denoised video based on the to-be-denoised video by using the video denoising model comprises: generating a plurality of to-be-denoised video segments corresponding to the to-be-denoised video based on the to-be-denoised video; respectively denoising the plurality of to-be-denoised video segments by using the video denoising model to obtain a plurality of noise-reduced video segments corresponding to the plurality of to-be-denoised video segments respectively; and splicing the plurality of noise-reduced video segments corresponding to the plurality of to-be-denoised video segments respectively to generate the noise-reduced video corresponding to the to-be-denoised video.

[0007] In some embodiments, determining the video denoising model comprises: generating a plurality of noise-added videos corresponding to the to-be-denoised video based on the to-be-denoised video; establishing an initial network model, and training the initial network model based on the plurality of noise-added videos and the to-be-denoised video to generate the video denoising model, wherein the video denoising model is used to denoise the to-be-denoised video to generate a denoised video corresponding to the to-be-denoised video.

[0008] In some embodiments, generating a plurality of noise-added videos corresponding to the to-be-denoised video based on the to-be-denoised video comprises: generating a plurality of noise-added videos corresponding to the to-be-denoised video based on the to-be-denoised video by using a noise model, wherein the plurality of noise-added videos contain different noise intensities, and the noise model comprises any one of the following models: an additive noise prior model, a multiplicative noise prior model, and an additive-multiplicative compound prior model.

[0009] In some embodiments, generating a plurality of noise-added videos corresponding to the to-be-denoised video based on the to-be-denoised video comprises: determining noise intensity estimation information of the to-be-denoised video based on the to-be-denoised video; and performing N times of noise addition operations on the to-be-denoised video based on the noise intensity estimation information to determine N noise-added videos corresponding to the to-be-denoised video, wherein N is a positive integer.

[0010] In some embodiments, determining noise intensity estimation information of the to-be-denoised video based on the to-be-denoised video comprises: determining at least one frame of video image corresponding to the to-be-denoised video based on the to-be-denoised video; generating a plurality of image regions corresponding to the at least one frame of video image based on the at least one frame of video image; calculating a gray mean value and a gray variance corresponding to each of the plurality of image regions; and determining the noise intensity estimation information based on the gray mean value and the gray variance corresponding to each of the plurality of image regions.

[0011] In some embodiments, determining the noise intensity estimation information based on the gray mean value and the gray variance corresponding to each of the plurality of image regions comprises: performing a linear regression operation on the gray mean value and the gray variance corresponding to each of the plurality of image regions to determine the noise intensity estimation information of the to-be-denoised video.

[0012] In some embodiments, the noise intensity estimation information of the to-be-de-noised video is determined based on the linear regression operation on the respective gray mean and the respective gray variance of the plurality of image regions, including: establishing a data point graph of the gray mean and the gray variance based on the respective gray mean and the respective gray variance of the plurality of image regions, the data point graph including a plurality of data points; performing a linear regression operation on the plurality of data points to determine a regression straight line function; and determining the noise intensity estimation information of the to-be-de-noised video based on the regression straight line function, wherein the noise intensity estimation information of the to-be-de-noised video includes a Poisson noise intensity, a Gaussian noise mean and a Gaussian noise variance.

[0013] In some embodiments, the initial network model is trained based on the plurality of noise-increased videos and the to-be-de-noised video to generate the video de-noising model, including: constructing a loss function of the initial network model, wherein the loss function includes a time domain smoothing kernel norm regular term and a spatial domain smoothing entropy regular term; and training the initial network model based on the plurality of noise-increased videos, the to-be-de-noised video and the loss function to generate the video de-noising model.

[0014] In some embodiments, the to-be-de-noised video includes a to-be-de-noised microscopic video.

[0015] In a second aspect, an embodiment of the present application provides a video de-noising device, including: a determination module configured to determine a video de-noising model, wherein the video de-noising model is obtained by training an initial network model based on a plurality of noise-increased videos corresponding to a to-be-de-noised video and the to-be-de-noised video; and a de-noising module configured to generate a de-noised video corresponding to the to-be-de-noised video based on the to-be-de-noised video by using the video de-noising model.

[0016] In a third aspect, an embodiment of the present application provides a computer readable storage medium, the storage medium storing instructions, when the instructions are executed by a processor of an electronic device, the electronic device is enabled to execute the video de-noising method mentioned in the first aspect.

[0017] In a fourth aspect, an embodiment of the present application provides an electronic device, including: a processor; a memory for storing computer executable instructions; and the processor is configured to execute the computer executable instructions to implement the video de-noising method mentioned in the first aspect.

[0018] The to-be-de-noised video is a video containing internal noise, and there is no noise-free ideal de-noising gold standard as a label in the training sample to train the initial network model. Therefore, the video de-noising method provided in the embodiment of the application determines a video de-noising model, wherein the video de-noising model is obtained by training an initial network model based on a plurality of noise-increased videos corresponding to the to-be-de-noised video and the to-be-de-noised video. The video de-noising model is used to generate a de-noised video corresponding to the to-be-de-noised video based on the to-be-de-noised video, so that the initial network model can learn the noise information of the to-be-de-noised video in the training, so that the generated video de-noising model can accurately de-noise the to-be-de-noised video. Therefore, the de-noised video corresponding to the to-be-de-noised video generated by the video de-noising model is more accurate. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 Fig. 1 shows an application scenario of a video de-noising method provided by an embodiment of the application.

[0020] Figure 2 Fig. 2 shows a flowchart of a video de-noising method provided by an embodiment of the application.

[0021] Figure 3 Fig. 3 shows a flowchart of a video de-noising method provided by another embodiment of the application.

[0022] Figure 4 Fig. 4 shows a flowchart of a video de-noising method provided by another embodiment of the application.

[0023] Figure 5 Fig. 5 shows a flowchart of a video de-noising method provided by another embodiment of the application.

[0024] Figure 6 Fig. 6 shows a flowchart of a video de-noising method provided by another embodiment of the application.

[0025] Figure 7 Fig. 7 shows a flowchart of a video de-noising method provided by another embodiment of the application.

[0026] Figure 8 Fig. 8 shows a flowchart of a video de-noising method provided by another embodiment of the application.

[0027] Figure 8a Fig. 9 shows a video image provided by an embodiment of the application.

[0028] Figure 9 Fig. 10 shows a flowchart of a video de-noising method provided by another embodiment of the application.

[0029] Figure 10 Fig. 11 shows a structural diagram of a video de-noising device provided by an embodiment of the application.

[0030] Figure 11 Fig. 2 shows a structural schematic diagram of a video denoising device according to another embodiment of the present application.

[0031] Figure 12 Fig. 2 shows a structural schematic diagram of a video denoising device according to another embodiment of the present application.

[0032] Figure 13 Fig. 2 shows a structural schematic diagram of a video denoising device according to another embodiment of the present application.

[0033] Figure 14 Fig. 2 shows a structural schematic diagram of a video denoising device according to another embodiment of the present application.

[0034] Figure 15 Fig. 2 shows a structural schematic diagram of a video denoising device according to another embodiment of the present application.

[0035] Figure 16 Fig. 2 shows a structural schematic diagram of a video denoising device according to another embodiment of the present application.

[0036] Figure 17 Fig. 2 shows a structural schematic diagram of a video denoising device according to another embodiment of the present application.

[0037] Figure 18 Fig. 2 shows a structural schematic diagram of a video denoising device according to another embodiment of the present application. DETAILED DESCRIPTION

[0038] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0039] Figure 1 Fig. 2 shows a structural schematic diagram of a video denoising device according to another embodiment of the present application. Figure 1 The scenario shown includes a server 110 and a video shooting device 120 in communication connection with the server 110. Specifically, the server 110 is configured to determine a video denoising model, wherein the video denoising model is obtained by training an initial network model based on a plurality of noise-increasing videos corresponding to a to-be-denoised video and the to-be-denoised video; and generate a noise-reduced video corresponding to the to-be-denoised video based on the to-be-denoised video by using the video denoising model.

[0040] Exemplarily, in the actual application process, the video shooting device 120 is configured to shoot a to-be-de-noised video, and send the acquired to-be-de-noised video to the server 110, and the server 110 generates a de-noised video corresponding to the to-be-de-noised video based on the received to-be-de-noised video.

[0041] Exemplary method

[0042] Figure 2 A flowchart of a video de-noising method provided by an embodiment of the present application is shown. As shown in the figure, Figure 2 The video de-noising method provided by the embodiment of the present application includes the following steps.

[0043] Step 210: determining a video de-noising model.

[0044] Exemplarily, the video de-noising model is obtained by training an initial network model based on a plurality of noise-increased videos corresponding to a to-be-de-noised video and the to-be-de-noised video.

[0045] Step 220: generating a de-noised video corresponding to the to-be-de-noised video based on the to-be-de-noised video by using the video de-noising model.

[0046] By acquiring a video de-noising model, wherein the video de-noising model is obtained by training an initial network model based on a plurality of noise-increased videos corresponding to a to-be-de-noised video and the to-be-de-noised video, and generating a de-noised video corresponding to the to-be-de-noised video based on the to-be-de-noised video by using the video de-noising model, the initial network model can learn the noise information of the to-be-de-noised video in the training, so that the generated video de-noising model can accurately de-noise the to-be-de-noised video. Therefore, the de-noised video corresponding to the to-be-de-noised video generated by using the video de-noising model is more accurate and clearer.

[0047] Figure 3 A flowchart of a video de-noising method provided by another embodiment of the present application is shown. Based on the embodiment of the present application Figure 2 Based on the embodiment of the present application, the video de-noising method provided by the embodiment of the present application includes the following steps. Figure 3 Based on the embodiment of the present application, the video de-noising method provided by the embodiment of the present application includes the following steps. Figure 3 The embodiment of the present application is different from Figure 2 The differences and similarities between the embodiments of the present application are not described again.

[0048] As shown in the figure, Figure 3 In the embodiment of the present application, the step of generating a de-noised video corresponding to the to-be-de-noised video based on the to-be-de-noised video by using the video de-noising model includes the following steps.

[0049] Step 310: generating a plurality of to-be-de-noised video segments corresponding to the to-be-de-noised video based on the to-be-de-noised video.

[0050] Exemplarily, the to-be-de-noised video segment can include 8 video images or 16 video images. Those skilled in the art can select the number of frames included in the to-be-de-noised video segment according to actual needs, which is not limited in the present application.

[0051] In step 320, the video de-noising model is used to de-noise the plurality of to-be-de-noised video segments respectively to obtain a plurality of de-noised video segments corresponding to the plurality of to-be-de-noised video segments respectively.

[0052] Specifically, the plurality of to-be-de-noised video segments can be sequentially input into the video de-noising model to output the plurality of de-noised video segments.

[0053] In step 330, the plurality of de-noised video segments corresponding to the plurality of to-be-de-noised video segments are spliced to generate a de-noised video corresponding to the to-be-de-noised video.

[0054] For example, the to-be-de-noised video is a 20-frame video. The 1st-16th frame of the to-be-de-noised video can be divided into a to-be-de-noised video segment, the 2nd-17th frame can be divided into a to-be-de-noised video segment, the 3rd-18th frame can be divided into a to-be-de-noised video segment, the 4th-19th frame can be divided into a to-be-de-noised video segment, and the 5th-20th frame can be divided into a to-be-de-noised video segment, thereby obtaining five to-be-de-noised video segments. Then, the five to-be-de-noised video segments are input into the video de-noising model to obtain five de-noised video segments. Finally, the five de-noised video segments are spliced to generate a de-noised video corresponding to the to-be-de-noised video. For the repeated video images in the five de-noised video segments, the average gray value of the repeated video images can be taken as the video image in the spliced de-noised video.

[0055] By dividing the to-be-de-noised video into a plurality of to-be-de-noised video segments, de-noising the to-be-de-noised video segments respectively, and then splicing the de-noised video segments to generate a de-noised video, the memory requirement of the hardware device in the video de-noising process can be reduced, and the video de-noising efficiency can be improved.

[0056] Figure 4 Fig. 1 shows a flowchart of a video de-noising method according to an embodiment of the present application. Based on the embodiment shown in Fig. 1, the present application Figure 2 extends to a video de-noising method according to another embodiment of the present application. Figure 4 The embodiment shown in Fig. 1 is different from the embodiment shown in Fig. 2 in that Figure 4 the embodiment shown in Fig. 1 is different from the embodiment shown in Fig. 3 in that Figure 2 the same parts are not described again.

[0057] As shown in Fig. 1, in the embodiment of the present application, the video de-noising model includes the following steps. Figure 4

[0058] In step 410, a to-be-de-noised video corresponding to the to-be-de-noised video is generated.

[0059] ​Specifically, the video to be denoised is a video containing internal noise. The video with added noise can be a video containing double noise. The double noise includes internal noise and external noise.

[0060] In an embodiment of the present application, the video to be denoised can be a microscopic video to be denoised. For example, the video to be denoised can be a calcium imaging video.

[0061] In an embodiment of the present application, a noise model can be used to generate a plurality of videos with added noise corresponding to the video to be denoised based on the video to be denoised. The plurality of videos with added noise contain different noise intensities. The noise model can be an additive noise prior model, a multiplicative noise prior model, or an additive-multiplicative compound prior model. The multiplicative noise prior model can be a Poisson noise model, the additive noise prior model can be a Gaussian noise model, and the additive-multiplicative compound prior model can be a compound model of Poisson noise and Gaussian noise.

[0062] Step 420, an initial network model is established, and the initial network model is trained based on the video with added noise and the video to be denoised to generate a video denoising model.

[0063] For example, the video denoising model is used to denoise the video to be denoised to generate a noise-reduced video corresponding to the video to be denoised. The video with added noise uses y * to represent, and the video to be denoised uses y to represent, thereby forming a training set {y * , y}. The training set {y * , y} is used to train the initial network model to generate the video denoising model.

[0064] Specifically, the video to be denoised can be a video containing internal noise, and the video with added noise can be a video containing double noise. The double noise includes internal noise and external noise. The noise-free ideal denoising gold standard refers to a video completely free of noise. Since there is no corresponding noise-free ideal denoising gold standard as a label in the training sample to train the initial network model, the present application generates a video with added noise corresponding to the video to be denoised based on the video to be denoised, and trains the initial network model based on the video with added noise and the video to be denoised to generate a video denoising model, so that the video to be denoised is used as a label of the video with added noise to train the initial network model, so that the initial network model can learn the noise information of the video to be denoised in the training, so that the generated video denoising model can accurately denoise the video to be denoised.

[0065] Figure 5 Fig. 2 shows a flowchart of a video denoising method provided by another embodiment of the present application. Based on the embodiment shown in Fig. 1, the embodiment shown in Fig. 2 extends the embodiment shown in Fig. 1. Figure 4 Fig. 3 shows a flowchart of a video denoising method provided by another embodiment of the present application. Based on the embodiment shown in Fig. 1, the embodiment shown in Fig. 3 extends the embodiment shown in Fig. 1. Figure 5 Fig. 4 shows a flowchart of a video denoising method provided by another embodiment of the present application. Based on the embodiment shown in Fig. 1, the embodiment shown in Fig. 4 extends the embodiment shown in Fig. 1. Figure 5 Fig. 5 shows a flowchart of a video denoising method provided by another embodiment of the present application. Based on the embodiment shown in Fig. 1, the embodiment shown in Fig. 5 extends the embodiment shown in Fig. 1. Figure 4The differences and similarities between the illustrated embodiments are not described in detail.

[0066] like Figure 5 As shown, in an embodiment of the present application, the step of generating multiple noise-enhanced videos corresponding to the video to be denoised based on the video to be denoised includes the following steps.

[0067] Step 510: Determine noise intensity estimation information of the video to be denoised based on the video to be denoised.

[0068] The noise intensity estimate is information obtained by estimating the internal noise contained in the video to be denoised. Specifically, the video to be denoised and the noise intensity estimate can be fused to add external noise to the video to generate a noise-enhanced video corresponding to the video to be denoised. In other words, the noise-enhanced video is obtained by adding external noise to the denoised video based on the noise intensity estimate.

[0069] Step 520 : Based on the noise intensity estimation information, perform N noise adding operations on the video to be denoised to determine N different noise-enhanced videos corresponding to the video to be denoised.

[0070] Specifically, each time a noise addition operation is performed, the noise intensity estimation information is added to the denoised video. N is a positive integer. N=1, that is, the noise addition operation is performed once on the denoised video to determine the first denoised video, N=2, that is, the noise addition operation is performed twice on the denoised video to determine the second denoised video, and so on, thereby obtaining the set of denoised videos {y *}. The set of denoised videos {y *}Includes N noise-enhanced videos.

[0071] By performing noise addition operations on the denoised video multiple times, multiple denoised videos are obtained, so that the obtained multiple denoised videos contain different amounts of noise intensity estimation information, which increases the richness of the set of denoised videos and thus increases the number of training sets {y * , y}, provides a rich data foundation for training the initial network model.

[0072] Figure 6 The figure shows a flow chart of a video denoising method provided by another embodiment of the present application. Figure 4 The present application is extended based on the embodiment shown Figure 6 The embodiment shown is described below in detail. Figure 6 The embodiment shown and Figure 4 The differences and similarities between the illustrated embodiments are not described in detail.

[0073] like Figure 6 As shown, in an embodiment of the present application, the step of determining noise intensity estimation information of the video to be denoised based on the video to be denoised includes the following steps.

[0074] At step 610, at least one frame of video image corresponding to the to-be-denoised video is determined based on the to-be-denoised video.

[0075] Specifically, the to-be-denoised video includes multiple frames of video image. The at least one frame of video image corresponding to the to-be-denoised video is determined based on the to-be-denoised video, which can be any one or more frames of video image selected from the multiple frames of video image included in the to-be-denoised video.

[0076] At step 620, multiple image regions corresponding to the at least one frame of video image are generated based on the at least one frame of video image.

[0077] Specifically, the at least one frame of video image can be one frame of video image or multiple frames of video image. The multiple image regions corresponding to the at least one frame of video image are generated based on the at least one frame of video image, which can be clipping one frame of video image into multiple image regions or clipping multiple frames of video image into multiple image regions. For example, one frame of video image can be clipped into k image regions, and multiple frames of video image can also be clipped into k image regions. The video image can be randomly clipped into multiple image regions, that is, the multiple image regions obtained by clipping can overlap or not overlap. The image regions obtained by clipping can be represented using Q k When k = 1, Q k = Q1 represents the first image region, when k = 2, Q k = Q2 represents the second image region, and so on.

[0078] At step 630, the gray mean value and the gray variance corresponding to each of the multiple image regions are calculated.

[0079] Exemplarily, the gray mean value is represented using E[Q k ], and the gray variance is represented using σ 2 [Q k ]. When k = 1, E[Q k ] = E[Q1] represents the gray mean value corresponding to the first image region, and σ 2 [Q k ] = σ 2 [Q1] represents the gray variance corresponding to the first image region; when k = 2, E[Q k ] = E[Q2] represents the gray mean value corresponding to the second image region, and σ 2 [Q k ] = σ 2 [Q2] represents the gray variance corresponding to the second image region, and so on.

[0080] At step 640, noise intensity estimation information is determined based on the gray mean value and the gray variance corresponding to each of the multiple image regions.

[0081] Specifically, a relationship function of the mean gray value and the gray variance corresponding to each of the plurality of image regions can be constructed, so as to calculate the noise intensity estimation information.

[0082] The embodiment of the present application determines at least one video image corresponding to the video to be denoised based on the video to be denoised, and performs gray analysis on the at least one video image to determine the noise intensity estimation information. Since the video image mainly represents noise in gray, the noise intensity estimation information is determined by performing gray analysis on the video image, thereby improving the accuracy of the noise intensity estimation information.

[0083] Figure 7 Fig. 6 shows a flowchart of a video denoising method according to another embodiment of the present application. The embodiment shown in Fig. 6 is extended on the basis of the embodiment shown in Fig. 5. Figure 6 Fig. 7 shows a flowchart of a video denoising method according to another embodiment of the present application. The embodiment shown in Fig. 7 is extended on the basis of the embodiment shown in Fig. 5. Figure 7 The embodiment shown in Fig. 7 is different from the embodiment shown in Fig. 5 in that the step of determining the noise intensity estimation information based on the mean gray value and the gray variance corresponding to each of the plurality of image regions includes the following steps. Figure 7 The embodiment shown in Fig. 7 is different from the embodiment shown in Fig. 5 in that the step of determining the noise intensity estimation information based on the mean gray value and the gray variance corresponding to each of the plurality of image regions includes the following steps. Figure 6 The embodiment shown in Fig. 7 is different from the embodiment shown in Fig. 5 in that the step of determining the noise intensity estimation information based on the mean gray value and the gray variance corresponding to each of the plurality of image regions includes the following steps.

[0084] As shown in Fig. 7, in the embodiment of the present application, the step of determining the noise intensity estimation information based on the mean gray value and the gray variance corresponding to each of the plurality of image regions includes the following steps. Figure 7 Step 710: performing linear regression operation based on the mean gray value and the gray variance corresponding to each of the plurality of image regions to determine the noise intensity estimation information of the video to be denoised.

[0085] Specifically, linear regression is a statistical analysis method for determining the quantitative relationship between two or more variables by using regression analysis in mathematical statistics.

[0086] By performing linear regression operation, the linear relationship between the mean gray value and the gray variance can be determined, so as to determine the noise intensity estimation information in the linear relationship. The method is simple and reliable, and has high efficiency.

[0087]

[0088] Fig. 8 shows a flowchart of a video denoising method according to another embodiment of the present application. The embodiment shown in Fig. 8 is extended on the basis of the embodiment shown in Fig. 7. Figure 8 Fig. 9 shows a flowchart of a video denoising method according to another embodiment of the present application. The embodiment shown in Fig. 9 is extended on the basis of the embodiment shown in Fig. 7. Figure 7 The embodiment shown in Fig. 9 is different from the embodiment shown in Fig. 7 in that the step of determining the noise intensity estimation information based on the mean gray value and the gray variance corresponding to each of the plurality of image regions includes the following steps. Figure 8 The embodiment shown in Fig. 9 is different from the embodiment shown in Fig. 7 in that the step of determining the noise intensity estimation information based on the mean gray value and the gray variance corresponding to each of the plurality of image regions includes the following steps. Figure 8 The embodiment shown in Fig. 9 is different from the embodiment shown in Fig. 7 in that the step of determining the noise intensity estimation information based on the mean gray value and the gray variance corresponding to each of the plurality of image regions includes the following steps. Figure 7 The embodiment shown in Fig. 9 is different from the embodiment shown in Fig. 7 in that the step of determining the noise intensity estimation information based on the mean gray value and the gray variance corresponding to each of the plurality of image regions includes the following steps.

[0089] As shown in Fig. 9, in the embodiment of the present application, the step of determining the noise intensity estimation information based on the mean gray value and the gray variance corresponding to each of the plurality of image regions includes the following steps. Figure 8As shown, in the embodiment of the present application, the step of determining the noise intensity estimation information of the to-be-denoised video based on the linear regression operation on the gray mean value and the gray variance corresponding to each of the plurality of image regions comprises the following steps.

[0090] At step 810, a data point diagram of the gray mean value and the gray variance is established based on the gray mean value and the gray variance corresponding to each of the plurality of image regions. The data point diagram comprises a plurality of data points.

[0091] Specifically, a coordinate system can be established with the gray mean value E[Q k ] as the horizontal axis and the gray variance σ 2 [Q k ] as the vertical axis. Then, a plurality of data points are determined according to the gray mean value and the gray variance corresponding to each of the plurality of image regions, so as to establish the data point diagram of the gray mean value and the gray variance. Each data point represents the gray mean value and the gray variance corresponding to one image region.

[0092] At step 820, a linear regression operation is performed on the plurality of data points to determine a regression straight line function.

[0093] Exemplarily, the linear regression operation on the plurality of data points can be a linear regression fitting on the plurality of data points, so as to obtain the regression straight line function.

[0094] At step 830, the noise intensity estimation information of the to-be-denoised video is determined based on the regression straight line function.

[0095] Exemplarily, the noise intensity estimation information of the to-be-denoised video comprises a Poisson noise intensity, a Gaussian noise mean value and a Gaussian noise variance. The Poisson noise intensity can be represented by a= , the Gaussian noise mean value can be represented by a= , and the Gaussian noise variance can be represented by a= .

[0096] The regression straight line function is as follows:

[0097]

[0098] As can be seen from the above formula, the slope a= of the regression straight line function is a= . The linear regression operation on the plurality of data points determines the regression straight line function, so as to determine the slope a and the intercept d of the regression straight line function. Therefore, the determination of the regression straight line function can directly obtain the Poisson noise intensity , and can determine the relationship between the Poisson noise intensity , the Gaussian noise mean value and the Gaussian noise variance . That is, after the determination of the regression straight line function, the Poisson noise intensity It is known that the Poisson noise intensity , Gaussian noise mean and Gaussian noise variance The relationship between is known, the Gaussian noise mean and Gaussian noise variance unknown.

[0099] Gaussian noise mean It can be calculated based on the grayscale of the video image. Figure 8a As shown in the figure, the video image includes white highlight areas with low grayscale values ​​and black areas with high grayscale values. It can be the grayscale average of the black area with higher grayscale value. The Gaussian noise mean can be obtained by calculating the grayscale average of the black area with higher grayscale value. Then, according to the formula d= The Gaussian noise variance can be calculated .

[0100] This embodiment first establishes a data point graph of the grayscale mean and grayscale variance based on the grayscale mean and grayscale variance corresponding to each of the multiple image regions. Then, a linear regression operation is performed on the multiple data points to determine the regression line function. Finally, based on the regression line function, the Poisson noise intensity, Gaussian noise mean, and Gaussian noise variance of the video to be denoised are determined. Since the internal noise contained in the video to be denoised is mainly Poisson-Gaussian mixed noise, and Poisson-Gaussian mixed noise is characterized by Poisson noise intensity, Gaussian noise mean, and Gaussian noise variance, the Poisson noise intensity, Gaussian noise mean, and Gaussian noise variance of the video to be denoised are determined, that is, the main internal noise of the video to be denoised is determined, thereby providing accurate training data for training the initial network model.

[0101] Figure 9 The figure shows a flow chart of a video denoising method provided by another embodiment of the present application. Figure 4 The present application is extended based on the embodiment shown Figure 9 The embodiment shown is described below in detail. Figure 9 The embodiment shown and Figure 4 The differences and similarities between the illustrated embodiments are not described in detail.

[0102] like Figure 9 As shown, in an embodiment of the present application, the steps of training an initial network model based on multiple denoised videos and videos to be denoised to generate a video denoising model include the following steps.

[0103] Step 910: construct a loss function of the initial network model.

[0104] Exemplarily, the loss function includes a time domain smoothed nuclear norm regularization term and a space domain smoothed entropy regularization term. The loss function is as follows:

[0105]

[0106] wherein, , ,

[0107] In the above formula, L is a loss value, L rec is a fidelity term, L nuclear is a time domain smooth kernel norm regular term, L entropy is a spatial domain smooth entropy regular term, λ1 is a weight of the time domain smooth kernel norm regular term, λ2 is a weight of the spatial domain smooth entropy regular term, is an output video of the initial network model in the process of training the initial network model, y is a video to be denoised, and ε is an infinitesimal, is a gray value of the output video of the initial network model . is a Hessian matrix of the output video of the initial network model . is a Hessian matrix of the output video of the initial network model Second-order derivatives are calculated in a, b, and c three different directions, The Hessian matrix of the output video of the initial network model

[0108]

[0109] Step 920, training the initial network model based on the plurality of noise-increasing videos, the video to be denoised, and the loss function to generate a video denoising model.

[0110] Specifically, the noise-increasing video y * is taken as the input of the initial network model, and the video to be denoised y is taken as the label of the noise-increasing video y * to train the initial network model. In the training process, the weight λ1 of the time domain smooth kernel norm regular term and the weight λ2 of the spatial domain smooth entropy regular term are constantly adjusted until the function value L of the loss function converges, thereby obtaining the video denoising model.

[0111] The video to be denoised can be a calcium imaging video of a neuron cell. In the calcium imaging process, the position of the cell body of the neuron cell does not change over time, but the gray value of the cell body changes, that is, the calcium imaging video contains both time and space information, thereby indicating that the calcium imaging video contains a spatio-temporal coupling structure. Most microscopic videos contain spatio-temporal coupling structures.

[0112] The time domain smoothing kernel norm regular term and the space domain smoothing entropy regular term are set in the loss function, low rank information in the space-time coupling structure can be extracted and noise signals changing too fast in the time domain can be eliminated. Therefore, by constructing the loss function, and training the initial network model based on the noise-increasing video, the video to be denoised and the loss function to generate the video denoising model, the initial network model can learn the space-time coupling structure and noise in the video to be denoised, so that the obtained video denoising model can more accurately retain the space-time coupling structure in the video to be denoised and remove the noise in the video to be denoised.

[0113] The method embodiments of the present application are described in detail above Figures 1-9 , and the device embodiments of the present application are described in detail below Figures 10-17 . It should be understood that the description of the method embodiments corresponds to the description of the device embodiments, and therefore, the parts not described in detail can be referred to the foregoing method embodiments.

[0114] Exemplary apparatus

[0115] Figure 10 Fig. 1 shows a structure schematic diagram of a video denoising device provided by an embodiment of the present application. As shown in Fig. 1, the video denoising device 1000 comprises: Figure 10

[0116] A determining module 1010 configured to determine a video denoising model, wherein the video denoising model is obtained by training an initial network model based on a plurality of noise-increasing videos corresponding to a video to be denoised and the video to be denoised.

[0117] A denoising module 1020 configured to generate a noise-reduced video corresponding to the video to be denoised based on the video to be denoised by using the video denoising model.

[0118] Figure 11 Fig. 2 shows a structure schematic diagram of a video denoising device provided by another embodiment of the present application. The embodiment shown in Fig. 2 extends from the embodiment shown in Fig. 1, and the differences between the embodiment shown in Fig. 2 and the embodiment shown in Fig. 1 will be mainly described below, and the same parts will not be described in detail. As shown in Fig. 2, the denoising module 1020 comprises: Figure 10 Figure 11 Figure 11 Figure 10 Figure 11

[0119] A video segment generating unit 1021 configured to generate a plurality of video segments to be denoised corresponding to the video to be denoised based on the video to be denoised;

[0120] A segment denoising unit 1022 configured to respectively denoise the plurality of video segments to be denoised by using the video denoising model to obtain a plurality of noise-reduced video segments corresponding to the plurality of video segments to be denoised respectively; ​​​​​​

[0121] The splicing unit 1023 is configured to splice the de-noising video corresponding to each of the plurality of to-be-de-noised video segments to generate a de-noising video corresponding to the to-be-de-noised video.

[0122] Figure 12 Fig. 2 shows a structural schematic diagram of a video de-noising device according to another embodiment of the present application. The video de-noising device shown in Fig. 2 is different from the video de-noising device shown in Fig. 1 in that the video de-noising device shown in Fig. 2 further comprises a generating module 1210. Figure 10 Based on the embodiment shown in Fig. 1, the embodiment shown in Fig. 2 extends the video de-noising device. Figure 12 The embodiment shown in Fig. 2 is different from the embodiment shown in Fig. 1 in that the embodiment shown in Fig. 2 further comprises the generating module 1210. Figure 12 The embodiment shown in Fig. 2 is different from the embodiment shown in Fig. 1 in that the embodiment shown in Fig. 2 further comprises the generating module 1210. Figure 10 The same parts of the embodiment shown in Fig. 2 and the embodiment shown in Fig. 1 will not be described again. As shown in Fig. 2, the generating module 1210 comprises: Figure 12 The generating module 1210 is configured to generate a plurality of noise-increasing videos corresponding to the to-be-de-noised video based on the to-be-de-noised video.

[0123] The training module 1220 is configured to establish an initial network model and train the initial network model based on the plurality of noise-increasing videos and the to-be-de-noised video to generate a video de-noising model, wherein the video de-noising model is used to de-noise the to-be-de-noised video to generate a de-noising video corresponding to the to-be-de-noised video.

[0124]

[0125] Fig. 3 shows a structural schematic diagram of a video de-noising device according to another embodiment of the present application. The video de-noising device shown in Fig. 3 is different from the video de-noising device shown in Fig. 2 in that the video de-noising device shown in Fig. 3 further comprises a determining unit 1211. Figure 13 Based on the embodiment shown in Fig. 2, the embodiment shown in Fig. 3 extends the video de-noising device. Figure 12 The embodiment shown in Fig. 3 is different from the embodiment shown in Fig. 2 in that the embodiment shown in Fig. 3 further comprises the determining unit 1211. Figure 13 The embodiment shown in Fig. 3 is different from the embodiment shown in Fig. 2 in that the embodiment shown in Fig. 3 further comprises the determining unit 1211. Figure 13 The embodiment shown in Fig. 3 is different from the embodiment shown in Fig. 2 in that the embodiment shown in Fig. 3 further comprises the determining unit 1211. Figure 12 The same parts of the embodiment shown in Fig. 3 and the embodiment shown in Fig. 2 will not be described again. As shown in Fig. 3, the determining unit 1211 comprises: Figure 13 The determining unit 1211 is configured to determine noise intensity estimation information of the to-be-de-noised video based on the to-be-de-noised video.

[0126] The noise-increasing unit 1212 is configured to perform N times of noise adding operations on the to-be-de-noised video based on the noise intensity estimation information to determine N noise-increasing videos corresponding to the to-be-de-noised video, wherein N is a positive integer.

[0127] In an embodiment of the present application, the generating module 1210 is further configured to generate a plurality of noise-increasing videos corresponding to the to-be-de-noised video based on the to-be-de-noised video by using a noise model, wherein the plurality of noise-increasing videos contain different noise intensities, and the noise model comprises any one of the following models: an additive noise prior model, a multiplicative noise prior model and an additive-multiplicative compound prior model.

[0128]

[0129] Figure 14 ​Fig. 1 shows a structural schematic diagram of a video denoising device according to an embodiment of the present application. The video denoising device shown in Fig. 1 can be used for the video denoising device shown in Fig. 1. Figure 12 Based on the embodiment shown in Fig. 1, the present application further provides a video denoising device. Figure 14 Based on the embodiment shown in Fig. 1, the present application further provides a video denoising device. Figure 14 Based on the embodiment shown in Fig. 1, the present application further provides a video denoising device. Figure 12 The differences between the embodiments shown in Fig. 1 and Fig. 2 will be described below, and the same parts will not be described again. As shown in Fig. 2, the video denoising device includes a video determining unit 1210, a noise intensity estimation information determining unit 1211, and a video denoising unit 1212. Figure 14 As shown in Fig. 2, the video determining unit 1210 includes:

[0130] The image determining sub-unit 1221 is configured to determine at least one frame of video image corresponding to the video to be denoised based on the video to be denoised.

[0131] The region determining sub-unit 1222 is configured to generate a plurality of image regions corresponding to the at least one frame of video image based on the at least one frame of video image.

[0132] The calculation sub-unit 1223 is configured to calculate the mean gray value and the gray variance corresponding to each of the plurality of image regions.

[0133] The information determining sub-unit 1224 is configured to determine the noise intensity estimation information based on the mean gray value and the gray variance corresponding to each of the plurality of image regions.

[0134] Figure 15 Fig. 3 shows a structural schematic diagram of a video denoising device according to another embodiment of the present application. Based on the embodiment shown in Fig. 1, the present application further provides a video denoising device. Figure 14 Based on the embodiment shown in Fig. 1, the present application further provides a video denoising device. Figure 15 Based on the embodiment shown in Fig. 1, the present application further provides a video denoising device. Figure 15 Based on the embodiment shown in Fig. 1, the present application further provides a video denoising device. Figure 14 The differences between the embodiments shown in Fig. 1 and Fig. 2 will be described below, and the same parts will not be described again. As shown in Fig. 2, the video denoising device includes a video determining unit 1210, a noise intensity estimation information determining unit 1211, and a video denoising unit 1212. Figure 15 As shown in Fig. 2, the information determining sub-unit 1224 includes:

[0135] The linear regression sub-unit 1510 is configured to perform linear regression operation based on the mean gray value and the gray variance corresponding to each of the plurality of image regions, and determine the noise intensity estimation information of the video to be denoised.

[0136] Figure 16 Fig. 3 shows a structural schematic diagram of a video denoising device according to another embodiment of the present application. Based on the embodiment shown in Fig. 1, the present application further provides a video denoising device. Figure 15 Based on the embodiment shown in Fig. 1, the present application further provides a video denoising device. Figure 16 Based on the embodiment shown in Fig. 1, the present application further provides a video denoising device. Figure 16 Based on the embodiment shown in Fig. 1, the present application further provides a video denoising device. Figure 15 The differences between the embodiments shown in Fig. 1 and Fig. 2 will be described below, and the same parts will not be described again. As shown in Fig. 2, the video denoising device includes a video determining unit 1210, a noise intensity estimation information determining unit 1211, and a video denoising unit 1212. Figure 16 As shown in Fig. 2, the information determining sub-unit 1224 includes:

[0137] The point map establishing subunit 1511 is configured to establish a data point map of grayscale mean and grayscale variance based on the grayscale mean and grayscale variance corresponding to each of the plurality of image regions, wherein the data point map includes a plurality of data points;

[0138] The function determination subunit 1512 is configured to perform a linear regression operation on a plurality of data points to determine a regression line function;

[0139] The estimation information determination subunit 1513 is configured to determine noise intensity estimation information of the video to be denoised based on the regression line function, wherein the noise intensity estimation information of the video to be denoised includes Poisson noise intensity, Gaussian noise mean, and Gaussian noise variance.

[0140] Figure 17 The figure shows a schematic diagram of the structure of a video denoising device provided by another embodiment of the present application. Figure 12 The present application is extended based on the embodiment shown Figure 17 The embodiment shown is described below in detail. Figure 17 The embodiment shown and Figure 12 The differences between the embodiments shown are omitted here for simplicity. Figure 17 As shown, the training module 1220 includes:

[0141] A function construction unit 1721 is configured to construct a loss function of the initial network model, wherein the loss function includes a time domain smoothed nuclear norm regularization term and a space domain smoothed entropy regularization term;

[0142] The model generation unit 1722 is configured to train an initial network model based on multiple denoised videos, videos to be denoised, and a loss function to generate a video denoising model.

[0143] Exemplary electronic device

[0144] Figure 18 The figure shows a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Figure 18 As shown, the electronic device 180 includes: one or more processors 1801 and a memory 1802; and computer program instructions stored in the memory 1802. When the computer program instructions are executed by the processor 1801, the processor 1801 executes the video denoising method as any of the above embodiments.

[0145] The processor 1801 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.

[0146] The memory 1802 can include one or more computer program products that can include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory, for example, can include random access memory (RAM), and / or a cache, etc. The non-volatile memory, for example, can include read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions can be stored on the computer-readable storage media, and the processor 1801 can execute the program instructions to implement the steps in the video denoising method of various embodiments of the present application described above and / or other desired functions.

[0147] In one example, the electronic device 180 can further include an input device 1803 and an output device 1804, which are interconnected through a bus system and / or other forms of connection mechanisms (not shown in the figure). Figure 18

[0148] In addition, the input device 1803 can further include, for example, a keyboard, a mouse, a microphone, etc.

[0149] The output device 1804 can output various information to the outside, which can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, etc.

[0150] Of course, in order to simplify, Figure 18 In the figure, only some of the components in the electronic device 180 related to the present application are shown, and components such as buses, input / output interfaces, etc. are omitted. In addition, the electronic device 180 can further include any other appropriate components according to specific application cases.

[0151] Exemplary computer-readable storage medium

[0152] In addition to the above-mentioned methods and devices, the embodiments of the present application can also be computer program products including computer program instructions, which, when executed by a processor, cause the processor to perform the steps in the video denoising method of any of the above embodiments.

[0153] The computer program product can be written in any combination of one or more programming languages, including object-oriented programming languages, such as Java, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on a user computing device, partially on a user device, as an independent software package, partially on a user computing device and partially on a remote computing device, or entirely on a remote computing device or server. ​

[0154] Furthermore, an embodiment of the present application can also be a computer readable storage medium, having stored thereon, a computer program comprising instructions which, when executed by a computer, cause the computer to carry out the steps described in the above "Exemplary Method" section of the present specification of the video denoising method according to various embodiments of the present application.

[0155] The computer readable storage medium can be a combination of one or more computer readable media. The computer readable media can be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium can include, for example but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium include an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0156] The above describes the basic principles of the present application in conjunction with specific embodiments, but it should be noted that the advantages, benefits, effects and the like mentioned in the present application are only examples and are not limiting, and these advantages, benefits, effects and the like cannot be considered as necessary for each embodiment of the present application. In addition, the above specific details are only for the purpose of example and understanding, and are not limiting, and the above details do not limit the present application to the above specific details.

[0157] The block diagrams of the devices, apparatuses, equipment, systems involved in the present application are only illustrative examples and are not intended to require or imply that the connections, arrangements, configurations must be as shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, equipment, systems can be connected, arranged, configured in any manner. Words such as "include", "contain", "have" and the like are open-ended words, meaning "including but not limited to", and can be used interchangeably. The words "or" and "and" used herein mean the word "and / or", and can be used interchangeably unless the context clearly indicates otherwise. The word "such as" used herein means the phrase "such as but not limited to", and can be used interchangeably.

[0158] It should also be noted that in the devices, apparatuses and methods of the present application, each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be considered as equivalent solutions of the present application.

[0159] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein can be applied to other aspects without departing from the scope of the application. Thus, the present application is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0160] The above description has been presented to enable any person skilled in the art to make or use the application. Numerous modifications and alterations to disclosed

[0161] The above description is presented to enable any person skilled in the art to make or use the application. Numerous modifications and alterations to this application will be readily apparent to those skilled in the art, and it is intended to include all such modifications and alterations insofar as they come within the scope of the general concepts defined herein and the following claims.

Claims

1. A method of video denoising, characterized in that, The method comprises the steps of: determining a video denoising model, wherein the video denoising model is obtained by training an initial network model based on a plurality of noise-increased videos corresponding to a to-be-denoised video and the to-be-denoised video, and the to-be-denoised video comprises a to-be-denoised microscopic video; generating a noise-reduced video corresponding to the to-be-denoised video based on the to-be-denoised video by using the video denoising model; wherein the determination of the video denoising model comprises: generating a plurality of noise-increased videos corresponding to the to-be-denoised video based on the to-be-denoised video; establishing an initial network model and training the initial network model based on the plurality of noise-increased videos and the to-be-denoised video to generate a video denoising model, wherein the video denoising model is used to denoise the to-be-denoised video to generate a noise-reduced video corresponding to the to-be-denoised video; the generation of the plurality of noise-increased videos corresponding to the to-be-denoised video based on the to-be-denoised video comprises: determining noise intensity estimation information of the to-be-denoised video based on the to-be-denoised video, wherein the noise intensity estimation information is estimation information obtained by estimating internal noise contained in the to-be-denoised video; performing N times of noise addition operations on the to-be-denoised video based on the noise intensity estimation information to determine N noise-increased videos corresponding to the to-be-denoised video, wherein N is a positive integer, and the plurality of noise-increased videos contain different amounts of noise intensity estimation information.

2. The method of video de-noising according to claim 1, wherein, the generation of the noise-reduced video corresponding to the to-be-denoised video based on the to-be-denoised video by using the video denoising model comprises: generating a plurality of to-be-denoised video segments corresponding to the to-be-denoised video based on the to-be-denoised video; denoising the plurality of to-be-denoised video segments respectively by using the video denoising model to obtain a plurality of noise-reduced video segments corresponding to the plurality of to-be-denoised video segments respectively; splicing the plurality of noise-reduced video segments corresponding to the plurality of to-be-denoised video segments respectively to generate a noise-reduced video corresponding to the to-be-denoised video.

3. The method of video de-noising according to claim 1, wherein, the generation of the plurality of noise-increased videos corresponding to the to-be-denoised video based on the to-be-denoised video comprises: generating a plurality of noise-increased videos corresponding to the to-be-denoised video based on the to-be-denoised video by using a noise model, wherein the plurality of noise-increased videos contain different noise intensities, and the noise model comprises any one of the following models: an additive noise prior model, a multiplicative noise prior model, and an additive-multiplicative compound prior model.

4. The method of video de-noising according to claim 1, wherein, the determination of the noise intensity estimation information of the to-be-denoised video based on the to-be-denoised video comprises: determining at least one video image corresponding to the to-be-denoised video based on the to-be-denoised video; generating a plurality of image regions corresponding to the at least one video image based on the at least one video image; calculating a gray mean value and a gray variance corresponding to each of the plurality of image regions; determining the noise intensity estimation information based on the gray mean value and the gray variance corresponding to each of the plurality of image regions.

5. The method of video de-noising according to claim 4, wherein, the determination of the noise intensity estimation information based on the gray mean value and the gray variance corresponding to each of the plurality of image regions comprises: performing a linear regression operation on the gray mean value and the gray variance corresponding to each of the plurality of image regions to determine the noise intensity estimation information of the to-be-denoised video.

6. The method of video de-noising according to claim 5, wherein, The linear regression operation is performed based on the gray mean value and the gray variance of each of the plurality of image regions to determine noise intensity estimation information of the video to be denoised, and the noise intensity estimation information of the video to be denoised includes: Based on the gray mean value and the gray variance of each of the plurality of image regions, a data point graph of the gray mean value and the gray variance is established, and the data point graph includes a plurality of data points; The linear regression operation is performed on the plurality of data points to determine a regression straight line function; Based on the regression straight line function, the noise intensity estimation information of the video to be denoised is determined, wherein the noise intensity estimation information of the video to be denoised includes Poisson noise intensity, Gaussian noise mean value and Gaussian noise variance.

7. The method of video de-noising according to claim 1, wherein, The initial network model is trained based on the plurality of noise-increased videos and the video to be denoised to generate a video denoising model, and the video denoising model includes: A loss function of the initial network model is constructed, wherein the loss function includes a time domain smoothing kernel norm regular term and a spatial domain smoothing entropy regular term; The initial network model is trained based on the plurality of noise-increased videos, the video to be denoised and the loss function to generate a video denoising model.

8. An apparatus for video denoising, the apparatus comprising: It includes: A determination module configured to determine a video denoising model, wherein the video denoising model is obtained by training an initial network model based on a plurality of noise-increased videos corresponding to a video to be denoised and the video to be denoised, and the video to be denoised includes a video to be denoised; A denoising module configured to utilize the video denoising model to generate a noise-reduced video corresponding to the video to be denoised based on the video to be denoised; Wherein, the determination of the video denoising model includes: Based on the video to be denoised, a plurality of noise-increased videos corresponding to the video to be denoised are generated; An initial network model is established, and the initial network model is trained based on the plurality of noise-increased videos and the video to be denoised to generate a video denoising model, wherein the video denoising model is used to denoise the video to be denoised to generate a noise-reduced video corresponding to the video to be denoised; The plurality of noise-increased videos corresponding to the video to be denoised are generated based on the video to be denoised, and the plurality of noise-increased videos include: Based on the video to be denoised, noise intensity estimation information of the video to be denoised is determined, and the noise intensity estimation information is estimation information obtained by estimating internal noise contained in the video to be denoised; Based on the noise intensity estimation information, the noise addition operation is performed on the video to be denoised N times to determine N noise-increased videos corresponding to the video to be denoised, wherein N is a positive integer, and the plurality of noise-increased videos contain different amounts of noise intensity estimation information.

9. A computer-readable storage medium, characterized in that, The storage medium stores instructions, and when the instructions are executed by the processor of the electronic device, the electronic device can execute the video denoising method of any one of claims 1 to 7.

10. An electronic device, comprising: The electronic device includes: A processor; A memory for storing computer executable instructions; The processor is configured to execute the computer executable instructions to implement the video denoising method of any one of claims 1 to 7.

Images