Video denoising with uncalibrated cameras

Abstract

According to an aspect, a method includes identifying frames of video data, identifying background portions in a subset of the frames, and determining an composite background based on the background portions for the subset. The method further includes determining at least one noise parameter for the video data based on the composite background and a model. The method also includes updating the video data based on the at least one noise parameter and the model.

Description

VIDEO DENOISING WITH UNCALIBRATEDCAMERASCROSS-REFERENCE TO RELATED APPLICATION

[0001] This application claims priority- to U.S. Provisional Patent Application No. 63 / 579,811, filed on August 30, 2023, entitled “FAST VIDEO DENOISING WITH UNCALIBRATED CAMERAS,” the disclosure of which is incorporated by reference herein in its entirety.BACKGROUND

[0002] In video data, noise refers to unwanted random variations in brightness or color, which degrade video quality-. Common ty pes include salt-and-pepper noise (random white and black pixels), Gaussian noise (following a Gaussian distribution affecting every pixel, often from electronic circuit noise), Poisson noise (statistical fluctuations in low-light conditions), speckle noise (granular interference noise common in radar and laser systems), and temporal noise (variations between frames due to lighting changes, movement, or camera electronics). Each type of noise can impact the clarity and visual quality- of the video.SUMMARY

[0003] This disclosure relates to systems and methods for correcting noise in video data. In at least one implementation, a computing device is configured to determine noise parameters that can be applied to frames of video data to correct noisy input. In determining the noise parameters, the computing device can select a subset of frames from the video data and process the frames to generate a composite background (e g., average background) for the frames. From the average background, the computing device can implement a denoising process on individual frames and compare the denoised individual frames to the average frame to update noise parameters for the individual video (or video camera). The noise parameters may include light level, sensor additive or read noise, a gamma correction coefficient, or some other parameters. In some implementations, the noise parameters provide normalization and inverse normalization on the frames used before and after the denoising model. In some examples, the computing device processing the video can be provided with a denoise model that is maintained on the device. At the same time, the normalizing andinverse normalizing functions are updated (via the noise parameters) to correct the individualized noise associated with the video source of the video. Once the parameters are updated, the system can process the frames of the video and provide an updated version of the frames for viewing that corrects the noise associated with the capturing camera.

[0004] In some aspects, the techniques described herein relate to a method including: identifying a plurality of frames of video data: identifying plurality of background portions in a subset of the plurality of frames; determining a composite background based on the plurality of background portions for the subset; determining at least one noise parameter for the video data based on the composite background and a model; and updating the video data based on the at least one noise parameter and the model.

[0005] In some aspects, the techniques described herein relate to a computing apparatus including: computer-readable storage media; at least one processor operatively coupled to the computer-readable storage media; and program instructions stored on the computer-readable storage media that, when executed by the at least one processor, direct the computing apparatus to perform a method, the method including: identify ing a plurality of frames of video data; identifying a plurality of background portions in a subset of the plurality of frames; determining a composite background based on the plurality of background portions for the subset; determining at least one noise parameter for the video data based on the composite background and a model; and updating the video data based on the at least one noise parameter and the model.

[0006] In some aspects, the techniques described herein relate to a computer-readable storage medium having program instructions stored thereon that, when executed by at least one processor, direct the at least one processor to perform a method, the method including: identify ing a plurality of frames of video data: identifying a plurality of background portions in a subset of the plurality of frames; determining a composite background based on the background portions for the subset; determining at least one noise parameter for the video data based on the composite background and a model, wherein determining the least one noise parameter for the video data based on the composite background and the model includes: applying a first noise parameter and the model to a first frame in the subset to generate a second frame; determining a difference between the first frame and the composite background; and updating the first noise parameter to a second noise parameter, the second noise parameter included as the at least one noise parameter; and updating the video data based on the at least one noise parameter and the model.

[0007] The accompanying drawings and the description below set forth the details of one or more implementations. Other features will be apparent from the description, drawings, and claims.BRIEF DESCRIPTION OF THE DRAWINGS

[0008] FIG. 1 illustrates a flow diagram for updating noise parameters associated with denoising video data according to an implementation.

[0009] FIG. 2 illustrates a method of operating a computing device to identify noise parameters and use the noise parameters to correct video data according to an implementation.

[0010] FIG. 3 illustrates an operational scenario

[0011] FIG. 4 illustrates a timing diagram for correcting noise in video data according to an implementation.

[0012] FIG. 5 illustrates an operational scenario of generating a denoising model according to an implementation.

[0013] FIG. 6 illustrates a computing system that determines noise parameters associated with denoising video data according to an implementation.DETAILED DESCRIPTION

[0014] This disclosure relates to systems and methods for denoising video data on computing devices. In video data, noise refers to unwanted random variations in brightness or color that degrade the video’s quality. This noise can originate from various sources, including the video capture process, transmission errors, and environmental factors. Common ty pes of noise include salt-and-pepper noise (random white and black pixels), Gaussian noise (random pixel variations from electronic sources), Poisson noise (statistical fluctuations in photon arrivals), speckle noise (granular interference in radar and laser systems), and temporal noise (frame-to-frame variations due to lighting or movement). These disturbances can affect video clarity and necessitate noise reduction techniques to improve visual quality.

[0015] In some implementations, computers can be configured to use algorithms to reduce or eliminate noise while preserving important details or features. Techniques for denoising can include spatial filtering, which smooths pixel values within individual frames to reduce noise, and temporal filtering, which uses information from adjacent frames to distinguish between noise and actual motion. Advanced methods like wavelet transforms, deep learning-based approaches, and non-local means filtering analyze patterns andcorrelations in the video to more effectively remove noise without blurring important details. The goal of denoising is to enhance video clarity and quality while maintaining the integrity of the original content. However, a technical problem exists in the computing resources required to perform the denoising process, including processing and memory resources.

[0016] As at least one technical solution, a computing device can be configured to deploy a denoising model that denoises video data based on parameters identified from a subset of the frames of the video data. In at least one example, the computing device can be configured to identify frames of video data and identify background portions in the video data. In some examples, the video data can correspond to a video with a stationary background (e.g., a video call). In some implementations, the device can identify the background portions by separating static elements from moving objects to distinguish the scene’s background. This process often uses algorithms that analyze pixel-level changes over time, applying operations like background subtraction, where a model of the background is continuously updated and compared against new frames to detect differences. Methods such as Gaussian Mixture Models (GMM), frame differencing, and optical flow can help identify and track static background elements by filtering out foreground objects. In some examples, more advanced approaches, including deep learning-based methods, can enhance accuracy by learning complex patterns and features from the video data to identify and separate the background from dynamic elements. Once the background portions are identified in the subset, the device can be configured to determine a composite background, which may comprise an average background in some examples, based on the background portions. For example, the composite background may be the average of the background portions. A pixel of the composite background may have a value that is an average of the values of the corresponding pixels of the background portions. The background portions may have the same shape and / or location in the respective frames. The averaged background can then be used as a ground-truth image that can be compared to denoised frames to determine updates to the at least one noise parameter associated with the denoising process.

[0017] As an illustrative example, a device can identify video data frames associated with a web camera for a video call. The device can select a subset of the frames (e.g., the first fifty frames) and extract the background portions from the subset of frames. In some implementations, the device will extract the full background. In other implementations, the device will extract a portion of the complete background. Once the backgrounds are extracted from the subset of frames, the device then averages the backgrounds to generate a groundtruth image. In machine learning, ground truth refers to the accurate, real-world data or labelsused as a benchmark to train and evaluate models. It represents the correct answers (i.e., target) that a model aims to predict and is crucial for assessing the performance and accuracy of the algorithm.

[0018] In some implementations, to generate the composite background, the device linearizes the noisy frames, averages the noisy frames, and then applies gamma correction to the averaged frame to generate the completed ground-truth average frame for the video data. Linearizing a frame of video data involves converting the pixel values from their onginal, often non-linear format (such as gamma-corrected or log-encoded) to a linear format, representing light’s true intensity. This process can include several steps. Initially, the device applies an inverse gamma correction if the frame uses gamma encoding, which involves raising pixel values to the power of the reciprocal of the gamma value. For other encodings, such as log encoding, the device applies an exponential function to revert the values to a linear scale. Once linearized, the pixel values more accurately reflect the actual light intensity.

[0019] After performing the linearization, the device can be configured to average the linearized frames. In some examples, frames are averaged by flattening each frame into a one-dimensional array of pixel values. Then, corresponding pixel values from each frame’s array are summed together and divided by the number of frames to compute the element- wise average. The result is a single one-dimensional array that represents the average of the linearized frames.

[0020] Once linearized and averaged, the device can be configured to perform gamma correction that adjusts brightness and contrast by compensating for the non-linear response of display devices or sensors. It involves applying a gamma curve to the pixel values, which can either brighten or darken the image to correct for these non-linearities and ensure an accurate representation of the visual information. The result is that the averaged background or the ground-truth image can then be used to compare and generate the noise parameters to correct the frames of video data. Although this is one example of generating a composite background, a composite background can be generated using various weightings for the pixel values from the various frames. For example, a system may use a most often occurring value or mode of the pixel values to select the values for the composite background. Other weightings can also be applied to the pixel values across multiple frames.

[0021] After the computing device determines the average background, at least one technical solution includes the device applying first noise parameters and a model (i.e., denoising model) to a frame to generate a denoised frame, comparing the denoised frame tothe average background and updating the first noise parameters to second noise parameters based on the comparison. The noise parameters can include light level, sensor additive noise, a gamma correction coefficient, or some other parameter that is used in association with normalizing or inverse normalizing the pixels associated with the frame. Light level refers to the overall brightness of an image or video frame, which is adjusted to ensure consistent exposure and illumination across different frames or scenes. Sensor additive noise refers to random variations in the electrical signal produced by a sensor, which are supenmposed on the actual signal, degrading the accuracy and quality of the captured data. The gamma correction coefficient is a numerical value that defines the non-linearity of the gamma curve applied to pixel values and is used to adjust the brightness and contrast of an image or video for accurate display or processing.

[0022] In some implementations, the device can be configured to normalize a frame of the video data using the noise parameters, denoise the normalized frame using the denoising model, and implement an inverse normalization on the denoised frame to generate a restored frame (i.e., a frame corrected using the denoising model). The device then compares elements in the restored frame to the ground-truth image, such as comparing the elements in the background to determine updates associated with the noise parameters for the normalization and denormalization operations. The process can be iteratively repeated to regressively update the parameters, such that the normalizing, denoising, and denormalization process accurately reflects the ground-truth image generated from the average of the frames.

[0023] Once the noise parameters are determined, the noise parameters and denoising model can be applied to the frames of the video. In at least one implementation, the device is configured to, for each frame, normalize the frame using the noise parameters, apply the denoising model to the frame, and inverse normalize the frame following the application of the denoising model. As at least one technical effect, the systems and methods permit a subset of frames to be used in configuring noise parameters to accurately denoise the overall video.

[0024] FIG. 1 illustrates a flow diagram 100 for updating noise parameters associated with denoising video data according to an implementation. Flow diagram 100 includes frame 1 10, normalize operation 120, denoise operation 122, inverse normalization operation 124, restored frame 112, noise parameters 150, compare operation 130, and averaged background 114. The operations in flow diagram 100 can be performed by one or more computing devices, such as desktop computers, server computers, laptop computers, smartphones, tablets, or other computing devices.

[0025] In flow diagram 100, normalize operation 120 is applied to frame 110 to generate a normalized frame. Normalizing a frame refers to the process of adjusting the values within a dataset or an image so that they fall within a standard range (e.g., between zero and one). This can be achieved by subtracting the minimum value and dividing by the range (maximum value minus minimum value). Normalization is used in fields such as image processing because it ensures that the data is scaled consistently, which can improve the performance and accuracy of algorithms. Normalizing the data can reduce differences in scale, making it easier to compare and process the data uniformly. In some implementations, the normalization includes applying one or more noise parameters 150 as part of a function to adjust the values for the various pixels in the image. Noise parameters 150 can include light level, sensor additive noise, a gamma correction coefficient, or some other parameter that is used in association with normalizing or inverse normalizing the pixels associated with the frame. Light level refers to the overall brightness of an image or video frame, which is adjusted to ensure consistent exposure and illumination across different frames or scenes. Sensor additive noise refers to random variations in the electrical signal produced by a sensor, which are superimposed on the true signal, degrading the accuracy and quality of the captured data. The gamma correction coefficient is a numerical value that defines the nonlinearity of the gamma curve applied to pixel values, used to adjust the brightness and contrast of an image or video for accurate display or processing.

[0026] After frame 1 10 is normalized, denoise operation 122 is applied to the normalized frame. Denoise operation 122 represents a model that processes the values of the normalized frame to remove noise associated with the frame. Noise in video refers to random variations in brightness or color information that appear as grainy or speckled artifacts, often caused by low light conditions, high ISO settings, or electronic interference. This unwanted visual distortion can degrade the quality and clarity of the video, making it harder to discern details and reducing overall visual fidelity. Denoise operation 122 represents a model designed to remove unwanted noise from video frames while preserving essential details and improving visual quality. Denoise operation 122, which can be based on machine learning techniques such as convolutional neural networks (CNNs) or autoencoders, is trained on datasets (or large datasets) of noisy and clean video pairs. The model can predict and subtract noise from new video frames by learning the noise patterns and underlying clean signals. The goal is to achieve a smoother, clearer output by effectively distinguishing between noise and actual video content, improving the overall viewing experience. In some implementations, the configuration of denoise operation 122 is maintained, while noise parameters 150 aredynamic to change the normalization and denormalization of the frame. Thus, rather than updating the model of denoise operation 122, the inputs (i. e. , normalized frame) are changed via noise parameters 150. The noise parameters can be updated so that denoise operation 122 can be applied to different cameras with different noise characteristics.

[0027] To update the parameters, after denoise operation 122 is applied, inverse normalization operation 124 is performed on the denoised frame to generate restored frame 112. Inverse normalization for a frame of video refers to the process of converting pixel values that have been normalized (e.g., zero and one) back to their original scale. This step is necessary' to return the processed frame to its original color or brightness levels. Once restored frame 112 is determined, compare operation 130 compares restored frame 112 to averaged background 114.

[0028] In some implementations, the device can be configured to select a subset of frames from the video data and determine an averaged background 114 from the subset of frames. In some examples, the device can extract the background from the subset of frames and average the background across the different frames to create an averaged background 114.

[0029] In some implementations, determining averaged background 114 includes linearizing the subset of frames (or the background extracted from the frames) by inverse gamma, averaging the linearized frames, and applying a gamma correction to the averaged frame to generate the averaged background 114. Linearizing by inverse gamma can include converting gamma-corrected pixel values from the frames to their original linear values. Gamma correction is applied to adjust the image’s brightness and contrast to match the nonlinear response of display devices. However, for precise editing or color grading, working with linear values is used because the values represent light intensity more accurately. To achieve this, the inverse gamma function can be applied to the gamma-corrected values, effectively reversing the correction and restoring a linear relationship between pixel values and light intensity7. Once linearized, the device can average the linearized frames and apply a gamma correction to generate the final averaged background 114 (i.e., ground-truth frame for the video). Gamma correction can demonstrate how colors and brightness are represented to ensure that images display correctly on screens and match human visual perception while preserving detail in both bright and dark areas. Gamma correction can be based on a gamma curve, which maps input values to output values in a way that compensates for the nonlinear response of display devices.

[0030] Once averaged background 114 is generated, the device can compare a restored frame 112 to averaged background 114 and update noise parameters 150 based on the comparison. The update is used to improve the abi li ty of a restored frame to match the averaged background 114. In some examples, the process is iteratively repeated to improve noise parameters 150. After iteratively improving noise parameters 150, the device can apply the updated noise parameters and denoise operation to any of the frames in the video data to remove the noise from the video data.

[0031] Although demonstrated in the example of FIG. 1 as using the entire background in comparison with restored frame 112, a smaller portion of the background can be compared between the restored frame and the averaged background in some examples. For example, the right side of the restored frame can be compared to the average right side of the background from the video data. Further, while demonstrated in FIG. 1 using an averaged background, a system can use any form of composite background. The composite background can use various weightings associated with the different pixel values in the backgrounds. For example, the composite background can be generated using the mode of the pixels to generate the ground-truth frame for companson. Thus, in some examples, different combinations or compositions of the backgrounds can be used to generate the composite frame.

[0032] FIG. 2 illustrates method 200 of operating a computing device (e.g.. computing device 600 of FIG. 6) to identify noise parameters and use the noise parameters to correct video data according to an implementation. The steps of method 200 can be performed by one or more computing devices, such as desktop computers, server computers, laptop computers, smartphones, tablets, or other computing devices.

[0033] Method 200 includes identifying frames of video data at step 201 and identifying background portions in a subset of the frames at step 202. In some implementations, the frames correspond to video with a consistent background, such as a video call where the user is in front of a consistent background. For example, the position of the computing device relative to its environment, e.g.. relative to the part of the environment representing the background, is constant over some or all the identified video frames. The background portions can correspond to the entire background (e g., all portions outside of the user) or a subset of the background. The background in the frames can be identified using advanced image processing algorithms and machine learning models. These techniques analyze various image elements, such as color, texture, and edges, to distinguish the subject from the background. In some examples. Convolutional Neural Netw orks (CNNs) can beused for this purpose, as they can be trained on large datasets to recognize patterns and separate the foreground from the background.

[0034] After the background portions are identified in the subset of frames, method 200 further includes determining a composite background based on the background portions for the subset at step 203. In some implementations, determining the composite background includes determining an average background of the selected frames. In some implementations, determining the average background includes linearizing the subset of frames by inverse gamma, averaging the linearized frames, and applying a gamma correction to the averaged frame to generate the averaged background. Linearizing by inverse gamma can include converting gamma-corrected pixel values from the frames to their original linear values. Gamma correction is applied to adjust the image’s brightness and contrast to match the nonlinear response of display devices. However, working with linear values can be used for precise editing or color grading because the values represent light intensity more accurately. To achieve this, the inverse gamma function can be applied to the gammacorrected values, effectively reversing the correction and restoring a linear relationship between pixel values and light intensity. Once linearized, the method can average the linearized frames and apply a gamma correction to generate the final averaged frame (i.e., ground-truth frame for the video). Gamma correction can demonstrate how colors and brightness are represented to ensure that images display correctly on screens and match human visual perception while preserving detail in both bright and dark areas. Gamma correction can be based on a gamma curve, which maps input values to output values in a way that compensates for the nonlinear response of display devices.

[0035] After the composite background is determined, method 200 further includes determining at least one noise parameter for the video data based on the composite background and a model at step 204. In some implementations, a frame (or a portion of a frame) is normalized. Normalizing a frame of video involves adjusting its pixel values to a consistent range to ensure uniformity across all frames and facilitate accurate processing. This process can start with converting the frame’s pixel values from their original format to a floating-point representation. Then, the pixel values are scaled based on the minimum and maximum values present in the frame or across a series of frames to standardize the brightness and contrast. In some implementations, the normalization can include a function that uses various noise parameters to normalize the noise associated with a corresponding pixel. The noise parameters can include light level, sensor additive noise, a gamma correction coefficient, or some other noise parameter.

[0036] In at least one example, the following equation can be used (y) = LyY+ cr2where y represents a noisy frame of the video data, L represents the light level, y represents the gamma correction coefficient, and <r represents the sensor additive noise standard deviation. The equation is used to change the values of the pixels such that they are normalized for processing by a denoising model.

[0037] Once a frame is normalized, the device or devices can implement the denoising operation on the normalized frame. The denoising operation can comprise a machine learning model trained to distinguish between noise and the actual signal of the desired image. In some implementations, the process starts by collecting a large dataset of paired noisy and clean frames. These frames are normalized to a standard range (e.g., zero to one) to ensure consistent input for the model. An example choice for such tasks is a CNN due to its effectiveness in handling spatial data. The network is trained using a loss function that penalizes differences between the denoised output and the clean reference frame, sometimes using metrics like mean squared error (MSE) or perceptual loss. During training, the model learns to map noisy inputs to clean outputs by identify ing and reducing noise patterns while preserving important features and details. After training, the model can be used on the frames of video data from different types of cameras. However, the noise parameters used to normalize the pixels of the frames can be adjusted, such that the same denoising model can be applied to different sensors and cameras.

[0038] After the frame is processed using the denoising model or operation, the denoised frame is then processed using an inverse normalization operation. The inverse normalization is used to revert the normalized pixel values back to their original scale after processing. Normalization can be used to scale pixel values to a standard range (e.g.. zero to one) to facilitate operations, such as machine learning processing to remove noise. Once these operations are completed, inverse normalization is applied to restore the pixel values to their original range. The inverse normalization can use the same noise parameters as the normalization process to return the pixels to the desired range. Once the inverse normalization operation is completed, the device or devices can compare the restored frame from the inverse normalization operation (e.g., the background from the restored frame) to the ground truth background determined from the averaging operation. The comparison can be used to update the at least one noise parameter associated with the normalization and denormalization process to improve the ability of the denoising operation to remove noise from the video. In some implementations, the process can be iteratively repeated, or aregressive operation is performed that applies the updated noise parameters to a new frame and compares the processed frame (e.g., denoised frame) to the averaged frame (i.e., groundtruth truth frame). The technical effect of performing regression allows the parameters to be updated over multiple iterations to more accurately generate a similar background to the ground truth. In some implementations, the improvement iterations can be quantified by an administrator (e.g.. 500 iterations). In some implementations, the improvement iterations (e.g.. the number thereof) can be determined based on how close a processed frame is to the ground -truth average frame. Accordingly, while starting with first values for the noise parameters, which can be provided with the denoising model, method 200 can update the first values to second values based on the comparison with the ground truth frame. Thus, determining one or more noise parameters for the video data based on the composite (e.g., average) background and the model (e.g., denoising model) may comprise generating a denoised frame using the model (e.g., after normalizing the frame and before inversenormalizing the denoised frame), and determining a difference between a resulting restored frame and the composite background. Therein, the one or more noise parameters may be used to determine the restored frame. By varying the one or more noise parameters the difference between the restored frame and the composite background may be minimized and the one or more noise parameters resulting in the minimal difference can be used as the determined one or more noise parameters.

[0039] Once the noise parameters are determined, method 200 further includes updating the video data based on the at least one noise parameter and the model at step 205. In some implementations, the device or devices can be configured to normalize a frame of the video data using the determined noise parameters, denoise the frame using the model, and inversely normalize the frame to generate an updated frame. The technical effect permits the noise parameters to be calculated using a subset or smaller portion of the frames from the video data and then update the entirety of the video data using the determined noise parameters. In some implementations, these operations permit the denoising parameters to be calculated from a small portion of the video and then applied to additional frames on the premise that the background does not change for the video data.

[0040] Although the above example demonstrates using an averaged frame as the composite frame, different composite frames can be used in some examples. The composite frame can use different weightings associated with the pixels to provide a combination or composite from the set of frames.

[0041] FIG. 3 illustrates an operational scenario 300 for generating an average background according to an implementation. Operational scenario 300 includes frames 310, backgrounds 312, operation 320, and averaged frame 330. Any device or system of computing devices, such as desktop computers, servers, laptop computers, tablets, smartphones, and the like can perform operational scenario 300, e.g., the computing device 600 of FIG. 6.

[0042] In operational scenario 300, frames 310 are extracted from a video. Frames 310 may comprise a set of first frames from the video, a set of randomly selected frames from the video, frames from the end of the video, or frames from some other portion of the video. Once frames 310 are selected, backgrounds 312 are extracted from frames 310. Background extraction from video frames involves separating the static parts of a scene from the moving elements. This can be accomplished in some implementations by creating a background model from several frames and comparing each new frame to this model. Pixels that differ significantly from the model are labeled as moving objects (foreground), while similar pixels are considered part of the static background. Adaptive techniques and advanced methods like Gaussian Mixture Models (GMM) or deep learning allow the background model to adjust over time, handling changes in lighting or new stationary objects to improve accuracy in dynamic environments. For example, in a video call, an application can identify a set of frames from a video and extract the user (i.e., non-background portion).

[0043] Once the backgrounds are extracted from the set of frames, the system then performs operation 320 on backgrounds 312. Operation 320 processes and averages the frames to generate averaged frame 330. In some examples, operation 320 linearizes the noisy frames of backgrounds 312, averages the noisy frames, and then applies gamma correction to the averaged frame to generate the completed ground-truth averaged frame 330 for the video data. Linearizing a frame of video data can involve converting the pixel values from their original, often non-linear format (such as gamma-corrected or log-encoded) to a linear format, representing light’s true intensify. This process can include several steps. Initially, the device applies an inverse gamma correction if the frame uses gamma encoding, which involves raising pixel values to the power of the reciprocal of the gamma value. The device applies an exponential function to revert the values to a linear scale for other encodings, such as log encoding. Once linearized, the pixel values more accurately reflect the actual light intensify.

[0044] After performing the linearization, operation 320 can average the linearized frames. In some examples, frames are averaged by flattening each frame into a one-dimensional array of pixel values. Then, corresponding pixel values from each frame’s array are summed together and divided by the number of frames to compute the element-wise average. The result is a single one-dimensional array that represents the average of the linearized frames.

[0045] Once linearized and averaged, operation 320 can be configured to perform gamma correction that adjusts brightness and contrast by compensating for the non-linear response of display devices or sensors. It involves applying a gamma curve to the pixel values, which can either brighten or darken the image to correct for these non-linearities and ensure an accurate representation of the visual information. The result, averaged background 330, can then be used as the ground-truth image to compare to other frames and update the noise parameters associated with the normalizing and inverse normalization operations as described further in FIG. 2.

[0046] In some implementations, the noise parameters include light level, sensor additive noise, a gamma correction coefficient, or some other parameter that is associated with normalizing or inverse normalizing the pixels associated with the frame. Light level refers to the overall brightness of an image or video frame, which is adjusted to ensure consistent exposure and illumination across different frames or scenes. Sensor additive noise refers to random variations in the electrical signal produced by a sensor, which are superimposed on the true signal, degrading the accuracy and quality of the captured data. The gamma correction coefficient is a numerical value that defines the non-linearity of the gamma curve applied to pixel values, used to adjust the brightness and contrast of an image or video for accurate display or processing.

[0047] FIG. 4 illustrates a timing diagram 400 for correcting noise in video data according to an implementation. Timing diagram 400 includes video data 405 with first frames and second frames 420, denoise application 430, and updated frames 440. Timing diagram 400 is representative of operations that can be performed by any computing device or combination of computing devices, e.g., the computing device 600 of FIG. 6.

[0048] In timing diagram 400, denoise application 430 identifies first frames 410 at step 1. Frames 410 can comprise the first frames of a video, the last frames of the video, randomly selected frames, or some other segment of frames from video data 405. For example, in a video streaming application, frames 410 can represent a first set of frames for the streamed video for the application. Denoise application 430 then determines the average frame or average background frame from first frames 410 (i.e., ground-truth frame). In some implementations, in determining the average frame, denoise application 430 can beconfigured to linearize the noisy frames, average the noisy frames, and then apply gamma correction to the averaged frame to generate the completed ground-truth average frame for the video data. Linearizing a frame of video data involves converting the pixel values from their original, often non-linear format (such as gamma-corrected or log-encoded) to a linear format, representing light’s true intensity'. This process can include several steps: first, the device applies an inverse gamma correction if the frame uses gamma encoding, which involves raising pixel values to the power of the reciprocal of the gamma value. For other encodings, such as log encoding, the device applies an exponential function to revert the values to a linear scale. Once linearized, the pixel values reflect the actual light intensity more accurately.

[0049] After performing the linearization, denoise application 430 can be configured to average the linearized frames. In some examples, frames are averaged by flattening each frame into a one-dimensional array of pixel values. Then, corresponding pixel values from each frame’s array are summed together and divided by the number of frames to compute the element-wise average. The result is a single one-dimensional array that represents the average of the linearized frames.

[0050] Once linearized and averaged, denoise application 430 can be configured to perform gamma correction that adjusts brightness and contrast by compensating for the nonlinear response of display devices or sensors. It involves applying a gamma curve to the pixel values, which can either brighten or darken the image to correct for these non-linearities and ensure an accurate representation of the visual information. The result (i.e., averaged background) can then be used as the ground-truth image to compare and generate the noise parameters associated with the normalizing and inverse normalization operations. Other operations can be used to determine the averaged background across multiple frames.

[0051] After denoise application 430 determines the average frame, denoise application 430 can further be configured to identify a first frame from first frames 410 (or a portion of the frame) at step 3 and use the first frame to update noise parameters at step 4. In some implementations, denoise application 430 can normalize the first frame and apply a denoising model to the normalized first frame. In some examples, the denoising model can comprise a denoising machine learning model. A machine learning model for denoising is designed to remove noise from data, such as images, to improve their quality'. These models can be configured to learn to distinguish between the underlying signal and noise by being trained on large datasets that contain both noisy and clean versions of the data. Common approaches include autoencoders, which compress and then reconstruct data, and CNNs,which can be effective in image denoising due to their ability to capture spatial patterns. Once trained, the model can predict the clean version of new, noisy data, effectively “denoising” it. Once a normalized frame is processed using the denoising model, the denoised frame can be processed using inverse normalization to generate a restored frame without noise.

[0052] Normalization and inverse normalization can be performed in some implementations using one or more noise parameters. The parameters are used to provide normalization before the model and inverse normalization following the model. The noise parameters can include light level, sensor additive noise, a gamma correction coefficient, or some other parameter that is used in associated with normalizing or inverse normalizing the pixels associated with the frame. Light level refers to the overall brightness of an image or video frame, which is adjusted to ensure consistent exposure and illumination across different frames or scenes. Sensor additive noise refers to random variations in the electrical signal produced by a sensor, which are superimposed on the true signal, degrading the accuracy and quality of the captured data. The gamma correction coefficient is a numerical value that defines the non-linearity of the gamma curve applied to pixel values, used to adjust the brightness and contrast of an image or video for accurate display or processing. The normalization values can be included as an equation to normalize the pixels from the frame before being processed by the model (and provide inverse normalization follow ing processing by the model).

[0053] After the restored frame is generated using the model and the normalization operations, the restored frame is compared to the averaged frame to update the noise parameters. In some examples, the update is part of a regression operation that compares restored frames (restored using improved noise parameters from a previous iteration) to the average frame. Regression can be used to update noise parameters associated with the normalization for the model by minimizing the difference between predicted and actual values by minimizing a loss function such as Mean Squared Error (MSE). In the training process, denoise application 430 iteratively adjusts the noise parameters (like w eights in linear regression) to find the optimal values that best fit the data. This can be achieved through optimization techniques such as gradient descent, which calculates the gradient of the loss function with respect to the parameters and updates them in the direction that reduces the error. This process continues until the model parameters converge to values that minimize the loss function, resulting in a model that can make accurate predictions on new data. The technical effect is that as the noise parameters are updated using additional comparisons ofrestored frames to the averaged frame, the noise parameters more closely resemble the average frame.

[0054] After the noise parameters are adjusted in association with normalizing and inverse normalizing functions, denoise application 430 processes frames from video data 405 to generate updated frames 440 at step 5. In some implementations, denoise application 430 will linearize the frames using the identified noise parameters from step 4, process the linearized frames using the denoising model, and inverse linearize the frames to generate updated frames 440.

[0055] FIG. 5 illustrates an operational scenario 500 of generating a denoising model according to an implementation. Operational scenario 500 includes noisy frames 510, clean frames 512, model generation application 530, and denoising model 540. Model generation application 530 represents an application that can be executed by one or more computing devices to generate denoising model 540, e.g., the computing device 600 of FIG. 6.

[0056] In operational scenario 500, model generation application 530 processes noisy frames 510 and clean frames 512 to generate denoising model 540. Denoising model 540 can represent a machine learning model designed to remove noise from video frames, enhancing their visual quality. These models can be built using deep learning architectures like CNNs that are trained to distinguish and eliminate noise while preserving important details in each frame. Denoising model 540 can be trained using a dataset of noisy and clean video frames. The clean frames can be obtained from high-quality video sources, while noisy frames can be generated by artificially adding various types of noise (e g., Gaussian, salt-and-pepper) to these clean frames. The model is then trained to map the noisy frames to their clean counterparts. During training, the model minimizes a loss function that quantifies the difference between the denoised output and the clean ground truth, typically using metrics like Mean Squared Error (MSE) or Structural Similarity Index (SSIM) to guide optimization.

[0057] Various techniques can be employed during training to enhance the effectiveness of denoising model 540. Data augmentation methods, such as adding noise at different levels and types, can help the model generalize better to various real-world scenarios. Additionally, temporal consistency can be enforced by incorporating information from multiple frames through three-dimensional convolutions or recurrent networks that can model the temporal dynamics in video sequences. This ensures that the model not only reduces noise in individual frames but also maintains the coherence and smoothness of the video over time. Regularization techniques like dropout and batch normalization may alsoprevent overfitting and improve the model’s generalization to unseen and noisy video content.

[0058] In some implementations, to implement a denoising model 540, a computing device or system of computing devices can be configured to normalize the data from a frame before implementing the model. Normalizing is used to modify the pixel values within the frame to fall within a specific range, such as 0 to 1 or 0 to 255, depending on the application. This process ensures consistent brightness and contrast across frames, making analyzing, processing, or comparing the video data easier for denoising model 540. In normalizing the values, a function can be applied to the pixel values, which include one or more noise parameters. The noise parameters can include light level, sensor additive noise, a gamma correction coefficient, or some other parameter that is used in association with normalizing or inverse normalizing the pixels of the frame. A frame can be processed using the normalization function, processed using the denoising model 540, and then processed using an inverse normalization function to generate a corrected frame.

[0059] In some implementations, during training the neural network can be represented as the following: arg nun £(fi(f(y)), f(x*)) where x* is the clean image and y is a normal frame. The model is used to take in noisy images with ground-truth or known noise parameters for normalization and generate a clean image.

[0060] Once the training model is generated, in some examples, the model is frozen or remains unchanged and the noise parameters are modified to correct for a particular camera capturing a video (e.g., web camera).

[0061] FIG. 6 illustrates a computing system that determines noise parameters associated with denoising video data according to an implementation. Computing system 600 can represent any computing system or systems with which the various operational architectures, processes, scenarios, and sequences disclosed herein for denoising video may be implemented. Computing system 600 includes storage system 645, processing system 650, communication interface 660, input / output (I / O) device(s) 670. Processing system 650 is operatively linked to communication interface 660, I / O device(s) 670, and storage system 645. In some implementations, communication interface 660 and / or I / O device(s) 670 may be communicatively linked to storage system 645. Computing system 600 may further include other components such as a battery and enclosure that are not shown for clarity.

[0062] Communication interface 660 comprises components that communicate over communication links, such as network cards, ports, radio frequency, processing circuitry andsoftware, or some other communication devices. Communication interface 660 may be configured to communicate over metallic, wireless, or optical links. Communication interface 660 may be configured to use Time Division Multiplex (TDM), Internet Protocol (IP), Ethernet, optical networking, wireless protocols, communication signaling, or some other communication format - including combinations thereof. Communication interface 660 may be configured to communicate with external devices, such as servers, user devices, or other computing devices.

[0063] I / O device(s) 670 may include computer peripherals that facilitate the interaction between the user and computing system 600. Examples of I / O device(s) 670 may include keyboards, mice, trackpads, monitors, displays, printers, cameras, microphones, external storage devices, and the like. In some implementations, I / O device(s) 670 are used to capture image data associated with a subject, wherein the subject may comprise a face, an animal, or some other object.

[0064] Processing system 650 comprises microprocessor circuitry (e.g., at least one processor) and other circuitry that retrieves and executes operating software (i.e.. program instructions) from storage system 645. Storage system 645 may include volatile and nonvolatile, removable, and non-removable media implemented in any method or technology for storage of information, such as computer-readable instructions, data structures, program modules, or other data. Storage system 645 may be implemented as a single storage device and across multiple storage devices or sub-systems. Storage system 645 may comprise additional elements, such as a controller to read operating software from the storage systems. Examples of storage media (also referred to as computer-readable storage media) include random access memoiy, read-only memory, magnetic disks, optical disks, and flash memory, as well as any combination or variation thereof or any other type of storage media. In some implementations, the storage media may be a non-transitory storage media. In some instances, at least a portion of the storage media may be transitory. In no case is the storage media a propagated signal.

[0065] Processing system 650 is typically mounted on a circuit board that may also hold the storage system. The operating software of storage system 645 comprises computer programs, firmware, or some other form of machine-readable program instructions. The operating software of storage system 645 comprises denoise application 624. The operating software on storage system 645 may further include an operating system, utilities, drivers, network interfaces, applications, or some other type of software. When read and executed by processing system 650 the operating software on storage system 645 directs computingsystem 600 to operate as a computing device as described herein. In at least one implementation, the operating software can provide the operations of FIGs. 1-5, including method 200 of FIG. 2.

[0066] In at least one implementation, denoise application 624 directs processing system 650 to identify frames of video data and identify background portions in a subset of the frames. In some examples, the subset of frames can include the first frames from a video, the last frames from a video, randomly selected frames from a video, or some other subset of frames from a video. In some examples, the background portions can comprise the entire background from each frame but can include smaller portions of the background in some examples. After the background portions are identified, denoise application 624 directs processing system 650 to determine an average background based on the background portions for the subset.

[0067] In some implementations, denoise application 624 extracts backgrounds by separating the static background from moving foreground objects or subjects. This can be done using various techniques, such as background subtraction, where a reference frame (often an average or median of multiple frames) is created to represent the background. Each frame of the video is then compared to this reference frame, and pixels that differ significantly are considered part of the foreground, while the remaining pixels are assumed to belong to the background. Advanced methods may involve machine learning models like semantic segmentation or neural networks trained to differentiate between foreground and background in complex scenarios.

[0068] In some examples, to determine the average frame, denoise application 624 linearizes the noisy frames, averages the noisy frames, and then applies gamma correction to the averaged frame to generate the completed ground-truth average frame for the video data. In some implementations, the average background can be determined by averaging the pixel values across multiple frames, smoothing out moving objects, and emphasizing the static elements. This approach can reduce noise in the background and highlight the consistent features in the scene, which can represent the background. The averaged background can then be used to define the noise parameters associated with normalizing a frame before performing the denoising model and inverse normalizing the denoised frame.

[0069] Once the average frame is determined, the system can apply the denoise model to a frame from the video data to generate a denoised frame and update noise parameters based on comparing the denoised frame and the average frame. The update increases the similarity between the average frame and frames updated via the model. In someimplementations, when applying the denoise model, the system will linearize the frame, denoise the linearized frame, and perform inverse linearization on the denoised frame to generate the final frame for comparison. Based on the comparison, the noise components can then be updated in association with the normalization function.

[0070] After the noise components are updated, denoise application 624 can direct processing system 650 to update frames of the video data using the model and the denoising model. In some implementations, for a frame in the video data, denoise application 624 normalizes the frame using the one or more noise parameters determined from the testing against the average frames. Once normalized, denoise application 624 applies the denoise model to the normalized frame and, subsequently, applies inverse normalization to the frame. This frame is then used as an updated frame for the video. The process can be repeated for remaining frames in the video. As a technical effect, a first set of frames can be processed by denoise application 624 to generate the noise parameters for normalization and inverse normalization, and then applied to the remaining frames to generate a video with filtered noise.

[0071] In some implementations, the noise model that is implemented by denoise application 624 includes a denoising model with parameters that are fixed. Rather than changing the parameters associated with the denoising model, denoise application 624 can use the test time regression (e.g., the average background compared to individual frames) to update the parameters associated with normalization and inverse normalization. In some implementations, the denoising model is trained using a supervised learning approach, where the model is trained on pairs of noisy and clean video frames. The model, which can include a CNN or a similar deep learning architecture, learns to map noisy input frames to their corresponding clean versions. During training, the model minimizes a loss function that measures the difference between its output and the ground truth clean frame. In some implementations, during training, the denoising model uses set noise parameters for the normalizing and inverse normalizing functions. These parameters are then modified during test time of a current video (e.g.. video from a web camera) using the previously described methods associated with the comparison to a ground-truth average frame.

[0072] Clause 1. A method comprising: identifying a plurality of frames of video data; identifying a plurality of background portions in a subset of the plurality of frames; determining a composite background based on the plurality of background portions for the subset; determining at least one noise parameter for the video data based on the compositebackground and a model; and updating the video data based on the at least one noise parameter and the model.

[0073] Clause 2. The method of clause 1, wherein the at least one noise parameter comprises light level or sensor additive noise.

[0074] Clause 3. The method of clause 1 or 2, wherein determining the at least one noise parameter for the video data based on the composite background and the model comprises: applying a first noise parameter and the model to a first frame in the subset to generate a second frame; determining a difference between the second frame and the composite background; and updating the first noise parameter to a second noise parameter based on the difference, the second noise parameter included as the at least one noise parameter.

[0075] Clause 4. The method of any of clauses 1-4, wherein updating the video data based on the at least one noise parameter and the model comprises: for a frame in the video data: applying a normalizing function to the frame to generate a normalized frame, the normalizing function including the at least one noise parameter; applying the model to the normalized frame to generate a denoised frame; and applying an inverse normalization function to the denoised frame to generate an updated frame.

[0076] Clause 5. The method of clause 4, wherein the inverse normalization function includes a noise parameter of the at least one noise parameter.

[0077] Clause 6. The method of any of clauses 1-5, wherein the model comprises a denoising model trained from known noisy and truth images.

[0078] Clause 7. The method of any of clauses 1-6, wherein the at least one noise parameter comprises a gamma correction coefficient.

[0079] Clause 8. A computing apparatus comprising: computer-readable storage media; at least one processor operatively coupled to the computer-readable storage media; and program instructions stored on the computer-readable storage media that, when executed by the at least one processor, direct the computing apparatus to perform a method, the method comprising: identifying a plurality of frames of video data; identifying a plurality of background portions in a subset of the plurality of frames; determining a composite background based on the plurality of background portions for the subset; determining at least one noise parameter for the video data based on the composite background and a model; and updating the video data based on the at least one noise parameter and the model.

[0080] Clause 9. The computing apparatus of clause 8, wherein the at least one noise parameter comprises light level or sensor additive noise.

[0081] Clause 10. The computing apparatus of clause 8 or 9, wherein determining the at least one noise parameter for the video data based on the composite background and the model comprises: applying a first noise parameter and the model to a first frame in the subset to generate a second frame; determining a difference between the second frame and the composite background; and updating the first noise parameter to a second noise parameter based on the difference, the second noise parameter included as the at least one noise parameter.

[0082] Clause 11. The computing apparatus of any of clauses 8-10, wherein updating the video data based on the at least one noise parameter and the model comprises: for a frame in the video data: applying a normalizing function to the frame to generate a normalized frame, the normalizing function including the at least one noise parameter; applying the model to the normalized frame to generate a denoised frame; and applying an inverse normalization function to the denoised frame to generate an updated frame.

[0083] Clause 12. The computing apparatus of clause 11, wherein the inverse normalization function includes a noise parameter of the at least one noise parameter.

[0084] Clause 13. The computing apparatus of any of clauses 8-12, wherein the model comprises a denoising model trained from known noisy and truth images.

[0085] Clause 14. The computing apparatus of any of clauses 8-13, wherein the at least one noise parameter comprises a gamma correction coefficient.

[0086] Clause 15. A computer-readable storage medium having program instructions stored thereon that, when executed by at least one processor, direct the at least one processor to perform a method, the method comprising: identifying a plurality of frames of video data; identify ing a plurality' of background portions in a subset of the plurality of frames; determining a composite background based on the plurality of background portions for the subset; determining at least one noise parameter for the video data based on the composite background and a model, wherein determining the least one noise parameter for the video data based on the composite background and the model comprises: applying a first noise parameter and the model to a first frame in the subset to generate a second frame; determining a difference between the second frame and the composite background; and updating the first noise parameter to a second noise parameter based on the difference, the second noise parameter included as the at least one noise parameter; and updating the video data based on the at least one noise parameter and the model.

[0087] Clause 16. The computer-readable storage medium of clause 15, wherein the at least one noise parameter comprises light level or sensor additive noise.

[0088] Clause 17. The computer-readable storage medium of clause 15 or 16, wherein updating the video data based on the at least one noise parameter and the model comprises: for a frame in the video data: applying a normalizing function to the frame to generate a normalized frame, the normalizing function including the at least one noise parameter; applying the model to the normalized frame to generate a denoised frame; and applying an inverse normalization function to the denoised frame to generate an updated frame.

[0089] Clause 18. The computer-readable storage medium of clause 17, wherein the inverse normalization function includes a noise parameter of the at least one noise parameter.

[0090] Clause 19. The computer-readable storage medium of any of clauses 15-18, wherein the model comprises a denoising model trained from known noisy and truth images.

[0091] Clause 20. The computer-readable storage medium of any of clauses 15-19. wherein the at least one noise parameter comprises a gamma correction coefficient.

[0092] In this specification and the appended claims, the singular forms "a," "an" and "the" do not exclude the plural reference unless the context clearly dictates otherwise. Further, conjunctions such as “and,” “or,” and “and / or” are inclusive unless the context clearly dictates otherwise. For example, “A and / or B” includes A alone, B alone, and A with B. Further, connecting lines or connectors shown in the various figures presented are intended to represent example functional relationships and / or physical or logical couplings between the various elements. Many alternative or additional functional relationships, physical connections or logical connections may be present in a practical device. Moreover, no item or component is essential to the practice of the implementations disclosed herein unless the element is specifically described as “essential” or “critical”.

[0093] Terms such as, but not limited to, approximately, substantially, generally, etc. are used herein to indicate that a precise value or range thereof is not required and need not be specified. As used herein, the terms discussed above will have ready and instant meaning to one of ordinary skill in the art.

[0094] Moreover, use of terms such as up, dow n, top, bottom, side, end, front, back, etc. herein are used with reference to a currently considered or illustrated orientation. If they are considered with respect to another orientation, such terms must be correspondingly modified.

[0095] Further, in this specification and the appended claims, the singular forms "a," "an" and "the" do not exclude the plural reference unless the context clearly dictates otherwise. Moreover, conjunctions such as “and,” “or,” and “and / or” are inclusive unless thecontext clearly dictates otherwise. For example, “A and / or B” includes A alone, B alone, and A with B.

[0096] Although certain example methods, apparatuses and articles of manufacture have been described herein, the scope of coverage of this patent is not limited thereto. It is to be understood that terminology' employed herein is for the purpose of describing aspects and is not intended to be limiting. On the contrary, this patent covers all methods, apparatus and articles of manufacture fairly falling within the scope of the claims of this patent.

Claims

WHAT IS CLAIMED IS:

1. A method comprising: identifying a plurality of frames of video data; identifying a plurality of background portions in a subset of the plurality of frames; determining a composite background based on the plurality of background portions for the subset; determining at least one noise parameter for the video data based on the composite background and a model; and updating the video data based on the at least one noise parameter and the model.

2. The method of claim 1 , wherein the at least one noise parameter comprises light level or sensor additive noise.

3. The method of claim 1, wherein determining the at least one noise parameter for the video data based on the composite background and the model comprises: applying a first noise parameter and the model to a first frame in the subset to generate a second frame; determining a difference between the second frame and the composite background; and updating the first noise parameter to a second noise parameter based on the difference, the second noise parameter included as the at least one noise parameter.

4. The method of claim 1, wherein updating the video data based on the at least one noise parameter and the model comprises: for a frame in the video data: applying a normalizing function to the frame to generate a normalized frame, the normalizing function including the at least one noise parameter; applying the model to the normalized frame to generate a denoised frame; and applying an inverse normalization function to the denoised frame to generate an updated frame.

5. The method of claim 4, wherein the inverse normalization function includes a noise parameter of the at least one noise parameter.

6. The method of claim 1, wherein the model comprises a denoising model trained from known noisy and truth images.

7. The method of claim 1, wherein the at least one noise parameter comprises a gamma correction coefficient.

8. A computing apparatus comprising: computer-readable storage media; at least one processor operatively coupled to the computer-readable storage media; and program instructions stored on the computer-readable storage media that, when executed by the at least one processor, direct the computing apparatus to perform a method, the method comprising: identifying a plurality of frames of video data; identifying a plurality of background portions in a subset of the plurality of frames; determining a composite background based on the plurality of background portions for the subset: determining at least one noise parameter for the video data based on the composite background and a model; and updating the video data based on the at least one noise parameter and the model.

9. The computing apparatus of claim 8, wherein the at least one noise parameter comprises light level or sensor additive noise.

10. The computing apparatus of claim 8, wherein determining the at least one noise parameter for the video data based on the composite background and the model comprises: applying a first noise parameter and the model to a first frame in the subset to generate a second frame; determining a difference between the second frame and the composite background; andupdating the first noise parameter to a second noise parameter based on the difference, the second noise parameter included as the at least one noise parameter.

11. The computing apparatus of claim 8, wherein updating the video data based on the at least one noise parameter and the model comprises: for a frame in the video data: applying a normalizing function to the frame to generate a normalized frame, the normalizing function including the at least one noise parameter; applying the model to the normalized frame to generate a denoised frame; and applying an inverse normalization function to the denoised frame to generate an updated frame.

12. The computing apparatus of claim 11, wherein the inverse normalization function includes a noise parameter of the at least one noise parameter.

13. The computing apparatus of claim 8. wherein the model comprises a denoising model trained from known noisy and truth images.

14. The computing apparatus of claim 8, wherein the at least one noise parameter comprises a gamma correction coefficient.

15. A computer-readable storage medium having program instructions stored thereon that, when executed by at least one processor, direct the at least one processor to perform a method, the method comprising: identifying a plurality of frames of video data; identify ing a plurality of background portions in a subset of the plurality of frames; determining a composite background based on the plurality of background portions for the subset; determining at least one noise parameter for the video data based on the composite background and a model, wherein determining the least one noise parameter for the video data based on the composite background and the model comprises: applying a first noise parameter and the model to a first frame in the subset to generate a second frame;determining a difference between the second frame and the composite background; and updating the first noise parameter to a second noise parameter based on the difference, the second noise parameter included as the at least one noise parameter; and updating the video data based on the at least one noise parameter and the model.

16. The computer-readable storage medium of claim 15, wherein the at least one noise parameter comprises light level or sensor additive noise.

17. The computer-readable storage medium of claim 15, wherein updating the video data based on the at least one noise parameter and the model comprises: for a frame in the video data: applying a normalizing function to the frame to generate a normalized frame, the normalizing function including the at least one noise parameter; applying the model to the normalized frame to generate a denoised frame; and applying an inverse normalization function to the denoised frame to generate an updated frame.

18. The computer-readable storage medium of claim 17, wherein the inverse normalization function includes a noise parameter of the at least one noise parameter.

19. The computer-readable storage medium of claim 15, wherein the model comprises a denoising model trained from known noisy and truth images.

20. The computer-readable storage medium of claim 15, wherein the at least one noise parameter comprises a gamma correction coefficient.

Images