Image denoising, filtering data processing method, device and computer equipment

Abstract

The application relates to an image denoising, filtering data processing method, device and computer equipment, which can be applied to video image denoising. The method comprises the following steps: acquiring a to-be-processed image and a reference image; determining the motion intensity of a to-be-processed block in the to-be-processed image according to the reference image; the to-be-processed block is obtained by dividing the to-be-processed image; acquiring filtering coefficient description information matched with the motion intensity, wherein the filtering coefficient description information is used for describing the corresponding relationship between a pixel difference value and a filtering coefficient representation value; acquiring a target pixel difference value between a pixel point in the to-be-processed block and a corresponding position pixel point in the reference image, determining a target filtering coefficient representation value corresponding to the target pixel difference value based on the filtering coefficient description information; and determining a target denoising image corresponding to the to-be-processed image based on the target filtering coefficient representation value. The method can improve the image denoising effect.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image noise reduction method, apparatus, computer equipment, storage medium and computer program product, as well as a filtered data processing method, apparatus, computer equipment, storage medium and computer program product. Background Technology

[0002] With the development of image processing technology, image denoising technology has emerged. This technology can reduce image noise caused by hardware errors and unstable light sources. Image denoising is required in many scenarios. For example, in video capture, various noises can be mixed in the captured image sequence. To improve the visual effect of video images, increase compression efficiency, or save bandwidth, video images need to be denoised.

[0003] In related technologies, poor noise reduction effect is often a common problem when performing noise reduction on images. Summary of the Invention

[0004] Therefore, it is necessary to provide an image denoising method, apparatus, computer equipment, computer-readable storage medium, and computer program product that can improve the image denoising effect, as well as a filtering data processing method, apparatus, computer equipment, computer-readable storage medium, and computer program product, in response to the above-mentioned technical problems.

[0005] On one hand, this application provides an image denoising method. The method includes: acquiring an image to be processed and a reference image; determining the motion intensity of a block to be processed in the image to be processed based on the reference image; the block to be processed is obtained by dividing the image to be processed; acquiring filter coefficient description information matching the motion intensity, the filter coefficient description information being used to describe the correspondence between pixel differences and filter coefficient characterization values; wherein, the filter coefficient characterization value is negatively correlated with the pixel differences, and the degree of change of the filter coefficient characterization value is positively correlated with the motion intensity; acquiring the target pixel difference between pixels in the block to be processed and corresponding pixels in the reference image, and determining a target filter coefficient characterization value corresponding to the target pixel difference based on the filter coefficient description information; and determining a target denoised image corresponding to the image to be processed based on the target filter coefficient characterization value.

[0006] On the other hand, this application also provides an image denoising apparatus. The apparatus includes: an image acquisition module for acquiring an image to be processed and a reference image; a motion intensity determination module for determining the motion intensity of a block to be processed in the image to be processed based on the reference image; the block to be processed is obtained by dividing the image to be processed; a description information acquisition module for acquiring filter coefficient description information matching the motion intensity, the filter coefficient description information describing the correspondence between pixel differences and filter coefficient characterization values; wherein the filter coefficient characterization value is negatively correlated with the pixel differences, and the degree of change of the filter coefficient characterization value is positively correlated with the motion intensity; a filter coefficient determination module for acquiring the target pixel difference between pixels in the block to be processed and corresponding pixels in the reference image, and determining a target filter coefficient characterization value corresponding to the target pixel difference based on the filter coefficient description information; and a denoised image determination module for determining a target denoised image corresponding to the image to be processed based on the target filter coefficient characterization value.

[0007] On the other hand, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the above-described image noise reduction method.

[0008] On the other hand, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the above-described image noise reduction method.

[0009] On the other hand, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps of the above-described image noise reduction method.

[0010] The aforementioned image denoising methods, apparatuses, computer devices, storage media, and computer program products, since the blocks to be processed are obtained by dividing the image to be processed, can determine different target filter coefficient characterization values ​​for different blocks to be processed in the image to be processed. The obtained target filter coefficient characterization values ​​can accurately match the motion of each region in the image, avoiding the problem of poor denoising effect in some regions caused by motion intensity estimation of the entire image, thus improving the denoising effect of the image to be processed. In addition, since the target filter coefficient characterization values ​​are determined by filter coefficient description information, which describes the correspondence between pixel differences and filter coefficient characterization values, and the filter coefficient characterization values ​​are negatively correlated with pixel differences and positively correlated with motion intensity, for each pixel value, a target filter coefficient characterization value matching that pixel can be obtained, further improving the denoising effect of the image to be processed.

[0011] On the other hand, this application provides a filtering data processing method. The method includes: determining multiple reference motion intensity information and determining the pixel difference distribution range; determining filter coefficient characterization values ​​for multiple pixel differences within the pixel difference distribution range based on each reference motion intensity information; establishing a correspondence between each filter coefficient characterization value and its corresponding pixel difference; and forming filter coefficient description information by combining the correspondences of filter coefficient characterization values ​​determined based on the same reference motion intensity information, thereby obtaining filter coefficient description information corresponding to each reference motion intensity information; wherein the filter coefficient description information is used for temporal filtering of the image to be processed.

[0012] On the other hand, this application also provides a filtering data processing apparatus. The apparatus includes: a reference motion intensity determination module, used to determine multiple reference motion intensity information and determine the pixel difference distribution range; a characterization value determination module, used to determine the characterization value of the filtering coefficients under multiple pixel differences within the pixel difference distribution range based on each reference motion intensity information; a correspondence establishment module, used to establish a correspondence between each filtering coefficient characterization value and its corresponding pixel difference; and a description information determination module, used to form filtering coefficient description information by combining the correspondences of the filtering coefficient characterization values ​​determined based on the same reference motion intensity information, thereby obtaining filtering coefficient description information corresponding to each reference motion intensity information; wherein, the filtering coefficient description information is used for temporal filtering of the image to be processed.

[0013] On the other hand, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the above-described filtering data processing method.

[0014] On the other hand, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the above-described filtering data processing method.

[0015] On the other hand, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps of the above-described filtering data processing method.

[0016] The aforementioned filtering data processing methods, apparatus, computer equipment, storage media, and computer program products, because the filtering coefficient description information describes the correspondence between pixel differences and filtering coefficient characterization values ​​under each reference motion intensity information, can directly use the pixel differences corresponding to the pixels in the image to be processed as an index to query the corresponding filtering coefficient characterization values ​​from the filtering coefficient description information when performing temporal filtering on the image to be processed. This avoids obtaining the filter coefficients through complex calculations and improves filtering efficiency. Attached Figure Description

[0017] Figure 1 This is an application environment diagram of the image noise reduction method and the filtered data processing method in one embodiment;

[0018] Figure 2 This is a flowchart illustrating an image noise reduction method in one embodiment;

[0019] Figure 3 This is a schematic diagram of the interface of an image noise reduction algorithm applied to a video conferencing application in one embodiment;

[0020] Figure 4 This is a schematic diagram illustrating the segmentation of the image to be processed in one embodiment;

[0021] Figure 5 This is a schematic diagram of a filtering function in one embodiment;

[0022] Figure 6 This is a schematic diagram illustrating the specific process of an image noise reduction method in another embodiment;

[0023] Figure 7 This is an application architecture diagram of an image noise reduction method in one embodiment;

[0024] Figure 8 This is a flowchart illustrating a filtering data processing method in one embodiment;

[0025] Figure 9 This is a schematic diagram illustrating the effect of an image noise reduction method in one embodiment;

[0026] Figure 10This is a schematic diagram illustrating the effect of the image noise reduction method in another embodiment;

[0027] Figure 11 This is a structural block diagram of an image noise reduction device in one embodiment;

[0028] Figure 12 This is a structural block diagram of a filtering data processing device in one embodiment;

[0029] Figure 13 This is an internal structural diagram of a computer device in one embodiment;

[0030] Figure 14 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0032] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0033] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, as well as machine learning / deep learning, autonomous driving, and intelligent transportation.

[0034] Computer vision (CV) is a science that studies how to enable machines to "see." More specifically, it refers to using cameras and computers to replace human eyes in recognizing and measuring targets, and then performing image processing to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), autonomous driving, intelligent transportation, and other technologies, as well as common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0035] Machine learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computer devices can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computer devices with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instructional learning.

[0036] The solutions provided in this application involve technologies such as computer vision and machine learning in artificial intelligence, and are specifically illustrated through the following embodiments:

[0037] The image denoising method and filtered data processing method provided in this application embodiment can be applied to, for example... Figure 1In the application environment shown, terminal 102 communicates with server 104 via a network. The data storage system can store the data that server 104 needs to process, such as the denoised target image obtained through noise reduction. The data storage system can be integrated on server 104, or it can be located in the cloud or on other servers. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal can be a smartphone, tablet, laptop, desktop computer, smart voice interaction device, smart home appliance, vehicle terminal, aircraft, etc., but is not limited to these. The terminal and server can be directly or indirectly connected via wired or wireless communication, which is not limited herein.

[0038] It is understood that the image denoising method and filtering data processing method provided in this application embodiment can be executed by terminal 102, server 104, or jointly by terminal 102 and server 104. For example, terminal 102 can acquire an image to be processed and send it to the server. The server further acquires a reference image corresponding to the image to be processed, then determines the motion intensity of the block to be processed in the image to be processed based on the reference image, acquires filtering coefficient description information matching the motion intensity, the filtering coefficient description information is used to describe the correspondence between pixel difference and filtering coefficient characterization value, acquires the target pixel difference between the pixel point in the block to be processed and the corresponding pixel point in the reference image, determines the target filtering coefficient characterization value corresponding to the target pixel difference based on the filtering coefficient description information, determines the target denoised image corresponding to the image to be processed based on the target filtering coefficient characterization value, and finally, server 104 can return the target denoised image to terminal 102.

[0039] In one embodiment, such as Figure 2 As shown, an image denoising method is provided, which is performed by a computer device, the computer device being... Figure 1 Terminal 102 in the middle can also be Figure 1 The server 104 in the diagram can also be a system consisting of the terminal 102 and the server 104. Specifically, the image noise reduction method includes the following steps:

[0040] Step 202: Obtain the image to be processed and the reference image.

[0041] In this context, the image to be processed refers to the image that needs to be denoised. The image to be processed can be an image captured by a computer device, or an image received by a computer device via a network and captured by another computer device. The image to be processed can be a video image. The reference image refers to the image used as a reference for temporal filtering of the image to be processed. The reference image and the image to be processed can include the same foreground object. The reference image can be an image obtained through denoising processing. When the image to be processed is an image in the target video to be denoised, the reference image can be an image obtained by denoising a preceding video frame of the image to be processed. The denoising processing here can be temporal filtering or spatial filtering. In other embodiments, the denoising processing can also be temporal filtering followed by spatial filtering.

[0042] Specifically, after determining the image to be processed, the computer device can acquire one or more reference images corresponding to the image to be processed, and then perform filtering processing on the image to be processed based on the reference images, thereby achieving noise reduction of the image to be processed. It should be noted that the "multiple" mentioned in the embodiments of this application refers to at least two.

[0043] In one embodiment, obtaining the image to be processed and the reference image includes: determining the target video to be denoised; using video frames in the target video as the image to be processed, determining the target video frame from the forward video frames corresponding to the image to be processed; obtaining the target denoised image corresponding to the target video frame, and determining the target denoised image corresponding to the target video frame as the reference image corresponding to the image to be processed.

[0044] In this context, the target video to be denoised refers to the video that requires denoising processing. The preceding video frame corresponding to the image to be processed refers to the video frame in the target video whose time point is before the image to be processed. For example, if the image to be processed is the 10th second video frame in the target video, then the video frames in the target video before the 10th second are the preceding video frames of the image to be processed. The target denoised image corresponding to the target video frame refers to the target denoised image obtained when the target video frame is used as the image to be processed. In this embodiment, denoising can be performed using a recursive filtering method. After obtaining the target denoised image using the image denoising method of this application for each video frame, it can be saved as a reference image for the following video frame.

[0045] Specifically, the computer device can sequentially determine each video frame after the first frame in the target video as the image to be processed. For each image to be processed, the computer device can determine one or more target video frames from the forward video frames corresponding to the image to be processed, obtain the target denoised image corresponding to the target video frame, and use the target denoised image as the reference image corresponding to the image to be processed.

[0046] In one specific embodiment, a video application can be installed on the computer device. This video application can be used to capture, play, or edit videos. Examples of video applications include live streaming applications, video editing applications, video monitoring applications, video conferencing applications, and so on. When the computer device plays a video through this application, for each frame of the played video, the computer device can sequentially use these frames as images to be processed and obtain a reference image for the image to be processed. This allows the target denoised image of the image to be processed to be obtained using the image denoising method provided in this application, thereby improving the video's visual quality and providing a better user experience. Here, "application" can refer to a client installed on a terminal (also known as an application client or APP client), which is a program installed and running on the terminal. An application can also refer to an installation-free application, i.e., an application that can be used without downloading and installing; these applications are commonly known as mini-programs, which typically run as subroutines on the client. An application can also refer to a web application opened through a browser; and so on.

[0047] For example, such as Figure 3 The image shown is a schematic diagram of the interface when the image denoising algorithm of this application is applied to a video conferencing application. (Reference) Figure 3 During a video conference, when a user selects the "video noise reduction" option in the video conferencing application interface, the video conferencing application can use the image noise reduction method provided in this application to reduce the noise in the video generated in the conference scenario.

[0048] In one embodiment, considering that the correlation between video frames with close intervals is greater than that between video frames with distant intervals, using video frames with close intervals as target video frames to obtain reference images can improve the noise reduction effect. Therefore, when selecting target video frames, the computer device can use one or a preset number of video frames adjacent to the image to be processed in the preceding video frames of the image to be processed as target video frames. Here, the preset number is less than a preset threshold, and the preset number can be, for example, 2. For example, assuming that the image to be processed is the 5th video frame in the target video frames, the 3rd and 4th video frames can be used as target video frames to obtain the reference image of the 5th video frame.

[0049] Step 204: Determine the motion intensity of the block to be processed in the image to be processed based on the reference image; the block to be processed is obtained by dividing the image to be processed.

[0050] Here, the "block to be processed" refers to the image block obtained by dividing the image to be processed. Dividing the image to be processed can yield multiple image blocks, each of which can serve as a block to be processed. Motion intensity characterizes the magnitude of the motion of the block to be processed relative to the reference image. Motion intensity is negatively correlated with the noise intensity of the block to be processed relative to the reference image and positively correlated with the degree of difference between the block to be processed and the reference image.

[0051] Specifically, considering that noise information exhibits a Gaussian distribution in the time domain, noise appearing at a certain position in the current frame of a video may not exist or have a weaker intensity at the same position in the preceding frame. Therefore, a weighted approach can be used for image fusion, such as the fused current frame = 0.2 * the previous two frames + 0.3 * the previous frame + 0.5 * the current frame. While this reduces noise, simple fusion can result in noticeable motion blur in moving scenes. Therefore, this embodiment employs a motion-adaptive temporal filtering method. When a large amount of motion is detected in the current frame, the weight of the current frame is increased, while the proportion of the previous frame is reduced. For example, when the amount of motion is large, the fused current frame = 0.05 * the previous two frames + 0.05 * the previous frame + 0.9 * the current frame, thus alleviating the motion blur problem. Furthermore, considering that the motion intensity in different regions of a video frame often varies significantly—for example, in a scene where only the person is moving while the background is stationary—calculating the motion intensity for the entire frame would result in a weaker temporal filtering intensity, leading to poor noise reduction in the background area.

[0052] Based on this, in this embodiment, the image to be processed can be divided into multiple image blocks, each image block is used as the block to be processed, and the motion intensity of each block to be processed is estimated based on the reference image. Thus, different temporal filters are used in different image blocks according to the estimated motion level, so that good noise reduction effect can still be achieved on a fixed background when the person is moving.

[0053] In one embodiment, when dividing an image to be processed, the computer device can use a uniform division method to divide the image into multiple blocks. Specifically, if the height and width of the image to be processed are H and W respectively, and the entire image is divided into m rows and n columns, then the height and width of each block are H / m and W / n respectively. For example, ... Figure 4 The diagram shown is a schematic representation of the segmentation of the image to be processed in a specific embodiment. (Refer to...) Figure 4 Each small grid represents a block to be processed. It is understood that in other embodiments, the computer device may also divide the image to be processed into uniform or overlapping sections. This application does not limit the specific division method.

[0054] In one embodiment, when there are multiple reference images, for each block to be processed in the image to be processed, the server calculates the motion intensity based on each reference image, so that multiple motion intensities can be obtained for each block to be processed.

[0055] Step 206: Obtain the filter coefficient description information that matches the motion intensity. The filter coefficient description information is used to describe the correspondence between the pixel difference and the filter coefficient characterization value. The filter coefficient characterization value is negatively correlated with the pixel difference, and the degree of change of the filter coefficient characterization value is positively correlated with the motion intensity.

[0056] Among them, the filter coefficient characterization value refers to the numerical value that characterizes the filter coefficient. Specifically, the filter coefficient characterization value can be the filter coefficient value or a value obtained by performing certain calculations on the filter coefficient value.

[0057] In the noise reduction process of this application embodiment, temporal filtering of the block to be processed can be performed according to the following formula, where i and j are pixel position indices in the image to be processed, and Y represents the pixel value in the block to be processed. The time-domain filtering result represents the current moment. This represents the pixel value in the reference image, which here uses the temporal filtering result at time t-1, and k represents the filter coefficients:

[0058] (1)

[0059] The above formula represents the current time-domain filtering result, which is equal to the weighted fusion of the current input value and the previous time-domain filtering result. The weighting coefficient k ranges from 0 to 1 and has the following relationship: Current input Compared with the time-domain filtering result of the previous moment The larger the pixel difference between the two (where pixel difference represents the absolute value of the difference), the smaller k is, and vice versa. Therefore, the filter coefficient k is related to... and Since the differences between the pixels are negatively correlated, in this embodiment of the application, a correspondence between the pixel difference and the filter coefficient characterization value can be established and saved as filter coefficient description information. Then, the filter coefficient characterization value can be determined based on the filter coefficient description information, and the time domain filtering result obtained by performing time domain filtering on the image to be processed can be determined through the filter coefficient characterization value.

[0060] Furthermore, since this application employs a motion-adaptive filtering method, different filtering effects are expected under different motion intensities. The greater the motion intensity, the weaker the filtering effect should be, and the smaller the motion intensity, the stronger the filtering effect should be. Therefore, this application can predetermine multiple filtering coefficient descriptions. The degree of change of the filtering coefficient representation value in different filtering coefficient descriptions is different, and the degree of change of the filtering coefficient representation value is set to be positively correlated with the motion intensity. The degree of change of the filtering coefficient representation value refers to the degree of change of the filtering coefficient representation value as the pixel difference changes. That is, the greater the motion intensity, the greater the degree of change of the filtering coefficient representation value. At this time, the filtering coefficient representation value decreases rapidly as the pixel difference increases. Conversely, the smaller the motion intensity, the smaller the degree of change of the filtering coefficient representation value. At this time, the weakening of the filtering coefficient representation value as the pixel difference increases can be slower.

[0061] In one embodiment, the filter coefficient description information can be a filter function, where the filter coefficients are the dependent variable and the pixel difference is the independent variable. For example, the filter function can be a decreasing exponential function of the filter coefficients with respect to the pixel difference. The computer device can divide the motion intensity into multiple intervals and set a corresponding filter function for each interval. The stronger the motion, the steeper the curve of the filter function, thus avoiding motion blur in the moving areas. Conversely, for image patches with weak motion, a gentler filter curve is used, achieving better noise reduction against a relatively fixed background. In a specific embodiment, the filter function can be an exponential function. For example, see [reference]. Figure 5 The image shows a schematic diagram of the filtering function in one embodiment. Figure 5 In the illustrated embodiment, the motion intensity is divided into two intervals: those below a preset motion intensity threshold are classified as low-intensity intervals, and those above a preset threshold are classified as high-intensity intervals. For the low-intensity interval, the designed filtering function is as follows: Figure 5 As shown by the dashed line, the designed filter function for the high motion intensity range is as follows: Figure 5 As shown by the solid line in the middle, by Figure 5 It can be seen that the function curve corresponding to high exercise intensity is steeper than the function curve corresponding to low exercise intensity.

[0062] Specifically, the computer device pre-stores multiple filter coefficient descriptions, each of which corresponds to a reference motion intensity. When filtering, the computer device can determine the reference motion intensity that matches the motion intensity of the block to be processed from the multiple reference motion intensity information, and then obtain the filter coefficient description information corresponding to the reference motion intensity information as the filter coefficient description information that matches the motion intensity.

[0063] In one embodiment, when there are multiple reference images, multiple motion intensities can be obtained for each block to be processed. In this case, for each motion intensity corresponding to each block to be processed, the computer device needs to obtain matching filter coefficient description information.

[0064] Step 208: Obtain the target pixel difference between the pixel in the block to be processed and the corresponding pixel in the reference image, and determine the target filter coefficient characterization value that corresponds to the target pixel difference based on the filter coefficient description information.

[0065] Specifically, for each pixel in the block to be processed, the computer device can calculate the difference between the pixel and the corresponding pixel in the reference image, take the absolute value of the difference as the target pixel difference, and then determine the target filter coefficient characterization value that corresponds to the target pixel difference based on the filter coefficient description information.

[0066] In one embodiment, when there are multiple reference images, since each block to be processed corresponds to multiple filter coefficient description information, for the target pixel difference of each pixel in the block to be processed, the computer device can determine the target filter coefficient characterization value that corresponds to the target pixel difference based on each filter coefficient description information, so that each pixel can obtain multiple target filter coefficient characterization values.

[0067] In one embodiment, when the filter coefficient description information is a filter function, the computer device can substitute the target pixel value into the filter function, and the calculated function value is the target filter coefficient representation value corresponding to the target pixel value.

[0068] Step 210: Based on the target filter coefficient characterization value, determine the target denoised image corresponding to the image to be processed.

[0069] Specifically, the computer device can determine the intermediate processed image obtained by performing temporal filtering on the image to be processed based on the target filtering coefficient characterization value corresponding to each pixel, and then obtain the target denoised image corresponding to the image to be processed based on the intermediate processed image.

[0070] In one embodiment, when the filter coefficient representation value is the filter coefficient value, the computer device can refer to the formula (1) above to perform temporal filtering on each pixel based on the target filter coefficient representation value corresponding to each pixel in the block to be processed, obtain the temporal filtering result of each pixel, and use the temporal filtering result of each pixel as the current pixel value in the image to be processed to update the image to be processed, and then obtain the intermediate processed image.

[0071] In one embodiment, after obtaining the intermediate processed image, the computer device can directly use the intermediate processed image as the target denoised image corresponding to the image to be processed. In another embodiment, considering that some large-amplitude noise that is difficult to filter out may still exist after temporal filtering, the computer device can continue to use spatial filtering after temporal operation, and use the image obtained by spatial filtering as the target denoised image. Here, spatial filtering is a smoothing method. Its principle is that the pixel values ​​of a natural image are relatively smooth and continuous in space. The image obtained by shooting is a natural image with noise added. Spatial filtering aims to eliminate the non-smooth noise and obtain a smooth natural image.

[0072] In one specific embodiment, spatial filtering can employ at least one of Gaussian filtering and bilateral filtering. In another specific embodiment, a computer device can acquire a trained deep learning model for spatial filtering, input an intermediate processed image into the deep learning model, and output the target denoised image through the deep learning model. The deep learning model for spatial filtering can be trained using a supervised training method, with the input samples during training being the original image without spatial filtering, and the training label being the target image obtained through spatial filtering.

[0073] In the above image denoising method, the motion intensity of the block to be processed in the image to be processed is determined based on the reference image. Filter coefficient description information matching the motion intensity is obtained. The target pixel difference between the pixels in the block to be processed and the corresponding pixels in the reference image is obtained. Based on the filter coefficient description information, a target filter coefficient representation value corresponding to the target pixel difference is determined. Based on the target filter coefficient representation value, the target denoised image corresponding to the image to be processed is determined. Since the block to be processed is obtained by dividing the image to be processed, different target filter coefficient representation values ​​can be determined for different blocks in the image to be processed. It can accurately match the motion of each region in the image, avoiding the problem of poor noise reduction in some areas caused by motion intensity estimation of the entire image, thus improving the noise reduction effect of the image to be processed. In addition, since the target filter coefficient characterization value is determined by the filter coefficient description information, which describes the correspondence between pixel difference and filter coefficient characterization value, and the filter coefficient characterization value is negatively correlated with the pixel difference, and the degree of change of the filter coefficient characterization value is positively correlated with the motion intensity, for each pixel value, the target filter coefficient characterization value matching that pixel can be obtained, further improving the noise reduction effect of the image to be processed.

[0074] In one embodiment, the method further includes: determining multiple reference motion intensity information and determining the pixel difference distribution range; determining filter coefficient characterization values ​​for multiple pixel differences within the pixel difference distribution range based on each reference motion intensity information; establishing a correspondence between each filter coefficient characterization value and its corresponding pixel difference; forming filter coefficient description information by combining the correspondence of filter coefficient characterization values ​​determined based on the same reference motion intensity information; and obtaining filter coefficient description information matching the motion intensity, including: determining target motion intensity information matching the motion intensity from multiple reference motion intensity information, and determining the filter coefficient description information corresponding to the target motion intensity information as the filter coefficient description information matching the motion intensity.

[0075] Motion intensity reference information refers to the information used as a reference to determine the description information of the filter coefficients. Motion intensity reference information can be a motion intensity range or a specific numerical value. Pixel difference distribution range refers to the range in which all possible pixel differences are distributed; the pixel difference distribution range can be [0, 255].

[0076] Specifically, after determining multiple reference motion intensity information and the distribution range of pixel differences, the computer device, for each reference motion intensity information, ensures that the filter coefficient representation value is negatively correlated with the pixel difference and that the degree of change of the filter coefficient representation value is positively correlated with the motion intensity. It then determines the filter coefficient representation value for each possible pixel difference within the pixel difference distribution range. For example, assuming the pixel difference is an integer, for each reference motion intensity information, the computer device can determine the filter coefficient representation value for 256 pixel differences within the range [0, 255], including 0, 1, 2…

[0077] Furthermore, the computer device can establish a correspondence between each filter coefficient representation value and its corresponding pixel difference. From these correspondences, the filter coefficient representation values ​​included are used to form filter coefficient description information based on the correspondence determined by the same reference motion intensity information. This filter coefficient description information is then used as the filter coefficient description information corresponding to the reference motion intensity information, thereby obtaining the filter coefficient description information corresponding to each reference motion intensity information. In the example above, assuming there are two reference motion intensity information values, represented as X1 and X2, the correspondence determined based on X1 is (0, b10), (1, b11), (255, b10), and (255, b11). 255 The correspondence determined based on X2 is (0, b20), (1, b21), (255, b2). 255 ), where b represents the pixel difference, then the computer device can distinguish (0, b10), (1, b11), (255, b10), where b represents the pixel difference.255 The filter coefficient description information corresponding to X1 is composed of (0, b20), (1, b21), and (255, b20). 255 The filter coefficient description information corresponding to X2 is composed of X2.

[0078] In a specific embodiment, for each filter coefficient description, the computer device can store the correspondence in the form of a table as follows, wherein the index value of the table is the pixel difference. Referring to Table 1, for example, when the target pixel difference is 3, the filter coefficient characterization value is 0.88.

[0079] Table 1

[0080] 1 0.96 0.92 0.88 0.84 0.80 0.23 0.20 0.18 ……

[0081] The computer device can further establish a correspondence between reference motion intensity information and their respective corresponding filter coefficient description information. Based on this correspondence, when it is necessary to obtain filter coefficient description information that matches the motion intensity of the block to be processed, the computer device can determine the target motion intensity information that matches the motion intensity from multiple reference motion intensity information, and determine the filter coefficient description information that has a correspondence with the target motion intensity information as the filter coefficient description information that matches the motion intensity.

[0082] In one specific embodiment, the reference motion intensity information can be a specific reference motion intensity, which is a specific numerical value. When the computer device determines the target motion intensity information that matches a certain motion intensity, it can determine the reference motion intensity with the smallest difference from the target motion intensity as the target reference motion intensity, and determine the filter coefficient description information that corresponds to the target reference motion intensity as the filter coefficient description information that matches the motion intensity. For example, assuming the reference motion intensity information includes 10, 20, and 30, if the motion intensity of a block to be processed is 12, then its corresponding reference motion intensity information can be determined to be 10, and the filter coefficient description information corresponding to 10 can be determined as the filter coefficient description information that matches the motion intensity. It is understood that in other embodiments, the computer device can set a larger number of reference motion intensities, establishing a correspondence between each filter coefficient description information and multiple reference motion intensities, thereby making the established correspondence more accurate. For example... The computer device can be set to 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, and 30. The 10, 12, 14, and 16 are associated with the first filter coefficient description information, the 18, 20, 22, 24, and 26 are associated with the second filter coefficient description information, and the 28 and 30 are associated with the third filter coefficient description information.

[0083] In the above embodiments, since the filter coefficient description information describes the correspondence between the pixel difference and the filter coefficient characterization value under each reference motion intensity information, when performing temporal filtering on the image to be processed, the pixel difference corresponding to the pixel point in the processed image can be directly used as an index to query the corresponding filter coefficient characterization value from the filter coefficient description information, avoiding the need to obtain the filter coefficient through complex calculations and improving the filtering efficiency.

[0084] In one embodiment, determining the filter coefficient representation value for multiple pixel differences within the pixel difference distribution range based on each reference motion intensity information includes: determining the target filter coefficients for multiple pixel differences within the pixel difference distribution range based on each reference motion intensity information; multiplying each target filter coefficient by its corresponding pixel difference to obtain the filter coefficient representation value for each pixel difference; and determining the target denoised image corresponding to the image to be processed based on the target filter coefficient representation value, including: determining the temporal filtering result of the pixel based on the magnitude relationship between the pixel value of the pixel and the pixel value of the corresponding position in the reference image; and determining the target denoised image corresponding to the image to be processed based on the temporal filtering result of the pixel.

[0085] Specifically, formula (1) above can be further transformed as follows:

[0086] (2)

[0087] in, Let k be the pixel difference, and k be the filter coefficient. As can be seen from formula (2), if the pixel difference is calculated in advance... Therefore, the filtering result at the current moment can be calculated by simple addition and subtraction, which can further improve the filtering efficiency. Based on this, in this embodiment, the computer device can first determine the target filter coefficients under multiple pixel differences within the pixel difference distribution range based on each reference motion intensity information. When determining the target filter coefficients, it is ensured that the filter coefficient characterization value is negatively correlated with the pixel difference, and the degree of change of the filter coefficient characterization value is positively correlated with the motion intensity.

[0088] After determining the target filter coefficients, each target filter coefficient is multiplied by its corresponding pixel difference to obtain the filter coefficient representation value for each pixel difference, i.e. As the representation value of the filter coefficient. For example, assuming that Table 1 stores the filter coefficients, the representation value of the filter coefficient obtained by multiplying each filter coefficient by its corresponding pixel difference is shown in Table 2 below. In Table 2, the index value is also the pixel difference. For example, assuming that the target pixel difference is 4, the representation value of the filter coefficient can be obtained by looking up the table as 3.36.

[0089] Table 2

[0090] 0 0.96 1.84 2.64 3.36 4.0 1.38 1.40 1.44 ……

[0091] It is understandable that the pixel difference multiplied by the target filter coefficients here is the absolute value of the pixel difference, therefore Whether the actual calculation result is positive or negative depends on the relationship between the pixel value of the pixel and the pixel value of the corresponding pixel in the reference image.

[0092] In one embodiment, determining the temporal filtering result of a pixel based on the relationship between the pixel value of the pixel and the pixel value of the corresponding pixel in the reference image includes: when the pixel value of the pixel is greater than or equal to the pixel value of the corresponding pixel in the reference image, subtracting the target filtering coefficient characterization value from the pixel value to obtain the temporal filtering result of the pixel; when the pixel value of the pixel is less than or equal to the pixel value of the corresponding pixel in the reference image, adding the target filtering coefficient characterization value to the pixel value to obtain the temporal filtering result of the pixel.

[0093] Specifically, when the pixel value of a pixel is greater than the pixel value of the corresponding pixel in the reference image, As can be seen from formula (2), the filtering result at this time is equivalent to subtracting the target filtering coefficient value from the pixel value of the pixel. When the pixel value of the pixel is less than the pixel value of the corresponding pixel in the reference image, When the value is positive, the filtering result is equivalent to adding the target filtering coefficient value to the pixel value of the pixel. When the pixel value of the pixel equals the pixel value of the corresponding pixel in the reference image, that is... ,at this time When the value is 0, the computer device subtracts the target filter coefficient value from the pixel value to obtain the temporal filtering result of the pixel, or adds the target filter coefficient value to the pixel value to obtain the temporal filtering result of the pixel. It can be seen that the temporal filtering result at this point is... or .

[0094] For example, as shown in Table 2, assuming the target pixel difference is 3, the target filtering coefficient is represented by a value of 2.64. Then, when... Time-domain filtering results ,when Time-domain filtering results .

[0095] Further, after obtaining the time-domain filtering results of each pixel point in the image to be processed, the computer device updates the pixel values in the image to be processed with the time-domain filtering results, that is, obtains an intermediate processed image. Based on this intermediate processed image, the computer device can determine the target denoised image corresponding to the image to be processed. For specific reference, see the description in the above embodiments.

[0096] In the above embodiments, by multiplying each target filter coefficient by its corresponding pixel difference, a filter coefficient characterization value for each pixel difference can be obtained, and complex function operations can be converted into simple addition and subtraction calculations, further improving the filtering efficiency in the time-domain filtering process, thereby improving the denoising efficiency.

[0097] In one embodiment, multiple reference motion intensity information is determined: the motion intensity distribution range is divided into multiple motion intensity intervals, and each motion intensity interval is used as a reference motion intensity information; the target motion intensity information matching the motion intensity is determined from the multiple reference motion intensity information, and the filter coefficient description information corresponding to the target motion intensity information is determined as the filter coefficient description information matching the motion intensity, including: determining the target interval to which the motion intensity belongs from the multiple motion intensity intervals, and determining the filter coefficient description information corresponding to the target interval as the filter coefficient description information matching the motion intensity.

[0098] Among them, the motion intensity distribution range refers to the range in which all possible motion intensities are distributed.

[0099] In this embodiment, the computer device can divide the motion intensity distribution range into multiple intervals to obtain multiple motion intensity intervals, use each motion intensity interval as a reference motion intensity information, establish a correspondence relationship between the motion intensity interval and the filter coefficient description information. Then, after determining the motion intensity of the block to be processed, the computer device can determine which motion intensity interval the motion intensity belongs to, determine the motion intensity interval to which the motion intensity belongs as the target interval, obtain the filter coefficient description information corresponding to the target interval, and determine the filter coefficient description information as the filter coefficient description information matching the motion intensity.

[0100] For example, assume that the motion intensity distribution range is [a, b], and the motion intensity distribution range can be divided into three intervals, namely [a, c], (c, d), [d, b], where a < c < d < b. If a certain motion intensity of the block to be processed belongs to (c, d), then (c, d) can be determined as the target interval, and the filter coefficient description information corresponding to it can be determined as the filter coefficient description information matching the motion intensity.

[0101] In the above embodiments, by dividing the range of motion intensity distribution into multiple intervals and using each motion intensity interval as a reference motion intensity information, the matching filter coefficient description information can be determined by the motion intensity interval to which the motion intensity belongs, thereby improving the accuracy of the determined filter coefficient description information.

[0102] In one embodiment, before determining the motion intensity of the block to be processed in the image to be processed based on the reference image, the method further includes: converting the three primary color channel data in the image to be processed into luminance channel data, chroma channel data, and saturation channel data and extracting the luminance channel data therefrom; converting the three primary color channel data in the reference image into luminance channel data, chroma channel data, and saturation channel data and extracting the luminance channel data therefrom; determining the motion intensity of the block to be processed in the image to be processed based on the reference image, including: determining the motion intensity of the block to be processed based on the luminance channel data of the reference image and the luminance channel data of the block to be processed; obtaining the target pixel difference between the pixels in the block to be processed and the corresponding pixels in the reference image, including: obtaining the pixel difference between the pixels in the block to be processed and the corresponding pixels in the reference image under the luminance channel data, and obtaining the target pixel difference; determining the target denoised image corresponding to the image to be processed based on the target filter coefficient characterization value, including: determining the target denoised image obtained by denoising the luminance channel data of the image to be processed based on the target filter coefficient characterization value.

[0103] The three primary color channels refer to the R (red), G (green), and B (blue) channels. Display technology achieves almost any color of visible light by combining different intensities of these three primary colors. In image storage, the method of recording images by recording the red, green, and blue intensities of each pixel is called the RGB model. Common image file formats such as PNG and BMP are based on the RGB model. Besides the RGB model, another widely used model is the YUV model, also known as the Luma-Chroma model. It uses mathematical transformations to convert the RGB three channels into one channel representing luminance (Y, also known as Luma) and two channels representing chromaticity (UV, together known as Chroma) to record the image.

[0104] In a specific embodiment, the formula for converting RGB three-channel data to YUV three-channel data is shown in the following equation:

[0105] Y=0.299*R+0.587*G+0.114*B (3)

[0106] U=-0.169*R-0.331*G+0.5*B (4)

[0107] V=0.5*R-0.419*G-0.081*B (5)

[0108] Where Y represents the luminance channel value, U represents the chroma channel value, V represents the saturation channel value, R represents the R channel value, G represents the G channel value, and B represents the B channel value.

[0109] Considering that formulas (3) to (5) above are floating-point operations, and that floating-point multiplication requires exponent and mantissa calculations within the computer, which is relatively time-consuming, this embodiment uses some mathematical transformations to convert floating-point operations into integer operations. Taking the Y channel as an example, the transformation process is as follows:

[0110] Y=0.299*R+0.587*G+0.114*B=128*(0.299*R+0.587*G+0.114*B)>>7=(38*R+75*B+15*B)>>7

[0111] The U and V channels can be converted in the same way as the Y channel. The amplified values ​​are generated using a four-input, five-output method. After this conversion, floating-point operations become integer multiplication and bit shifting operations. Bit shifting operations are very efficient, and integer multiplication is also more efficient than floating-point operations. Therefore, the obtained Y channel data, U channel data, and V channel data are all integer values.

[0112] In this embodiment, based on the fact that the human eye is more sensitive to the luminance channel components in an image, the computer device can convert the three primary color channel data in the image to be processed into Y channel data, U channel data, and V channel data, and then extract only the Y channel data for noise reduction processing. Similarly, the three primary color channel data in the reference image are converted into Y channel data, U channel data, and V channel data, and only the Y channel data is extracted. Thus, the motion intensity of the luminance channel data of the block to be processed can be determined based on the luminance channel data of the reference image, and the pixel difference between the pixel point in the block to be processed and the corresponding pixel point in the reference image under the luminance channel data can be obtained to obtain the target pixel difference. Finally, based on the target filter coefficient characterization value, the target denoised image obtained by denoising the luminance channel data of the image to be processed is determined.

[0113] In the above embodiments, the image to be processed is converted from the RGB domain to the YUV domain, and then only the luminance channel Y is denoised, which saves the amount of computation in the denoising process and improves the efficiency of image denoising.

[0114] In one embodiment, the method for determining the target denoised image obtained by denoising the luminance channel data of the image to be processed based on the target filter coefficient characterization value includes: determining intermediate processing data obtained by temporal filtering of the luminance channel data in the image to be processed based on the target filter coefficient characterization value; performing spatial denoising based on the intermediate processing data to obtain the target luminance data corresponding to the image to be processed; and combining the target luminance data with the chroma channel data and density channel data of the image to be processed and converting them into three primary color channel data to obtain the target denoised image.

[0115] In this embodiment, since the luminance channel data in the image to be processed is extracted, after the computer device determines the target filtering coefficient characterization value of each pixel, it can determine the intermediate processing data obtained by performing temporal filtering on the luminance channel data in the image to be processed. Furthermore, spatial denoising can be performed based on the intermediate processing data to obtain the target luminance data in the luminance channel. Finally, the target luminance data and the chroma channel data and density channel data previously separated from the image to be processed are combined and then converted into three primary color channel data to obtain the target denoised image.

[0116] In the above embodiments, after performing temporal and spatial denoising on the Y channel data, the Y channel data, U channel data, and V channel data are further combined and converted into RGB data, and the resulting target denoised image can better meet the requirements.

[0117] In one embodiment, the image denoising method further includes: determining the luminance representation value of the block to be processed based on the luminance channel data of the block to be processed; if the luminance representation value is less than or equal to a preset luminance threshold, determining the motion intensity of the block to be processed based on the luminance channel data of the reference image and the luminance channel data of the block to be processed; if the luminance representation value is greater than the preset luminance threshold, using the luminance channel data of the block to be processed as intermediate processing data, and performing spatial denoising based on the intermediate processing data to obtain the target luminance data corresponding to the image to be processed.

[0118] The brightness characterization value is used to characterize the overall brightness of the block to be processed. The brightness characterization value can be obtained by statistically analyzing the Y channel values ​​of each pixel in the block to be processed. This statistical analysis can be one of summation, averaging, or median calculation. In a specific embodiment, assuming the current block is Y with width and height h and w respectively, the brightness characterization value can be calculated using the following formula (6):

[0119] (6)

[0120] Based on visual characteristics, when the image brightness is high, the signal-to-noise ratio is also very high. At this time, the human eye cannot observe the noise signal. Therefore, denoising areas with high brightness values ​​will lead to unnecessary performance waste. In this embodiment, the brightness characterization value of the block to be processed is obtained by statistics. Blocks with brightness characterization values ​​higher than a certain threshold are not subject to temporal domain denoising to achieve the purpose of saving performance.

[0121] In one specific embodiment, reference Figure 6 This is a schematic diagram illustrating the specific process of an image denoising method, which includes the following steps:

[0122] Step 602: Convert RGB to YUV and extract the Y channel.

[0123] Specifically, computer equipment can convert the image to be processed from RGB format to YUV format and extract the Y channel data, and convert the reference image from RGB format to YUV format and extract the Y channel data.

[0124] Step 604: Divide the material into blocks.

[0125] Specifically, computer equipment can divide the image to be processed into multiple blocks.

[0126] Step 606: Determine whether the average brightness within the block exceeds the threshold. If yes, proceed to step 612; otherwise, proceed to step 608.

[0127] Specifically, the computer device calculates the average brightness of each block to be processed. For each block to be processed, if the average brightness is greater than the brightness threshold, proceed to step 612; if the average brightness is less than or equal to the brightness threshold, proceed to step 608.

[0128] Step 608: Estimate the intensity of motion within the block.

[0129] Specifically, for a block to be processed whose average brightness is less than or equal to a brightness threshold, the computer device can determine the motion intensity of the block to be processed based on the Y channel data of the reference image.

[0130] Step 610: Select different time-domain filters for different blocks based on motion intensity.

[0131] Specifically, after determining the motion intensity of the block to be processed, the computer device can acquire filter coefficient description information matching the motion intensity. Thus, for each pixel in the block to be processed whose average brightness is less than or equal to the brightness threshold, the computer device can acquire the target pixel difference between the pixel and the corresponding pixel in the reference image. Based on the filter coefficient description information, the target filter coefficient characterization value corresponding to the target pixel difference is determined, thereby obtaining the target filter coefficient characterization value of each pixel. Based on each target filter coefficient characterization value, the computer device can determine the temporal filtering result obtained by temporal filtering the Y channel data of each pixel in the block to be processed. The temporal filtering results of each pixel constitute the intermediate processing data of the block to be processed.

[0132] Step 612: Combine the blocks.

[0133] Step 614: Perform spatial filtering.

[0134] Specifically, the computer device can combine intermediate processing data from various blocks to be processed, including intermediate processing data from blocks with an average brightness greater than a preset brightness threshold, and intermediate processing data from blocks with an average brightness less than or equal to the preset brightness threshold. Spatial domain noise reduction is then performed on the combined intermediate processing data to obtain the target brightness data corresponding to the image to be processed. For blocks with an average brightness greater than the preset brightness threshold, since no temporal domain noise reduction is performed, the intermediate processing data is the original brightness channel data of that block.

[0135] Step 616: Combine the Y and UV channels and complete the YUV to RGB conversion.

[0136] Specifically, the computer device combines the target brightness data with the U-channel and V-channel data, then converts it into RGB format to obtain the target denoised image.

[0137] In the above embodiments, since RGB format data is converted to YUV format data, noise reduction is only performed on the Y channel data, and no temporal noise reduction is performed on the blocks to be processed with an average brightness greater than a preset brightness threshold, the performance of computer equipment can be saved while satisfying the noise reduction effect, thus avoiding performance waste.

[0138] In one embodiment, determining the motion intensity of a block to be processed in an image to be processed based on a reference image includes: determining the difference degree of the block to be processed relative to the reference image, and the noise intensity of the block to be processed relative to the reference image; determining the motion intensity of the block to be processed based on the difference degree and the noise intensity; wherein the motion intensity and the difference degree are positively correlated, and the motion intensity and the noise intensity are negatively correlated.

[0139] Here, the difference degree is used to characterize the magnitude of the difference between the block to be processed and the reference image; the greater the difference, the greater the difference degree. The noise intensity is used to characterize the noise level of the block to be processed relative to the reference image; the greater the noise, the greater the noise intensity.

[0140] Specifically, for a block to be processed in an image, the computer device can determine the degree of difference between the block to be processed and the corresponding image block in the reference image, as well as the noise intensity of the block to be processed relative to the corresponding image block in the reference image. Based on the degree of difference and the noise intensity, the motion intensity of the block to be processed is determined. Here, motion intensity and degree of difference are positively correlated, and motion intensity and noise intensity are negatively correlated.

[0141] Positive correlation means that, all other things being equal, two variables change in the same direction; when one variable decreases, the other also decreases. It's important to understand that positive correlation means the directions of change are consistent, but it doesn't require that a slight change in one variable necessarily means a change in the other. For example, we can set variable a to 100 when variable a is between 10 and 20, and variable b to 100 when variable a is between 20 and 30. In this case, the direction of change for both a and b is that as a increases, b also increases. However, when a is between 10 and 20, b may remain unchanged. Negative correlation means that when one variable decreases, the other also increases; that is, the directions of change for the two variables are opposite.

[0142] As can be seen, in this embodiment, the greater the difference, the greater the motion intensity, and the smaller the difference, the smaller the motion intensity; the greater the noise intensity, the smaller the motion intensity, and the smaller the noise intensity, the greater the motion intensity.

[0143] In one embodiment, the computer device can obtain the motion intensity by calculating the ratio of difference to noise intensity, i.e., motion intensity = difference / noise intensity.

[0144] In the above embodiments, since the motion intensity of the block to be processed can be determined based on the difference degree and the noise intensity, and the motion intensity is positively correlated with the difference degree and negatively correlated with the noise intensity, the motion intensity can accurately reflect the noise situation of the image to be processed, thereby improving the accuracy of image denoising.

[0145] In one embodiment, determining the difference degree of the block to be processed relative to the reference image, and the noise intensity of the block to be processed relative to the reference image, includes: obtaining the pixel difference value of each pixel in the block to be processed relative to the corresponding pixel in the reference image; determining the noise pixels and moving pixels in the block to be processed based on the pixel difference values ​​of each pixel; wherein, moving pixels are pixels with pixel differences greater than a preset difference threshold, and noise pixels are pixels with pixel differences less than or equal to the preset difference threshold; calculating the pixel difference value of each noise pixel to obtain the noise intensity, and calculating the pixel difference value of each moving pixel to obtain the difference degree.

[0146] In this context, for a pixel in the image to be processed, the pixel corresponding to that position in the reference image refers to the pixel with the same pixel coordinates. For example, if the pixel coordinates of a pixel in the image to be processed are (x1, y1), then the pixel coordinates of the pixel corresponding to that position in the reference image are also (x1, y1).

[0147] Specifically, considering that the movement amplitude of noise is generally small, in this embodiment, a difference threshold N can be preset. For each pixel in the block to be processed, when the pixel difference between the pixel and the corresponding pixel in the reference image is less than or equal to the preset difference threshold, it means that the pixel value at that position is not significantly different between the previous and next frames, and is very likely a noise signal. In this case, the computer device can identify the pixel as a noise pixel. Conversely, when the pixel difference between the pixel and the corresponding pixel in the reference image is greater than the preset difference threshold, it means that the pixel at that position is very likely to have moved, and the computer device can identify the pixel as a moving pixel.

[0148] Computer devices can calculate the noise intensity by analyzing the pixel differences between various noisy pixels in the block to be processed. For example, the computer device can sum the pixel differences between various noisy pixels to obtain the noise intensity. Similarly, computer devices can calculate the difference in motion pixels in the block to be processed by analyzing the difference in motion pixels to obtain the degree of difference.

[0149] It should be noted that in this embodiment, when the computer device obtains the pixel difference between each pixel in the block to be processed and the corresponding pixel in the reference image, the pixel difference refers to the absolute difference. That is, if the pixel value of a certain pixel in the block to be processed is X and the pixel value of the pixel in the reference image corresponding to the position of that pixel is Y, then the pixel difference is |XY|.

[0150] In the above embodiments, noise pixels and moving pixels are determined based on the relationship between pixel differences and preset difference thresholds. Then, the noise intensity can be obtained by statistically analyzing the pixel differences of each noise pixel, and the difference degree can be obtained by statistically analyzing the pixel differences of each moving pixel. Since the pixel difference can reflect whether the pixel at each position has moved, the accuracy of motion intensity calculation is improved.

[0151] In one embodiment, determining the target denoised image corresponding to the image to be processed based on the target filter coefficient characterization value includes: determining an intermediate processing image obtained by temporal filtering of the image to be processed based on the target filter coefficient characterization value, and using the intermediate processing image as the input image and the guide image respectively; downsampling the input image to obtain a first sampled image, downsampling the guide image to obtain a second sampled image; performing guide filtering on the first sampled image based on the second sampled image to obtain the target image; and upsampling the target image according to the size of the input image to obtain a target denoised image with the same size as the input image.

[0152] In this embodiment, after obtaining the intermediate processed image, the computer device uses the intermediate processed image as the input image and the guide image respectively. The input image is further downsampled to reduce its size according to the target scaling ratio to obtain the first sampled image. The guide image is then downsampled to reduce its size according to the target scaling ratio to obtain the second sampled image. Then, guided filtering is performed on the first sampled image based on the second sampled image to obtain the target image. The target image is still a scaled-down image. Therefore, the computer device further upsamples the target image according to the size of the input image to enlarge the target image according to the target scaling ratio, thereby obtaining a target noise-reduced image with the same size as the input image.

[0153] In the above embodiments, when performing guided filtering, since the input image and the guided image are the same image, details can be preserved at the image edges, while smoothing is performed in flat areas, thus avoiding image blurring caused by noise reduction as much as possible. At the same time, since the input image and the guided image are downsampled during the filtering process, the computational complexity of the filtering process can be reduced, saving noise reduction costs.

[0154] In one specific embodiment, the image denoising method of this application can be applied as follows: Figure 7In the illustrated architecture, the image sequence captured by the camera is sequentially used as the image to be processed. After obtaining the target denoised image using the image denoising method provided in this application, video encoding is performed to obtain encoded data. The encoded data is then sent to the cloud, where it decodes the video data and displays the decoded video stream to the user. Alternatively, the computer device can also decode the encoded data locally and display the decoded video stream to the user. The camera can be either a built-in camera or an external camera. The original image captured by the camera contains noisy signals; the image denoising processing in this application produces a clean image, improving image quality. Furthermore, because noise is filtered out, subsequent video encoding stages do not need to encode irregular noise signals, thus producing smaller encoded files and saving bandwidth and storage space.

[0155] In one embodiment, such as Figure 8 As shown, this application also provides a method for processing filtered data, which is executed by a computer device, the computer device being... Figure 1 Terminal 102 in the middle can also be Figure 1 The server 104 can also be a system consisting of terminal 102 and server 104. Specifically, the filtered data processing method includes the following steps:

[0156] Step 802: Obtain multiple reference motion intensity information and determine the pixel difference distribution range.

[0157] Step 804: Based on each reference motion intensity information, determine the filter coefficient characterization value under multiple pixel differences within the pixel difference distribution range.

[0158] Motion intensity reference information refers to the information used as a reference to determine the description information of the filter coefficients. Motion intensity reference information can be a motion intensity range or a specific numerical value. Pixel difference distribution range refers to the range in which all possible pixel differences are distributed; the pixel difference distribution range can be [0, 255].

[0159] Specifically, after acquiring multiple reference motion intensity information and determining the distribution range of pixel differences, the computer device, for each reference motion intensity information, determines the filter coefficient characterization value for each possible pixel difference within the pixel difference distribution range, provided that the filter coefficient characterization value is negatively correlated with the pixel difference and the degree of change of the filter coefficient characterization value is positively correlated with the motion intensity.

[0160] Step 806: Establish the correspondence between the characterization values ​​of each filter coefficient and their corresponding pixel differences.

[0161] Step 808: The corresponding relationship of the filter coefficient characterization values ​​determined based on the same reference motion intensity information is used to form the filter coefficient description information, so as to obtain the filter coefficient description information corresponding to each reference motion intensity information.

[0162] The filter coefficient description information is used to perform temporal filtering on the image to be processed.

[0163] Specifically, the computer device can establish a correspondence between each filter coefficient characterization value and its corresponding pixel difference. From these correspondences, the filter coefficient characterization values ​​included are used to form filter coefficient description information based on the correspondence determined by the same reference motion intensity information. This filter coefficient description information is then used as the filter coefficient description information corresponding to the reference motion intensity information, thereby obtaining the filter coefficient description information corresponding to each reference motion intensity information.

[0164] In one embodiment, the computer device can further establish a correspondence between reference motion intensity information and their respective corresponding filter coefficient description information. Based on this correspondence, when temporal filtering of the image to be processed is required, when obtaining filter coefficient description information that matches the motion intensity of the block to be processed in the image to be processed, the computer device can determine the target motion intensity information that matches the motion intensity from multiple reference motion intensity information, and determine the filter coefficient description information that has a correspondence with the target motion intensity information as the filter coefficient description information that matches the motion intensity, thereby determining temporal filtering of the image to be processed based on the filter coefficient description information.

[0165] In the above-mentioned filtering data processing method, since the filter coefficient description information describes the correspondence between the pixel difference and the filter coefficient characterization value under each reference motion intensity information, when performing temporal filtering on the image to be processed, the pixel difference corresponding to the pixel point in the image to be processed can be directly used as an index to query the corresponding filter coefficient characterization value from the filter coefficient description information, avoiding the need to obtain the filter coefficient through complex calculations and improving the filtering efficiency.

[0166] In one embodiment, determining the characterization value of the filter coefficient under multiple pixel differences within the pixel difference distribution range based on each reference motion intensity information includes: determining the target filter coefficient under multiple pixel differences within the pixel difference distribution range based on each reference motion intensity information; and multiplying each target filter coefficient by its corresponding pixel difference to obtain the characterization value of the filter coefficient under each pixel difference.

[0167] In one embodiment, the above-described filtering data processing method further includes: acquiring a to-be-processed image and a reference image; determining the motion intensity of a block to be processed in the to-be-processed image based on the reference image; the block to be processed is obtained by dividing the to-be-processed image; acquiring filtering coefficient description information matching the motion intensity, wherein the filtering coefficient description information is used to describe the correspondence between pixel differences and filtering coefficient characterization values; wherein the filtering coefficient characterization values ​​are negatively correlated with pixel differences, and the degree of change of the filtering coefficient characterization values ​​is positively correlated with motion intensity; acquiring the target pixel difference between pixels in the to-be-processed block and corresponding pixels in the reference image, and determining the target filtering coefficient characterization value corresponding to the target pixel difference based on the filtering coefficient description information; and determining the target denoised image corresponding to the to-be-processed image based on the target filtering coefficient characterization value.

[0168] It should be noted that the specific description of this embodiment can be found in the description of the embodiments above, and will not be repeated here.

[0169] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0170] In one embodiment, this application also provides an application scenario in which the image denoising method of this application is applied to a video conferencing application to perform denoising processing on video frames in a video conference. Specifically, the computer device can take the video frame sequence captured during the conference, starting from the second frame, and sequentially use them as images to be processed. For each image to be processed, the following steps are performed:

[0171] 1. Convert the image to be processed from RGB format to YUV format.

[0172] 2. Divide the image to be processed into multiple blocks.

[0173] Specifically, assuming the height and width of the image to be processed are H and W respectively, the image can be divided into m rows and n columns, and the height and width of each block are H / m and W / n respectively.

[0174] 3. Take each block in the image to be processed as the current block to be processed. For the current block Y, calculate the average brightness within block Y and determine whether the average brightness exceeds a preset brightness threshold. If it does, it means that the brightness of the block is very high and the noise cannot be perceived by the human eye. In this case, the block does not need to be processed and proceeds directly to step 9. Otherwise, subsequent temporal denoising is required, and proceeds to step 4.

[0175] 4. Perform intra-block motion intensity estimation. First, calculate the frame difference D between each block to be processed at the current time (current frame) t and the previous time (previous frame) t-1, which can be formulated as D(t) = |Y(t) - Y(t-1)|. Considering that the amplitude of noise is generally small, a noise threshold N is artificially set, and let (i, j) be the position index. When the pixel value at coordinates (i, j) is small between frames, it is likely a noise signal; conversely, when the pixel value at coordinates (i, j) is large, it indicates that the pixel value at coordinates (i, j) is not significantly different between frames. At this time, the pixel representing that position has likely moved. Define matrices VN and VS, whose size is consistent with the current block size Y. hour, and .when hour, and Finally, the motion intensity S of the block to be processed can be calculated using the following formula (7):

[0176] (7)

[0177] 5. Determine the target interval to which the exercise intensity belongs from multiple exercise intensity intervals, and determine the filter coefficient description information corresponding to the target interval as the filter coefficient description information that matches the exercise intensity.

[0178] The filter coefficient description information is obtained in advance through the following steps and stored in tabular form:

[0179] 5.1 Divide the range of exercise intensity distribution to obtain multiple exercise intensity intervals.

[0180] 5.2 Determine the pixel difference distribution range. Based on each motion intensity interval, determine the target filter coefficients for multiple pixel differences within the pixel difference distribution range.

[0181] The pixel difference distribution range is [0, 255], and multiple pixel differences within this range are integer values ​​within [0, 255]. When determining the target filter coefficients, it is ensured that the filter coefficients are negatively correlated with the pixel differences, and that the degree of change in the filter coefficients is positively correlated with the motion intensity values ​​within the motion intensity range.

[0182] 5.3 Multiply each target filter coefficient by its corresponding pixel difference to obtain the filter coefficient representation value for each pixel difference.

[0183] 5.4 Establish the correspondence between the characterization values ​​of each filter coefficient and their corresponding pixel differences.

[0184] 5.5. The corresponding relationships of the filter coefficient characterization values ​​determined based on the same motion intensity range are used to form the filter coefficient description information, thus obtaining the filter coefficient description information corresponding to each motion intensity range.

[0185] Specifically, the filter coefficient description information can be stored in tabular form, with the pixel difference serving as the index value of the table. See Table 2 above for details.

[0186] 6. Obtain the target pixel difference between each pixel in the current block to be processed and the corresponding pixel in the reference image. Use each target pixel difference as an index value to query the target filter coefficient characterization value that corresponds to each target pixel difference from the filter coefficient description information that matches the motion intensity of the current block to be processed.

[0187] The reference image is the intermediate processing image corresponding to the video frame at time t-1 (i.e., the previous frame). That is, when the video frame at time t-1 is used as the image to be processed, the image is obtained by performing temporal filtering on the image to be processed. In this embodiment, the temporal filtering process adopts recursive filtering. After each video frame obtains the intermediate processing image of the temporal filtering output by the image denoising method provided in this embodiment, the intermediate processing image is saved as the reference image of the next video frame.

[0188] 7. For each pixel in the current block to be processed, when the pixel value of the pixel is greater than or equal to the pixel value of the corresponding pixel in the reference image, subtract the target filtering coefficient characterization value from the pixel value to obtain the temporal filtering result of the pixel. When the pixel value of the pixel is less than or equal to the pixel value of the corresponding pixel in the reference image, add the target filtering coefficient characterization value to the pixel value to obtain the temporal filtering result of the pixel.

[0189] 8. Determine the intermediate processing data obtained by performing time-domain filtering on the current block to be processed based on the time-domain filtering results of each pixel.

[0190] 9. Combine the intermediate data to be processed, and perform spatial domain noise reduction on the combined intermediate data to obtain the target brightness data corresponding to the image to be processed.

[0191] For blocks whose average brightness is greater than the preset brightness threshold, since no temporal noise reduction processing is performed, the intermediate processed data is the original brightness channel data of the block to be processed.

[0192] It is understandable that the image formed by combining the intermediate processing data here is the intermediate processing image corresponding to the image to be processed. This intermediate processing image can be saved as a reference image for the next frame of the image to be processed.

[0193] 10. Combine the target brightness data and the U-channel and V-channel data of the image to be processed, and convert them into RGB data to obtain the target denoised image.

[0194] In another embodiment, this application also provides another application scenario. In this application scenario, the image denoising method of this application is applied to a video live streaming application to denoise video frames during the live streaming process. Specifically, the computer device can take the video frame sequence acquired during the live streaming process and take it as the image to be processed sequentially starting from the second frame. For each image to be processed, the image denoising method provided in this application embodiment is executed to obtain the target denoised image, thereby improving the video visual quality during the live streaming process. In this embodiment, the filter coefficient description information stores the correspondence between filter coefficients and pixel differences. For each pixel in the block to be processed with a brightness mean greater than or equal to a preset brightness mean, after the computer device obtains the pixel difference corresponding to each pixel, it can query the target filter coefficient from the filter coefficient description information that matches the motion intensity of the block to be processed, and then calculate the temporal filtering result of each pixel using the formula (1) above.

[0195] In a specific embodiment, such as Figure 9 The image shown is a schematic diagram illustrating the effect of the image noise reduction method provided in this embodiment. Figure 9 Figure (a) shows the image before noise reduction. Figure 9 Figure (b) in the diagram is a schematic of the image after noise reduction. To better demonstrate the effect, Figure 9 The image has been enlarged in some areas (the area within the box in the original image). Figure 9 As can be seen, the image before denoising contains many noise particles. After denoising using the image denoising method provided in this application embodiment, the particles are reduced, and the image becomes clearer and smoother. Furthermore, the image denoising method provided in this application embodiment can achieve good denoising effects on both background and foreground regions.

[0196] In a specific embodiment, such as Figure 10 The image shown is a schematic diagram illustrating the effect of the image noise reduction method provided in this embodiment. Figure 10 Figure (a) shows the image before noise reduction. Figure 10 Figure (b) in the diagram shows the image after noise reduction. Figure 10 As can be seen, the image before denoising contains frequently jittering noise points, causing discomfort. After denoising is performed using the image denoising method provided in this application embodiment, these jittering points will disappear.

[0197] Based on the same inventive concept, this application also provides an image denoising apparatus for implementing the image denoising method described above, and a filtering data processing apparatus for implementing the filtering data processing method described above. The solution provided by this apparatus is similar to the solution described in the above methods; therefore, the specific limitations in one or more image denoising apparatus and filtering data processing apparatus embodiments provided below can be found in the limitations in the method embodiments above.

[0198] In one embodiment, such as Figure 11 As shown, an image noise reduction device 1100 is provided, comprising:

[0199] Image acquisition module 1102 is used to acquire the image to be processed and the reference image;

[0200] The motion intensity determination module 1104 is used to determine the motion intensity of the block to be processed in the image to be processed based on the reference image; the block to be processed is obtained by dividing the image to be processed.

[0201] The description information acquisition module 1106 is used to acquire filter coefficient description information that matches the motion intensity. The filter coefficient description information is used to describe the correspondence between pixel difference and filter coefficient characterization value. Among them, the filter coefficient characterization value is negatively correlated with pixel difference, and the degree of change of filter coefficient characterization value is positively correlated with motion intensity.

[0202] The filter coefficient determination module 1108 is used to obtain the target pixel difference between the pixel point in the block to be processed and the corresponding pixel point in the reference image, and to determine the target filter coefficient characterization value that corresponds to the target pixel difference based on the filter coefficient description information.

[0203] The denoised image determination module 1110 is used to determine the target denoised image corresponding to the image to be processed based on the target filter coefficient characterization value.

[0204] In the aforementioned image denoising device, since the blocks to be processed are obtained by dividing the image to be processed, different target filter coefficient representation values ​​can be determined for different blocks to be processed in the image to be processed. The obtained target filter coefficient representation values ​​can be accurately matched with the motion of each region in the image, avoiding the problem of poor denoising effect in some regions caused by motion intensity estimation of the entire image, thus improving the denoising effect of the image to be processed. In addition, since the target filter coefficient representation values ​​are determined by the filter coefficient description information, which describes the correspondence between pixel difference and filter coefficient representation values, and the filter coefficient representation values ​​are negatively correlated with pixel difference and positively correlated with motion intensity, for each pixel value, a target filter coefficient representation value matching that pixel can be obtained, further improving the denoising effect of the image to be processed.

[0205] In one embodiment, the image processing apparatus further includes: a description information determination module, configured to determine multiple reference motion intensity information and determine the pixel difference distribution range; determine the filter coefficient characterization value under multiple pixel differences within the pixel difference distribution range based on each reference motion intensity information; establish a correspondence between each filter coefficient characterization value and its corresponding pixel difference; and form filter coefficient description information by combining the correspondence of the filter coefficient characterization values ​​determined based on the same reference motion intensity information to obtain the filter coefficient description information corresponding to each reference motion intensity information; the description information acquisition module 1106 is further configured to determine the target motion intensity information that matches the motion intensity from the multiple reference motion intensity information, and determine the filter coefficient description information corresponding to the target motion intensity information as the filter coefficient description information that matches the motion intensity.

[0206] In one embodiment, the description information determination module is further configured to determine target filter coefficients for multiple pixel differences within the pixel difference distribution range based on each reference motion intensity information; multiply each target filter coefficient by its corresponding pixel difference to obtain the filter coefficient characterization value for each pixel difference; and the denoised image determination module is further configured to determine the temporal filtering result of the pixel based on the size relationship between the pixel value of the pixel and the pixel value of the corresponding pixel in the reference image, and determine the target denoised image corresponding to the image to be processed based on the temporal filtering result of the pixel.

[0207] In one embodiment, the noise-reduced image determination module is further configured to: subtract the target filtering coefficient characterization value from the pixel value of the pixel when the pixel value of the pixel is greater than or equal to the pixel value of the corresponding pixel in the reference image, to obtain the temporal filtering result of the pixel; and add the target filtering coefficient characterization value to the pixel value when the pixel value of the pixel is less than or equal to the pixel value of the corresponding pixel in the reference image, to obtain the temporal filtering result of the pixel.

[0208] In one embodiment, the description information determination module is further configured to divide the motion intensity distribution range into multiple motion intensity intervals, and use each motion intensity interval as a reference motion intensity information; the description information acquisition module 1106 is further configured to determine the target interval to which the motion intensity belongs from the multiple motion intensity intervals, and determine the filter coefficient description information corresponding to the target interval as the filter coefficient description information that matches the motion intensity.

[0209] In one embodiment, the apparatus further includes a format conversion module, configured to convert the three primary color channel data in the image to be processed into luminance channel data, chroma channel data, and saturation channel data and extract the luminance channel data therefrom; and to convert the three primary color channel data in the reference image into luminance channel data, chroma channel data, and saturation channel data and extract the luminance channel data therefrom; a motion intensity determination module, configured to determine the motion intensity of the block to be processed based on the luminance channel data of the reference image and the luminance channel data of the block to be processed; a filter coefficient determination module, configured to obtain the pixel difference between the pixel point in the block to be processed and the corresponding pixel point in the reference image under the luminance channel data, and obtain the target pixel difference; and a denoised image determination module, configured to determine the target denoised image obtained by denoising the luminance channel data of the image to be processed based on the target filter coefficient characterization value.

[0210] In one embodiment, the denoised image determination module is further configured to determine, based on the target filter coefficient characterization value, intermediate processing data obtained by temporal filtering of the luminance channel data in the image to be processed; perform spatial denoising based on the intermediate processing data to obtain target luminance data corresponding to the image to be processed; and combine the target luminance data with the chroma channel data and density channel data of the image to be processed and convert them into three primary color channel data to obtain the target denoised image.

[0211] In one embodiment, the above apparatus further includes: a luminance characterization value determination module, configured to determine the luminance characterization value of the block to be processed based on the luminance channel data of the block to be processed; if the luminance characterization value is less than or equal to a preset luminance threshold, proceed to determine the motion intensity of the block to be processed based on the luminance channel data of the reference image and the luminance channel data of the block to be processed; if the luminance characterization value is greater than the preset luminance threshold, use the luminance channel data of the block to be processed as intermediate processing data, and proceed to perform spatial noise reduction based on the intermediate processing data to obtain the target luminance data corresponding to the image to be processed.

[0212] In one embodiment, the motion intensity determination module is further configured to determine the degree of difference between the block to be processed and the reference image, and the noise intensity of the block to be processed relative to the reference image; and to determine the motion intensity of the block to be processed based on the degree of difference and the noise intensity; the motion intensity and the degree of difference are positively correlated, and the motion intensity and the noise intensity are negatively correlated.

[0213] In one embodiment, the motion intensity determination module is further configured to acquire the pixel difference between each pixel in the block to be processed and the corresponding pixel in the reference image; determine the noise pixels and moving pixels in the block to be processed based on the pixel differences of each pixel; wherein, the moving pixels are pixels whose pixel differences are greater than a preset difference threshold, and the noise pixels are pixels whose pixel differences are less than or equal to the preset difference threshold; calculate the noise intensity by calculating the pixel differences of each noise pixel, and calculate the difference degree by calculating the pixel differences of each moving pixel.

[0214] In one embodiment, the denoised image determination module is further configured to determine an intermediate processed image obtained by temporal filtering of the image to be processed based on the target filter coefficient characterization value, and use the intermediate processed image as the input image and the guide image respectively; downsample the input image to obtain a first sampled image, downsample the guide image to obtain a second sampled image; perform guide filtering on the first sampled image based on the second sampled image to obtain a target image; and upsample the target image according to the size of the input image to obtain a target denoised image with the same size as the input image.

[0215] In one embodiment, the image acquisition module is further configured to determine the target video to be denoised; use the video frames in the target video as the images to be processed, determine the target video frame from the forward video frames corresponding to the images to be processed; acquire the target denoised image corresponding to the target video frame, and determine the target denoised image corresponding to the target video frame as the reference image corresponding to the image to be processed.

[0216] In one embodiment, such as Figure 12 As shown, a filtered data processing device 1200 is provided, comprising:

[0217] The reference motion intensity determination module 1202 is used to determine multiple reference motion intensity information and determine the distribution range of pixel differences;

[0218] The characterization value determination module 1204 is used to determine the characterization value of the filter coefficient under multiple pixel differences within the pixel difference distribution range based on each reference motion intensity information.

[0219] The correspondence establishment module 1206 is used to establish the correspondence between the characterization values ​​of each filter coefficient and their corresponding pixel differences;

[0220] The description information determination module 1208 is used to form the filter coefficient description information by combining the correspondence of the filter coefficient characterization values ​​determined based on the same reference motion intensity information, so as to obtain the filter coefficient description information corresponding to each reference motion intensity information; wherein, the filter coefficient description information is used to perform temporal filtering on the image to be processed.

[0221] The aforementioned filtering data processing methods, apparatus, computer equipment, storage media, and computer program products, because the filtering coefficient description information describes the correspondence between pixel differences and filtering coefficient characterization values ​​under each reference motion intensity information, can directly use the pixel differences corresponding to the pixels in the image to be processed as an index to query the corresponding filtering coefficient characterization values ​​from the filtering coefficient description information when performing temporal filtering on the image to be processed. This avoids obtaining the filter coefficients through complex calculations and improves filtering efficiency.

[0222] In one embodiment, the characterization value determination module is used to determine the target filtering coefficients under multiple pixel differences within the pixel difference distribution range based on each reference motion intensity information; and to multiply each target filtering coefficient by its corresponding pixel difference to obtain the characterization value of the filtering coefficient under each pixel difference.

[0223] Each module in the aforementioned image noise reduction device and filtering data processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0224] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 13 As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores image data, filter coefficient description information, and other data. The I / O interfaces are used for information exchange between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements an image denoising method or a filtering data processing method.

[0225] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 14 As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements an image noise reduction method or a filtered data processing method. The display unit of the computer device is used to form a visually visible image. It can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0226] Those skilled in the art will understand that Figure 13 and Figure 14 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0227] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described image denoising method or filtered data processing method.

[0228] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the above-described image denoising method or filtering data processing method.

[0229] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the above-described image denoising method or filtered data processing method.

[0230] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0231] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0232] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0233] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An image denoising method, characterized in that, The method includes: Obtain the image to be processed and the reference image; Determine the degree of difference between the block to be processed and the reference image, and the noise intensity of the block to be processed relative to the reference image; Based on the difference degree and the noise intensity, the motion intensity of the block to be processed is determined; the motion intensity and the difference degree are positively correlated, and the motion intensity and the noise intensity are negatively correlated; the block to be processed is obtained by dividing the image to be processed. Obtain filter coefficient description information that matches the motion intensity, wherein the filter coefficient description information is used to describe the correspondence between pixel difference and filter coefficient characterization value; The filter coefficient characterization value is a numerical value that characterizes the filter coefficient. The filter coefficient characterized by the filter coefficient characterization value is negatively correlated with the pixel difference. The degree of change of the filter coefficient characterized by the filter coefficient characterization value is positively correlated with the motion intensity. Obtain the target pixel difference between the pixel in the block to be processed and the corresponding pixel in the reference image, and query the target filter coefficient characterization value that corresponds to the target pixel difference from the filter coefficient description information; Based on the target filter coefficient characterization value, the target denoised image corresponding to the image to be processed is determined.

2. The method according to claim 1, characterized in that, The method further includes: Multiple reference motion intensity information are determined, and the distribution range of pixel differences is determined; Based on each reference motion intensity information, determine the filter coefficient characterization value for multiple pixel differences within the pixel difference distribution range; Establish the correspondence between the characterization values ​​of each filter coefficient and their corresponding pixel differences; The corresponding relationship of the filter coefficient characterization values ​​determined based on the same reference motion intensity information is used to form the filter coefficient description information, thus obtaining the filter coefficient description information corresponding to each reference motion intensity information. The step of obtaining the filter coefficient description information matching the motion intensity includes: From the plurality of reference motion intensity information, a target motion intensity information matching the motion intensity is determined, and the filter coefficient description information corresponding to the target motion intensity information is determined as the filter coefficient description information matching the motion intensity.

3. The method according to claim 2, characterized in that, The step of determining the filter coefficient representation values ​​for multiple pixel differences within the pixel difference distribution range based on each reference motion intensity information includes: Based on each reference motion intensity information, the target filter coefficients for multiple pixel differences within the pixel difference distribution range are determined. Multiply each target filter coefficient by its corresponding pixel difference to obtain the filter coefficient representation value for each pixel difference; The step of determining the target denoised image corresponding to the image to be processed based on the target filter coefficient characterization value includes: Based on the relationship between the pixel value of the pixel and the pixel value of the corresponding pixel in the reference image, the temporal filtering result of the pixel is determined, and the target denoising image corresponding to the image to be processed is determined based on the temporal filtering result of the pixel.

4. The method according to claim 3, characterized in that, The step of determining the temporal filtering result of the pixel based on the relationship between the pixel value of the pixel and the pixel value of the corresponding pixel in the reference image includes: When the pixel value of the pixel is greater than or equal to the pixel value of the corresponding pixel in the reference image, the pixel value is subtracted from the target filtering coefficient characterization value to obtain the temporal filtering result of the pixel. When the pixel value of the pixel is less than or equal to the pixel value of the corresponding pixel in the reference image, the pixel value is added to the target filtering coefficient characterization value to obtain the temporal filtering result of the pixel.

5. The method according to claim 2, characterized in that, The determination of multiple reference motion intensity information includes: The range of exercise intensity distribution is divided into multiple exercise intensity intervals, and each exercise intensity interval is used as a reference exercise intensity information. The step of determining the target motion intensity information that matches the motion intensity from the plurality of reference motion intensity information, and determining the filter coefficient description information corresponding to the target motion intensity information as the filter coefficient description information that matches the motion intensity, includes: From the plurality of motion intensity intervals, determine the target interval to which the motion intensity belongs, and determine the filter coefficient description information corresponding to the target interval as the filter coefficient description information that matches the motion intensity.

6. The method according to claim 1, characterized in that, Before determining the difference between the block to be processed and the reference image, and the noise intensity of the block to be processed relative to the reference image, the method further includes: The three primary color channel data in the image to be processed are converted into luminance channel data, chroma channel data and saturation channel data, and the luminance channel data is extracted. The three primary color channel data in the reference image are converted into luminance channel data, chroma channel data and saturation channel data, and the luminance channel data is extracted. Determining the difference degree of the block to be processed relative to the reference image, and the noise intensity of the block to be processed relative to the reference image, includes: The motion intensity of the block to be processed is determined based on the luminance channel data of the reference image and the luminance channel data of the block to be processed. The step of obtaining the target pixel difference between the pixels in the block to be processed and the corresponding pixels in the reference image includes: The pixel difference between the pixel in the block to be processed and the corresponding pixel in the reference image is obtained under the brightness channel data to obtain the target pixel difference. The step of determining the target denoised image corresponding to the image to be processed based on the target filter coefficient characterization value includes: Based on the target filter coefficient characterization value, the target denoised image obtained by denoising the brightness channel data of the image to be processed is determined.

7. The method according to claim 6, characterized in that, The step of determining the target denoised image obtained by denoising the luminance channel data of the image to be processed based on the target filter coefficient characterization value includes: Based on the target filter coefficient characterization value, the intermediate processing data obtained by performing temporal filtering on the brightness channel data in the image to be processed is determined. Spatial domain noise reduction is performed based on the intermediate processing data to obtain the target brightness data corresponding to the image to be processed. The target brightness data and the chroma channel data and density channel data of the image to be processed are combined and converted into three primary color channel data to obtain the target noise-reduced image.

8. The method according to claim 7, characterized in that, The method further includes: The brightness characterization value of the block to be processed is determined based on the brightness channel data of the block to be processed; If the brightness characterization value is less than or equal to a preset brightness threshold, the process proceeds to determine the motion intensity of the block to be processed based on the brightness channel data of the reference image and the brightness channel data of the block to be processed. If the brightness characterization value is greater than a preset brightness threshold, the brightness channel data of the block to be processed is used as intermediate processing data, and the process proceeds to the step of performing spatial noise reduction based on the intermediate processing data to obtain the target brightness data corresponding to the image to be processed.

9. The method according to claim 1, characterized in that, Determining the difference degree of the block to be processed relative to the reference image, and the noise intensity of the block to be processed relative to the reference image, includes: Obtain the pixel difference between each pixel in the block to be processed and the corresponding pixel in the reference image; Based on the pixel difference corresponding to each pixel, noise pixels and moving pixels in the block to be processed are determined; wherein, the moving pixels are pixels with a pixel difference greater than a preset difference threshold, and the noise pixels are pixels with a pixel difference less than or equal to the preset difference threshold. The noise intensity is obtained by calculating the pixel difference of each noisy pixel, and the difference is obtained by calculating the pixel difference of each moving pixel.

10. The method according to claim 1, characterized in that, The step of determining the target denoised image corresponding to the image to be processed based on the target filter coefficient characterization value includes: Based on the target filter coefficient characterization value, an intermediate processing image obtained by performing temporal filtering on the image to be processed is determined, and the intermediate processing image is used as the input image and the guide image, respectively. The input image is downsampled to obtain a first sampled image, and the guide image is downsampled to obtain a second sampled image; The target image is obtained by performing guided filtering on the first sampled image based on the second sampled image. The target image is upsampled according to the size of the input image to obtain a target denoised image with the same size as the input image.

11. The method according to any one of claims 1 to 10, characterized in that, The acquisition of the image to be processed and the reference image includes: Identify the target video to be denoised; The video frames in the target video are used as images to be processed, and the target video frames are determined from the forward video frames corresponding to the images to be processed. Obtain the target denoised image corresponding to the target video frame, and determine the target denoised image corresponding to the target video frame as the reference image corresponding to the image to be processed.

12. A method for processing filtered data, characterized in that, The method includes: Multiple reference motion intensity information are determined, and the distribution range of pixel differences is determined; Based on each reference motion intensity information, filter coefficient characterization values ​​are determined for multiple pixel differences within the pixel difference distribution range; wherein, the filter coefficient characterization value is a numerical value that characterizes the filter coefficient, the filter coefficient characterized by the filter coefficient characterization value is negatively correlated with the pixel difference within the pixel difference distribution range, and the degree of change of the filter coefficient characterized by the filter coefficient characterization value is positively correlated with the motion intensity described by the reference motion intensity information; Establish the correspondence between the characterization values ​​of each filter coefficient and their corresponding pixel differences; The corresponding relationship of the filter coefficient characterization values ​​determined based on the same reference motion intensity information is used to form the filter coefficient description information, thus obtaining the filter coefficient description information corresponding to each reference motion intensity information. The filtering coefficient description information is used to perform temporal filtering on the image to be processed.

13. The method according to claim 12, characterized in that, The step of determining the filter coefficient representation values ​​for multiple pixel differences within the pixel difference distribution range based on each reference motion intensity information includes: Based on each reference motion intensity information, the target filtering coefficients for multiple pixel differences within the pixel difference distribution range are determined. Each target filter coefficient is multiplied by its corresponding pixel difference to obtain the filter coefficient representation value for each pixel difference.

14. An image noise reduction device, characterized in that, The device includes: The image acquisition module is used to acquire the image to be processed and the reference image; A motion intensity determination module is used to determine the difference degree of the block to be processed relative to the reference image, and the noise intensity of the block to be processed relative to the reference image; based on the difference degree and the noise intensity, the motion intensity of the block to be processed is determined; the motion intensity and the difference degree are positively correlated, and the motion intensity and the noise intensity are negatively correlated; the block to be processed is obtained by dividing the image to be processed; The description information acquisition module is used to acquire filter coefficient description information matching the motion intensity. The filter coefficient description information is used to describe the correspondence between pixel difference and filter coefficient characterization value. The filter coefficient characterization value is a numerical value that characterizes the filter coefficient. The filter coefficient characterization value is negatively correlated with the pixel difference, and the degree of change of the filter coefficient characterization value is positively correlated with the motion intensity. The filter coefficient determination module is used to obtain the target pixel difference between the pixel point in the block to be processed and the corresponding pixel point in the reference image, and to query the target filter coefficient characterization value that corresponds to the target pixel difference from the filter coefficient description information. The denoised image determination module is used to determine the target denoised image corresponding to the image to be processed based on the target filter coefficient characterization value.

15. The apparatus according to claim 14, characterized in that, The device further includes: a description information determination module, configured to determine multiple reference motion intensity information and determine the pixel difference distribution range; determine the filter coefficient characterization value under multiple pixel differences within the pixel difference distribution range based on each reference motion intensity information; establish a correspondence between each filter coefficient characterization value and its corresponding pixel difference; and form filter coefficient description information by combining the correspondence of filter coefficient characterization values ​​determined based on the same reference motion intensity information to obtain filter coefficient description information corresponding to each reference motion intensity information; the description information acquisition module is further configured to: determine target motion intensity information matching the motion intensity from the multiple reference motion intensity information, and determine the filter coefficient description information corresponding to the target motion intensity information as the filter coefficient description information matching the motion intensity.

16. The apparatus according to claim 15, characterized in that, The description information determination module is further configured to: determine the target filter coefficients under multiple pixel differences within the pixel difference distribution range based on each reference motion intensity information; and multiply each target filter coefficient by its corresponding pixel difference to obtain the filter coefficient characterization value under each pixel difference. The noise reduction image determination module is further configured to determine the temporal filtering result of the pixel based on the size relationship between the pixel value of the pixel and the pixel value of the corresponding position pixel in the reference image, and to determine the target noise reduction image corresponding to the image to be processed based on the temporal filtering result of the pixel.

17. The apparatus according to claim 16, characterized in that, The noise-reduced image determination module is further configured to: when the pixel value of the pixel is greater than or equal to the pixel value of the corresponding pixel in the reference image, subtract the target filtering coefficient characterization value from the pixel value to obtain the temporal filtering result of the pixel; and when the pixel value of the pixel is less than or equal to the pixel value of the corresponding pixel in the reference image, add the target filtering coefficient characterization value to the pixel value to obtain the temporal filtering result of the pixel.

18. The apparatus according to claim 15, characterized in that, The description information determination module is also used to divide the range of motion intensity distribution into multiple motion intensity intervals, and use each motion intensity interval as a reference motion intensity information. The description information acquisition module is also used to determine the target interval to which the motion intensity belongs from the plurality of motion intensity intervals, and to determine the filter coefficient description information corresponding to the target interval as the filter coefficient description information that matches the motion intensity.

19. The apparatus according to claim 14, characterized in that, The device further includes: a format conversion module, used to convert the three primary color channel data in the image to be processed into luminance channel data, chroma channel data, and saturation channel data and extract the luminance channel data therefrom; and to convert the three primary color channel data in the reference image into luminance channel data, chroma channel data, and saturation channel data and extract the luminance channel data therefrom; a motion intensity determination module, used to determine the motion intensity of the block to be processed based on the luminance channel data of the reference image and the luminance channel data of the block to be processed; a filter coefficient determination module, used to obtain the pixel difference between the pixel point in the block to be processed and the corresponding pixel point in the reference image under the luminance channel data, and obtain the target pixel difference; and a denoised image determination module, used to determine the target denoised image obtained by denoising the luminance channel data of the image to be processed based on the target filter coefficient characterization value.

20. The apparatus according to claim 19, characterized in that, The noise reduction image determination module is further configured to determine, based on the target filter coefficient characterization value, intermediate processing data obtained by performing temporal filtering on the brightness channel data in the image to be processed; Spatial domain noise reduction is performed based on the intermediate processing data to obtain the target brightness data corresponding to the image to be processed. The target brightness data and the chroma channel data and density channel data of the image to be processed are combined and converted into three primary color channel data to obtain the target noise-reduced image.

21. The apparatus according to claim 20, characterized in that, The device further includes a brightness characterization value determination module, configured to: determine the brightness characterization value of the block to be processed based on the brightness channel data of the block to be processed; if the brightness characterization value is less than or equal to a preset brightness threshold, proceed to the step of determining the motion intensity of the block to be processed based on the brightness channel data of the reference image and the brightness channel data of the block to be processed; if the brightness characterization value is greater than the preset brightness threshold, use the brightness channel data of the block to be processed as intermediate processing data, and proceed to the step of performing spatial noise reduction based on the intermediate processing data to obtain the target brightness data corresponding to the image to be processed.

22. The apparatus according to claim 14, characterized in that, The motion intensity determination module is further configured to acquire the pixel difference between each pixel in the block to be processed and the corresponding pixel in the reference image; based on the pixel difference between each pixel, determine the noise pixels and moving pixels in the block to be processed; wherein, the moving pixels are pixels with a pixel difference greater than a preset difference threshold, and the noise pixels are pixels with a pixel difference less than or equal to the preset difference threshold; calculate the pixel difference between each noise pixel to obtain the noise intensity, and calculate the pixel difference between each moving pixel to obtain the difference degree.

23. The apparatus according to claim 14, characterized in that, The noise reduction image determination module is further configured to: determine an intermediate processing image obtained by performing temporal filtering on the image to be processed based on the target filtering coefficient characterization value; use the intermediate processing image as the input image and the guide image respectively to downsample the input image to obtain a first sampled image; and downsample the guide image to obtain a second sampled image. Guided filtering is performed on the first sampled image based on the second sampled image to obtain the target image; the target image is upsampled according to the size of the input image to obtain a target denoised image with the same size as the input image.

24. The apparatus according to any one of claims 14 to 23, characterized in that, The image acquisition module is further configured to: determine the target video to be denoised; use the video frames in the target video as images to be processed, and determine the target video frame from the forward video frames corresponding to the images to be processed; acquire the target denoised image corresponding to the target video frame, and determine the target denoised image corresponding to the target video frame as the reference image corresponding to the image to be processed.

25. A filtered data processing device, characterized in that, The device includes: The reference motion intensity determination module is used to determine multiple reference motion intensity information and determine the distribution range of pixel differences; The characterization value determination module is used to determine the characterization value of the filter coefficient under multiple pixel differences within the pixel difference distribution range based on each reference motion intensity information; wherein, the characterization value of the filter coefficient is a numerical value that characterizes the filter coefficient, the filter coefficient represented by the characterization value is negatively correlated with the pixel difference within the pixel difference distribution range, and the degree of change of the filter coefficient represented by the characterization value is positively correlated with the motion intensity described by the reference motion intensity information; The correspondence establishment module is used to establish the correspondence between the characterization values ​​of each filter coefficient and their corresponding pixel differences; The description information determination module is used to form the filter coefficient description information by combining the correspondence of the filter coefficient characterization values ​​determined based on the same reference motion intensity information, so as to obtain the filter coefficient description information corresponding to each reference motion intensity information; wherein, the filter coefficient description information is used to perform temporal filtering on the image to be processed.

26. The apparatus according to claim 25, characterized in that, The characterization value determination module is used to: determine the target filtering coefficients under multiple pixel differences within the pixel difference distribution range based on each reference motion intensity information; and multiply each target filtering coefficient by its corresponding pixel difference to obtain the characterization value of the filtering coefficient under each pixel difference.

27. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 11 or 12 to 13.

28. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 11 or 12 to 13.

29. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 11 or 12 to 13.

Images