Image denoising method and electronic device

Abstract

The application discloses an image denoising method and an electronic device. The image denoising method comprises the following steps: acquiring a to-be-denoised image frame and a neighboring image frame corresponding to the to-be-denoised image frame; extracting a feature description vector corresponding to a current pixel point in the to-be-denoised image frame; selecting a pixel point from the neighboring image frame, wherein the feature description vector of the pixel point and the current pixel point satisfy a preset condition, and obtaining an inter-frame matching point corresponding to the current pixel point; and performing denoising processing on the current pixel point based on the inter-frame matching point, and obtaining a denoised image frame corresponding to the to-be-denoised image frame. The calculation amount can be effectively reduced under the premise of ensuring matching accuracy.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image noise reduction method and electronic device. Background Technology

[0002] Images have a wide range of applications, including biomedicine, military, transportation security, and machine vision. The quality of an image directly affects its effectiveness in these fields. However, images are inevitably affected by noise during acquisition, processing, and transmission. Therefore, filtering out noise from images is of paramount importance.

[0003] Currently, inter-frame matching can be performed using methods such as optical flow and block matching to achieve motion-compensated image denoising. However, optical flow matching methods lack sufficient accuracy in matching texture details, affecting the denoising effect; while block matching methods also suffer from insufficient accuracy when using small blocks, and using large blocks increases computational resource requirements. Summary of the Invention

[0004] To address the aforementioned problems, this application provides at least one image noise reduction method and electronic device.

[0005] The first aspect of this application provides an image denoising method, the method comprising: acquiring an image frame to be denoised and adjacent image frames corresponding to the image frame to be denoised; extracting the pixel features of the current pixel in the image frame to be denoised relative to the pixel in the image frame to be denoised, and constructing a feature description vector corresponding to the current pixel; selecting pixels from adjacent image frames whose feature description vector similarity with the current pixel satisfies a preset condition, and obtaining an inter-frame matching point corresponding to the current pixel; and performing denoising processing on the current pixel based on the inter-frame matching point to obtain a denoised image frame corresponding to the image frame to be denoised.

[0006] In one embodiment, extracting the pixel features of the current pixel in the image frame to be denoised relative to the pixel features of the image frame to be denoised, and constructing the feature description vector corresponding to the current pixel, includes: calculating the pixel value similarity between the current pixel and other pixels in the image frame to be denoised; selecting pixels in the image frame to be denoised whose pixel value similarity with the current pixel meets a preset condition to obtain the intra-frame matching point corresponding to the current pixel; and constructing the feature description vector corresponding to the current pixel using the pixel value information of the current pixel and the intra-frame matching point.

[0007] In one embodiment, calculating the pixel value similarity between the current pixel and other pixels in the image frame to be denoised includes: dividing the image frame to be denoised into an intra-frame point retrieval region corresponding to the current pixel based on the position of the current pixel in the image frame to be denoised; using the pixels in the intra-frame point retrieval region as intra-frame candidate points, and obtaining the pixel value information of the current pixel and the intra-frame candidate points respectively; and calculating the pixel value similarity between the current pixel and the intra-frame candidate points based on the pixel value information of the current pixel and the intra-frame candidate points.

[0008] In one embodiment, obtaining pixel value information of the current pixel and intra-frame candidate points respectively includes: determining the neighboring pixels corresponding to the current pixel and the neighboring pixels corresponding to the intra-frame candidate points; extracting the pixel values ​​of the current pixel and the neighboring pixels corresponding to the current pixel to obtain the pixel value information of the current pixel; and extracting the pixel values ​​of the intra-frame candidate points and the neighboring pixels corresponding to the intra-frame candidate points to obtain the pixel value information of the intra-frame candidate points.

[0009] In one embodiment, the pixel value similarity between the current pixel and the intra-frame candidate points is calculated based on the pixel value information of the current pixel and the intra-frame candidate points, including: performing a difference calculation on the pixel value information of the current pixel and the pixel value information of the intra-frame candidate points; and obtaining the pixel value similarity between the current pixel and the intra-frame candidate points based on the calculation result of the difference calculation.

[0010] In one embodiment, the feature description vector corresponding to the current pixel is constructed using the pixel value information of the current pixel and the intra-frame matching point, including: obtaining the pixel value information of the current pixel and the intra-frame matching point respectively; constructing description information corresponding to the current pixel and the intra-frame matching point respectively based on the pixel value information of the current pixel and the intra-frame matching point; and constructing the feature description vector corresponding to the current pixel by combining the description information of the current pixel and the intra-frame matching point.

[0011] In one embodiment, the pixel value information corresponding to the current pixel contains the pixel value of the current pixel and the pixel values ​​of the adjacent pixels corresponding to the current pixel, and the pixel value information corresponding to the intra-frame matching point contains the pixel value of the intra-frame matching point and the pixel values ​​of the adjacent pixels corresponding to the intra-frame matching point. Based on the pixel value information of the current pixel and the intra-frame matching point, description information corresponding to the current pixel and the intra-frame matching point is constructed respectively, including: calculating the difference between the pixel value of the current pixel and the pixel values ​​of the adjacent pixels corresponding to the current pixel to obtain the pixel difference of the current point; and calculating the difference between the pixel value of the intra-frame matching point and the pixel values ​​of the adjacent pixels corresponding to the intra-frame matching point to obtain the pixel difference of the intra-frame point; constructing description information corresponding to the current pixel based on the pixel value of the current pixel and the pixel difference of the current point; and constructing description information corresponding to the intra-frame matching point based on the pixel value of the intra-frame matching point and the pixel difference of the intra-frame point.

[0012] In one embodiment, selecting pixels from adjacent image frames whose feature description vector similarity with the current pixel satisfies a preset condition to obtain inter-frame matching points corresponding to the current pixel includes: dividing the adjacent image frames into inter-frame point retrieval regions corresponding to the current pixel based on the position of the current pixel relative to the adjacent image frames; using the pixels in the inter-frame point retrieval regions as inter-frame candidate points and obtaining the feature description vectors of the inter-frame candidate points; calculating the similarity between the feature description vector of the current pixel and the feature description vector of the inter-frame candidate points to obtain the feature description vector similarity; and selecting inter-frame candidate points in the inter-frame point retrieval regions whose feature description vector similarity satisfies a preset condition to obtain inter-frame matching points corresponding to the current pixel.

[0013] In one embodiment, denoising the current pixel based on inter-frame matching points to obtain a denoised image frame corresponding to the image frame to be denoised includes: constructing an inter-frame correlation vector corresponding to the current pixel based on each inter-frame matching point; using the similarity of the feature description vector between each inter-frame matching point and the current pixel as a weight parameter of the inter-frame correlation vector, and weighting the inter-frame correlation vector to obtain a weighted inter-frame vector; and denoising the current pixel based on the weighted inter-frame vector to obtain a denoised image frame corresponding to the image frame to be denoised.

[0014] A second aspect of this application provides an image denoising apparatus, comprising: a frame acquisition module for acquiring an image frame to be denoised and adjacent image frames corresponding to the image frame to be denoised; a description vector construction module for extracting pixel features of the current pixel in the image frame to be denoised relative to the pixel in the image frame to be denoised, and constructing a feature description vector corresponding to the current pixel; an inter-frame matching module for selecting pixels from adjacent image frames whose feature description vector similarity with the current pixel meets a preset condition, and obtaining an inter-frame matching point corresponding to the current pixel; and a denoising module for performing denoising processing on the current pixel based on the inter-frame matching point, and obtaining a denoised image frame corresponding to the image frame to be denoised.

[0015] A third aspect of this application provides an electronic device, including a memory and a processor, wherein the processor is configured to execute program instructions stored in the memory to implement the above-described image noise reduction method.

[0016] The fourth aspect of this application provides a computer-readable storage medium having program instructions stored thereon, which, when executed by a processor, implement the above-described image denoising method.

[0017] The above scheme obtains the image frame to be denoised and its adjacent image frames; extracts the pixel features of the current pixel in the image frame to be denoised relative to the pixel features of the image frame to be denoised, and constructs the feature description vector corresponding to the current pixel; selects pixels from the adjacent image frames whose feature description vectors have a similarity to the current pixel that meets a preset condition, and obtains the inter-frame matching point corresponding to the current pixel; performs denoising processing on the current pixel based on the inter-frame matching point to obtain the denoised image frame corresponding to the image frame to be denoised, which can reduce the amount of computation while ensuring matching accuracy.

[0018] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this application. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application.

[0020] Figure 1 This is a schematic diagram of the implementation environment of a solution provided in one embodiment of this application;

[0021] Figure 2 This is a flowchart illustrating an exemplary embodiment of the image noise reduction method of this application;

[0022] Figure 3 This is a schematic diagram illustrating the acquisition of pixel value information in an exemplary embodiment of this application;

[0023] Figure 4 This is a schematic diagram illustrating an intra-frame matching point in an exemplary embodiment of this application;

[0024] Figure 5 This is a schematic diagram illustrating the acquisition of descriptive information of the current pixel point, as shown in an exemplary embodiment of this application;

[0025] Figure 6 This is a schematic diagram illustrating the division of the inter-frame point retrieval region as an exemplary embodiment of this application;

[0026] Figure 7 This is a schematic diagram illustrating inter-frame matching points in an exemplary embodiment of this application;

[0027] Figure 8 This is a block diagram illustrating an image noise reduction apparatus according to an exemplary embodiment of this application;

[0028] Figure 9 This is a schematic diagram of the structure of an electronic device shown in an exemplary embodiment of this application;

[0029] Figure 10 This is a schematic diagram illustrating the structure of a computer-readable storage medium, as shown in an exemplary embodiment of this application. Detailed Implementation

[0030] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0031] In the following description, specific details such as particular system architectures, interfaces, and technologies are presented for illustrative purposes rather than for limiting purposes, in order to provide a thorough understanding of this application.

[0032] In this document, the term "and / or" is merely a description of the association information of related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "many" in this document means two or more. Moreover, the term "at least one" in this document means any combination of at least two of any one or more of a plurality of elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0033] The image noise reduction method provided in the embodiments of this application will be described below.

[0034] Please refer to Figure 1 The diagram illustrates an implementation environment provided in one embodiment of this application. This implementation environment may include a terminal 110 and a server 120, which are interconnected.

[0035] The number of terminals 110 can be one or more. Terminal 110 can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, etc., but is not limited to these.

[0036] Server 120 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0037] In one example, server 120 can perform noise reduction processing on the image frame sequence obtained from terminal 110 to obtain a noise-reduced image frame sequence. Then, server 120 can store the noise-reduced image frame sequence locally, send it back to terminal 110, or transmit it to other terminals 110.

[0038] In one example, a client running a target application is installed on terminal 110. This target application could be an application that provides image denoising functionality, used to denoise image frame sequences. Server 120 could be a backend server for this target application, providing background services to the client of the target application.

[0039] The image noise reduction method provided in this application embodiment can be executed by a terminal 110, such as the client of the target application installed and running on the terminal 110, or by a server 120, or by the interaction and cooperation between the terminal 110 and the server 120. That is, some steps of the method are executed by the terminal 110 and other steps are executed by the server 120. This application does not limit this.

[0040] Please see Figure 2 , Figure 2 This is a flowchart illustrating an exemplary embodiment of an image denoising method according to this application. This image denoising method can be applied to… Figure 1 The implementation environment shown is specifically executed by a server within that implementation environment. It should be understood that this method can also be applied to other exemplary implementation environments and executed by devices in other implementation environments; this embodiment does not limit the implementation environment to which the method is applicable.

[0041] like Figure 2 As shown, the image denoising method includes at least steps S210 to S240, which are described in detail below:

[0042] Step S210: Obtain the image frame to be denoised and its adjacent image frames.

[0043] Among them, the image frame to be denoised refers to the image frame that needs to be denoised. It can be the image frame that is traversed and denoised for each image frame in the image frame sequence. The image frame to be denoised is the image frame that is currently traversed.

[0044] Among them, the adjacent image frame corresponding to the image frame to be denoised refers to the image frame that is adjacent to the image frame to be denoised in the image frame sequence.

[0045] For example, if an image frame sequence contains N image frames ordered chronologically, and the image frames to be denoised are represented as F... n Then, the adjacent image frames corresponding to the image frame to be denoised can be represented as F. n-1 F n+1 , where n = 1, 2, ..., N.

[0046] Currently, in addition to using the frame immediately before and after the image frame to be denoised as adjacent image frames in the above example, the selection range of adjacent image frames can be expanded. For example, the frame frames before and after the image frame to be denoised can be used as adjacent image frames. That is, all image frames in the image frame sequence between n-FR and n+FR are used as adjacent image frames corresponding to the image frame to be denoised. The value of FR can be preset or flexibly calculated according to the current situation, such as based on the denoising accuracy, the image quality of the image frame to be denoised, etc. This application does not limit this.

[0047] Step S220: Extract the pixel value features of the current pixel in the image frame to be denoised relative to the image frame to be denoised, and construct the feature description vector corresponding to the current pixel.

[0048] Here, the current pixel is any pixel in the image frame to be denoised.

[0049] Obtain the pixel value feature of the current pixel in the image frame to be denoised relative to the image frame to be denoised. This pixel value feature reflects the feature information of the current pixel in the image frame to be denoised.

[0050] For example, one or more combinations of the position information, pixel value information, and gradient information of the current pixel in the image frame to be denoised are extracted to obtain the pixel features of the current pixel in the image frame to be denoised, and then the feature description vector corresponding to the current pixel is constructed.

[0051] For example, the correlation features between the current pixel and other pixels in the image frame to be denoised can also be extracted, and the feature description vector corresponding to the current pixel can be constructed by combining the correlation features.

[0052] For example, pixels similar to the current pixel are retrieved from the image frame to be denoised, and the feature description vector corresponding to the current pixel is constructed by combining the pixel value information of the current pixel and the pixel value information of the similar pixels.

[0053] It is understandable that the more comprehensive the extracted pixel features are, the more accurate the constructed feature description vector will be, and the higher the accuracy of subsequent matching of the current pixel will be. The specific construction method of the feature description vector can be flexibly selected according to the actual application situation, and this application does not limit it.

[0054] Step S230: Select pixels from adjacent image frames whose feature description vector similarity with the current pixel satisfies a preset condition, and obtain the inter-frame matching point corresponding to the current pixel.

[0055] Each pixel in an adjacent image frame also has a corresponding feature description vector, constructed in the same way as the current pixel.

[0056] Calculate the similarity between the feature description vectors of pixels in adjacent image frames and the feature description vectors of the feature description vectors to obtain the feature description vector similarity. Select pixels from adjacent image frames whose feature description vector similarity meets the preset conditions to obtain the inter-frame matching point corresponding to the current pixel.

[0057] The similarity of feature description vectors can be obtained by calculating the distance between them. The shorter the distance between feature description vectors, the higher the similarity. The longer the distance between feature description vectors, the lower the similarity.

[0058] The pixel whose feature description vector similarity satisfies the preset condition can be: the pixel with the highest feature description vector similarity in adjacent image frames; the pixel whose feature description vector similarity satisfies the preset condition can also be: the pixel whose feature description vector similarity in adjacent image frames is greater than the similarity threshold, and this application does not limit this.

[0059] Step S240: Perform noise reduction processing on the current pixel based on the inter-frame matching points to obtain the noise-reduced image frame corresponding to the image frame to be denoised.

[0060] After selecting inter-frame matching points from adjacent image frames, the pixel values ​​of the inter-frame matching points and the pixel value of the current pixel are fused to obtain the noise reduction result of the current pixel.

[0061] Repeatedly traverse each pixel in the image frame to be denoised to obtain the denoised image frame corresponding to the image frame to be denoised.

[0062] Compared to existing block-matching-based image denoising methods, this application performs pixel matching between adjacent image frames by describing the features corresponding to the pixels, which can reduce the computational load while ensuring matching accuracy.

[0063] The following describes some embodiments of this application in detail.

[0064] In some implementations, step S220 extracts the pixel value features of the current pixel in the image frame to be denoised relative to the image frame to be denoised, and constructs the feature description vector corresponding to the current pixel, including:

[0065] Step S221: Calculate the pixel value similarity between the current pixel and other pixels in the image frame to be denoised.

[0066] This involves comparing the similarity between the current pixel and other pixels in the image frame to be denoised, based on pixel values.

[0067] For example, calculating the pixel value similarity between the current pixel and other pixels in the image frame to be denoised includes: dividing the image frame to be denoised into an intra-frame point retrieval region corresponding to the current pixel based on the position of the current pixel in the image frame to be denoised; taking the pixels in the intra-frame point retrieval region as intra-frame candidate points, and obtaining the pixel value information of the current pixel and the intra-frame candidate points respectively; and calculating the pixel value similarity between the current pixel and the intra-frame candidate points based on the pixel value information of the current pixel and the intra-frame candidate points.

[0068] To further reduce computation while ensuring the accuracy of the constructed feature description vector, an intra-frame point retrieval region corresponding to the current pixel can be divided in the image frame to be denoised, so as to compare the pixel value similarity between the pixels in the intra-frame point retrieval region and the current pixel.

[0069] For example, taking the current pixel position as the center, the image range of a preset size is divided in the image frame to be denoised, and this image range is used as the intra-frame point retrieval region.

[0070] For example, the retrieval region requirement for the current pixel is obtained, and the range size is flexibly determined based on this requirement. Using the current pixel's position as the center, an image range of the corresponding size is divided within the image frame to be denoised, and this image range is used as the intra-frame point retrieval region. The retrieval region requirement can be flexibly determined based on the denoising accuracy requirements of the current image frame, the available computing resources, and the image quality of the current image frame. For instance, the higher the denoising accuracy requirement of the current image frame, the larger the retrieval region requirement and the larger the corresponding range size.

[0071] For example, the intra-frame point retrieval region obtained by segmentation can also be irregular. For instance, the intra-frame point retrieval region can be segmented based on the texture features of the image.

[0072] The method for dividing the intra-frame point retrieval region can be flexibly selected according to the actual application scenario, and this application does not impose any restrictions on it.

[0073] Pixels in the intra-frame point retrieval region are used as intra-frame candidate points, and pixel value information of the current pixel and intra-frame candidate points are obtained respectively.

[0074] The pixel value information can be the pixel value of the current pixel or the candidate pixel within the frame itself, or it can be the pixel value of the current pixel or the candidate pixel within the frame itself and the pixel values ​​of its neighboring pixels.

[0075] For example, determine the neighboring pixels corresponding to the current pixel and the neighboring pixels corresponding to the intra-frame candidate points; extract the pixel values ​​of the current pixel and its corresponding neighboring pixels to obtain the pixel value information of the current pixel; and extract the pixel values ​​of the intra-frame candidate points and their corresponding neighboring pixels to obtain the pixel value information of the intra-frame candidate points.

[0076] It should be noted that adjacent pixels refer to pixels that are adjacent to the current pixel. Pixels within a preset distance from the current pixel can be considered as its corresponding adjacent pixels.

[0077] For example, please refer to Figure 3 , Figure 3 This is a schematic diagram illustrating the acquisition of pixel value information in an exemplary embodiment of this application, as shown below. Figure 3 As shown, the current pixel is pixel C. The image range with pixel C as the center and radius SR is taken as the intra-frame point retrieval region. SR is 3, so there are 48 intra-frame candidate points in the intra-frame point retrieval region.

[0078] For each candidate point and pixel C within a frame, its corresponding pixel value information is obtained. Taking point B1 as an example, a 3*3 pixel block is extracted with B1 as the center. This pixel block contains the pixel value of B1 and the pixel values ​​of its neighboring pixels to obtain the pixel value information corresponding to B1. The same method is used to extract the pixel value information corresponding to each candidate point and pixel C within a frame. The size of the pixel block can be flexibly adjusted, and this application does not limit the size of the pixel block.

[0079] Then, based on the pixel value information of the current pixel and the candidate points in the frame, the pixel value similarity between the current pixel and the candidate points in the frame is calculated.

[0080] For example, the pixel value information of the current pixel and the pixel value information of the candidate points in the frame are calculated by difference; based on the calculation result of the difference, the pixel value similarity between the current pixel and the candidate points in the frame is obtained.

[0081] Combination Figure 3 Taking candidate point B1 within the frame and current pixel C as an example, the formula for calculating pixel value similarity can be expressed as Formula 1:

[0082]

[0083] In Formula 1, diff1 represents the pixel value similarity between B1 and C, and B... 1,i C is the value of the i-th pixel in the pixel block corresponding to B1. i Let be the value of the i-th pixel in the pixel block corresponding to C, where i can be 1, 2, ..., 9.

[0084] Of course, if the pixel value information can be the pixel value of the current pixel or the pixel value of the candidate point in the frame, then the pixel value similarity can be obtained by directly subtracting the pixel value of the current pixel and the pixel value of the candidate point in the frame.

[0085] Step S222: Select pixels from the image frame to be denoised that have pixel value similarity with the current pixel that meets the preset conditions, and obtain the intra-frame matching point corresponding to the current pixel.

[0086] The pixels whose pixel value similarity meets the preset conditions can be: sort the pixels in the image frame to be denoised in descending order according to the pixel value similarity, and select the top M pixels as the pixels whose pixel value similarity meets the preset conditions.

[0087] Pixels whose pixel value similarity meets the preset conditions can also be: pixels whose pixel value similarity is greater than the similarity threshold are considered as pixels whose pixel value similarity meets the preset conditions.

[0088] For example, please refer to Figure 4 , Figure 4 This is a schematic diagram illustrating an intra-frame matching point as an exemplary embodiment of this application, such as... Figure 4 As shown, the current pixel is pixel C, the intra-frame point retrieval area is the image range with a radius of 3 centered on pixel C, and the selected pixel points whose pixel value similarity meets the preset conditions include A1, A2, A3, A4, and A5.

[0089] Optionally, an intra-frame correlation vector corresponding to the current pixel can be established by combining the position information of the current pixel and the intra-frame matching point, so as to represent the positional relationship between the intra-frame matching point and the current pixel through the intra-frame correlation vector.

[0090] For example, the intra-frame correlation vector of the current pixel C is represented as V = (vC, vA1, vA2, vA3, vA4, vA5), where vC represents the coordinates (i, j) of the current pixel C in the frame of the image to be denoised, and vA1, vA2, vA3, vA4, vA5 represent the position offsets of the intra-frame matching points A1, A2, A3, A4, A5 relative to the current pixel C. Then V can be represented as V = ((i, j), (-2, -2), (-3, 1), (1, -2), (3, 0), (2, 3)).

[0091] Step S223: Construct the feature description vector corresponding to the current pixel using the pixel value information of the current pixel and the matching points within the frame.

[0092] Extract the pixel value information of the current pixel and the matching points within the frame, and construct the feature description vector corresponding to the current pixel.

[0093] For example, the pixel value information of the current pixel and the intra-frame matching point are obtained respectively; based on the pixel value information of the current pixel and the intra-frame matching point, the description information corresponding to the current pixel and the intra-frame matching point is constructed respectively; and the feature description vector corresponding to the current pixel is constructed by combining the description information of the current pixel and the intra-frame matching point.

[0094] The pixel value information can be the pixel value of the current pixel or the candidate pixel within the frame itself, or it can be the pixel value of the current pixel or the candidate pixel within the frame itself and the pixel values ​​of its neighboring pixels.

[0095] For example, the pixel value information corresponding to the current pixel contains the pixel value of the current pixel and the pixel values ​​of the adjacent pixels corresponding to the current pixel; the pixel value information corresponding to the intra-frame matching point contains the pixel value of the intra-frame matching point and the pixel values ​​of the adjacent pixels corresponding to the intra-frame matching point. Based on the pixel value information of the current pixel and the intra-frame matching point, description information corresponding to the current pixel and the intra-frame matching point is constructed respectively, including: calculating the difference between the pixel value of the current pixel and the pixel values ​​of the adjacent pixels corresponding to the current pixel to obtain the pixel difference of the current point; and calculating the difference between the pixel value of the intra-frame matching point and the pixel values ​​of the adjacent pixels corresponding to the intra-frame matching point to obtain the pixel difference of the intra-frame matching point; description information corresponding to the current pixel is constructed based on the pixel value of the current pixel and the pixel difference of the current point; and description information corresponding to the intra-frame matching point is constructed based on the pixel value of the intra-frame matching point and the pixel difference of the intra-frame matching point.

[0096] For an example using the current pixel, please refer to [link / reference]. Figure 5 , Figure 5 This is a schematic diagram illustrating the acquisition of descriptive information of the current pixel in an exemplary embodiment of this application, as shown below. Figure 5As shown, obtain the description radius (MBR) of the current pixel (e.g. Figure 5 In the MBR (Mean Differential Ratio) setting, the description range of the current pixel is divided based on its position and the description radius. The values ​​of pixels within this range are extracted to obtain the pixel value information corresponding to the current pixel. This information includes both the current pixel's value and the values ​​of its adjacent pixels. Then, the difference between the current pixel's value and its adjacent pixels is calculated to obtain the pixel difference. Based on this difference, the description information for the current pixel is constructed.

[0097] For example, Figure 5 The description information corresponding to the current pixel C in the formula can be expressed as Equation 2:

[0098]

[0099] In Formula 2, MC represents the descriptive information corresponding to the current pixel C, and VA... C This is the pixel value corresponding to the current pixel point C. These are the pixel values ​​corresponding to adjacent pixels D1, D2, ..., D8 within the description range.

[0100] Descriptive information for each intra-frame matching point is constructed in the same manner.

[0101] Optionally, when constructing the description information of intra-frame matching points, the description radius (MBR) value corresponding to each intra-frame matching point can be different, allowing for flexible calculation of the description radius for each intra-frame matching point. For example, the description radius of the intra-frame matching point can be calculated based on the distance and / or pixel value similarity between the intra-frame matching point and the current pixel. A larger description radius indicates a greater distance and lower pixel value similarity, while a larger description radius indicates a closer distance and higher pixel value similarity.

[0102] Of course, when constructing the description information of intra-frame matching points, the description radius (MBR) value corresponding to each intra-frame matching point can also be the same, and this application does not impose any restrictions on this.

[0103] By combining the description information of the current pixel and the matching points within the frame, a feature description vector corresponding to the current pixel is constructed. For example, the description information of the current pixel and the description information of each matching point within the frame are concatenated to obtain the feature description vector corresponding to the current pixel.

[0104] If the number of intra-frame matching points is M, the feature description vector constructed according to the above embodiment includes (1+M)×(MBR+MBR+1)×(MBR+MBR+1) elements. This feature description vector implicitly contains the position information of the current pixel and the intra-frame matching points, as well as the pixel value gradient information of the current pixel and the intra-frame matching points and their neighboring pixels. This can improve the subsequent matching accuracy while reducing the matching computation.

[0105] To remove redundant information between frames and perform temporal denoising, it is necessary to establish the correlation between pixels between frames. Therefore, it is necessary to select the inter-frame matching point corresponding to the current pixel from the adjacent image frames corresponding to the image frame to be denoised.

[0106] In some implementations, step S230 involves selecting pixels from adjacent image frames whose feature description vector similarity to the current pixel satisfies a preset condition, thereby obtaining the inter-frame matching point corresponding to the current pixel. This includes:

[0107] Step S231: Based on the position of the current pixel relative to the adjacent image frames, divide the inter-frame point retrieval region corresponding to the current pixel in the adjacent image frames.

[0108] Based on the position of the current pixel relative to adjacent image frames, the inter-frame point retrieval region corresponding to the current pixel is divided in the adjacent image frames to improve retrieval efficiency.

[0109] For example, please refer to Figure 6 , Figure 6 This is a schematic diagram illustrating the division of the inter-frame point retrieval region as an exemplary embodiment of this application, as shown below. Figure 6 As shown, the image frame to be denoised is represented as F. n The adjacent image frames corresponding to the image frame to be denoised can be represented as F. n-1 F n+1 The current pixel position is represented as (n, i, j). Based on the current pixel position relative to adjacent image frames, coordinates (n-1, i, j) and (n+1, i, j) are mapped in adjacent image frames to correspond to the current pixel's spatial position. Then, using the mapped coordinates as the center, a search radius is obtained, dividing the inter-frame point search region. The value of the search radius can be flexibly set according to actual conditions, and this application does not impose any limitations on it.

[0110] Step S232: Use the pixels in the inter-frame point retrieval region as inter-frame candidate points and obtain the feature description vector of the inter-frame candidate points.

[0111] The method for obtaining the feature description vector of inter-frame candidate points is the same as that for the feature description vector of inter-frame candidate points of the current pixel, and will not be repeated here.

[0112] Step S233: Calculate the similarity between the feature description vector of the current pixel and the feature description vector of the inter-frame candidate points to obtain the feature description vector similarity.

[0113] It can be used to calculate the distance between two feature description vectors to obtain the feature description vector similarity.

[0114] For example, the formula for calculating the similarity of feature description vectors can be expressed as Formula 3:

[0115]

[0116] In Formula 3, TV n,i,j Indicates the image frame F to be denoised n The feature description vector of the current pixel (i, j); TV n-1,i+offi,j+offj Representing adjacent image frames F n-1 The feature description vector of the inter-frame candidate point (i+offi, j+offj), where offi and offj are the position offsets relative to the center point of the inter-frame point retrieval region; diffF n,n-1,i,j,offi,offj The distance between two feature description vectors is represented by diffF. n,n-1,i,j,offi,offj The smaller the value, the higher the similarity of the feature description vectors.

[0117] Step S234: Select inter-frame candidate points in the inter-frame point retrieval region whose feature description vector similarity meets the preset conditions, and obtain the inter-frame matching point corresponding to the current pixel.

[0118] For example, please see Figure 7 , Figure 7 This is a schematic diagram illustrating an inter-frame matching point as shown in an exemplary embodiment of this application, such as... Figure 7 As shown, for any adjacent image frame, the inter-frame candidate point with the highest similarity of feature description vectors is selected as the inter-frame matching point corresponding to the current pixel.

[0119] Then, noise reduction is performed on the current pixel based on the inter-frame matching points to obtain the noise-reduced image frame corresponding to the image frame to be denoised.

[0120] In some implementations, step S240 involves performing noise reduction on the current pixel based on inter-frame matching points to obtain a denoised image frame corresponding to the image frame to be denoised, including:

[0121] Step S241: Construct the inter-frame correlation vector corresponding to the current pixel based on each inter-frame matching point.

[0122] Among them, the inter-frame correlation vector is used to reflect the position information of the inter-frame matching points.

[0123] For example, the inter-frame correlation vector corresponding to the current pixel can be expressed as Equation 4:

[0124] PV=((I n-1 J n-1 ),(i n-1 ,j n-1 ),(I n+1 J n+1 ))

[0125] In Formula 4, (I n-1 J n-1 ) represents adjacent image frames F n-1 The coordinates of the inter-frame matching points in the (i n-1 ,j n-1 ) represents the image frame F to be denoised. n The coordinates of the current pixel in (I n+1 J n+1 ) represents adjacent image frames F n+1 The coordinates of the inter-frame matching points.

[0126] Of course, if there are more adjacent image frames, there will be more inter-frame matching points, and the constructed inter-frame correlation vector will have more elements.

[0127] Step S242: Use the similarity of the feature description vector between each inter-frame matching point and the current pixel point as the weight parameter of the inter-frame correlation vector, and weight the inter-frame correlation vector to obtain the weighted inter-frame vector.

[0128] Among them, the similarity of the feature description vectors of the inter-frame matching points is positively correlated with their corresponding weight parameters.

[0129] For example, adjacent image frames F n-FR In the inter-frame correlation vector (I) n-FR J n-FR For example, the corresponding weight parameters can be obtained using Formula 5:

[0130] w n-FR =max(sigma-diffF) n,n-FR ,0)

[0131] In formula 5, w n-FR For adjacent image frames F n-FR The weight parameters for the inter-frame matching points, sigma is a preset parameter value; diffF n,n-FR For adjacent image frames F n-FR Inter-frame matching points and the image frame to be denoised F n The similarity of feature description vectors between current pixels is calculated as shown in Formula 3 above.

[0132] The inter-frame correlation vectors are then weighted to obtain a weighted inter-frame vector, which can be expressed as Equation 6:

[0133]

[0134] In Formula 6, PV is the inter-frame correlation vector corresponding to the current pixel; WV is the weight value of the element corresponding to PV, and the element in WV can be represented as WV = (w n-FR … w n … w n+FR ); 1 is a row vector, represented as 1 = (1 … 1 … 1), with the same dimensions as WV.

[0135] By weighting the inter-frame correlation vectors using feature description vector similarity, the image noise reduction effect can be improved.

[0136] Step S243: Perform noise reduction processing on the current pixel based on the weighted inter-frame vector to obtain the noise-reduced image frame corresponding to the image frame to be denoised.

[0137] For example, the pixel values ​​of multiple inter-frame matching points and the current pixel are obtained based on the weighted inter-frame vector, and the pixel values ​​are weighted according to the weighting parameters to obtain the denoised pixel value corresponding to the current pixel.

[0138] Repeatedly traverse the pixels in the image frame to be denoised to perform denoising processing, and obtain the denoised image frame corresponding to the image frame to be denoised.

[0139] The image denoising method provided in this application obtains the image frame to be denoised and its adjacent image frames; extracts the pixel features of the current pixel in the image frame to be denoised relative to the pixel features of the image frame to be denoised, and constructs the feature description vector corresponding to the current pixel; selects pixels from the adjacent image frames whose feature description vectors have a similarity to the current pixel that meets a preset condition, and obtains the inter-frame matching point corresponding to the current pixel; and performs denoising processing on the current pixel based on the inter-frame matching point to obtain the denoised image frame corresponding to the image frame to be denoised. This method can reduce the amount of computation while ensuring matching accuracy.

[0140] Figure 8 This is a block diagram illustrating an image noise reduction apparatus according to an exemplary embodiment of this application. Figure 8 As shown, the exemplary image noise reduction device 800 includes: a frame acquisition module 810, a description vector construction module 820, an inter-frame matching module 830, and a noise reduction module 840. Specifically:

[0141] The frame acquisition module 810 is used to acquire the image frame to be denoised and the adjacent image frames corresponding to the image frame to be denoised.

[0142] The description vector construction module 820 is used to extract the pixel features of the current pixel in the image frame to be denoised, and construct the feature description vector corresponding to the current pixel.

[0143] The inter-frame matching module 830 is used to select pixels from adjacent image frames whose feature description vector similarity with the current pixel satisfies a preset condition, and obtain the inter-frame matching point corresponding to the current pixel.

[0144] The noise reduction module 840 is used to perform noise reduction processing on the current pixel based on the inter-frame matching point to obtain the noise-reduced image frame corresponding to the image frame to be denoised.

[0145] It should be noted that the image denoising device and the image denoising method provided in the above embodiments belong to the same concept. The specific operation methods of each module and unit have been described in detail in the method embodiments and will not be repeated here. In practical applications, the image denoising device provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. This is not a limitation.

[0146] Please see Figure 9 , Figure 9 This is a schematic diagram of the structure of an embodiment of the electronic device of this application. The electronic device 900 includes a memory 901 and a processor 902. The processor 902 is used to execute program instructions stored in the memory 901 to implement the steps in any of the above-described image noise reduction method embodiments. In a specific implementation scenario, the electronic device 900 may include, but is not limited to, a microcomputer or a server. In addition, the electronic device 900 may also include mobile devices such as laptops and tablets, which are not limited here.

[0147] Specifically, processor 902 controls itself and memory 901 to implement the steps in any of the above-described image denoising method embodiments. Processor 902 can also be referred to as a Central Processing Unit (CPU). Processor 902 may be an integrated circuit chip with signal processing capabilities. Processor 902 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor. Furthermore, processor 902 can be implemented using integrated circuit chips.

[0148] Please see Figure 10 , Figure 10 This is a schematic diagram of a computer-readable storage medium according to an embodiment of the present application. The computer-readable storage medium 1000 stores program instructions 1010 that can be executed by a processor. The program instructions 1010 are used to implement the steps in any of the above-described image noise reduction method embodiments.

[0149] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0150] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0151] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0152] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. An image denoising method, characterized in that, The method includes: Obtain the image frame to be denoised and its adjacent image frames; Calculate the pixel value similarity between the current pixel and other pixels in the image frame to be denoised; From the image frame to be denoised, select the pixel points whose pixel value similarity with the current pixel point meets the preset conditions, and obtain the intra-frame matching point corresponding to the current pixel point; The pixel value of the current pixel is calculated by comparing the pixel value of the current pixel with the pixel value of the adjacent pixel corresponding to the current pixel; and the pixel value of the intra-frame matching point is calculated by comparing the pixel value of the intra-frame matching point with the pixel value of the adjacent pixel corresponding to the intra-frame matching point. Based on the pixel value of the current pixel and the pixel difference of the current pixel, a description of the current pixel is constructed; and based on the pixel value of the intra-frame matching point and the pixel difference of the intra-frame matching point, a description of the intra-frame matching point is constructed. By combining the description information of the current pixel and the intra-frame matching point, a feature description vector corresponding to the current pixel is constructed; wherein, the feature description vector also contains the position information of the current pixel and the intra-frame matching point; From the adjacent image frames, select the pixels whose feature description vector similarity with the current pixel satisfies a preset condition, and obtain the inter-frame matching point corresponding to the current pixel. Based on the inter-frame matching points, the current pixel is denoised to obtain the denoised image frame corresponding to the image frame to be denoised.

2. The method according to claim 1, characterized in that, The calculation of pixel value similarity between the current pixel and other pixels in the image frame to be denoised includes: Based on the position of the current pixel in the image frame to be denoised, an intra-frame point retrieval region corresponding to the current pixel is divided in the image frame to be denoised. The pixels in the intra-frame point retrieval region are used as intra-frame candidate points, and the pixel value information of the current pixel and the intra-frame candidate points are obtained respectively. Based on the pixel value information of the current pixel and the intra-frame candidate points, the pixel value similarity between the current pixel and the intra-frame candidate points is calculated.

3. The method according to claim 2, characterized in that, The step of obtaining the pixel value information of the current pixel and the intra-frame candidate points respectively includes: Determine the neighboring pixels corresponding to the current pixel and the neighboring pixels corresponding to the intra-frame candidate points; Extract the pixel values ​​of the current pixel and its corresponding neighboring pixels to obtain the pixel value information of the current pixel; and extract the pixel values ​​of the intra-frame candidate point and its corresponding neighboring pixels to obtain the pixel value information of the intra-frame candidate point.

4. The method according to claim 2, characterized in that, The step of calculating the pixel value similarity between the current pixel and the intra-frame candidate points based on the pixel value information of the current pixel and the intra-frame candidate points includes: Calculate the difference between the pixel value information of the current pixel and the pixel value information of the candidate points within the frame; Based on the calculation result of the difference, the pixel value similarity between the current pixel and the intra-frame candidate point is obtained.

5. The method according to claim 1, characterized in that, The step of selecting pixels from the adjacent image frames whose feature description vector similarity with the current pixel meets a preset condition to obtain the inter-frame matching point corresponding to the current pixel includes: Based on the position of the current pixel relative to the adjacent image frames, the inter-frame point retrieval region corresponding to the current pixel is divided in the adjacent image frames; The pixels in the inter-frame point retrieval region are used as inter-frame candidate points, and the feature description vector of the inter-frame candidate points is obtained. Calculate the similarity between the feature description vector of the current pixel and the feature description vector of the inter-frame candidate point to obtain the feature description vector similarity. Select inter-frame candidate points in the inter-frame point retrieval region whose feature description vector similarity meets preset conditions to obtain the inter-frame matching point corresponding to the current pixel.

6. The method according to claim 5, characterized in that, The step of performing noise reduction processing on the current pixel based on the inter-frame matching points to obtain the noise-reduced image frame corresponding to the image frame to be noise-reduced includes: Based on each inter-frame matching point, the inter-frame correlation vector corresponding to the current pixel is constructed; The similarity of the feature description vector between each inter-frame matching point and the current pixel point is used as the weight parameter of the inter-frame correlation vector. The inter-frame correlation vector is weighted to obtain a weighted inter-frame vector. The current pixel is denoised based on the weighted inter-frame vector to obtain the denoised image frame corresponding to the image frame to be denoised.

7. An electronic device, characterized in that, The electronic device includes a memory and a processor, the processor being configured to execute program instructions stored in the memory to implement the steps of the method as described in any one of claims 1-6.

Images