Image processing method and device, computer readable storage medium and electronic device

Abstract

The present disclosure provides an image processing method, an image processing device, a computer-readable storage medium and an electronic device, and relates to the field of image processing technology. The image processing method comprises: performing multiple rounds of image processing processes on an image to be processed to obtain a denoised image corresponding to the image to be processed; wherein each round of image processing process adopts a different or not completely the same image segmentation method, and except for the first round of image processing process, the input image of each round of image processing process is the output image of the previous round of image processing process of the image processing process; each round of image processing process comprises: performing image segmentation on the input image to obtain multiple target image blocks; performing denoising on each target image block respectively, and generating an output image according to the result of the denoising process. The present disclosure can improve the image denoising effect.

Description

Technical Field

[0001] The present disclosure relates to the technical field of image processing, and in particular to an image processing method, an image processing device, a computer-readable storage medium, and an electronic device. Background Art

[0002] With the popularity of electronic devices with shooting functions, more and more users use electronic devices to capture images. However, due to the inadequacy of electronic device hardware or the influence of shooting environment, the obtained images will have noise. Although some image denoising solutions have emerged, there is still the problem of poor denoising effect. Summary of the invention

[0003] The present disclosure provides an image processing method, an image processing device, a computer-readable storage medium and an electronic device, thereby overcoming the problem of poor image denoising effect at least to a certain extent.

[0004] According to a first aspect of the present disclosure, an image processing method is provided, comprising: performing multiple rounds of image processing processes on an image to be processed to obtain a denoised image corresponding to the image to be processed; wherein the image segmentation method adopted in each round of image processing process is different or not completely the same, and except for the first round of image processing process, the input image of each round of image processing process is the output image of the previous round of image processing process of the image processing process; each round of image processing process comprises: performing image segmentation on the input image to obtain multiple target image blocks; performing denoising processing on each target image block respectively, and generating an output image according to the result of the denoising processing.

[0005] According to a second aspect of the present disclosure, an image processing device is provided, comprising: the image processing device is configured to perform multiple rounds of image processing processes on an image to be processed to obtain a denoised image corresponding to the image to be processed; wherein the image segmentation method adopted in each round of the image processing process is different or not completely the same, and except for the first round of the image processing process, the input image of each round of the image processing process is the output image of the previous round of the image processing process of the image processing process; each round of the image processing process comprises: performing image segmentation on the input image to obtain multiple target image blocks; performing denoising on each target image block respectively, and generating an output image according to the result of the denoising process.

[0006] According to a third aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the above-mentioned image processing method is implemented.

[0007] According to a fourth aspect of the present disclosure, an electronic device is provided, comprising a processor; and a memory for storing one or more programs, wherein when the one or more programs are executed by the processor, the processor implements the above-mentioned image processing method.

[0008] In the technical solutions provided by some embodiments of the present disclosure, multiple rounds of image processing processes are performed on the image to be processed to obtain a denoised image corresponding to the image to be processed. The image segmentation method used in each round of image processing process is different or not completely the same, and except for the first round of image processing process, the input image of each round of image processing process is the output image of the previous round of image processing process of the image processing process. On the one hand, since the image segmentation method used in each round of processing process is different or not completely the same, the segmented target image blocks are different or not completely the same. Therefore, in the process of continuously implementing image block denoising, the edge processing effect is optimized, the generation of block effect is effectively suppressed, and the image denoising effect is improved; on the other hand, the disclosed solution adopts image segmentation and multiple cascade processing processes to continuously optimize the denoising results, which can further improve the image denoising effect.

[0009] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present disclosure, and together with the specification, are used to explain the principles of the present disclosure. Obviously, the drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0011] Figure 1 A schematic diagram showing an exemplary system architecture of an image processing solution according to an embodiment of the present disclosure;

[0012] Figure 2 A schematic diagram of the structure of an electronic device suitable for implementing an embodiment of the present disclosure is shown;

[0013] Figure 3 A flowchart schematically shows an image processing method according to an exemplary embodiment of the present disclosure;

[0014] Figure 4 A flowchart of performing a round of image processing according to an exemplary embodiment of the present disclosure is schematically shown;

[0015] Figure 5 A schematic diagram showing the overlapping manner of image blocks after two image segmentations in two rounds of image processing;

[0016] Figure 6 Another schematic diagram showing the overlapping manner of image blocks after two image segmentations in two rounds of image processing;

[0017] Figure 7 Another schematic diagram showing the overlapping manner of image blocks after two image segmentations in two rounds of image processing;

[0018] Figure 8 A schematic diagram of determining a reference image block is shown;

[0019] Fig. 9 A schematic diagram showing an image segmentation method according to an embodiment of the present disclosure is shown;

[0020] Fig.10 A schematic diagram showing the entire process of the image processing solution of an embodiment of the present disclosure. DETAILED DESCRIPTION

[0021] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as being limited to the examples set forth herein; on the contrary, these embodiments are provided so that the present disclosure will be more comprehensive and complete, and the concepts of the example embodiments are fully conveyed to those skilled in the art. The described features, structures, or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known technical solutions are not shown or described in detail to avoid obscuring various aspects of the present disclosure.

[0022] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0023] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the steps. For example, some steps may be decomposed, while some steps may be combined or partially combined, so the actual execution order may change according to the actual situation.

[0024] Figure 1A schematic diagram showing an exemplary system architecture of an image processing solution according to an embodiment of the present disclosure.

[0025] like Figure 1 As shown, the system architecture may include a terminal device 1100 and a server 1200. The terminal device 1100 and the server 1200 may be connected via a network, and the connection type of the network may include, for example, a wired or wireless communication link or an optical fiber cable.

[0026] It should be understood that the number of terminal devices 1100 and servers 1200 is only illustrative. According to implementation requirements, there may be any number of terminal devices and servers. For example, the server 1200 may be a server cluster composed of multiple servers. The server 1200 may also be referred to as a cloud or cloud server.

[0027] The terminal device 1100 can interact with the server 1200 through the network to receive or send messages, etc. Figure 1 In the figure, a smart phone is used as an example, however, the terminal device 1100 also includes a tablet computer, a smart wearable device, a personal computer and other devices with a shooting function. Among them, the terminal device 1100 can also be called a terminal, a mobile terminal, a mobile terminal, a smart terminal, etc.

[0028] When the image processing process of the exemplary embodiment of the present disclosure is performed by the terminal device 1100, first, the terminal device 1100 may obtain an image to be processed. The image to be processed described in the embodiment of the present disclosure may be an image captured by the terminal device 1100 through its camera module, or an image obtained by the terminal device 1100 from the server 1200 or other devices. Specifically, it may be a video frame image or a photo.

[0029] Next, the terminal device 1100 may perform multiple rounds of image processing on the image to be processed to obtain a denoised image corresponding to the image to be processed, wherein each round of image processing uses a different or not completely identical image segmentation method, and except for the first round of image processing, the input image of each round of image processing is the output image of the previous round of image processing of the image processing.

[0030] Each round of image processing may include: performing image segmentation on the input image to obtain multiple target image blocks, performing denoising on each target image block, and generating an output image based on the denoising result. Specifically, the embodiment of the present disclosure may combine the NLM (Non-Local Means) denoising scheme to achieve denoising of the target image block.

[0031] In addition, the terminal device 1100 can save the denoised image to an album.

[0032] In the case where the image processing process according to the exemplary embodiment of the present disclosure is performed by the server 1200 , first, the server 1200 may acquire an image to be processed from the terminal device 1100 or other devices.

[0033] Next, the server 1200 may perform multiple rounds of image processing on the image to be processed to obtain a denoised image corresponding to the image to be processed, wherein each round of image processing uses a different or not completely identical image segmentation method, and except for the first round of image processing, the input image of each round of image processing is the output image of the previous round of image processing of the image processing.

[0034] Each round of image processing may include: performing image segmentation on the input image to obtain multiple target image blocks, performing denoising on each target image block, and generating an output image based on the denoising result. Specifically, the embodiment of the present disclosure may use the denoising scheme of NLM to achieve denoising of the target image block.

[0035] In addition, the server 1200 may also feed back the processed image to the terminal device 1100 for display and / or storage.

[0036] It should be noted that any of the above steps of processing the image can be performed by the terminal device 1100 or the server 1200. For example, after the terminal device 1100 performs image segmentation, the target image block is sent to the server 1200, and the server 1200 performs denoising. The present disclosure does not limit this.

[0037] Figure 2 Schematic diagram of an electronic device suitable for implementing the exemplary embodiment of the present disclosure is shown. The terminal device of the exemplary embodiment of the present disclosure may be configured as follows Figure 2 It should be noted that Figure 2 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0038] The electronic device of the present disclosure includes at least a processor and a memory, wherein the memory is used to store one or more programs. When the one or more programs are executed by the processor, the processor can implement the image processing method of the exemplary embodiment of the present disclosure.

[0039] Specifically, Figure 2As shown, the electronic device 200 may include: a processor 210, an internal memory 221, an external memory interface 222, a Universal Serial Bus (USB) interface 230, a charging management module 240, a power management module 241, a battery 242, an antenna 1, an antenna 2, a mobile communication module 250, a wireless communication module 260, an audio module 270, a speaker 271, a receiver 272, a microphone 273, an earphone interface 274, a sensor module 280, a display screen 290, a camera module 291, an indicator 292, a motor 293, a button 294, and a Subscriber Identification Module (SIM) card interface 295, etc. The sensor module 280 may include a depth sensor, a pressure sensor, a gyroscope sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, and a bone conduction sensor, etc.

[0040] It is to be understood that the structure illustrated in the embodiment of the present disclosure does not constitute a specific limitation on the electronic device 200. In other embodiments of the present disclosure, the electronic device 200 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0041] The processor 210 may include one or more processing units, for example, the processor 210 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor and / or a neural network processor (NPU). Different processing units may be independent devices or integrated into one or more processors. In addition, a memory may be provided in the processor 210 for storing instructions and data.

[0042] The electronic device 200 can realize the shooting function through the ISP, the camera module 291, the video codec, the GPU, the display screen 290 and the application processor. In some embodiments, the electronic device 200 may include 1 or N camera modules 291, where N is a positive integer greater than 1. If the electronic device 200 includes N cameras, one of the N cameras is a main camera.

[0043] The internal memory 221 can be used to store computer executable program codes, which include instructions. The internal memory 221 can include a program storage area and a data storage area. The external memory interface 222 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 200.

[0044] The present disclosure also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiments; or may exist independently without being assembled into the electronic device.

[0045] Computer readable storage media may be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or components, or any combination thereof. More specific examples of computer readable storage media may include, but are not limited to, an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, device, or device.

[0046] Computer-readable storage media can send, propagate or transmit programs for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable storage medium can be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0047] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by an electronic device, the electronic device implements the method described in the following embodiments.

[0048] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0049] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, and the units described may also be arranged in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0050] The following describes the steps of the image processing method of the present disclosure executed by a terminal device. In this case, the image processing apparatus of the embodiment of the present disclosure can be configured in the terminal device.

[0051] Figure 3 The flowchart of the image processing method according to the exemplary embodiment of the present disclosure is schematically shown. Figure 3 , the image processing method may include the following steps:

[0052] S32. Obtain an image to be processed.

[0053] In an exemplary embodiment of the present disclosure, the image to be processed may be an image captured by the terminal device through its camera module, or an image acquired by the terminal device from other devices. The image to be processed may be a video frame image or a separate photo. The present disclosure does not limit the acquisition method, type, format, size, etc. of the image to be processed.

[0054] S34. Perform multiple rounds of image processing on the image to be processed to obtain a denoised image corresponding to the image to be processed.

[0055] In an exemplary embodiment of the present disclosure, on the one hand, multiple rounds of image processing processes can be configured in a cascade manner, that is, except for the first round of image processing process, the input image of each round of image processing process is the output image of the previous round of image processing process of the image processing process. For example, the input image of the i-th round of image processing process, that is, the image to be processed by the i-th round of image processing process, is the image output by the i-1-th round of image processing process, where i is a positive integer greater than or equal to 2. It can be understood that the input image of the first round of image processing process is the image to be processed.

[0056] On the other hand, the image segmentation methods used in each round of image processing are different or not completely the same. The image segmentation methods include the size of the image blocks segmented and / or the position of the image blocks segmented. In other words, different image segmentation methods segment the image blocks with different sizes and / or different positions.

[0057] In some embodiments of the present disclosure, the image segmentation methods used in each of the multiple image processing processes may be different. For example, a total of 10 image processing processes are configured, and there are 10 image segmentation methods and none of the 10 methods are repeated.

[0058] In other embodiments of the present disclosure, the image segmentation methods used in each of the multiple rounds of image processing may not be completely the same. For example, a total of 10 rounds of image processing are configured, the image segmentation methods used in the first and fourth rounds are the same, the image segmentation methods used in the third and seventh rounds are the same, and the remaining image segmentation methods are different. It can be seen that these 10 rounds of image processing use 8 different image segmentation methods.

[0059] In addition, the present disclosure does not limit the number of image processing processes (ie, the number of rounds) included in the solution, which can be configured by the developer in combination with the processing capability of the terminal device.

[0060] For each round of image processing, refer to Figure 4 Provide explanation.

[0061] S42. Perform image segmentation on the input image to obtain multiple target image blocks.

[0062] According to some embodiments of the present disclosure, a terminal device may respond to an image block configuration operation of a user to segment an input image. The image block configuration operation may include size information and position information of the image block. Specifically, a user may input the size information and position information of the image block on an operation interface provided by the terminal device, and after obtaining the size information and position information, the terminal device may execute a corresponding program to segment the input image into a plurality of target image blocks.

[0063] According to other embodiments of the present disclosure, the terminal device may determine the size information and the position information in a random manner or in a predetermined configuration manner, and use the information determined by itself to perform an image segmentation operation to obtain a plurality of target image blocks.

[0064] In view of the fact that the image segmentation methods in the two rounds of image processing may be different, the interleaving (or overlapping) between the images segmented in the two rounds of processing will be described below. In the following illustrated description, the image block Blk0 and the image block Blk1 respectively represent the image blocks segmented by the two image segmentation methods corresponding to the two rounds of image processing.

[0065] refer to Figure 5 , Figure 6 and Figure 7 , the different situations when the image block Blk0 and the image block Blk1 have the same size are described.

[0066] refer to Figure 5 , the image block Blk1 is interleaved with four image blocks Blk0, that is, the image block Blk1 spans four image blocks Blk0.

[0067] refer to Figure 6 In the horizontal direction, the image block Blk1 is interlaced with two image blocks Blk0, that is, the image block Blk1 spans the two horizontal image blocks Blk0.

[0068] refer to Figure 7 In the vertical direction opposite to the horizontal direction, the image block Blk1 is interlaced with two image blocks Blk0, that is, the image block Blk1 spans over the two vertical image blocks Blk0.

[0069] In addition to being divided into image blocks of the same size, the image blocks divided by the image segmentation method used in the two rounds of image processing can also have different sizes. That is, the size of the image block Blk1 is different from that of the image block Blk0. It is conceivable that, in some cases, the image block Blk1 can span multiple image blocks Blk0; in other cases, the image block Blk1 can be inside the image block Blk0; in still other cases, there can also be the above-mentioned Figures 5 to 7 The present disclosure does not impose any restrictions on the interleaving method.

[0070] S44. Perform denoising processing on each target image block respectively, and generate an output image according to the result of the denoising processing.

[0071] For each target image block determined in step S42, a denoising process is performed.

[0072] First, a search window is established with the target image block as the center.

[0073] like Figure 8 As shown, the center of the established search window is the target image block, that is, the center point of the search window coincides with the center point of the target image block. The present disclosure does not limit the size of the search window. It is understandable that the larger the size of the search window, the more reference image blocks are subsequently obtained, and the better the denoising effect. However, the computer resource consumption is also greater, and the developer can flexibly configure it according to the actual hardware situation.

[0074] Next, at least one reference image block corresponding to the target image block is determined in the search window.

[0075] The size of the reference image block may be the same as the size of the target image block.

[0076] In some embodiments of the present disclosure, a search may be performed within a search window with fixed horizontal and vertical step sizes to obtain at least one reference image block. In these embodiments, there is no restriction on the values ​​of the horizontal and vertical step sizes.

[0077] In some other embodiments of the present disclosure, the horizontal step size and vertical step size of searching for the reference image block in the search window can be determined according to the size of the target image block. For example, the horizontal step size is equal to the width of the target image block, and the vertical step size is equal to the height of the target image block; for another example, the horizontal step size is equal to the sum of the width of the target image block and a width threshold, and the vertical step size is equal to the sum of the height of the target image block and a height threshold, wherein the width threshold and the height threshold can be the same or different, and the present disclosure does not limit this.

[0078] At least one reference image block is searched in the search window using the determined horizontal step size and vertical step size.

[0079] Then, the target image block and the determined at least one reference image block may be used to generate a denoised image block corresponding to the target image block.

[0080] The process of generating denoised image blocks is described in detail below.

[0081] First, the terminal device can determine the adjustment parameter value of the target image block to be denoised, and the adjustment parameter value includes one or more of the pixel difference parameter value between the target image block and the reference image block, the distance parameter value between the target image block and the reference image block, the texture direction difference parameter value between the target image block and the reference image block, the gradient complexity parameter value of the target image block, and the gradient complexity parameter value of the reference image block.

[0082] Specifically, the pixel difference parameter value can be represented by DIF, which can be obtained by calculation methods such as SAD (sum of absolute difference) and MSE (mean square error); the distance reference value can be represented by DST, which can be obtained by any parameter representing the distance such as the square of the pixel distance; the texture direction difference parameter value can be represented by DIR, which can be the angle of the texture direction between the target image block and the reference image block; the gradient complexity parameter value of the target image block can be represented by CGRAD, which can be calculated using operators such as Canny operator, Sobel operator, Laplace operator, etc.; the gradient complexity parameter value of the reference image block can be represented by RGRAD, which can also be calculated using operators such as Canny operator, Sobel operator, Laplace operator, etc.

[0083] Next, the terminal device may calculate the pixel values ​​of the pixels on the denoised image block corresponding to the target image block based on the pixel values ​​of the pixels on the target image block, the pixel values ​​of the pixels on the reference image block and the determined adjustment parameter values.

[0084] Taking the adjustment parameter values ​​including the pixel difference parameter value DIF, the distance reference value DST, the texture direction difference parameter value DIR, the gradient complexity parameter value CGRAD of the target image block and the gradient complexity parameter value RGRAD of the reference image block as an example, the pixel value of each pixel point on the denoised image block can be calculated using Formula 1.

[0085]

[0086] Among them, out_blk i,j is the pixel value of the pixel at position (i, j) in the denoised image block, cur_blk i,j is the pixel value of the pixel at position (i, j) in the target image block before denoising, is the pixel value of the pixel at position (i, j) in the kth reference image block, N is the number of reference image blocks, and a, b, c, d, and e are manually set coefficients.

[0087] It should be noted that Formula 1 is only an example description. It is easy to understand that when the adjustment parameter values ​​include one or more of the pixel difference parameter value DIF, the distance reference value DST, the texture direction difference parameter value DIR, the gradient complexity parameter value CGRAD of the target image block, and the gradient complexity parameter value RGRAD of the reference image block, Formula 1 can also be used for modification, and the present disclosure does not impose any restrictions on this.

[0088] It should be noted that, according to some embodiments of the present disclosure, for each round of image processing, the adjustment parameter types corresponding to the adjustment parameter values ​​determined by the terminal device are different or not completely the same.

[0089] Among them, the adjustment parameter type refers to the type corresponding to the parameter value, for example, the pixel difference corresponding to the pixel difference parameter value DIF is one type of adjustment parameter, the distance corresponding to the distance reference value DST is one type of adjustment parameter, the texture direction difference corresponding to the texture direction difference parameter value DIR is one type of adjustment parameter, the gradient complexity corresponding to the gradient complexity parameter value CGRAD of the target image block is one type of adjustment parameter, and the gradient complexity corresponding to the gradient complexity parameter value RGRAD of the reference image block can be another type of adjustment parameter.

[0090] According to other embodiments of the present disclosure, in each round of image processing, the adjustment parameter types corresponding to the adjustment parameter values ​​determined by the terminal device may be the same. For example, the adjustment parameter values ​​include a pixel difference parameter value DIF, a distance reference value DST, a texture direction difference parameter value DIR, a gradient complexity parameter value CGRAD of a target image block, and a gradient complexity parameter value RGRAD of a reference image block.

[0091] In addition, the present disclosure also provides a solution for determining whether a target image block needs to be denoised and denoising the target image block that needs to be denoised.

[0092] Specifically, whether the target image block needs to be denoised may be determined according to the adjustment parameter value of the target image block.

[0093] For example, in the case where the adjustment parameter value only includes the pixel difference parameter value, if it is determined that the pixel difference parameter value between the target image block and the reference image block is less than a difference threshold, the target image block may not be subjected to denoising and may be directly applied to the subsequent output image splicing process; if it is determined that the pixel difference parameter value between the target image block and the reference image block is greater than or equal to the difference threshold, the target image block may be subjected to the above denoising. The present disclosure does not limit the specific value of the difference threshold.

[0094] For another example, in the case where the adjustment parameter value only includes the gradient complexity parameter value of the target image block, if it is determined that the gradient complexity parameter value is less than a gradient complexity threshold, it means that the gradient complexity of the target image block is small, and the above-mentioned denoising process can be performed on the target image block; if it is determined that the gradient complexity parameter value is greater than or equal to the gradient complexity threshold, it means that the gradient complexity of the target image block is large, and the target image block may contain more complex detail information. At this time, the target image block may not be denoised, and can be directly applied to the subsequent output image splicing process to avoid losing image details. The present disclosure does not limit the specific value of the gradient complexity threshold.

[0095] In addition, a mapping relationship table between the adjustment parameter value and whether denoising is required can be pre-configured. Whether denoising is required can be represented by 0 and 1, for example, 0 means denoising is required and 1 means denoising is not required; or 0 means denoising is not required and 1 means denoising is required. After determining the adjustment parameter value of the target image block, the terminal device can directly determine whether denoising is required for the target image block by querying the mapping relationship table. If denoising is not required, the target image block can be directly applied to the stitching process of the subsequent output image. If denoising is required, the above-mentioned denoising process can be used to obtain the denoised image block.

[0096] After obtaining the denoised image block corresponding to the target image block, an output image can be generated and sent to the next round of image processing. In this round, which is the last round of image processing, the image is directly output as a processed image corresponding to the image to be processed.

[0097] Specifically, the denoised image block corresponding to each target image block may be determined, and the denoised image blocks corresponding to each target image block may be spliced ​​to generate an output image.

[0098] An embodiment of a total of three rounds of image processing will be described below.

[0099] In the first round of image processing, the image to be processed is divided into m0*n0 target image blocks, and the eight image blocks adjacent to the target image block can be used as reference image blocks of the target image block, and the adjustment parameter value of the target image block is determined to be the pixel difference parameter value DIF.

[0100] Formula 2 calculates the pixel value of each pixel on the denoised image block:

[0101]

[0102] Among them, out_blk i,j is the pixel value of the pixel at position (i, j) in the denoised image block, cur_blk i,j is the pixel value of the pixel at position (i, j) in the target image block before denoising, is the pixel value of the pixel at position (i, j) in the kth reference image block.

[0103] The denoised image blocks are spliced ​​together to obtain the output image of the first round of image processing.

[0104] In the second round of image processing, the output image of the first round of image processing is divided into target image blocks of m0*n0, and compared with the image blocks divided in the first round of image processing, the phase is 1 / 2 in both the horizontal and vertical directions to form a positive interlacing. The four image blocks immediately above, below, left and right of the target image block of the second round of image processing can be used as reference image blocks of the target image block, and the adjustment parameters of the target image block are determined to be the pixel difference parameter value DIF and the texture direction difference parameter value DIR. Therefore, the pixel value of each pixel on the image block after denoising in the second round of image processing can be calculated using formula 3:

[0105]

[0106] Among them, out_blk i,j is the pixel value of the pixel at position (i, j) in the denoised image block, cur_blk i,j is the pixel value of the pixel at position (i, j) in the target image block before denoising, is the pixel value of the pixel at position (i, j) in the kth reference image block.

[0107] The denoised image blocks are spliced ​​together to obtain the output image of the second round of image processing.

[0108] In the third round of image processing, the output image of the second round of image processing is divided into target image blocks of m1*n1, where m1 is greater than 4*m0 and n1 is greater than 4*n0. The image block determined in the search window can be used as a reference image block of the target image block, and the adjustment parameter values ​​of the target image block are determined to include the pixel difference parameter value DIF, the distance reference value DST, the texture direction difference parameter value DIR, the gradient complexity parameter value CGRAD of the target image block, and the gradient complexity parameter value RGRAD of the reference image block. Therefore, the pixel value of each pixel on the denoised image block can be calculated using Formula 4:

[0109]

[0110] Among them, out_blk i,j is the pixel value of the pixel at position (i, j) in the denoised image block, cur_blk i,j is the pixel value of the pixel at position (i, j) in the target image block before denoising, is the pixel value of the pixel at position (i, j) in the kth reference image block, and N is the number of reference image blocks.

[0111] The denoised image blocks are spliced ​​together to obtain the output image of the third round of image processing, which is the denoised image corresponding to the image to be processed.

[0112] An embodiment of a two-round image processing process is described below.

[0113] In the first round of image processing, the image to be processed is divided into target image blocks of m0*n0, and the four image blocks adjacent to the target image block, the upper, lower, left and right, can be used as reference image blocks of the target image block, and the adjustment parameter values ​​of the target image block are determined, including the pixel difference parameter value DIF, the distance reference value DST, the texture direction difference parameter value DIR, the gradient complexity parameter value CGRAD of the target image block, and the gradient complexity parameter value RGRAD of the reference image block. Therefore, the pixel value of each pixel on the denoised image block can be calculated using the above formula 1. The denoised image blocks are spliced ​​to obtain the output image of the first round of image processing.

[0114] In the second round of image processing, the output image of the first round of image processing is divided into m1*n1 target image blocks, where m1 is equal to 4*m0 and n1 is equal to 4*n0. Fig. 9 As shown, m1*n1 includes 16 m0*n0. The four image blocks immediately adjacent to the target image block, namely, the upper left, upper right, lower left, and lower right, can be used as reference image blocks of the target image block, and the adjustment parameter value of the target image block is determined as the pixel difference parameter value DIF. Therefore, the pixel value of each pixel on the image block after denoising in the second round of image processing can be calculated using formula 5:

[0115]

[0116] Among them, out_blk i,j is the pixel value of the pixel at position (i, j) in the denoised image block, cur_blk i,j is the pixel value of the pixel at position (i, j) in the target image block before denoising, is the pixel value of the pixel at position (i, j) in the kth reference image block.

[0117] The denoised image blocks are spliced ​​together to obtain the output image of the second round of image processing, which is the denoised image corresponding to the image to be processed.

[0118] The following will refer to Fig.10 The entire process of the image processing solution of the embodiment of the present disclosure is described.

[0119] The image to be processed is used as the input image of the first round of image processing process, and the first round of image processing process is performed. Specifically, first, the image block segmentation method is determined, including determining the size and position of the image block. Next, NLM denoising is performed on each image block, and the denoised image blocks are spliced ​​to obtain the denoising result of the first round, that is, the output image of the first round of image processing process.

[0120] Then, the output image of the first round of image processing is used as the input image of the second round of image processing, and the second round of image processing is performed until the Nth round of image processing is performed, and finally a denoised image corresponding to the image to be processed is obtained. The present disclosure does not limit the specific value of N, which can be a positive integer greater than or equal to 2.

[0121] In summary, the embodiments of the present disclosure provide a multi-round multilateral NLM denoising scheme based on interlaced image blocks. On the one hand, compared with the pixel-level NLM scheme of some technologies, it can greatly reduce the computational complexity and eliminate the block effect to a certain extent; on the other hand, not only can denoising be performed through multiple rounds of NLM processing based on interlaced image blocks, but also multilateral similarity measurement can be performed by combining factors such as distance, direction, and pixel difference, which can further improve the image denoising effect.

[0122] It should be noted that although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0123] Furthermore, this exemplary embodiment also provides an image processing device.

[0124] The image processing device of the exemplary embodiment of the present disclosure can be configured as follows: performing multiple rounds of image processing processes on the image to be processed to obtain a denoised image corresponding to the image to be processed; wherein the image segmentation method adopted in each round of image processing process is different or not completely the same, and except for the first round of image processing process, the input image of each round of image processing process is the output image of the previous round of image processing process of the image processing process; each round of image processing process includes: performing image segmentation on the input image to obtain multiple target image blocks; performing denoising on each target image block respectively, and generating an output image according to the result of the denoising process.

[0125] According to an exemplary embodiment of the present disclosure, the process of an image processing device performing denoising on a target image block can be configured to execute: establishing a search window centered on the target image block; determining at least one reference image block corresponding to the target image block in the search window; and generating a denoised image block corresponding to the target image block using the target image block and at least one reference image block.

[0126] According to an exemplary embodiment of the present disclosure, a process in which an image processing apparatus generates a denoised image block corresponding to a target image block using a target image block and at least one reference image block can be configured to perform: determining an adjustment parameter value of the target image block, the adjustment parameter value including one or more of a pixel difference parameter value between the target image block and the reference image block, a distance parameter value between the target image block and the reference image block, a texture direction difference parameter value between the target image block and the reference image block, a gradient complexity parameter value of the target image block, and a gradient complexity parameter value of the reference image block; and calculating the pixel value of a pixel on the denoised image block corresponding to the image block based on the pixel value of the pixel on the target image block, the pixel value of the pixel on the reference image block, and the adjustment parameter value.

[0127] According to an exemplary embodiment of the present disclosure, the types of adjustment parameters corresponding to the adjustment parameter values ​​determined in each round of image processing are different or not completely the same.

[0128] According to an exemplary embodiment of the present disclosure, the process of an image processing device generating an output image based on the result of denoising processing can be configured to perform: determining the denoised image block corresponding to each target image block; and splicing the denoised image blocks corresponding to each target image block to generate an output image.

[0129] According to an exemplary embodiment of the present disclosure, the process of an image processing device determining at least one reference image block corresponding to a target image block in a search window can be configured to perform: determining a horizontal step and a vertical step for searching for a reference image block in the search window according to the size of the target image block; searching for at least one reference image block in the search window using the horizontal step and the vertical step; wherein the size of the reference image block is the same as the size of the target image block.

[0130] According to an exemplary embodiment of the present disclosure, the image segmentation method includes segmenting a size of a target image block and / or segmenting a position of a target image block.

[0131] Since the functional modules of the image processing device in the embodiment of the present disclosure are the same as those in the above-mentioned method embodiment, they will not be described in detail here.

[0132] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0133] In addition, the above-mentioned figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0134] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0135] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing what is disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and embodiments are to be considered merely as exemplary, and the true scope and spirit of the present disclosure are indicated by the claims.

[0136] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An image processing method, characterized in that: include: Performing multiple rounds of image processing on the image to be processed to obtain a denoised image corresponding to the image to be processed; wherein each round of image processing adopts a different or partially the same image segmentation method, and except for the first round of image processing, the input image of each round of image processing is the output image of the previous round of image processing of the image processing process; Each round of image processing includes: Perform image segmentation on the input image to obtain multiple target image blocks; De-noising is performed on each of the target image blocks respectively, and an output image is generated according to the result of the de-noising.

2. The image processing method according to claim 1, characterized in that: The target image block is subjected to denoising processing, comprising: Establishing a search window with the target image block as the center; In the search window, determining at least one reference image block corresponding to the target image block; A denoised image block corresponding to the target image block is generated by using the target image block and the at least one reference image block.

3. The image processing method according to claim 2, characterized in that: The step of generating a denoised image block corresponding to the target image block by using the target image block and the at least one reference image block comprises: determining an adjustment parameter value of the target image block, wherein the adjustment parameter value includes one or more of a pixel difference parameter value between the target image block and the reference image block, a distance parameter value between the target image block and the reference image block, a texture direction difference parameter value between the target image block and the reference image block, a gradient complexity parameter value of the target image block, and a gradient complexity parameter value of the reference image block; Based on the pixel values ​​of the pixels on the target image block, the pixel values ​​of the pixels on the reference image block and the adjustment parameter value, the pixel values ​​of the pixels on the denoised image block corresponding to the target image block are calculated.

4. The image processing method according to claim 3, characterized in that: The adjustment parameter types corresponding to the adjustment parameter values ​​determined in each round of image processing are different or partially the same.

5. The image processing method according to claim 3, characterized in that: Based on the results of the denoising process, an output image is generated, including: Determine a denoised image block corresponding to each of the target image blocks; The denoised image blocks corresponding to each of the target image blocks are spliced ​​to generate an output image.

6. The image processing method according to claim 2, characterized in that: Determining at least one reference image block corresponding to the target image block in the search window includes: Determining, according to the size of the target image block, a horizontal step size and a vertical step size for searching for a reference image block in the search window; Searching for at least one reference image block in the search window using the horizontal step size and the vertical step size; The size of the reference image block is the same as the size of the target image block.

7. The image processing method according to any one of claims 1 to 6, characterized in that: The image segmentation method includes segmenting the size of the target image block and / or segmenting the position of the target image block.

8. An image processing device, characterized in that: include: The image processing device is configured to perform multiple rounds of image processing processes on the image to be processed to obtain a denoised image corresponding to the image to be processed; wherein each round of image processing process adopts a different or partially the same image segmentation method, and except for the first round of image processing process, the input image of each round of image processing process is the output image of the previous round of image processing process of the image processing process; Each round of image processing includes: Perform image segmentation on the input image to obtain multiple target image blocks; De-noising is performed on each of the target image blocks respectively, and an output image is generated according to the result of the de-noising.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the image processing method according to any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: include: processor; A memory for storing one or more programs, which, when executed by the processor, enables the processor to implement the image processing method according to any one of claims 1 to 7.

Images