Image processing method and device and electronic equipment
By judging and correcting the sharpening effect of the target pixel point in image processing, the problem of sharpening edges and enhancing noise in the prior art is solved, effectively weakening Gaussian noise and Poisson noise, reducing resource occupancy and energy Consumption.
Patent Information
- Application Number
- CN202411161016.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-22
- Publication Date
- 2024-12-03
AI Technical Summary
The existing image sharpening algorithm fails to effectively consider Gaussian noise and Poisson noise when processing images, resulting in sharpening edges simultaneously enhancing noise, and high resource occupancy and high energy consumption.
An image processing method is proposed, by determining the target image block with the target pixel point as the center, determining whether there is an edge, and performing sharpening processing in the presence of an edge. If oversharpening is found, the initial pixel of the surrounding pixel point is based on The value is corrected to reduce the sharpening effect of Gaussian noise and Poisson noise, reduce the number of transition pixels, and reduce resource occupancy and energy consumption.
Effectively weaken the sharpening effect of oversharpened target pixel points, reduce the sharpening of Gaussian noise and Poisson noise, reduce the number of transition pixels, and save resource occupancy and energy consumption.
Smart Images

Figure CN119067883A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of image processing technology, and in particular, relates to an image processing method, device and electronic device. Background Art
[0002] Images captured by a camera generally contain Poisson noise and Gaussian noise. In order to obtain a clearer image, related technologies use an image sharpening algorithm to sharpen the image to obtain clearer edges. However, existing image sharpening algorithms often do not take into account the Gaussian noise and Poisson noise problems of the image, resulting in sharpening of the noise while sharpening the edges. Summary of the invention
[0003] The present application aims to solve at least one of the technical problems existing in the related art. To this end, the present application proposes an image processing method, device, electronic device, computer-readable storage medium and computer program product, which can weaken the sharpening effect of over-sharpened target pixels and reduce the sharpening of Gaussian noise and Poisson noise.
[0004] In a first aspect, the present application provides an image processing method, the method comprising:
[0005] For a target pixel point in the image to be processed, determining a target image block centered at the target pixel point;
[0006] When it is determined that the target image block has an edge, the initial pixel value of each pixel in the target image block is sharpened to obtain a sharpened pixel value of the target pixel;
[0007] When it is determined that the target pixel is over-sharpened based on the initial pixel values of the surrounding pixels of the target pixel, the sharpened pixel value of the target pixel is corrected to obtain and output the corrected pixel value of the target pixel.
[0008] According to the image processing method of the present application, when it is determined that a target image block has an edge, the initial pixel value of each pixel in the target image block is sharpened to obtain the sharpened pixel value of the target pixel, and the image block without an edge is not sharpened. In addition, when it is determined that the target pixel is over-sharpened based on the initial pixel values of the surrounding pixels of the target pixel, the sharpened pixel value of the target pixel is corrected to obtain and output the corrected pixel value of the target pixel, thereby weakening the sharpening effect of the over-sharpened target pixel, reducing the sharpening of Gaussian noise and Poisson noise, reducing the number of transition pixels, and being able to reduce resource occupancy and save energy consumption.
[0009] According to an embodiment of the present application, the resolution of the target image block is n*n, where n is greater than or equal to 5; determining that the target pixel is over-sharpened includes:
[0010] Determine a pixel array of the target image block according to the edge included in the target image block;
[0011] Determining a distribution trend of average pixel values of each pixel array;
[0012] In the case where it is determined that the distribution trend characterizes that the average pixel values of each pixel array decrease, determining a first difference between the average pixel values of two pixel arrays with the largest average pixel values in the decreasing direction, and / or a second difference between the average pixel values of two pixel arrays with the smallest average pixel values;
[0013] Do at least one of the following:
[0014] In the case where it is determined that the first difference is less than the first target difference and the sharpened pixel value of the target pixel is greater than the average pixel value of the previous pixel array in the decreasing direction, it is determined that the target pixel is over-sharpened;
[0015] When it is determined that the second difference is smaller than the second target difference and the sharpened pixel value of the target pixel is smaller than the average pixel value of the subsequent pixel array in the decreasing direction, it is determined that the target pixel is over-sharpened.
[0016] According to one embodiment of the present application, when the sharpened pixel value of the target pixel point is greater than the average pixel value of the previous pixel array in the decreasing direction, the sharpened pixel value of the target pixel point is corrected, including:
[0017] Correcting the sharp pixel value of the target pixel point according to the average pixel value of the previous pixel array;
[0018] When the sharpened pixel value of the target pixel point is less than the average pixel value of the subsequent pixel array in the decreasing direction, the sharpened pixel value of the target pixel point is corrected, including:
[0019] The sharpened pixel value of the target pixel point is corrected according to the average pixel value of the latter pixel array.
[0020] According to one embodiment of the present application, the pixel array is a pixel row or a pixel column;
[0021] According to the edge contained in the target image block, a pixel array of the target image block is determined, including:
[0022] In the case where it is determined that the target image block includes a vertical edge, each pixel array of the target image block is determined to be a pixel column;
[0023] When it is determined that the target image block does not include a vertical edge but includes a horizontal edge, each pixel array of the target image block is determined to be a pixel row.
[0024] According to an embodiment of the present application, determining whether a target image block has an edge includes:
[0025] Determine a block average pixel value of a target image block, a row average pixel value of each pixel row in the target image block, and a column average pixel value of each pixel column;
[0026] Obtaining a third target difference and a fourth target difference according to the block average pixel value and the first coefficient;
[0027] Determine a first column mean difference between column mean pixel values of a first pixel column and a last pixel column, and a first row mean difference between row mean pixel values of a first pixel row and a last pixel row of the target image block;
[0028] When it is determined that the first column mean difference is greater than the third target difference, it is determined that a vertical edge exists in the target image block;
[0029] When it is determined that the first row mean value difference is greater than the fourth target difference, it is determined that a horizontal edge exists in the target image block.
[0030] According to an embodiment of the present application, performing a sharpening operation on the initial pixel value of each pixel point in the target image block includes:
[0031] Filtering the initial pixel value of each pixel in the target image block to obtain a mask layer of the target image block;
[0032] Subtract the pixel value of each pixel in the mask layer from the initial pixel value of each pixel in the target image block to obtain the image detail layer of the target image block;
[0033] According to the initial pixel value of each pixel in the target image block and the pixel value of each pixel in the image detail layer, the sharpened pixel value of each pixel in the target image block is obtained.
[0034] According to an embodiment of the present application, determining a target image block centered at a target pixel point includes:
[0035] Perform sliding window processing on the image to be processed with the target pixel as the center to obtain a sliding window with the target pixel as the center;
[0036] The sliding window centered on the target pixel is used as the target image block centered on the target pixel.
[0037] According to one embodiment of the present application, after obtaining the sharpened pixel value of the target pixel point, the method further includes:
[0038] When it is determined that the target pixel point is not over-sharpened, a sharpened pixel value is output.
[0039] In a second aspect, the present application provides an image processing device, the device comprising:
[0040] A first processing module is used to determine a target image block centered on a target pixel point in an image to be processed;
[0041] The second processing module is used to perform sharpening processing on the initial pixel value of each pixel point in the target image block to obtain the sharpened pixel value of the target pixel point when it is determined that the target image block has an edge;
[0042] The third processing module is used to correct the sharpened pixel value of the target pixel point when it is determined that the target pixel point is over-sharpened based on the initial pixel values of the surrounding pixels of the target pixel point, and obtain and output the corrected pixel value of the target pixel point.
[0043] According to the image processing device of the present application, when it is determined that a target image block has an edge, the initial pixel value of each pixel in the target image block is sharpened to obtain the sharpened pixel value of the target pixel, and the image block without an edge is not sharpened. In addition, when it is determined that the target pixel is over-sharpened based on the initial pixel values of the surrounding pixels of the target pixel, the sharpened pixel value of the target pixel is corrected to obtain and output the corrected pixel value of the target pixel, thereby weakening the sharpening effect of the over-sharpened target pixel, reducing the sharpening of Gaussian noise and Poisson noise, reducing the number of transition pixels, and being able to reduce resource occupancy and save energy.
[0044] In a third aspect, the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the image processing method of the first aspect described above when executing the computer program.
[0045] In a fourth aspect, the present application provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image processing method provided in the first aspect above.
[0046] In a fifth aspect, the present application provides a chip, the chip including a processor and a communication interface, the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the image processing method provided in the first aspect.
[0047] In a sixth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the image processing method provided in the first aspect above.
[0048] The above one or more technical solutions in the embodiments of the present application have at least one of the following technical effects:
[0049] In the embodiment of the present application, when it is determined that a target image block has an edge, the initial pixel value of each pixel in the target image block is sharpened to obtain the sharpened pixel value of the target pixel, and the image block without an edge is not sharpened, the sharpening of Gaussian noise and Poisson noise is reduced, and the number of transition pixels is reduced.
[0050] In addition, when it is determined that the target pixel is over-sharpened based on the initial pixel values of the surrounding pixels of the target pixel, the sharpened pixel value of the target pixel is corrected to obtain and output the corrected pixel value of the target pixel. This can also weaken the sharpening effect of the over-sharpened target pixel, reduce the sharpening of Gaussian noise and Poisson noise, reduce the number of transition pixels, and can reduce resource occupancy and save energy consumption.
[0051] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0052] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:
[0053] Figure 1 This is one of the flowcharts of the image processing method provided in the embodiment of the present application;
[0054] Figure 2 is a schematic diagram of the pixel values of each pixel point in each pixel column in the image to be processed provided by an embodiment of the present application;
[0055] Figure 3 is a schematic diagram of expanding the pixel columns of an image to be processed by mirror expansion provided by an embodiment of the present application;
[0056] Figure 4 Schematic diagram of extending the pixel column of the image to be processed by copying and extending according to an embodiment of the present application
[0057] Figure 5 is a schematic diagram of a sliding window centered on a target pixel point provided in an embodiment of the present application;
[0058] Figure 6 is a schematic diagram of correcting the sharpened pixel value of a target pixel point provided by an embodiment of the present application;
[0059] Figure 7 is a structural schematic diagram of an image processing device provided in an embodiment of the present application;
[0060] Figure 8 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0061] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.
[0062] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.
[0063] Transition pixels are pixels in edge areas of an image that change significantly during the sharpening process.
[0064] When collecting images through camera sensors, Poisson noise and Gaussian noise are generally present. At the same time, due to other reasons such as improper lens and focal length, the camera image has more transition pixels at the edge of the object. In order to make the camera image clear, image sharpening algorithms are generally used to reduce the number of transition pixels. At present, most traditional sharpening algorithms use algorithm templates for convolution operations, which will enhance the noise together with the image edges. At the same time, because the resources of linear array cameras are limited, they cannot bear the resources consumed by convolution operations.
[0065] To solve the above problems, embodiments of the present application provide an image processing method, an image processing device, an electronic device, and a readable storage medium.
[0066] The image processing method, image processing device, electronic device and readable storage medium provided in the embodiments of the present application are described in detail below with reference to the accompanying drawings through specific embodiments and their application scenarios.
[0067] The image processing method may be applied to a terminal, and may be specifically executed by hardware or software in the terminal.
[0068] The image processing method provided in the embodiment of the present application may be executed by an electronic device or a functional module or functional entity in the electronic device that can implement the image processing method. The electronic devices mentioned in the embodiment of the present application include but are not limited to mobile phones, tablet computers, computers, cameras, and wearable devices. The image processing method provided in the embodiment of the present application is described below using the electronic device as an example of the execution subject.
[0069] like Figure 1 As shown, the image processing method includes: step 110, step 120 and step 130.
[0070] Step 110: for a target pixel point in the image to be processed, determine a target image block centered on the target pixel point.
[0071] The image to be processed in the embodiment of the present application may be an image acquired by a camera, and the camera may be a line array camera or an area array camera.
[0072] When the camera is a linear array camera, the acquired multiple frame images can be stitched. For example, for the current frame image, the current frame image and the frame images before and after the current frame image (m is a positive integer) are acquired and cached. The frame images can be stitched in the acquisition order of each frame image to obtain the image to be processed. At this time, the resolution of the image to be processed is * linear array resolution (n is an odd number, n=2*m+1).
[0073] In the embodiment of the present application, the resolution of the image to be processed is N*N, where N is a positive integer.
[0074] The target pixel point may be any pixel point in the image to be processed. Of course, in some scenarios, each pixel point in the image to be processed may be traversed, and the currently traversed pixel point may be used as the target pixel point.
[0075] In some embodiments, before determining the target image block centered at the target pixel, the method further includes:
[0076] Get the image to be processed;
[0077] Perform sliding window processing on the image to be processed with the target pixel as the center to obtain a sliding window with the target pixel as the center;
[0078] The sliding window centered on the target pixel is used as the target image block centered on the target pixel.
[0079] In order to make each pixel point in the image to be processed serve as the center of the sliding window, the image to be processed may be pixel expanded, and the left and right sides of the image to be processed may be expanded by m pixel columns, and the top and bottom of the image to be processed may be expanded by m pixel rows to obtain the expanded image to be processed.
[0080] In the embodiments of the present application, the methods for performing pixel expansion on the image to be processed include but are not limited to mirror expansion and copy expansion.
[0081] Mirror extension is to extend the pixel rows or pixel columns inside the image with the boundary pixel rows or pixel columns of the image as the central axis, and the extended pixel rows or pixel columns are symmetrical with the pixel rows or pixel columns inside the image.
[0082] like Figure 2 The figure is a schematic diagram of the pixel values of each pixel point in each pixel column in the image to be processed provided by an embodiment of the present application, wherein the pixel values of each pixel point in the first pixel column of the image to be processed are 1, 4, and 7 respectively; the pixel values of each pixel point in the second pixel column are 2, 5, and 8 respectively; and the pixel values of each pixel point in the third pixel column are 3, 6, and 9 respectively.
[0083] Continuing with the above example, see Figure 3 , Figure 3 A schematic diagram of extending the pixel columns of an image to be processed by mirroring extension provided in an embodiment of the present application, Figure 2 The first pixel column of the image to be processed is used as the central axis to expand the image to be processed by one pixel column, and the pixel values of each pixel point in the expanded one-column item sequence are 2, 5, and 8 respectively, which is symmetrical with the original second pixel column.
[0084] Copy extension means copying the boundary pixel rows or boundary pixel columns of the image.
[0085] See also Figure 4 , Figure 4 A schematic diagram of extending the pixel column of an image to be processed by copying and extending provided in an embodiment of the present application, copying Figure 2 The first pixel column of the image to be processed shown is used to expand the image to be processed by 1 pixel column. The pixel values of each pixel point in the expanded 1 pixel column are 1, 4, and 7, respectively, which are consistent with the pixel values of each pixel point in the original first pixel column.
[0086] The above-mentioned embodiment has explained that by taking the currently traversed pixel point as the target pixel point, sliding window processing can be performed on the image to be processed with the target pixel point as the center to obtain a sliding window with the target pixel point as the center, and the sliding window with the target pixel point as the center can be used as the target image block with the target pixel point as the center.
[0087] like Figure 5 As shown, Figure 5 A schematic diagram of a sliding window centered on a target pixel point provided in an embodiment of the present application, wherein the resolution of the sliding window is 5*5, the central pixel of the sliding window is the target pixel point, and the other pixels in the sliding window except the target pixel point are the surrounding pixels of the target pixel point.
[0088] Step 120: When it is determined that the target image block has an edge, the initial pixel value of each pixel in the target image block is sharpened to obtain a sharpened pixel value of the target pixel.
[0089] In order to weaken Poisson noise and Gaussian noise, the embodiment of the present application needs to first determine whether the target image block has an edge when sharpening the target image block.
[0090] In order to reduce the sharpening of noise, the embodiment of the present application first determines whether the target pixel block has an edge. If the target image block has an edge, the initial pixel value of each pixel in the target image block is sharpened to obtain the sharpened pixel value of the target pixel.
[0091] The edge of an image is an area in the image where the grayscale or color value changes significantly. Specifically, an edge is usually where the brightness or color changes sharply from one value to another in an image. Edges play an important role in image processing because they usually correspond to the outlines of objects, object boundaries, or structural features in the image.
[0092] The embodiment of the present application can compare the pixel value changes between the first pixel column and the last pixel column of the target image block, and the pixel value changes between the first pixel row and the last pixel row of the target pixel block to determine whether there is an edge in the target pixel block. The detailed process is shown in the subsequent content.
[0093] The pixel value is the numerical representation of a single pixel in an image, usually used to describe color. When the image to be processed is a grayscale image, the pixel value directly represents the grayscale value, that is, the brightness of the pixel, ranging from black to white.
[0094] In some embodiments, performing a sharpening operation on the initial pixel value of each pixel in the target image block includes:
[0095] Filtering the initial pixel value of each pixel in the target image block to obtain a mask layer of the target image block;
[0096] Subtract the pixel value of each pixel in the mask layer from the initial pixel value of each pixel in the target image block to obtain the image detail layer of the target image block;
[0097] According to the initial pixel value of each pixel in the target image block and the pixel value of each pixel in the image detail layer, the sharpened pixel value of each pixel in the target image block is obtained.
[0098] Specifically, the initial pixel value of each pixel in the target image block can be filtered by some preset filtering algorithms. The filtering algorithm can be mean filtering, Gaussian filtering, guided filtering, etc. Not only a single-layer filtering method can be used, but also a multi-layer filtering method can be used for superposition processing, and there is no limitation on this.
[0099] The embodiment of the present application filters the initial pixel value of each pixel in the target image block to obtain a mask layer of the target image block. The mask layer can be used to highlight or hide specific areas of the image, and is usually used to achieve specific visual effects or adjustments.
[0100] After obtaining the mask layer, the embodiment of the present application subtracts the pixel value of each pixel in the mask layer from the initial pixel value of each pixel in the target image block to obtain the image detail layer of the target image block, that is, the initial pixel value of each pixel is subtracted from the pixel value of each pixel in the mask layer to obtain the pixel value of each pixel in the detail layer.
[0101] According to the initial pixel value of each pixel in the target image block and the pixel value of each pixel in the image detail layer, the sharpened pixel value of each pixel in the target image block is obtained. Specifically, the sharpened pixel value = the pixel value of the pixel in the detail layer * the second coefficient thr + the initial pixel value, 0 < the second coefficient thr < 1, thr is usually 0.5.
[0102] The sharpened pixel value of each pixel in the target image block includes the sharpened pixel value of the target pixel.
[0103] Step 130: When it is determined that the target pixel is over-sharpened based on the initial pixel values of the surrounding pixels of the target pixel, the sharpened pixel value of the target pixel is corrected to obtain and output the corrected pixel value of the target pixel.
[0104] After obtaining the sharpened pixel value of the target pixel, the embodiment of the present application needs to determine whether the sharpened pixel value of the target pixel is over-sharpened based on the initial pixel values of the surrounding pixels of the target pixel.
[0105] When it is determined that the target pixel point is not over-sharpened, the sharpened pixel point of the target pixel point is directly output.
[0106] When it is determined that the target pixel is over-sharpened, the sharpened pixel value of the target pixel is corrected according to the initial pixel values of the surrounding pixels of the target pixel, and the corrected pixel value of the target pixel is obtained and output.
[0107] In the embodiment of the present application, when it is determined that a target image block has an edge, the initial pixel value of each pixel in the target image block is sharpened to obtain the sharpened pixel value of the target pixel, and the image block without an edge is not sharpened, the sharpening of Gaussian noise and Poisson noise is reduced, and the number of transition pixels is reduced.
[0108] In addition, when it is determined that the target pixel is oversharpened according to the initial pixel values of the surrounding pixels of the target pixel, the sharpened pixel value of the target pixel is corrected, and the corrected pixel value of the target pixel is obtained and output, which can also weaken the sharpening effect of the oversharpened target pixel, reduce the sharpening of Gaussian noise and Poisson noise, reduce the number of transitional pixels, and can reduce the resource occupancy and save energy consumption.
[0109] In some embodiments, determining that the target image block has an edge includes:
[0110] Determining the block average pixel value of the target image block, the row average pixel value of each pixel row in the target image block, and the column average pixel value of each pixel column;
[0111] Obtaining a third target difference and a fourth target difference according to the block average pixel value and the first coefficient;
[0112] Determining a first column mean difference between the column average pixel values of the first column pixel column and the last column pixel column of the target image block, and a first row mean difference between the row average pixel values of the first row pixel row and the last row pixel row;
[0113] When it is determined that the first column mean difference is greater than the third target difference, it is determined that the target image block has a vertical edge;
[0114] When it is determined that the first row mean difference is greater than the fourth target difference, it is determined that the target image block has a horizontal edge.
[0115] In the embodiments of the present application, to determine whether the target image block has an edge, it is necessary to determine the block average pixel value of the target image block, the row average pixel value of each pixel row in the target image block, and the column average pixel value of each pixel column.
[0116] The block average pixel value is the mean of the pixel values of all pixel points in the entire target image block.
[0117] The row average pixel value is the mean of the pixel values of the pixel points in a pixel row.
[0118] The column average pixel value is the mean of the pixel values of the pixel points in a pixel column.
[0119] The third target difference and the fourth target difference can be obtained according to the block average pixel value and the first coefficient A. Both the third target difference and the fourth target difference can be thresholds. The third target difference = block average pixel value * first coefficient A, 0 < A < 1, A is usually 0.5, the fourth target difference = block average pixel value * second coefficient A', 0 < A' < 1, A' is usually 0.5. The first coefficient A and the second coefficient A' can be the same or different, that is, the third target difference and the fourth target difference can be the same or different.
[0120] The embodiment of the present application can determine the first column mean difference between the column average pixel values of the first column pixel column and the last column pixel column of the target image block. If the first column mean difference is greater than the third target difference, it indicates that the target image block has a vertical edge.
[0121] The embodiment of the present application can also determine the first row mean difference between the row average pixel values of the first row of pixels and the last row of pixels of the target image block. If the first row mean difference is greater than the fourth target difference, it indicates that the target image block has a horizontal edge.
[0122] If the above two conditions are met at the same time, it is a special edge.
[0123] If the target image block has any kind of edge, the target image block needs to be sharpened. If none of them are satisfied, the initial pixel value is directly output without sharpening.
[0124] In some embodiments, the resolution of the target image block is n*n, where n is greater than or equal to 5.
[0125] In some embodiments, determining that the target pixel is over-sharpened includes:
[0126] Determine a pixel array of the target image block according to the edge included in the target image block;
[0127] Determining a distribution trend of average pixel values of each pixel array;
[0128] In the case where it is determined that the distribution trend characterizes that the average pixel values of each pixel array decrease, determining a first difference between the average pixel values of two pixel arrays with the largest average pixel values in the decreasing direction, and / or a second difference between the average pixel values of two pixel arrays with the smallest average pixel values;
[0129] Do at least one of the following:
[0130] In the case where it is determined that the first difference is less than the first target difference and the sharpened pixel value of the target pixel is greater than the average pixel value of the previous pixel array in the decreasing direction, it is determined that the target pixel is over-sharpened;
[0131] When it is determined that the second difference is smaller than the second target difference and the sharpened pixel value of the target pixel is smaller than the average pixel value of the subsequent pixel array in the decreasing direction, it is determined that the target pixel is over-sharpened.
[0132] It is understandable that if the types of edges contained in the target image block are different, then the methods of determining whether the target pixel point is over-sharpened may be different. Therefore, it is necessary to determine the pixel array of the target image block based on the edges contained in the target image block.
[0133] In some embodiments, the pixel array is a pixel row or a pixel column, that is, the pixel array is a single row of pixel rows or a single column of pixel columns.
[0134] In some embodiments, determining a pixel array of a target image block according to an edge included in the target image block includes:
[0135] In the case where it is determined that the target image block includes a vertical edge, each pixel array of the target image block is determined to be a pixel column;
[0136] When it is determined that the target image block does not include a vertical edge but includes a horizontal edge, each pixel array of the target image block is determined to be a pixel row.
[0137] When it is determined that the target pixel block contains a vertical edge, each pixel array of the target image block is determined to be a pixel column. In this case, the average pixel value of the pixel array is the column average pixel value of the pixel column; when the target image block does not contain a vertical edge but contains a horizontal edge, each pixel array of the target image block is determined to be a pixel row. In this case, the average pixel value of the pixel array is the row average pixel value of the pixel row.
[0138] The distribution trend of the average pixel value in the embodiment of the present application is used to characterize the change pattern of the average pixel value in a certain direction (for example, from top to bottom, from bottom to top, from left to right, and from right to left, etc.).
[0139] When the target image block contains a vertical edge, it is necessary to determine a first distribution trend of column average pixel values of each pixel column. The first distribution trend may be a change rule of the column average pixel value from left to right or from right to left.
[0140] When the target image block does not contain vertical edges but contains horizontal edges, it is necessary to determine the second distribution trend of the row average pixel values of each pixel row. The second distribution trend may be a change rule of the row average pixel values from top to bottom or from bottom to top.
[0141] It can be understood that, when the column average pixel values of the pixel columns decrease or increase, the two pixel columns with the largest or smallest column average pixel values are the two leftmost pixel columns or the two rightmost pixel columns.
[0142] When the row average pixel values of pixel rows decrease or increase, the two pixel rows with the largest or smallest row average pixel values are the top two pixel rows or the bottom two pixel rows.
[0143] In the embodiment of the present application, when determining that the distribution trend characterizes that the average pixel values of each pixel array are decreasing, it is necessary to combine the first difference between the average pixel values of the two pixel arrays with the largest average pixel values in the decreasing direction and the size relationship between the sharpened pixel value of the target pixel point and the average pixel value of the previous pixel array in the decreasing direction to determine whether the sharpened pixel value of the target pixel point is over-sharpened.
[0144] In addition, whether the sharpened pixel value of the target pixel point is over-sharpened can be determined by combining the second difference between the average pixel values of the two pixel arrays with the smallest average pixel values and the size relationship between the sharpened pixel value of the target pixel point and the average pixel value of the subsequent pixel array in the decreasing direction.
[0145] Specifically, perform at least one of the following:
[0146] When it is determined that the first difference is less than the first target difference and the sharpened pixel value of the target pixel is greater than the average pixel value of the previous pixel array in the decreasing direction, it is determined that the target pixel is over-sharpened; in this case, the sharpened pixel value of the target pixel is corrected according to the average pixel value of the previous pixel array, specifically, the average pixel value of the previous pixel array is used as the corrected pixel value of the target pixel.
[0147] When it is determined that the second difference is smaller than the second target difference and the sharpened pixel value of the target pixel is smaller than the average pixel value of the subsequent pixel array in the decreasing direction, it is determined that the target pixel is over-sharpened; in this case, the sharpened pixel value of the target pixel is corrected according to the average pixel value of the subsequent pixel array, and specifically, the average pixel value of the subsequent pixel array is used as the corrected pixel value of the target pixel.
[0148] like Figure 6 The figure shows a schematic diagram of correcting the sharpened pixel value of the target pixel point provided by an embodiment of the present application, determining the type of edges included, and for a target image block including vertical edges, determining the decreasing direction of the column average pixel values of each pixel column, and when the decreasing direction is determined to be decreasing from left to right, the column average pixel values of the two leftmost pixel columns are the largest, and the column average pixel values of the two rightmost pixel columns are the smallest.
[0149] You can do at least one of the following:
[0150] The first difference a between the column average pixel values of the two leftmost pixel columns can be determined. If the first difference a is smaller than the set first target difference B (the first target difference B is a preset threshold value, such as B=4), it is determined whether the sharpened pixel value of the target pixel point is greater than the column average pixel value of the previous pixel column where the target pixel point is located in the decreasing direction. If it is greater, the column average pixel value of the previous pixel column is used as the corrected pixel value of the target pixel point.
[0151] A second difference b between the column average pixel values of the two rightmost pixel columns can be determined. If the second difference b is smaller than a set second target difference C (the first target difference C is a preset threshold value, such as C=4), it is determined whether the sharpened pixel value of the target pixel point is smaller than the column average pixel value of the pixel column following the pixel column where the target pixel point is located in a decreasing direction. If it is smaller, the column average pixel value of the following pixel column is used as the corrected pixel value of the target pixel point.
[0152] continue Figure 6 , when it is determined that the decreasing direction is decreasing from right to left, the column average pixel value of the two rightmost pixel columns is the largest, and the column average pixel value of the two leftmost pixel columns is the smallest.
[0153] You can do at least one of the following:
[0154] The first difference c between the column average pixel values of the two rightmost pixel columns can be determined. If the first difference c is smaller than the set first target difference B, it is judged whether the sharpened pixel value of the target pixel point is greater than the column average pixel value of the previous pixel column where the target pixel point is located in the decreasing direction. If it is greater, the column average pixel value of the previous pixel column is used as the corrected pixel value of the target pixel point.
[0155] The second difference d between the column average pixel values of the two leftmost pixel columns can be determined. If the second difference d is smaller than the set second target difference C, it is judged whether the sharpened pixel value of the target pixel point is smaller than the column average pixel value of the pixel column following the pixel column where the target pixel point is located in the decreasing direction. If it is smaller, the column average pixel value of the following pixel column is used as the corrected pixel value of the target pixel point.
[0156] The processing principle in the horizontal direction is the same as above.
[0157] continue Figure 6 For a target image block that contains horizontal edges but not vertical edges, the decreasing direction of the row average pixel values of each pixel row is determined. When the decreasing direction is determined to be decreasing from top to bottom, the row average pixel values of the top two pixel rows are the largest, and the row average pixel values of the bottom two pixel rows are the smallest.
[0158] You can do at least one of the following:
[0159] The first difference e between the row average pixel values of the top two pixel rows can be determined. If the first difference e is smaller than the set first target difference B, it is judged whether the sharpened pixel value of the target pixel point is greater than the row average pixel value of the previous pixel row where the target pixel point is located in the decreasing direction. If it is greater, the row average pixel value of the previous pixel row is used as the corrected pixel value of the target pixel point.
[0160] The second difference f between the row average pixel values of the two bottom pixel rows can be determined. If the second difference f is smaller than the set second target difference C, it is judged whether the sharpened pixel value of the target pixel point is smaller than the row average pixel value of the pixel row following the target pixel point in the decreasing direction. If it is smaller, the row average pixel value of the pixel row following the target pixel point is used as the corrected pixel value of the target pixel point.
[0161] continue Figure 6 , when it is determined that the decreasing direction is decreasing from bottom to top, the row average pixel value of the two pixel rows at the bottom is the largest, and the row average pixel value of the two pixel rows at the top is the smallest.
[0162] You can do at least one of the following:
[0163] The first difference g between the row average pixel values of the two bottom pixel rows can be determined. If the first difference g is smaller than the set first target difference B, it is judged whether the sharpened pixel value of the target pixel point is greater than the row average pixel value of the previous pixel row where the target pixel point is located in the decreasing direction. If it is greater, the row average pixel value of the previous pixel row is used as the corrected pixel value of the target pixel point.
[0164] The second difference h between the row average pixel values of the top two pixel rows can be determined. If the second difference h is smaller than the set second target difference C, it is judged whether the sharpened pixel value of the target pixel point is smaller than the row average pixel value of the pixel row after the target pixel point in the decreasing direction. If it is smaller, the row average pixel value of the pixel row after the target pixel point is located is used as the corrected pixel value of the target pixel point.
[0165] Through the above method, it is possible to accurately determine whether the sharpened pixel value of the target pixel point is over-sharpened. If it is over-sharpened, the sharpened pixel value is corrected in time to reduce the over-sharpening.
[0166] In some embodiments, after obtaining the sharpened pixel value of the target pixel, the method further includes:
[0167] When it is determined that the target pixel point is not over-sharpened, a sharpened pixel value is output.
[0168] When any of the following conditions is met, it is determined that the sharpened pixel value of the target pixel point is not over-sharpened:
[0169] (1) The distribution trend is neither decreasing nor increasing;
[0170] (2) the first difference is not less than the first target difference, and the second difference is not less than the second target difference;
[0171] (3) The sharpened pixel value of the target pixel is not greater than the average pixel value of the previous pixel array in the decreasing direction, and the sharpened pixel value of the target pixel is not less than the average pixel value of the next pixel array in the decreasing direction.
[0172] When any one of the above (1), (2) and (3) is met, it is determined that the sharpened pixel value of the target pixel point is not over-sharpened, and the sharpened pixel value of the target pixel point can be directly output.
[0173] The embodiment of the present application determines that the image processing is completed after outputting the pixel value of each target pixel point (any one of the original pixel value, the sharpened pixel value, and the corrected pixel value).
[0174] The image processing method provided in the embodiment of the present application can be executed by an image processing device. In the embodiment of the present application, an image processing device executing the image processing method is taken as an example to illustrate the image processing device provided in the embodiment of the present application.
[0175] The embodiment of the present application also provides an image processing device.
[0176] like Figure 7 As shown, the image processing device includes: a first processing module 710 , a second processing module 720 and a third processing module 730 .
[0177] The first processing module 710 is used to determine a target image block centered on the target pixel point in the image to be processed;
[0178] The second processing module 720 is used to perform sharpening processing on the initial pixel value of each pixel in the target image block to obtain a sharpened pixel value of the target pixel when it is determined that the target image block has an edge;
[0179] The third processing module 730 is used to correct the sharpened pixel value of the target pixel point when it is determined that the target pixel point is over-sharpened based on the initial pixel values of the surrounding pixels of the target pixel point, and obtain and output the corrected pixel value of the target pixel point.
[0180] According to the image processing device of the present application, when it is determined that a target image block has an edge, the initial pixel value of each pixel in the target image block is sharpened to obtain the sharpened pixel value of the target pixel, and the image block without an edge is not sharpened. In addition, when it is determined that the target pixel is over-sharpened based on the initial pixel values of the surrounding pixels of the target pixel, the sharpened pixel value of the target pixel is corrected to obtain and output the corrected pixel value of the target pixel, thereby weakening the sharpening effect of the over-sharpened target pixel, reducing the sharpening of Gaussian noise and Poisson noise, reducing the number of transition pixels, and being able to reduce resource occupancy and save energy.
[0181] In some embodiments, the resolution of the target image block is n*n, where n is greater than or equal to 5; the third processing module includes:
[0182] A first processing submodule, used for determining a pixel array of a target image block according to an edge included in the target image block;
[0183] A second processing submodule, used to determine the distribution trend of the average pixel value of each pixel array;
[0184] A third processing submodule is used to determine, when it is determined that the distribution trend characterizes that the average pixel values of each pixel array are decreasing, a first difference between the average pixel values of two pixel arrays with the largest average pixel values in a decreasing direction, and / or a second difference between the average pixel values of two pixel arrays with the smallest average pixel values;
[0185] The fourth processing submodule is configured to perform at least one of the following:
[0186] In the case where it is determined that the first difference is less than the first target difference and the sharpened pixel value of the target pixel is greater than the average pixel value of the previous pixel array in the decreasing direction, it is determined that the target pixel is over-sharpened;
[0187] When it is determined that the second difference is smaller than the second target difference and the sharpened pixel value of the target pixel is smaller than the average pixel value of the subsequent pixel array in the decreasing direction, it is determined that the target pixel is over-sharpened.
[0188] In some embodiments, the third processing module further includes:
[0189] A fifth processing submodule, used for correcting the sharpened pixel value of the target pixel point according to the average pixel value of the previous pixel array;
[0190] The fifth processing submodule is further used to correct the sharpening pixel value of the target pixel point according to the average pixel value of the subsequent pixel array.
[0191] In some embodiments, the pixel array is a row or column of pixels;
[0192] The first processing submodule is specifically used to determine that each pixel array of the target image block is each pixel column when it is determined that the target image block contains a vertical edge; and to determine that each pixel array of the target image block is each pixel row when it is determined that the target image block does not contain a vertical edge but contains a horizontal edge.
[0193] In some embodiments, the second processing module is specifically configured to:
[0194] Determine a block average pixel value of a target image block, a row average pixel value of each pixel row in the target image block, and a column average pixel value of each pixel column;
[0195] Obtaining a third target difference and a fourth target difference according to the block average pixel value and the first coefficient;
[0196] Determine a first column mean difference between column mean pixel values of a first pixel column and a last pixel column, and a first row mean difference between row mean pixel values of a first pixel row and a last pixel row of the target image block;
[0197] When it is determined that the first column mean difference is greater than the third target difference, it is determined that a vertical edge exists in the target image block;
[0198] When it is determined that the first row mean value difference is greater than the fourth target difference, it is determined that a horizontal edge exists in the target image block.
[0199] In some embodiments, the second processing module is further configured to:
[0200] Filtering the initial pixel value of each pixel in the target image block to obtain a mask layer of the target image block;
[0201] Subtract the pixel value of each pixel in the mask layer from the initial pixel value of each pixel in the target image block to obtain the image detail layer of the target image block;
[0202] According to the initial pixel value of each pixel in the target image block and the pixel value of each pixel in the image detail layer, the sharpened pixel value of each pixel in the target image block is obtained.
[0203] In some embodiments, the first processing module is specifically used to perform sliding window processing on the image to be processed with the target pixel point as the center to obtain a sliding window with the target pixel point as the center;
[0204] The sliding window centered on the target pixel is used as the target image block centered on the target pixel.
[0205] In some embodiments, the apparatus further comprises:
[0206] The fourth processing module is used to output a sharpened pixel value when it is determined that the target pixel point is not over-sharpened.
[0207] The image processing device in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.
[0208] The image processing device in the embodiment of the present application may be a device having an operating system. The operating system may be a Microsoft (Windows) operating system, an Android (Android) operating system, an IOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.
[0209] The image processing device provided in the embodiment of the present application can achieve Figures 1 to 6 To avoid repetition, the various processes implemented by the method embodiment are not described here.
[0210] In some embodiments, Figure 8 As shown, an embodiment of the present application also provides an electronic device 800, including a processor 801, a memory 802, and a computer program stored in the memory 802 and executable on the processor 801. When the program is executed by the processor 801, each process of the above-mentioned image processing method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.
[0211] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.
[0212] An embodiment of the present application also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the above-mentioned image processing method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.
[0213] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.
[0214] An embodiment of the present application also provides a computer program product, including a computer program, which implements the above-mentioned image processing method when executed by a processor.
[0215] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.
[0216] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned image processing method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0217] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.
[0218] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.
[0219] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the relevant technology, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.
[0220] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.
[0221] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.
[0222] Although the embodiments of the present application have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present application, and that the scope of the present application is defined by the claims and their equivalents.
Claims
1. An image processing method, It is characterized in that include: For a target pixel point in the image to be processed, determining a target image block centered on the target pixel point; When it is determined that the target image block has an edge, sharpening is performed on the initial pixel value of each pixel in the target image block to obtain a sharpened pixel value of the target pixel; When it is determined that the target pixel is over-sharpened based on the initial pixel values of the surrounding pixels of the target pixel, the sharpened pixel value of the target pixel is corrected to obtain and output the corrected pixel value of the target pixel.
2. The image processing method according to claim 1, It is characterized in that The resolution of the target image block is n*n, where n is greater than or equal to 5; and determining that the target pixel point is over-sharpened includes: Determining a pixel array of the target image block according to an edge included in the target image block; Determining a distribution trend of average pixel values of each pixel array; In the case where it is determined that the distribution trend indicates that the average pixel values of each pixel array are decreasing, determining a first difference between the average pixel values of two pixel arrays with the largest average pixel values in the decreasing direction, and / or a second difference between the average pixel values of two pixel arrays with the smallest average pixel values; Do at least one of the following: In a case where it is determined that the first difference is less than the first target difference and the sharpened pixel value of the target pixel point is greater than the average pixel value of the previous pixel array in the decreasing direction, determining that the target pixel point is over-sharpened; When it is determined that the second difference is smaller than the second target difference and the sharpened pixel value of the target pixel point is smaller than the average pixel value of the subsequent pixel array in the decreasing direction, it is determined that the target pixel point is over-sharpened.
3. The image processing method according to claim 2, It is characterized in that In a case where the sharpened pixel value of the target pixel point is greater than the average pixel value of a previous pixel array in a decreasing direction, correcting the sharpened pixel value of the target pixel point comprises: Correcting the sharpened pixel value of the target pixel point according to the average pixel value of the previous pixel array; In a case where the sharpened pixel value of the target pixel point is less than the average pixel value of a subsequent pixel array in a decreasing direction, correcting the sharpened pixel value of the target pixel point comprises: The sharpened pixel value of the target pixel point is corrected according to the average pixel value of the latter pixel array.
4. The image processing method according to claim 2, It is characterized in that The pixel array is a pixel row or a pixel column; The step of determining the pixel array of the target image block according to the edge included in the target image block comprises: In a case where it is determined that the target image block includes a vertical edge, determining each pixel array of the target image block to be each pixel column; In the case that it is determined that the target image block does not include a vertical edge but includes a horizontal edge, each pixel array of the target image block is determined to be a pixel row.
5. The image processing method according to claim 2, It is characterized in that The determining that the target image block has an edge includes: Determine a block average pixel value of the target image block, a row average pixel value of each pixel row in the target image block, and a column average pixel value of each pixel column; Obtaining a third target difference and a fourth target difference according to the block average pixel value and the first coefficient; Determine a first column mean difference between column mean pixel values of a first pixel column and a last pixel column, and a first row mean difference between row mean pixel values of a first pixel row and a last pixel row of the target image block; In a case where it is determined that the first column mean difference is greater than the third target difference, determining that a vertical edge exists in the target image block; When it is determined that the first row mean value difference is greater than the fourth target difference, it is determined that a horizontal edge exists in the target image block.
6. The image processing method according to any one of claims 1 to 5, It is characterized in that The sharpening operation is performed on the initial pixel value of each pixel point in the target image block, comprising: Performing filtering processing on the initial pixel value of each pixel point in the target image block to obtain a mask layer of the target image block; Subtracting the pixel value of each pixel in the mask layer from the initial pixel value of each pixel in the target image block to obtain an image detail layer of the target image block; According to the initial pixel value of each pixel in the target image block and the pixel value of each pixel in the image detail layer, the sharpened pixel value of each pixel in the target image block is obtained.
7. The image processing method according to any one of claims 1 to 5, It is characterized in that The determining of a target image block centered at the target pixel point comprises: Performing sliding window processing on the image to be processed with the target pixel point as the center to obtain a sliding window with the target pixel point as the center; The sliding window centered on the target pixel is used as the target image block centered on the target pixel.
8. The image processing method according to any one of claims 1 to 5, It is characterized in that After obtaining the sharpened pixel value of the target pixel point, the method further includes: When it is determined that the target pixel point is not over-sharpened, the sharpened pixel value is output.
9. An image processing device, It is characterized in that include: A first processing module, configured to determine, for a target pixel point in an image to be processed, a target image block centered on the target pixel point; A second processing module is used for performing a sharpening process on the initial pixel value of each pixel point in the target image block to obtain a sharpened pixel value of the target pixel point when it is determined that the target image block has an edge; The third processing module is used to correct the sharpened pixel value of the target pixel point when it is determined that the target pixel point is over-sharpened based on the initial pixel values of the surrounding pixels of the target pixel point, and obtain and output the corrected pixel value of the target pixel point.
10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the program, the image processing method according to any one of claims 1 to 8 is implemented.
