Image adaptive sharpening method and system based on block blur estimation

Abstract

The embodiment of the application discloses an image self-adaptive sharpening method and system based on block blur degree estimation. The technical scheme provided by the embodiment of the application comprises the following steps: obtaining a Y channel image of an original image; determining a blur map of the Y channel image based on blur degree estimation of a pixel block, wherein the blur map is used to represent block blur degrees of each pixel block of the Y channel image; then calculating enhancement coefficients of each pixel block of the original image based on the block blur degrees of the blur map, and obtaining an enhancement coefficient matrix; and finally performing sharpening enhancement processing on a detail layer of the Y channel image based on the enhancement coefficient matrix, and obtaining a sharpened image of the original image based on the Y channel image after the sharpening enhancement processing. By using the above technical scheme, appropriate sharpening processing can be ensured for different regions of the image, the sharpening coefficient is increased for a weak texture region to enhance image details, the sharpening coefficient is reduced for a noise region to suppress noise enhancement, the image display effect is improved, and the user's picture experience is improved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of image processing, and particularly relate to an image adaptive sharpening method and system based on block blur degree estimation. BACKGROUND

[0002] At present, in order to improve image quality, for some low-definition blurred images caused by limitations of acquisition equipment or limited traffic bandwidth, the edge details and other high-frequency information of the original image are usually enhanced through image sharpening, so as to improve the definition of the image. When performing image sharpening processing, the difference between the original image and the blurred image is taken as the detail part to be enhanced, multiplied by an amplification factor, and then superimposed on the original image, so as to enhance the edges of the image.

[0003] However, simply applying the same amplification factor to the entire image for sharpening enhancement of the same intensity will cause the image background noise to be sharpened and enhanced as well, so that the main body of the image cannot be highlighted, and the user's picture experience is relatively poor. SUMMARY

[0004] Embodiments of the present application provide an image adaptive sharpening method and system based on block blur degree estimation, which can perform adaptive sharpening on different regions of an image, improve the user's picture experience, and solve the technical problem of lack of regional detail adaptive processing in image sharpening.

[0005] In a first aspect, embodiments of the present application provide an image adaptive sharpening method based on block blur degree estimation, comprising:

[0006] obtaining a Y channel image of an original image, determining a blur map of the Y channel image based on blur degree estimation of a pixel block, and the blur map is used to represent the block blur degree of each pixel block of the Y channel image;

[0007] calculating an enhancement coefficient of each pixel block of the original image based on the block blur degree of the blur map, and obtaining an enhancement coefficient matrix;

[0008] performing sharpening enhancement processing on a detail layer of the Y channel image based on the enhancement coefficient matrix, and obtaining a sharpened image of the original image based on the Y channel image after the sharpening enhancement processing.

[0009] In a second aspect, embodiments of the present application provide an image adaptive sharpening system based on block blur degree estimation, comprising:

[0010] a blur degree calculation module configured to obtain a Y channel image of an original image, determine a blur map of the Y channel image based on blur degree estimation of a pixel block, and the blur map is used to represent the block blur degree of each pixel block of the Y channel image;

[0011] The matrix conversion module is configured to calculate enhancement coefficients of each pixel block of the original image based on the block blur degree of the blur map, and obtain an enhancement coefficient matrix.

[0012] The sharpening processing module is configured to perform sharpening enhancement processing on the detail layer of the Y channel image based on the enhancement coefficient matrix, and obtain a sharpened image of the original image based on the Y channel image after the sharpening enhancement processing.

[0013] In a third aspect, an image adaptive sharpening device based on block blur degree estimation is provided, including:

[0014] a memory and one or more processors;

[0015] The memory is configured to store one or more programs.

[0016] When the one or more programs are executed by the one or more processors, the one or more processors implement the image adaptive sharpening method based on block blur degree estimation as described in the first aspect.

[0017] In a fourth aspect, a computer readable storage medium is provided, which stores computer executable instructions, and the computer executable instructions, when executed by a computer processor, are configured to perform the image adaptive sharpening method based on block blur degree estimation as described in the first aspect.

[0018] In a fifth aspect, a computer program product is provided, which contains instructions, and when the instructions are run on a computer or processor, the computer or processor performs the image adaptive sharpening method based on block blur degree estimation as described in the first aspect.

[0019] The embodiments of the present application obtain the Y channel image of the original image, determine the blur map of the Y channel image based on the blur degree estimation of the pixel block, the blur map is used to represent the block blur degree of each pixel block of the Y channel image, then calculate the enhancement coefficients of each pixel block of the original image based on the block blur degree of the blur map, and obtain the enhancement coefficient matrix, then perform sharpening enhancement processing on the detail layer of the Y channel image based on the enhancement coefficient matrix, and obtain the sharpened image of the original image based on the Y channel image after the sharpening enhancement processing. By using the above technical means, the enhancement coefficient matrix is constructed by the blur degree of different regions of the corresponding image, the sharpening intensity of different regions of the image is adaptively adjusted based on the enhancement coefficient matrix, and the sharpening processing is performed on different regions of the image. In this way, appropriate sharpening processing can be ensured for different regions of the image, the sharpening coefficient is increased for the weak texture region to enhance the image details, and the sharpening coefficient is reduced for the noise region to suppress noise enhancement. The image display effect is improved through adaptive image sharpening, and the user's picture experience is improved. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 is a flow chart of an image adaptive sharpening method based on block blur estimation provided by an embodiment of the present application;

[0021] Figure 2 is a flow chart of block blur calculation of a pixel block in an embodiment of the present application;

[0022] Figure 3 is a flow chart of enhancement coefficient matrix calculation in an embodiment of the present application;

[0023] Figure 4 is a flow chart of image sharpening in an embodiment of the present application;

[0024] Figure 5 is a structural schematic diagram of an image adaptive sharpening system based on block blur estimation provided by an embodiment of the present application;

[0025] Figure 6 is a structural schematic diagram of an image adaptive sharpening device based on block blur estimation provided by an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical scheme and advantages of the present application more clear, the specific embodiments of the present application are described in more detail below in combination with the drawings. It can be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application. In addition, it should be noted that, for the convenience of description, only the parts related to the present application are shown in the drawings, but not all the contents. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted by flow charts. Although the flow chart describes each operation (or step) as a sequential process, many of the operations can be implemented in parallel, concurrently or simultaneously. In addition, the order of the operations can be rearranged. The process can be terminated when its operations are completed, but can also have additional steps not included in the drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0027] The image adaptive sharpening method based on block blur estimation provided by the present application aims to construct an enhancement coefficient matrix by the blur of different regions of a corresponding image, to adaptively adjust the sharpening intensity of different regions of the image based on the enhancement coefficient matrix, and to perform sharpening processing on different regions of the image. In this way, the image display effect is improved, and the user's picture experience is improved.

[0028] For the existing image sharpening scheme, since the image sharpening algorithm generally performs sharpening enhancement with equal intensity for different images, it cannot guarantee that images with different clarity are appropriately enhanced. And applying the same magnification factor to the entire image to perform sharpening enhancement with equal intensity will cause the image background to be enhanced together with the image subject, the noise in the image background is enhanced, and the image subject cannot be highlighted. Based on this, the technical problem of the image adaptive sharpening method based on block blur degree estimation provided in the embodiments of the present application is to lack regional detail adaptive processing of image sharpening.

[0029] Embodiments:

[0030] Figure 1 A flowchart of the image adaptive sharpening method based on block blur degree estimation provided in the embodiments of the present application is given. The image adaptive sharpening method based on block blur degree estimation provided in the embodiments can be executed by an image adaptive sharpening device based on block blur degree estimation. The image adaptive sharpening device based on block blur degree estimation can be realized by software and / or hardware. The image adaptive sharpening device based on block blur degree estimation can be composed of two or more physical entities, or one physical entity. Generally, the image adaptive sharpening device based on block blur degree estimation can be a server host, a computer, a mobile phone, a tablet, and the like.

[0031] The following describes the image adaptive sharpening method based on block blur degree estimation by taking the image adaptive sharpening device based on block blur degree estimation as an example. Referring to Figure 1 , the image adaptive sharpening method based on block blur degree estimation specifically includes:

[0032] S110, obtaining a Y channel image of an original image, determining a blur map of the Y channel image based on blur degree estimation of a pixel block, and the blur map is used to represent the block blur degree of each pixel block of the Y channel image;

[0033] S120, calculating an enhancement coefficient of each pixel block of the original image based on the block blur degree of the blur map, and obtaining an enhancement coefficient matrix;

[0034] S130, performing sharpening enhancement processing on the detail layer of the Y channel image based on the enhancement coefficient matrix, and obtaining a sharpened image of the original image based on the Y channel image after the sharpening enhancement processing.

[0035] The embodiment of the present application performs image adaptive sharpening, detects the block blur degree of each pixel of a Y channel of an image block by block, maps the detected block blur degree to an enhancement coefficient of each pixel block, and the enhancement coefficient is used to represent the sharpening intensity of the corresponding pixel block. The enhancement coefficient matrix constructed based on the enhancement coefficient adaptively adjusts the sharpening intensity of different regions of the image, increases the sharpening coefficient of the weak texture region to enhance the image details, and reduces the sharpening coefficient of the background noise region to suppress noise enhancement. In this way, adaptive sharpening of different regions of the image is realized, and a better picture experience effect is obtained.

[0036] Specifically, when performing image sharpening processing, the image to be sharpened is defined as an original image, and the Y channel image of the original image is obtained to perform adaptive sharpening processing based on the Y channel image. The original image can be an image in YUV format, an image in RGB format, or an image in other formats. When the Y channel image of the original image is obtained, it is first determined whether the format of the current original image is YUV format. If the format of the original image is YUV format, the original image is split into Y, U, and V component images, and the Y component image is the Y channel image required for sharpening processing. If the original image is an image in RGB format or other formats, the original image needs to be converted into a YUV format image first, and then the Y channel image is obtained by splitting the component images from the YUV format image. There are many ways to convert the image format and split the YUV format image, which are not described here.

[0037] Further, based on the obtained Y channel image, the block blur degree of each pixel block is calculated to determine the enhancement coefficient for sharpening enhancement based on the block blur degree. It can be understood that the background part of an image is relatively blurred, and a small enhancement coefficient needs to be set for this part to suppress the sharpening intensity of the noise region. The main part of the image is relatively clear, and a large enhancement coefficient needs to be set for this part to enhance the image details of the main region. Therefore, the embodiment of the present application determines the positively correlated enhancement coefficient according to the block blur degree of different regions of the image, so as to determine the sharpening intensity of different regions according to the size of the enhancement coefficient.

[0038] Then, based on the determined enhancement coefficient of each pixel block, an enhancement coefficient matrix is constructed, and the enhancement coefficient matrix is applied to the detail layer of the Y channel image to perform sharpening enhancement processing on different regions of the image. Then, the sharpened Y channel image and the original UV component image are combined to obtain a sharpened YUV format image. Then, according to the format of the original image, the YUV format image is converted or kept to output. Thus, the image adaptive sharpening based on block blur degree estimation of the embodiment of the present application is completed.

[0039] Specifically, as Figure 2As shown, the embodiment of the present application includes the following steps when determining the blur map of the Y channel image based on the blur estimation of the pixel block:

[0040] S1101, determining the pixel block gradient of each fixed size pixel block in the Y channel image.

[0041] S1102, calculating the gradient mean and gradient variance of the pixel block gradient, and calculating the block blur of the corresponding pixel block based on the gradient mean and the gradient variance.

[0042] When calculating the blur of the Y channel image, the image is divided into pixel blocks (such as 8x8 pixel blocks) according to a fixed size, and the block blur is calculated for different pixel blocks, which is used for subsequent adaptive sharpening of different regions of the image corresponding to different pixel blocks. Wherein, the internal pixel gradient blk_gradient of each pixel block is calculated. The first order difference between all adjacent two pixels inside the pixel block is calculated, and the internal pixel gradient blk_gradient is obtained by superimposing. Then, the gradient mean blk_mean and the gradient variance blk_std of the internal pixel gradient blk_gradient of the pixel block are calculated. According to the calculated gradient mean blk_mean and gradient variance blk_std, the block blur of the corresponding pixel block is calculated combined with the preset weight. The block blur calculation formula is represented as:

[0043] blk blur = 0.2 * blk std + 0.8 * blk mean

[0044] Wherein, 0.2 and 0.8 are the weight coefficients corresponding to the gradient mean and the gradient variance, which are set according to the influence of the gradient mean and the gradient variance on the value of the block blur. In the actual block blur calculation process, other characteristics of the image pixels can also be combined for block blur estimation, and the specific block blur representation method is not limited in the embodiment of the present application, which will not be described here.

[0045] Based on the determined block blur of each pixel block, the blur map corresponding to the entire original image can be obtained, and the blur of each pixel block represented by the blur map can be calculated to calculate the enhancement coefficient of each pixel block for sharpening and enhancement, which is used for subsequent regional sharpening and enhancement. Moreover, before calculating the enhancement coefficient of each pixel block of the original image based on the block blur of the blur map, it also includes: smoothing filtering the blur map based on the weighted least squares filter, and outputting the smoothed filtered blur map.

[0046] The obtained blur map is filtered by weighted least square (WLS), which is a smoothing filter that can make the image as smooth as possible while keeping the image edges. The purpose of applying WLS to the blur map is to make the blur degree of the image excessively flat and the blur degree level of the same region close. A smooth and stable image sharpening effect is achieved.

[0047] Then, the enhancement coefficient is calculated using the smoothed blur map, wherein, as shown in Figure 3 the enhancement coefficient of each pixel block of the original image is calculated based on the blur degree of each block of the blur map to obtain an enhancement coefficient matrix, including:

[0048] S1201, mapping parameters of the blur degree of each block are calculated based on the set mapping width, mapping conversion speed and transverse translation coefficient of the enhancement coefficient curve;

[0049] S1202, the enhancement coefficient matrix is calculated according to the mapping parameters and the value range influence parameter of the enhancement coefficient.

[0050] The calculation formula of the enhancement coefficient matrix is as follows:

[0051] scaling factor =ρ*Gb(M)+η

[0052] where

[0053]

[0054] In the above formula, ρ and η represent the value range influence parameter of the enhancement coefficient, which is used to dynamically affect the value range of the enhancement coefficient; M represents a value of the blur map, i.e. the block blur degree of the corresponding pixel block; α represents the set mapping width, β represents the set mapping conversion speed, and μ represents the transverse translation coefficient of the enhancement coefficient curve, which is used to control the translation of the enhancement coefficient curve on the x-axis; the blur degree mapping result Gb(M) ∈ (0, 1), so the enhancement coefficient matrix scaling_factor will fall within the range of (η, η+ρ].

[0055] The mapping relationship between the blur map and the enhancement coefficient matrix is designed in this way to reduce the enhancement coefficient in the region lacking effective high-frequency information and gradually increase the enhancement coefficient in the low-texture region. Moreover, the maximum enhancement coefficient is applied in the region with rich high-frequency information of the picture, and the enhancement coefficient is gradually reduced in the region with very rich high-frequency information. In this way, the sharpening intensity of different regions of the image is adaptively adjusted according to the blur degree, so that the effective details of the overall image are greatly enhanced without over-sharpening, and the enhancement of the background noise is suppressed.

[0056] Optionally, the value range of the enhancement coefficient is set according to the resolution of the original image, and the value range of the enhancement coefficient is positively correlated with the resolution of the original image. It can be understood that the value range of the enhancement coefficient is dynamically adjusted according to the resolution, and the value range of the enhancement coefficient is adjusted and limited according to the resolution of the original image. The resolution of the image is greater, the value of the enhancement coefficient is greater, and the resolution of the image is smaller, and the enhancement coefficient is appropriately reduced to avoid excessive sharpening caused by high-frequency stacking to generate artifacts. In this way, a smooth and natural image sharpening effect can be further achieved.

[0057] Based on the determined enhancement coefficient matrix, the pixel blocks of different regions are adaptively sharpened according to the enhancement coefficients of different regions. Referring to Figure 4 , the detail layer of the Y channel image is sharpened and enhanced based on the enhancement coefficient matrix, and the sharpened image of the original image is obtained based on the Y channel image after the sharpening and enhancement processing, including:

[0058] S1301, extracting the base layer and the detail layer of the Y channel image, multiplying the enhancement coefficient matrix with the detail layer based on pixel-by-pixel multiplication, and superimposing the multiplication result with the base layer to obtain the Y channel image after sharpening and enhancement processing;

[0059] S1302, merging the Y channel image after sharpening and enhancement processing with the UV channel image of the original image to obtain the sharpened image of the original image.

[0060] Corresponding to the Y channel image, the image base layer is obtained by Gaussian filtering, and then the original Y channel image is subtracted from the image obtained after Gaussian filtering (i.e. the base layer), that is, the detail layer of the image is obtained.

[0061] Further, based on the above-mentioned detail layer, base layer and determined enhancement coefficient matrix, image sharpening processing is performed, and the sharpening processing formula is represented as:

[0062]

[0063] Wherein is the enhanced image, B is the base layer of the Y channel image, T is the detail layer of the Y channel image, and A is the determined enhancement coefficient matrix, represents pixel-by-pixel multiplication. In this way, the Y channel image after sharpening and enhancement processing is obtained by multiplying the calculated detail layer with the enhancement coefficient matrix and adding the base layer. Further, the Y channel image after sharpening and enhancement processing is combined with the UV component image separated from the original image to obtain the sharpened image of the original image.

[0064] In an embodiment, considering that most of the existing image adaptive intensity sharpening schemes are to evaluate and set the sharpening intensity on a pixel-by-pixel basis, the calculation amount is relatively large. Based on this, in the image sharpening processing of the embodiment of the present application, the blur map is determined through downsampling, and after the enhancement coefficient matrix is calculated, the enhancement coefficient matrix is upsampled according to the corresponding multiple, so as to reduce the calculation amount. Wherein, the blur map of the Y channel image is determined based on the blur estimation of the pixel block, including: downsampling the Y channel image to obtain the corresponding downsampled image, and estimating the blur of the pixel block of the downsampled image to obtain the blur map of the Y channel image; correspondingly, the enhancement coefficient of each pixel block of the original image is calculated based on the blur of each block of the blur map, to obtain the enhancement coefficient matrix, including: calculating the enhancement coefficient of each pixel block of the original image based on the blur of each block of the blur map to obtain the initial coefficient matrix, and upsampling the initial coefficient matrix to obtain the enhancement coefficient matrix.

[0065] For example, the downsampled image after 1 / 4 downsampling is obtained by 1 / 2 downsampling the Y channel image twice. Downsampling the downsampled image can eliminate part of the noise to obtain a smoothed denoised image, and reduce the amount of data that needs to be calculated for block blur in the subsequent process. It should be noted that direct 1 / 4 downsampling is easy to cause sawtooth noise, and two times of 1 / 2 downsampling can reduce the sawtooth noise. Then, the downsampled image is traversed block by block with a fixed size (such as 8x8) of pixel block size, and the block blur of each pixel block is calculated according to the above block blur calculation method, and the block blur of all blocks is composed into a blur map. Then, the initial enhancement coefficient matrix is calculated using the blur map. Since the blur is obtained from the downsampled image, the initial enhancement coefficient matrix calculated based on the blur needs to be upsampled by a factor corresponding to the downsampling to restore the initial enhancement coefficient matrix to the original image size, so as to obtain the enhancement coefficient matrix corresponding to the original image, and then the image sharpening enhancement processing is performed through the enhancement coefficient matrix.

[0066] In addition, in an embodiment, before obtaining the Y channel image of the original image, it further includes: identifying the scene type of the original image based on a pre-constructed classification network model; correspondingly, in the case that the scene type is a specified type, performing sharpening enhancement processing on the detail layer of the Y channel image based on a set enhancement factor, and obtaining a sharpened image of the original image based on the Y channel image after the sharpening enhancement processing.

[0067] Considering that the existing adaptive intensity sharpening algorithm does not consider the needs of different scenes for the sharpening algorithm, such as game live pictures, which often need to be globally uniformly sharpened due to rich picture information, adaptive sharpening for such scenes is a waste of computing power. Therefore, the embodiment of the present application corresponds to an original image to be sharpened, and inputs the original image into a pre-trained scene classification network model to output a scene classification value through a deep network. Before that, training samples are constructed according to images of different scene types, and the scene classification values of the training samples are labeled for training, so that the classification network model has the ability to identify the scene type of the image. Then, according to the scene classification value of the current original image, the scene type of the image is determined, and if it is a game live / large amount of text scene, adaptive sharpening is not needed. At this time, the enhancement coefficient matrix of the whole image is set to a fixed value t, and the sharpening processing formula is represented as: Referring to the above-mentioned image sharpening enhancement processing method based on the enhancement coefficient matrix, a fixed value t is used as the enhancement coefficient of all pixel blocks, and the sharpened image obtained by the enhancement processing is obtained.

[0068] Exemplarily, in a video live scene, the behavior of the host starting to broadcast to the user watching roughly goes through the following link: video image acquisition-end encoding-video processing-transcoding and issuing-user decoding. Through the image adaptive sharpening enhancement scheme of the embodiment of the present application, in the video processing link after the end encoding, the image is adaptively sharpened to improve the definition of the image, so as to improve the image picture experience of the user end.

[0069] The above-mentioned method comprises the following steps: obtaining a Y channel image of an original image; determining a blur map of the Y channel image based on a blur degree estimation of a pixel block, the blur map being used to represent a block blur degree of each pixel block of the Y channel image; calculating an enhancement coefficient of each pixel block of the original image based on each block blur degree of the blur map, to obtain an enhancement coefficient matrix; and performing sharpening enhancement processing on a detail layer of the Y channel image based on the enhancement coefficient matrix, to obtain a sharpened image of the original image based on the Y channel image after the sharpening enhancement processing. By using the above-mentioned technical means, the enhancement coefficient matrix is constructed according to the blur degrees of different regions of the image, so as to adaptively adjust the sharpening intensity of different regions of the image based on the enhancement coefficient matrix, and to perform sharpening processing on different regions of the image. In this way, appropriate sharpening processing can be ensured for different regions of the image, the sharpening coefficient of the weak texture region is increased to enhance the image details, and the sharpening coefficient of the noise region is reduced to suppress noise enhancement. Through adaptive image sharpening, the image display effect is improved, and the user picture experience is further improved.

[0070] On the basis of the above-mentioned embodiment, Figure 5 A structure diagram of an image adaptive sharpening system based on block blur degree estimation is provided in the present application. Referring to Figure 5The image self-adaptive sharpening system based on block blur degree estimation provided by the embodiment specifically comprises a blur degree calculation module 21, a matrix conversion module 22 and a sharpening processing module 23.

[0071] The blur degree calculation module 21 is configured to acquire a Y channel image of an original image, determine a blur map of the Y channel image based on a blur degree estimation of a pixel block, and the blur map is used to represent the block blur degree of each pixel block of the Y channel image.

[0072] The matrix conversion module 22 is configured to calculate an enhancement coefficient of each pixel block of the original image based on each block blur degree of the blur map, and obtain an enhancement coefficient matrix.

[0073] The sharpening processing module 23 is configured to perform sharpening enhancement processing on a detail layer of the Y channel image based on the enhancement coefficient matrix, and obtain a sharpened image of the original image based on the Y channel image after the sharpening enhancement processing.

[0074] Specifically, the determination of the blur map of the Y channel image based on the blur degree estimation of the pixel block comprises:

[0075] determining a pixel block gradient of each pixel block of a fixed size in the Y channel image;

[0076] calculating a gradient mean and a gradient variance of the pixel block gradient, and calculating the block blur degree of the corresponding pixel block based on the gradient mean and the gradient variance.

[0077] Specifically, the calculation of the enhancement coefficient of each pixel block of the original image based on each block blur degree of the blur map to obtain the enhancement coefficient matrix comprises:

[0078] calculating a mapping parameter of each block blur degree based on a set mapping width, a mapping conversion speed and a lateral translation coefficient of the enhancement coefficient curve;

[0079] calculating the enhancement coefficient matrix according to each mapping parameter and a value range influence parameter of the enhancement coefficient.

[0080] The value range influence parameter of the enhancement coefficient is set according to the resolution of the original image, and the value range of the enhancement coefficient is positively correlated with the resolution of the original image.

[0081] And before the calculation of the enhancement coefficient of each pixel block of the original image based on each block blur degree of the blur map, the method further comprises:

[0082] performing a smoothing filtering processing on the blur map based on a weighted least square filter, and outputting the blur map after the smoothing filtering processing.

[0083] Specifically, the determination of the blur map of the Y channel image based on the blur degree estimation of the pixel block comprises:

[0084] down-sampling the Y channel image to obtain a corresponding down-sampled image, and performing blur estimation on the down-sampled image in pixel blocks to obtain a blur map of the Y channel image;

[0085] calculating enhancement coefficients of each pixel block of the original image based on the blur degree of each block of the blur map to obtain an enhancement coefficient matrix, including:

[0086] calculating enhancement coefficients of each pixel block of the original image based on the blur degree of each block of the blur map to obtain an initial coefficient matrix, and up-sampling the initial coefficient matrix to obtain the enhancement coefficient matrix.

[0087] Specifically, performing sharpening enhancement processing on the detail layer of the Y channel image based on the enhancement coefficient matrix, and obtaining a sharpened image of the original image based on the Y channel image after the sharpening enhancement processing, including:

[0088] extracting the base layer and the detail layer of the Y channel image, multiplying the enhancement coefficient matrix and the detail layer based on pixel-by-pixel multiplication, and superimposing the multiplication result and the base layer to obtain the Y channel image after the sharpening enhancement processing;

[0089] merging the Y channel image after the sharpening enhancement processing and the UV channel image of the original image to obtain the sharpened image of the original image.

[0090] Before obtaining the Y channel image of the original image, further including:

[0091] recognizing the scene type of the original image based on a pre-constructed classification network model;

[0092] Correspondingly, in the case that the scene type is a specified type, performing sharpening enhancement processing on the detail layer of the Y channel image based on a set enhancement factor, and obtaining a sharpened image of the original image based on the Y channel image after the sharpening enhancement processing.

[0093] The above, by obtaining the Y channel image of the original image, determining the blur map of the Y channel image based on the blur estimation of the pixel blocks, the blur map is used to represent the block blur degree of each pixel block of the Y channel image; then, calculating the enhancement coefficients of each pixel block of the original image based on the blur degree of each block of the blur map to obtain the enhancement coefficient matrix; and then, performing sharpening enhancement processing on the detail layer of the Y channel image based on the enhancement coefficient matrix, and obtaining a sharpened image of the original image based on the Y channel image after the sharpening enhancement processing. By using the above technical means, the enhancement coefficient matrix is constructed by the blur degree of different regions of the corresponding image, so as to adaptively adjust the sharpening intensity of different regions of the image based on the enhancement coefficient matrix, and perform sharpening processing on different regions of the image. In this way, appropriate sharpening processing can be ensured for different regions of the image, the sharpening coefficient is increased for the weak texture region to enhance the image details, and the sharpening coefficient is reduced for the noise region to suppress noise enhancement. Through adaptive image sharpening, the image display effect is improved, and the user's picture experience is further improved.

[0094] The image adaptive sharpening system based on block blur degree estimation provided by the embodiments of the present application can be configured to perform the image adaptive sharpening method based on block blur degree estimation provided by the above embodiments, and has the corresponding functions and beneficial effects.

[0095] Based on the above actual examples, the embodiments of the present application further provide an image adaptive sharpening device based on block blur degree estimation, which refers to Figure 6 The image adaptive sharpening device based on block blur degree estimation includes a processor 31, a memory 32, a communication module 33, an input device 34, and an output device 35. The memory 32 is a computer readable storage medium, which can be configured to store software programs, computer executable programs, and modules, such as program instructions / modules corresponding to the image adaptive sharpening method based on block blur degree estimation (for example, blur degree calculation module, matrix conversion module, and sharpening processing module in the image adaptive sharpening system based on block blur degree estimation) described in any embodiment of the present application. The communication module 33 is configured to perform data transmission. The processor 31 performs various functional applications and data processing of the device by running the software programs, instructions, and modules stored in the memory, that is, implements the above-mentioned image adaptive sharpening method based on block blur degree estimation. The input device 34 can be configured to receive input digital or character information, and generate key signal input related to user settings and function control of the device. The output device 35 can include a display device such as a display screen. The image adaptive sharpening device based on block blur degree estimation provided above can be configured to perform the image adaptive sharpening method based on block blur degree estimation provided by the above embodiments, and has the corresponding functions and beneficial effects.

[0096] Based on the above embodiments, the embodiments of the present application further provide a computer readable storage medium, which stores computer executable instructions. The computer executable instructions are configured to perform an image adaptive sharpening method based on block blur degree estimation when executed by a computer processor. The storage medium can be any various types of memory device or storage device. Of course, the computer executable instructions of the computer readable storage medium provided by the embodiments of the present application are not limited to the image adaptive sharpening method based on block blur degree estimation as described above, but can also perform related operations in the image adaptive sharpening method based on block blur degree estimation provided by any embodiment of the present application.

[0097] On the basis of the above-mentioned embodiments, the embodiments of the present application further provide a computer program product, the technical solution of the present application or the whole or part of the contribution to the prior art can be embodied in the form of a software product, the computer program product is stored in a storage medium, and includes a plurality of instructions to make a computer device, a mobile terminal or a processor therein execute all or part of the steps of the image adaptive sharpening method based on block blur estimation described in various embodiments of the present application.

Claims

1. A method of image adaptive sharpening based on block blur estimation, characterized in that, The method comprises the following steps: obtaining a Y channel image of an original image, determining a blur map of the Y channel image based on blur degree estimation of a pixel block, the blur map being used to represent block blur degrees of each pixel block of the Y channel image; calculating enhancement coefficients of each pixel block of the original image based on each block blur degree of the blur map, to obtain an enhancement coefficient matrix; wherein, the block blur degree is calculated in a weighted manner according to a gradient mean and a gradient variance of a pixel block gradient of the pixel block; a smaller enhancement coefficient is set for a relatively blurred part of the image; a larger enhancement coefficient is set for a relatively clear part of the image; mapping parameters of each block blur degree are calculated based on a set mapping width, a mapping conversion speed and a lateral translation coefficient of an enhancement coefficient curve, the lateral translation coefficient being used to control the translation of the enhancement coefficient curve on the x-axis; the enhancement coefficient matrix is calculated according to each mapping parameter and a value range influence parameter of the enhancement coefficient. performing sharpening enhancement processing on a detail layer of the Y channel image based on the enhancement coefficient matrix, and obtaining a sharpened image of the original image based on the Y channel image after the sharpening enhancement processing.

2. The method for image self-adaptive sharpening based on block blur degree estimation according to claim 1, characterized in that, Before calculating the enhancement coefficients of each pixel block of the original image based on each block blur degree of the blur map, the method further comprises the following steps: performing smoothing filtering processing on the blur map based on a weighted least square filter, and outputting the blur map after the smoothing filtering processing.

3. The method for image self-adaptive sharpening based on block blur estimation according to claim 1, characterized in that, The value range influence parameter of the enhancement coefficient is set according to the resolution of the original image, and the value range of the enhancement coefficient is positively correlated with the resolution of the original image.

4. The method for image self-adaptive sharpening based on block blur degree estimation according to claim 1, characterized in that, The method of determining the blur map of the Y channel image based on the blur degree estimation of the pixel block comprises the following steps: performing down-sampling on the Y channel image to obtain a corresponding down-sampled image, and performing blur degree estimation of a pixel block on the down-sampled image to obtain the blur map of the Y channel image. The method of calculating the enhancement coefficients of each pixel block of the original image based on each block blur degree of the blur map to obtain an enhancement coefficient matrix comprises the following steps: calculating the enhancement coefficients of each pixel block of the original image based on each block blur degree of the blur map to obtain an initial coefficient matrix, and performing up-sampling on the initial coefficient matrix to obtain the enhancement coefficient matrix.

5. The method for image self-adaptive sharpening based on block blur estimation according to claim 1, characterized in that, The method of performing sharpening enhancement processing on a detail layer of the Y channel image based on the enhancement coefficient matrix, and obtaining a sharpened image of the original image based on the Y channel image after the sharpening enhancement processing comprises the following steps: extracting a base layer and a detail layer of the Y channel image, multiplying the enhancement coefficient matrix and the detail layer based on pixel-by-pixel multiplication, and superimposing the multiplication result and the base layer to obtain the Y channel image after the sharpening enhancement processing; merging the Y channel image after the sharpening enhancement processing and UV channel images of the original image to obtain the sharpened image of the original image.

6. The method for image self-adaptive sharpening based on block blur estimation according to claim 1, characterized in that, Before obtaining the Y channel image of the original image, the method further comprises the following steps: recognizing a scene type of the original image based on a pre-constructed classification network model. Correspondingly, in the case that the scene type is a specified type, a detail layer of the Y channel image is sharpened and enhanced based on a set enhancement factor, and a sharpened image of the original image is obtained based on the Y channel image after the sharpening and enhancement.

7. An image self-adaptive sharpening system based on block blur degree estimation, characterized in that, The method comprises: a blur calculation module configured to obtain a Y channel image of an original image, and determine a blur map of the Y channel image based on block blur estimation of pixel blocks, the blur map being used to represent block blur degrees of each pixel block of the Y channel image; a matrix conversion module configured to calculate enhancement coefficients of each pixel block of the original image based on each block blur degree of the blur map, and obtain an enhancement coefficient matrix; wherein the matrix conversion module is specifically configured to calculate the block blur degree in a weighted manner according to a gradient mean and a gradient variance of a pixel block gradient of the pixel block; a smaller enhancement coefficient is set for a relatively blurred part of the image; a larger enhancement coefficient is set for a relatively clear part of the image; mapping parameters of each block blur degree are calculated based on a set mapping width, a mapping conversion speed, and a horizontal translation coefficient of an enhancement coefficient curve, the horizontal translation coefficient being used to control translation of the enhancement coefficient curve on an x-axis; and the enhancement coefficient matrix is calculated according to each mapping parameter and a value range influence parameter of the enhancement coefficient; a sharpening processing module configured to perform sharpening and enhancement processing on a detail layer of the Y channel image based on the enhancement coefficient matrix, and obtain a sharpened image of the original image based on the Y channel image after the sharpening and enhancement processing.

8. An image self-adaptive sharpening device based on block blur degree estimation, characterized in that, The method comprises: a memory and one or more processors; the memory is configured to store one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the image adaptive sharpening method based on block blur estimation according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer executable instructions, which, when executed by a computer processor, are configured to perform the image adaptive sharpening method based on block blur estimation according to any one of claims 1-6.

10. A computer program product, characterised in that, The computer program product contains instructions, which, when executed on a computer or processor, cause the computer or processor to perform the image adaptive sharpening method based on block blur estimation according to any one of claims 1-6.

Images