Image processing method and device, electronic equipment and storage medium

Abstract

The application discloses an image processing method and device, electronic equipment and storage medium, and belongs to the technical field of image processing. The method comprises the following steps: performing layered processing on a to-be-processed image to obtain a base layer image and a detail layer image corresponding to the to-be-processed image; performing contrast enhancement processing on the base layer image to obtain a base layer enhanced image; determining a target scene parameter corresponding to the to-be-processed image based on feature information of the base layer enhanced image; performing enhancement processing on the detail layer image based on the target scene parameter to obtain a detail layer enhanced image; and obtaining a target image based on the base layer enhanced image, the detail layer enhanced image and the target scene parameter. The method can adapt to an image scene, perform enhancement processing on the image, effectively improve the image quality and improve the visual perception of the human eye.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, and in particular relates to an image processing method, apparatus, electronic device and storage medium. Background Technology

[0002] Images acquired by image detectors often lack clear detail due to various factors such as inclement weather and dynamic range compression, affecting human observation and judgment. Current image processing methods primarily employ layered filtering to process low-frequency and high-frequency images separately, aiming to improve image contrast and enhance details.

[0003] However, the goal of these processing methods is to maximize image contrast and highlight details. This can lead to over-enhancement in some scenes with a small dynamic range, such as a clear sky or a calm lake, which does not conform to human visual observation. At the same time, it can also amplify image noise and affect the image quality. Summary of the Invention

[0004] This application aims to address at least one of the technical problems existing in the prior art. To this end, this application proposes an image processing method, apparatus, electronic device, and storage medium that can adaptively enhance images based on image scenes, effectively improving image quality and enhancing human visual perception.

[0005] In a first aspect, this application provides an image processing method, the method comprising:

[0006] The image to be processed is subjected to layer processing to obtain the base layer image and detail layer image corresponding to the image to be processed;

[0007] The base layer image is subjected to contrast enhancement processing to obtain the base layer enhanced image;

[0008] Based on the feature information of the base layer enhanced image, the target scene parameters corresponding to the image to be processed are determined;

[0009] Based on the target scene parameters, the detail layer image is enhanced to obtain a detail layer enhanced image;

[0010] The target image is obtained based on the base layer enhanced image, the detail layer enhanced image, and the target scene parameters.

[0011] According to the image processing method of this application, by acquiring the feature information of the base layer enhanced image, the image to be processed is divided into scenes, the target scene parameters corresponding to the image to be processed are determined, and the detail layer image is enhanced based on the target scene parameters to obtain the detail layer enhanced image. This enhanced image is then fused with the base layer enhanced image to obtain the target image. Based on the feature information of the base layer enhanced image, the image is enhanced adaptively to the image scene.

[0012] It can avoid excessive contrast enhancement during image enhancement, effectively improve image quality, and enhance the visual experience of the human eye.

[0013] According to one embodiment of this application, the target scene parameters include detail enhancement gain coefficient and fused image dynamic range.

[0014] According to one embodiment of this application, the step of enhancing the detail layer image based on the target scene parameters to obtain a detail layer enhanced image includes:

[0015] Based on the detail enhancement gain coefficient, the detail layer image is enhanced to obtain the detail layer enhanced image.

[0016] According to one embodiment of this application, the step of enhancing the detail layer image based on the detail enhancement gain coefficient to obtain the detail layer enhanced image includes:

[0017] The edge coefficients of the detail layer image are extracted to obtain the edge coefficient matrix;

[0018] Based on the detail enhancement gain coefficient and the edge coefficient matrix, a point-by-point sharpening coefficient matrix is ​​obtained;

[0019] The enhanced detail layer image is obtained based on the point-by-point sharpening coefficient matrix and the detail layer image.

[0020] According to one embodiment of this application, the detail enhancement gain coefficient includes a strong detail enhancement gain coefficient and a weak detail enhancement gain coefficient, and the step of obtaining a point-by-point sharpening coefficient matrix based on the detail enhancement gain coefficient and the edge coefficient matrix includes:

[0021] Application formula

[0022] G = G1 * E + G2

[0023] Wherein, G is the point-by-point sharpening coefficient matrix, G1 is the strong detail enhancement gain coefficient, G2 is the weak detail enhancement gain coefficient, and E is the edge coefficient matrix.

[0024] According to one embodiment of this application, obtaining the target image based on the base layer enhanced image, the detail layer enhanced image, and the target scene parameters includes:

[0025] The base layer enhanced image and the detail layer enhanced image are fused together, and the dynamic range of the image is adjusted to the dynamic range of the fused image to obtain the target image.

[0026] According to one embodiment of this application, the feature information of the base layer enhancement image includes at least one of the image dynamic range, local variance, standard deviation, and gradient of the base layer enhancement image.

[0027] According to one embodiment of this application, the step of performing contrast enhancement processing on the base layer image to obtain a base layer enhanced image includes:

[0028] The base layer image is divided into multiple image sub-blocks;

[0029] Each of the image sub-blocks is subjected to grayscale mapping to obtain multiple image enhancement sub-blocks;

[0030] The base layer enhanced image is obtained by stitching together the multiple image enhancement sub-blocks.

[0031] Secondly, this application provides an image processing apparatus, the apparatus comprising:

[0032] The first processing module is used to perform layered processing on the image to be processed, to obtain the base layer image and detail layer image corresponding to the image to be processed.

[0033] The second processing module is used to perform contrast enhancement processing on the base layer image to obtain an enhanced base layer image;

[0034] The third processing module is used to determine the target scene parameters corresponding to the image to be processed based on the feature information of the base layer enhanced image;

[0035] The fourth processing module is used to perform enhancement processing on the detail layer image based on the target scene parameters to obtain a detail layer enhanced image;

[0036] The fifth processing module is used to obtain the target image based on the base layer enhanced image, the detail layer enhanced image, and the target scene parameters.

[0037] According to the image processing apparatus of this application, by acquiring the feature information of the base layer enhanced image, the image to be processed is divided into scenes, the target scene parameters corresponding to the image to be processed are determined, and the detail layer image is enhanced based on the target scene parameters to obtain the detail layer enhanced image. The detail layer enhanced image is then fused with the base layer enhanced image to obtain the target image. Based on the feature information of the base layer enhanced image, the image is enhanced by adapting to the image scene. This can avoid excessive contrast enhancement during the image enhancement process, effectively improve image quality, and enhance the visual experience of the human eye.

[0038] Thirdly, this application provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the image processing method as described in the first aspect above.

[0039] Fourthly, this application provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image processing method as described in the first aspect above.

[0040] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the image processing method as described in the first aspect above.

[0041] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0042] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0043] Figure 1 This is one of the schematic flowcharts of the image processing method provided in the embodiments of this application;

[0044] Figure 2 This is a second schematic flowchart of the image processing method provided in the embodiments of this application;

[0045] Figure 3 This is a schematic diagram of grayscale interpolation provided in an embodiment of this application;

[0046] Figure 4 This is a schematic diagram of the outdoor street view image to be processed provided in an embodiment of this application;

[0047] Figure 5 This is a schematic diagram of an enhanced outdoor street view image provided in an embodiment of this application;

[0048] Figure 6 This is a schematic diagram of the seascape image to be processed provided in an embodiment of this application;

[0049] Figure 7 This is a schematic diagram of the enhanced seascape image provided in the embodiments of this application;

[0050] Figure 8 This is a schematic diagram of the structure of the image processing apparatus provided in the embodiments of this application;

[0051] Figure 9This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0052] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0053] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0054] The image processing method, image processing apparatus, electronic device, and readable storage medium provided in this application will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios.

[0055] The image processing method can be applied to the terminal, and can be executed by the hardware or software in the terminal.

[0056] The terminal includes, but is not limited to, portable communication devices such as mobile phones or tablets with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads). It should also be understood that, in some embodiments, the terminal may not be a portable communication device, but rather a desktop computer with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads).

[0057] The following embodiments describe a terminal including a display and a touch-sensitive surface. However, it should be understood that the terminal may include one or more other physical user interface devices such as a physical keyboard, mouse, and joystick.

[0058] The image processing method provided in this application embodiment can be executed by an electronic device or a functional module or entity in an electronic device that can implement the image processing method. The electronic devices mentioned in this application embodiment include, but are not limited to, mobile phones, tablets, computers, cameras, and wearable devices. The image processing method provided in this application embodiment will be described below using an electronic device as the execution subject as an example.

[0059] Image processing methods are used to process images, improve image contrast, enhance image details, and obtain clear images that conform to human visual perception.

[0060] like Figure 1 As shown, the image processing method includes steps 110 to 150.

[0061] Step 110: Perform layer processing on the image to be processed to obtain the base layer image and detail layer image corresponding to the image to be processed.

[0062] The base layer image is a low-frequency image obtained after layering the image to be processed, and the detail layer image is a high-frequency image obtained after layering the image to be processed.

[0063] In practice, spatial filtering can be used to process the image to be processed in layers to obtain the base layer image corresponding to the image to be processed, i.e., the low-frequency image.

[0064] Spatial filtering can include, but is not limited to, combinations of one or more filtering methods such as guided filtering, bilateral filtering, Gaussian filtering, and mean filtering.

[0065] By subtracting the base layer image from the image to be processed, the detail layer image, i.e., the high-frequency image, corresponding to the image to be processed is obtained.

[0066] In this embodiment, by inputting an image to be processed, the image is processed in layers to obtain the base layer image and detail layer image required for image processing.

[0067] It should be noted that when the original image is a black and white image acquired by a visible light or infrared detector, the acquired original image can be directly used as the image to be processed for layering.

[0068] When the original image is a visible light color image, color conversion processing can be performed on the original image, and the processed image can be used as the image to be processed for layered processing.

[0069] For example, processing visible light color images can be done by converting them from the RGB domain to the YUV domain. The formula for obtaining a YUV domain image through color conversion is as follows:

[0070] Y(i,j)=0.299R(i,j)+0.587G(i,j)+0.114B(i,j)

[0071] V(i,j)=0.713(R(i,j)-Y(i,j))+128

[0072] U(i,j)=0.564(B(i,j)-Y(i,j))+128

[0073] Where R, G, B are the RGB component values ​​of the original image, Y, U, V are the converted YUV component values, (i,j) are the row and column values ​​of the image, and the converted Y component is used as the image to be processed for layered processing.

[0074] Step 120: Perform contrast enhancement processing on the base layer image to obtain the base layer enhanced image.

[0075] In this step, contrast enhancement processing is performed on the base layer image to enhance the details of the base layer image and improve the visual experience for the human eye.

[0076] In practice, the contrast of the base layer image can be enhanced by performing grayscale remapping on the base layer image, resulting in an enhanced base layer image.

[0077] The mapping methods include, but are not limited to, histogram equalization (HE), plateau histogram equalization (PHE), piecewise linear mapping, gamma mapping, and sigmoid mapping.

[0078] Step 130: Determine the target scene parameters corresponding to the image to be processed based on the feature information of the base layer enhanced image.

[0079] It is understandable that performing contrast enhancement on the base layer image results in enhanced details in the enhanced base layer image, which can be used to determine the target scene corresponding to the image to be processed.

[0080] In this embodiment, based on the feature information of the base layer enhanced image, the target scene corresponding to the image to be processed can be divided, the type of the target scene can be determined, and then the target scene parameters corresponding to different target scenes can be obtained. The target scene parameters represent the image processing parameters used to process images of different target scenes.

[0081] For example, when the target scene is a clear sky, scene judgment is performed based on the feature information of the base enhanced image to determine that the target scene is of type 1, and the target scene parameter 1 corresponding to the target scene is obtained.

[0082] For example, when the target scene is a calm lake, scene judgment is performed based on the feature information of the base layer enhanced image to determine that the target scene is type 2, and the target scene parameter 2 corresponding to the target scene is obtained.

[0083] Step 140: Based on the target scene parameters, perform enhancement processing on the detail layer image to obtain the enhanced detail layer image.

[0084] In practice, the enhanced detail layer image can be calculated by extracting the sharpness information of the detail layer image and based on the target scene parameters.

[0085] In this embodiment, the detail layer image is enhanced based on the target scene parameters. According to the target scene corresponding to the image to be processed, the high-frequency detail layer image is enhanced in a targeted manner, which can avoid the phenomenon of excessive contrast enhancement during the detail layer image enhancement process and improve the adaptive capability of image enhancement.

[0086] For example, when the target scene is a clear sky, the target scene parameter 1 corresponding to the clear sky image is obtained. Based on the target scene parameter 1, the detail layer image is enhanced to obtain the enhanced detail layer image. By enhancing the detail layer image using the target scene parameter 1, over-enhancement of the clear sky image can be avoided when enhancing the details of images with a small dynamic range, such as clear sky images.

[0087] Step 150: Obtain the target image based on the base layer enhanced image, the detail layer enhanced image, and the target scene parameters.

[0088] The target image is the processed image output after image enhancement processing of the image to be processed. Compared with the image to be processed, the target image has improved image contrast and more prominent details.

[0089] In this step, the base layer enhancement image and the detail layer enhancement image are fused according to the target scene parameters to obtain the final target image.

[0090] In this embodiment, when fusing the base layer enhanced image and the detail layer enhanced image, the image fusion process is adjusted based on the target scene parameters so that the fused target image matches the target scene corresponding to the image to be processed, effectively improving image quality and enhancing human visual perception.

[0091] It should be noted that when the original image is a black and white image acquired by a visible light or infrared detector, the target image is directly output based on the base layer enhancement image, the detail layer enhancement image, and the target scene parameters.

[0092] When the original image is a visible light color image, after fusing the base layer enhancement image and the detail layer enhancement image according to the target scene parameters, color conversion processing is then performed to obtain the final target image.

[0093] In this embodiment, by converting the image from the YUV domain to the RGB domain, the enhancement processing result of the Y component is... out The process involves color conversion formulas to obtain the three components R of the output color image. out G out B outThe calculation formula is as follows:

[0094] R out (i,j)=Y out (i,j)+1.403(V(i,j)-128)

[0095] G out (i,j)=Y out (i,j)-0.344(V(i,j)-128)-0.714(U(i,j)-128)

[0096] B out (i,j)=Y out (i,j)+1.773(U(i,j)-128)

[0097] Where U and V are the UV component values ​​after conversion and calculation of the visible light color image.

[0098] In related technologies, guided filtering layers are performed using adaptive filtering parameters to distinguish noise and edge details, which can reduce over-enhancement of high-frequency details but cannot solve the problem of over-enhancement of low-frequency contrast. In this embodiment, the target scene corresponding to the image to be processed is determined by the feature information of the base layer enhanced image, and the target scene parameters are obtained. The target scene parameters are used for the enhancement of the detail layer image and the dynamic range adjustment of the final fused enhanced image. By adapting to the image scene, the image is enhanced, effectively improving image details while avoiding over-enhancement, resulting in a clear target image that conforms to human visual observation.

[0099] According to the image processing method provided in the embodiments of this application, by obtaining the feature information of the base layer enhanced image, the image to be processed is divided into scenes, the target scene parameters corresponding to the image to be processed are determined, and the detail layer image is enhanced based on the target scene parameters to obtain the detail layer enhanced image. The image is then fused with the base layer enhanced image to obtain the target image. Based on the feature information of the base layer enhanced image, the image is enhanced in an adaptive manner according to the image scene. This can avoid the phenomenon of excessive contrast enhancement during the image enhancement process, effectively improve image quality, and improve the visual experience of the human eye.

[0100] In some embodiments, the target scene parameters include detail enhancement gain coefficient and fused image dynamic range.

[0101] Among them, the detail enhancement gain coefficient refers to the gain coefficient used to enhance details. The detail enhancement gain coefficient can be used to enhance the details of the detail layer image.

[0102] Dynamic range describes the range of light intensity distribution in an image, from the darkest shadows to the brightest highlights. This target scene parameter, dynamic range, can be used to adjust the dynamic range during the fusion process of base layer enhancement images and detail layer enhancement images.

[0103] In this embodiment, by obtaining the enhancement gain coefficient and the dynamic range of the fused image from the target scene parameters, the detail layer image can be enhanced, which can solve the problem of over-enhancement of contrast in the existing image enhancement process and effectively improve image quality.

[0104] In some embodiments, the detail layer image is enhanced based on target scene parameters to obtain a detail layer enhanced image, including:

[0105] Based on the detail enhancement gain coefficient, the detail layer image is enhanced to obtain the detail layer enhanced image.

[0106] In this embodiment, by using the obtained detail enhancement gain coefficient to enhance the detail layer image, the over-enhancement of contrast during the image enhancement process can be effectively avoided, making the output image more suitable for human visual observation.

[0107] In some embodiments, the detail layer image is enhanced based on the detail enhancement gain coefficient to obtain a detail layer enhanced image, including:

[0108] The edge coefficients of the detail layer image are extracted to obtain the edge coefficient matrix;

[0109] Based on the detail enhancement gain coefficient and edge coefficient matrix, the point-by-point sharpening coefficient matrix is ​​obtained;

[0110] Based on the point-by-point sharpening coefficient matrix and the detail layer image, the detail layer enhanced image is obtained.

[0111] The edge coefficient is used to describe the sharpness information of the detail layer image.

[0112] The point-by-point sharpening factor is used to represent the degree of contrast enhancement processing performed point-by-point on the detail layer image.

[0113] It should be noted that when enhancing a detail layer image, limiting the image size helps maintain its stability during the enhancement process.

[0114] Based on the point-by-point sharpening coefficient matrix and the detail layer image after amplitude limiting, an enhanced detail layer image is obtained.

[0115] In this embodiment, the edge coefficient matrix is ​​obtained, and the point-by-point sharpening coefficient matrix is ​​calculated based on the detail enhancement gain coefficient and the edge coefficient matrix. Contrast enhancement processing is performed on the detail layer image. By enhancing the detail layer image through the detail enhancement gain coefficient, the phenomenon of over-enhancement of contrast during image enhancement can be effectively avoided, adapting to the image scene and effectively improving image quality.

[0116] In some embodiments, the detail enhancement gain coefficients include strong detail enhancement gain coefficients and weak detail enhancement gain coefficients. Based on the detail enhancement gain coefficients and the edge coefficient matrix, a point-by-point sharpening coefficient matrix is ​​obtained, including:

[0117] Application formula

[0118] G = G1 * E + G2

[0119] Where G is the point-by-point sharpening coefficient matrix, G1 is the strong detail enhancement gain coefficient, G2 is the weak detail enhancement gain coefficient, and E is the edge coefficient matrix.

[0120] In this embodiment, the point-by-point sharpening coefficient matrix G is calculated by obtaining the edge coefficient matrix E, the detail enhancement gain coefficient G1, and the weak detail enhancement gain coefficient G2, and the detail layer image is enhanced by the point-by-point sharpening coefficient matrix.

[0121] In some embodiments, a target image is obtained based on a base layer enhanced image, a detail layer enhanced image, and target scene parameters, including:

[0122] The base layer enhancement image and the detail layer enhancement image are fused together, and the dynamic range of the image is adjusted to the dynamic range of the fused image to obtain the target image.

[0123] In practice, a fused enhanced image is obtained by fusing the base layer enhancement image and the detail layer enhancement image.

[0124] The dynamic range of the fused enhanced image is adjusted based on the dynamic range of the target scene parameters to obtain the target image.

[0125] In this embodiment, by fusing and enhancing images to adjust the dynamic range, more image information can be clearly displayed while maintaining a smaller dynamic range, so that the image quality of the target image conforms to human visual observation.

[0126] In some embodiments, the feature information of the base layer enhancement image includes at least one of the image dynamic range, local variance, standard deviation, and gradient of the base layer enhancement image.

[0127] In this embodiment, by acquiring the feature information of the base layer enhancement image, including at least one of the image dynamic range, local variance, standard deviation and gradient, the target scene parameters corresponding to the image to be processed can be determined.

[0128] In some embodiments, the base layer image is subjected to contrast enhancement processing to obtain a base layer enhanced image, including:

[0129] The base layer image is divided into multiple image sub-blocks;

[0130] Each image sub-block is mapped to grayscale to obtain multiple image enhancement sub-blocks;

[0131] Multiple image enhancement sub-blocks are stitched together to obtain the base layer enhanced image.

[0132] In this embodiment, by dividing the base layer image into blocks, performing grayscale mapping on each image sub-block to obtain image enhancement sub-blocks, and then re-stitching the processed image enhancement sub-blocks, the base layer image is enhanced by using a block processing method, which can obtain more local details by increasing local contrast.

[0133] In practice, the base layer image can be divided into M×N sub-blocks, and grayscale remapping can be performed on each sub-block. The mapping relationship of each sub-block is denoted as f. (m,n) (Ib), where Ib represents the base layer image, m,n represents the mapping of the m-th row and n-th column sub-block, m≦M, n≦N.

[0134] The mapping methods include, but are not limited to, histogram equalization (HE), plateau histogram equalization (PHE), piecewise linear mapping, gamma mapping, and sigmoid mapping.

[0135] The following is a specific example.

[0136] like Figure 2 As shown, the 14-bit black and white image obtained by the infrared detector is enhanced in detail and compressed to 8-bit for display.

[0137] The image to be processed is filtered and layered. The 14-bit image to be processed, I, is filtered by Gaussian to obtain the base layer image Ib. The detail layer image Id is obtained by subtracting I from Ib.

[0138] The base layer image is processed by dividing it into 4×4 sub-blocks, and histogram statistics are performed on each sub-block.

[0139] The new grayscale mapping relationship f is determined using gamma correction. (m,n) (Ib), (m≦4, n≦4), the specific formula is:

[0140]

[0141] Among them, Ib max Ib min Ib mean F represents the maximum, minimum, and mean values ​​of the base layer image Ib. max F min It is the grayscale range after mapping.

[0142] like Figure 3 As shown, let (x1,y1), (x2,y1), (x1,y2), and (x2,y2) be the coordinates of the center points of the four sub-blocks associated with the current pixel Ib(i,j), and the grayscale mapping relationship of the four sub-blocks is f. (m,n) (Ib), f (m,n+1) (Ib), f (m+1,n) (Ib), f (m+1,n+1) (Ib) is determined by bilinear interpolation of gray levels. The calculation formula is as follows:

[0143] Ibe(i,j)=(f (m,n) (Ib(i,j)×(x2-j)×(y2-i)+f (m,n+1) (Ib(i,j))×(j-x1)×(y2-i))+f (m+1,n) (Ib(i,j))×(x2-j)×(i-y1)+f (m+1,n+1) (Ib(i,j))×(j-x1)×(i-y1)) / ((x2-x1)×(y2-y1))

[0144] The base layer enhancement image Ibe is obtained through a calculation formula.

[0145] Scene determination is performed on the base layer enhanced image Ibe. Based on the preset window size, the local variance matrix Ibe_var is calculated, and the number of scenes greater than the target threshold T is recorded as the target scene.

[0146] Based on the n categories of scenarios, n variance thresholds T1, T2...Tn are set to automatically determine which scenario the target scenario belongs to.

[0147] Calculate the dynamic range Ibe_range of the base layer enhanced image Ibe, and determine different detail gain coefficients G1, G2 and the dynamic range R of the fused image based on the built-in scene parameter table.

[0148] Table 1 shows the different detail gain coefficients G1, G2 and dynamic range R of the fused image determined according to different scene classifications in the embodiments of this application, taking outdoor street scene, indoor portrait and seascape as examples.

[0149] Table 1

[0150] Scene Classification threshold G1 G2 R Outdoor street scene T3 3 2 Ibe_range×1.5 Indoor portrait T2 2 1 Ibe_range×1 Sea and sky scenery T1 1 1 Ibe_range×0.5

[0151] The thresholds T, T1, T2...Tn and their corresponding parameters are derived from classification and debugging experience in various practical application scenarios.

[0152] Sharpness information of the detail layer image is extracted, and the Sobel operator is used to extract the edges of the detail layer image Id to form a normalized coefficient matrix E.

[0153] Based on the obtained gain coefficients G1 and G2, the point-by-point sharpening coefficient matrix G is calculated using the following formula:

[0154] G = G1 * E + G2

[0155] Where G is the point-by-point sharpening coefficient matrix, G1 is the strong detail enhancement gain coefficient, G2 is the weak detail enhancement gain coefficient, and E is the edge coefficient matrix.

[0156] After limiting the range of the detail layer image Id data, multiply it by the point-by-point sharpening coefficient matrix G to obtain the detail layer enhanced image Ide.

[0157] The obtained base layer enhancement image Ibe and detail layer enhancement image Ide are fused, and the maximum value of their sum is Max. (Ibe+Ide) Normalization is performed to adjust the dynamic range to R, and the calculation formula is as follows:

[0158]

[0159] Among them, I out For the output target image, Ibe is the base layer enhancement image, Ide is the detail layer enhancement image, and Max is the detail layer enhancement image. (Ibe+Ide) R is the maximum value of the fusion of the base layer enhancement image and the detail layer enhancement image, and R is the dynamic range of the fused image.

[0160] Take the enhancement processing of outdoor street scene images as an example.

[0161] Figure 4 The image shown is a schematic diagram of an outdoor street view image to be processed provided in an embodiment of this application. Figure 4 As shown, the contrast between the background and objects in the outdoor street scene image to be processed is not obvious, and the details of the objects cannot be identified.

[0162] Figure 5 The image shown is a schematic diagram of an enhanced outdoor street scene image provided in an embodiment of this application. Figure 5As shown, after enhancing the outdoor street scene image to be processed according to the target scene parameters of the adaptive scene, the contrast between the background and objects in the outdoor street scene image is obvious, and the details of objects such as houses, windows, wires, and power towers can be identified. The image quality is high and the visual experience of the human eye is good.

[0163] Take the enhancement processing of seascape images as an example.

[0164] Figure 6 The image shown is a schematic diagram of the seascape image to be processed provided in an embodiment of this application. Figure 6 As shown, the contrast between the background and objects in the seascape image to be processed is not obvious, and the details of the objects cannot be identified.

[0165] Figure 7 The image shown is a schematic diagram of an enhanced outdoor street scene image provided in an embodiment of this application. Figure 7 As shown, after enhancing the seascape image according to the target scene parameters of the adaptive scene, the contrast between the background and objects in the seascape image is obvious, and the details of objects such as the sky and kites can be identified. The obtained image quality is high and conforms to the visual perception of the human eye.

[0166] The image processing method provided in this application can be executed by an image processing device. This application uses an image processing device executing the image processing method as an example to illustrate the image processing device provided in this application.

[0167] This application also provides an image processing apparatus.

[0168] like Figure 8 As shown, the image processing apparatus includes:

[0169] The first processing module 810 is used to perform layer processing on the image to be processed to obtain the base layer image and detail layer image corresponding to the image to be processed.

[0170] The second processing module 820 is used to perform contrast enhancement processing on the base layer image to obtain the base layer enhanced image;

[0171] The third processing module 830 is used to determine the target scene parameters corresponding to the image to be processed based on the feature information of the base layer enhanced image;

[0172] The fourth processing module 840 is used to enhance the detail layer image based on the target scene parameters to obtain an enhanced detail layer image;

[0173] The fifth processing module 850 is used to obtain the target image based on the base layer enhanced image, the detail layer enhanced image, and the target scene parameters.

[0174] According to the image processing apparatus provided in the embodiments of this application, by acquiring the feature information of the base layer enhanced image, the image to be processed is divided into scenes, the target scene parameters corresponding to the image to be processed are determined, and the detail layer image is enhanced based on the target scene parameters to obtain the detail layer enhanced image. The image is then fused with the base layer enhanced image to obtain the target image. Based on the feature information of the base layer enhanced image, the image is enhanced by adapting to the image scene. This can avoid excessive contrast enhancement during the image enhancement process, effectively improve image quality, and enhance the visual experience of the human eye.

[0175] In some embodiments, the target scene parameters include detail enhancement gain coefficient and fused image dynamic range.

[0176] In some embodiments, the fourth processing module 840 is configured to perform enhancement processing on the detail layer image based on target scene parameters to obtain a detail layer enhanced image, including:

[0177] Based on the detail enhancement gain coefficient, the detail layer image is enhanced to obtain the detail layer enhanced image.

[0178] In some embodiments, the fourth processing module 840 is configured to perform enhancement processing on the detail layer image based on the detail enhancement gain coefficient to obtain a detail layer enhanced image, including:

[0179] The edge coefficients of the detail layer image are extracted to obtain the edge coefficient matrix;

[0180] Based on the detail enhancement gain coefficient and edge coefficient matrix, the point-by-point sharpening coefficient matrix is ​​obtained;

[0181] Based on the point-by-point sharpening coefficient matrix and the detail layer image, the detail layer enhanced image is obtained.

[0182] In some embodiments, the detail enhancement gain coefficients include strong detail enhancement gain coefficients and weak detail enhancement gain coefficients. Based on the detail enhancement gain coefficients and the edge coefficient matrix, a point-by-point sharpening coefficient matrix is ​​obtained, including:

[0183] Application formula

[0184] G = G1 * E + G2

[0185] Where G is the point-by-point sharpening coefficient matrix, G1 is the strong detail enhancement gain coefficient, G2 is the weak detail enhancement gain coefficient, and E is the edge coefficient matrix.

[0186] In some embodiments, the fifth processing module 850 is used to obtain a target image based on the base layer enhanced image, the detail layer enhanced image, and target scene parameters, including:

[0187] The base layer enhancement image and the detail layer enhancement image are fused together, and the dynamic range of the image is adjusted to the dynamic range of the fused image to obtain the target image.

[0188] In some embodiments, the feature information of the base layer enhancement image includes at least one of the image dynamic range, local variance, standard deviation, and gradient of the base layer enhancement image.

[0189] In some embodiments, the second processing module 820 is configured to perform contrast enhancement processing on the base layer image to obtain a base layer enhanced image, including:

[0190] The base layer image is divided into multiple image sub-blocks;

[0191] Each image sub-block is mapped to grayscale to obtain multiple image enhancement sub-blocks;

[0192] Multiple image enhancement sub-blocks are stitched together to obtain the base layer enhanced image.

[0193] The image processing device in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television set (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the device.

[0194] The image processing device in this application embodiment can be a device with an operating system. The operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit the specific operating system.

[0195] The image processing apparatus provided in this application embodiment can achieve... Figures 1 to 7 The various processes implemented in the method implementation examples will not be described again here to avoid repetition.

[0196] In some embodiments, such as Figure 9 As shown, this application embodiment also provides an electronic device 900, including a processor 901, a memory 902, and a computer program stored in the memory 902 and executable on the processor 901. When the program is executed by the processor 901, it implements the various processes of the above-described image processing method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0197] It should be noted that the electronic devices in the embodiments of this application include the aforementioned mobile electronic devices and non-mobile electronic devices.

[0198] This application also provides a non-transitory computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the above-described image processing method embodiments and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0199] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0200] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described image processing method.

[0201] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0202] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface and the processor are coupled. The processor is used to run programs or instructions to implement the various processes of the above-described image processing method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0203] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0204] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0205] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of this application.

[0206] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

[0207] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0208] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

Claims

1. An image processing method, characterized by, The method comprises the following steps: performing hierarchical processing on a to-be-processed image to obtain a base layer image and a detail layer image corresponding to the to-be-processed image; performing contrast enhancement processing on the base layer image to obtain a base layer enhanced image; determining a target scene parameter corresponding to the to-be-processed image based on feature information of the base layer enhanced image; performing enhancement processing on the detail layer image based on the target scene parameter to obtain a detail layer enhanced image; obtaining a target image based on the base layer enhanced image, the detail layer enhanced image and the target scene parameter; The method comprises the following steps: determining the target scene parameter corresponding to the to-be-processed image based on feature information of the base layer enhanced image, comprising: determining n variance thresholds according to n grades of scene settings, performing scene judgment on the base layer enhanced image, calculating a local variance matrix according to a preset window size, and determining the target scene corresponding to the to-be-processed image based on the variance matrix and the corresponding variance threshold; The target scene parameter comprises a detail enhancement gain coefficient and a fusion image dynamic range; 2. The image processing method of claim 1, wherein, The fusion image dynamic range is the dynamic range of the base layer enhanced image multiplied by a preset proportion coefficient corresponding to the target scene. The method comprises the following steps:

3. The image processing method of claim 2, wherein, performing enhancement processing on the detail layer image based on the detail enhancement gain coefficient to obtain the detail layer enhanced image. The method comprises the following steps: extracting an edge coefficient of the detail layer image to obtain an edge coefficient matrix; obtaining a point-by-point sharpening coefficient matrix based on the detail enhancement gain coefficient and the edge coefficient matrix; 4. The image processing method of claim 3, wherein, obtaining the detail layer enhanced image based on the point-by-point sharpening coefficient matrix and the detail layer image. The detail enhancement gain coefficient comprises a strong detail enhancement gain coefficient and a weak detail enhancement gain coefficient, and the method comprises the following steps: wherein, is the point-wise sharpening coefficient matrix, is the strong detail enhancement gain coefficient, is the weak detail enhancement gain coefficient, is the edge coefficient matrix.

5. The image processing method of claim 1, wherein, applying the formula The method comprises the following steps:

6. The image processing method of any of claims 1-5, wherein, performing fusion processing on the base layer enhanced image and the detail layer enhanced image, adjusting the image dynamic range to the fusion image dynamic range, and obtaining the target image.

7. The image processing method according to any one of claims 1-5, characterized in that, The feature information of the base layer enhanced image comprises at least one of an image dynamic range and a local variance of the base layer enhanced image. The method comprises the following steps: dividing the base layer image into a plurality of image subblocks; performing gray mapping on each of the image subblocks to obtain a plurality of image enhancement subblocks; 8. An image processing apparatus characterized by comprising: splicing the plurality of image enhancement subblocks to obtain the base layer enhanced image. The method comprises the following steps: a first processing module is configured to perform hierarchical processing on a to-be-processed image to obtain a base layer image and a detail layer image corresponding to the to-be-processed image; The second processing module is configured to perform a contrast enhancement processing on the base layer image to obtain a base layer enhanced image. The third processing module is configured to determine a target scene parameter corresponding to the image to be processed based on feature information of the base layer enhanced image. The fourth processing module is configured to perform an enhancement processing on the detail layer image based on the target scene parameter to obtain a detail layer enhanced image. The fifth processing module is configured to obtain a target image based on the base layer enhanced image, the detail layer enhanced image and the target scene parameter.

9. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the image processing method according to any one of claims 1-7 when executing the program.

Images