Image processing method, device and computer storage medium

Abstract

The present invention discloses an image processing method, device and computer storage medium, wherein the image processing method comprises: obtaining a brightness parameter of an image to be processed; dynamically adjusting the brightness of the image to be processed based on the brightness parameter, so as to obtain a target image with overall dynamic improvement in brightness. The image processing method, device and computer storage medium provided by the present invention can improve the overall dynamic range of the brightness of the image by dynamically adjusting the brightness of a single-frame image, thereby improving the image quality, and the operation is convenient and fast, thereby improving the user experience.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to an image processing method, device and computer storage medium. Background Art

[0002] The dynamic range of an image is one of the main indicators of an imaging system. By improving the dynamic range of an image, more image details can be captured. During the image capture process, the relevant technology can detect whether there is a face, form multiple exposure images at different exposures according to the scene, and synthesize multiple exposure images to improve the overall dynamic range of the image. However, the above method requires multiple exposure images, resulting in a long imaging delay time, and the improvement of the overall picture brightness is not high enough, which affects the user experience. Summary of the invention

[0003] The object of the present invention is to provide an image processing method, device and computer storage medium, which can improve the overall dynamic range of the image, improve the image quality, and are convenient and quick to operate, thereby improving the user experience.

[0004] To achieve the above object, the technical solution of the present invention is achieved as follows:

[0005] In a first aspect, an embodiment of the present invention provides an image processing method, including:

[0006] Get the brightness parameter of the image to be processed;

[0007] The brightness of the image to be processed is dynamically adjusted based on the brightness parameter to obtain a target image with overall dynamically improved brightness.

[0008] As one implementation manner, the step of obtaining the brightness parameter of the image to be processed includes:

[0009] Extracting brightness component data and component data other than the brightness component data of the image to be processed;

[0010] The step of dynamically adjusting the brightness of the image to be processed based on the brightness parameter to obtain a target image with overall dynamically improved brightness includes:

[0011] Performing brightness stretching processing on the brightness component data to obtain the brightness component data after brightness stretching processing;

[0012] The brightness component data after the brightness stretching process and the component data other than the brightness component data are merged to obtain a target image with overall dynamic brightness enhancement.

[0013] As one implementation manner, the extracting brightness component data and component data other than the brightness component data of the image to be processed includes:

[0014] If the image to be processed is in RGB color mode, the image to be processed is converted into a YUV color space, and brightness component data and component data other than the brightness component data of the image to be processed in the YUV color mode are extracted.

[0015] As one implementation manner, performing brightness stretching processing on the brightness component data to obtain the brightness component data after brightness stretching processing includes:

[0016] Filtering the brightness component data to obtain a background layer and a detail layer of the brightness component data;

[0017] Performing contrast-limited adaptive histogram equalization processing on the background layer to obtain a target background layer;

[0018] The detail layer and the target background layer are merged to obtain the brightness component data after brightness stretching processing.

[0019] As one implementation manner, performing contrast-limited adaptive histogram equalization processing on the background layer to obtain a target background layer includes:

[0020] Evenly dividing the background layer into blocks according to at least two different image segmentation methods, to obtain at least two background layers divided into blocks;

[0021] Performing histogram equalization processing on the at least two background layers after being divided into blocks, respectively, to obtain at least two background layers after being divided into blocks and subjected to histogram equalization processing;

[0022] The at least two background layers that have been segmented into blocks and processed by histogram equalization are weightedly fused to obtain a target background layer.

[0023] As one implementation manner, the image segmentation method includes a first image segmentation method for segmenting the background layer into N1*N1 blocks and a second image segmentation method for segmenting the background layer into N2*N2 blocks, and N1 and N2 are positive integers and N1 is less than N2; the at least two background layers after segmentation into blocks and histogram equalization processing include a first background layer and a second background layer; the weighted fusion of the at least two background layers after segmentation into blocks and histogram equalization processing to obtain a target background layer includes:

[0024] Obtaining a first weight matrix of the first background layer and a second weight matrix of the second background layer, wherein each element in the first weight matrix represents a weight of a corresponding pixel point in the first background layer, and each element in the second weight matrix represents a weight of a corresponding pixel point in the second background layer;

[0025] The first background layer and the second background layer are weightedly fused according to the first weight matrix and the second weight matrix to obtain a target background layer.

[0026] As one implementation manner, the obtaining of a first weight matrix of the first background layer and a second weight matrix of the second background layer includes:

[0027] Displaying a weight setting interface including the first background layer and / or the second background layer to the user, wherein the weight setting interface includes a block selection option and a weight setting option;

[0028] Acquire a user's selection operation on the block selection option and a setting operation on the weight setting option in the weight setting interface;

[0029] A first weight matrix for the first background layer and / or a second weight matrix for the second background layer are correspondingly generated according to the selection operation and the setting operation of the user.

[0030] As one implementation manner, after dynamically adjusting the brightness of the image to be processed based on the brightness parameter to obtain a target image with overall dynamically improved brightness, the method further includes:

[0031] Extracting a region image corresponding to a target object from the target image;

[0032] Performing light effect enhancement processing on the area image corresponding to the target object to obtain the area image corresponding to the target object after the light effect enhancement processing;

[0033] The regional image corresponding to the target object after the light effect enhancement processing is weightedly superimposed on the target image to obtain the target image after regional enhancement.

[0034] As one implementation manner, before extracting the area image corresponding to the target object from the target image, the method further includes:

[0035] Detecting whether a target object exists in the target image;

[0036] When it is determined that the target object exists in the target image, a step of extracting a region image corresponding to the target object from the target image is performed.

[0037] In one implementation manner, the target object includes a face; and extracting a region image corresponding to the target object from the target image includes:

[0038] Obtaining facial region coordinates and facial key point data in the target image;

[0039] Based on the key point data of the human face, a region growing method is used to determine the contour of the human face region;

[0040] A region image corresponding to the face is extracted from the target image according to the face region coordinates and the face region contour.

[0041] As one implementation manner, performing light effect enhancement processing on the area image corresponding to the target object to obtain the area image corresponding to the target object after the light effect enhancement processing includes:

[0042] Performing light effect enhancement from strong to weak on the regional image corresponding to the face region from the center to the edge to obtain the regional image corresponding to the face after light effect enhancement processing; or,

[0043] The regional image corresponding to the face region is subjected to light effect enhancement from weak to strong from the edge to the center to obtain the regional image corresponding to the face after light effect enhancement processing.

[0044] In one implementation manner, the target object includes the moon; and extracting a region image corresponding to the target object from the target image includes:

[0045] Detecting the position of the moon in the target image;

[0046] Based on the position of the moon, a segmentation algorithm is used to segment the area image corresponding to the moon from the target image.

[0047] As one implementation manner, performing light effect enhancement processing on the area image corresponding to the target object to obtain the area image corresponding to the target object after the light effect enhancement processing includes:

[0048] A deblurring algorithm and a super-resolution algorithm are used to enhance the light effect of the image of the area corresponding to the moon, so as to obtain the image of the area corresponding to the moon after the light effect enhancement processing.

[0049] In a second aspect, an embodiment of the present invention provides an image processing device, which includes a processor and a storage device for storing programs; when the program is executed by the processor, the processor implements the image processing method as described in the first aspect.

[0050] In a third aspect, an embodiment of the present invention provides a computer storage medium storing a computer program, wherein when the computer program is executed by a processor, the image processing method as described in the first aspect is implemented.

[0051] The image processing method, device and computer storage medium provided by the embodiments of the present invention include: obtaining the brightness parameters of the image to be processed; dynamically adjusting the brightness of the image to be processed based on the brightness parameters to obtain a target image with overall dynamic brightness improvement. In this way, by dynamically adjusting the brightness of a single-frame image, the overall dynamic range of the image brightness is improved, including suppressing the brightness of high-brightness areas in the image and / or improving the brightness of low-brightness areas, thereby improving the image quality, and there is no need to use multiple frames of exposure images to achieve dynamic range adjustment, the operation is convenient and fast, and the user experience is improved. In addition, it is capable of processing images taken or obtained in various complex light environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 A schematic diagram of a flow chart of an image processing method provided by an embodiment of the present invention;

[0053] Figure 2 A schematic diagram of a specific process of an image processing method provided by an embodiment of the present invention;

[0054] Figure 3 is a schematic diagram of a background layer after being divided into blocks in an embodiment of the present invention;

[0055] Figure 4 is a schematic diagram of a setting interface of a background layer in an embodiment of the present invention;

[0056] Figure 5 A schematic diagram of the structure of an image processing device provided by an embodiment of the present invention;

[0057] Figure 6 Schematic diagram of comparison before and after image processing in an embodiment of the present invention. DETAILED DESCRIPTION

[0058] The technical solution of the present invention is further elaborated in detail below in conjunction with the accompanying drawings and specific embodiments of the specification. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the technical field of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items.

[0059] See also Figure 1, is an image processing method provided by an embodiment of the present invention. The image processing method can be applied to the case of improving the dynamic range of an image. The image processing method can be executed by an image processing device provided by an embodiment of the present invention. The image processing device can be implemented in software and / or hardware. In a specific application, the image processing device can be a mobile terminal such as a smart phone, a personal digital assistant, a tablet computer, etc. The image processing method includes the following steps:

[0060] Step S101: obtaining brightness parameters of the image to be processed;

[0061] Step S102: dynamically adjusting the brightness of the image to be processed based on the brightness parameter to obtain a target image with overall dynamically improved brightness.

[0062] Here, the image to be processed is a single-frame image, which may be a frame of image extracted from a video currently being shot or already shot by a photographing device such as a camera, or a single frame of image currently being shot or already shot by a photographing device such as a camera, or an image acquired by other means. The brightness parameter is used to characterize the brightness information of the image, and may be the brightness component data of the image, etc. The brightness of the image to be processed is dynamically adjusted based on the brightness parameter, such as suppressing the brightness of a high-brightness area in the image and / or increasing the brightness of a low-brightness area, which can improve the overall dynamic range of the brightness of the image to achieve improved image quality.

[0063] Optionally, taking the step of obtaining the brightness parameter of the image to be processed, including extracting the brightness component data of the image to be processed and the component data other than the brightness component data as an example, Figure 2 FIG. 1 is a schematic diagram of a specific flow chart of an image processing method provided by an embodiment of the present invention, including:

[0064] Step S201: extracting brightness component data and component data other than the brightness component data of the image to be processed;

[0065] Here, in order to improve the overall dynamic range of the image to be processed, the brightness component data of the image to be processed can be subjected to dynamic range stretching processing, so it is necessary to extract the brightness component data of the image to be processed. In this embodiment, the image to be processed in YUV color mode is preferably used. If the space of the image to be processed itself is RGB (Red, Green, Blue) color space or HSV (Hue, Saturation, Value, hue, saturation, brightness) color space, etc., the image space can be converted into YUV (Luma, Chroma, brightness, chroma) color space through space conversion, that is, the color mode of the image to be processed is converted into YUV color mode, wherein the Y component data is the brightness component data, and the U component data and the V component data are the chroma component data. It should be noted that in actual applications, the Y component data can also be called Y channel data, the U component data can be called U channel data, and the V component data can be called V channel data. Here, the brightness component data of the image to be processed is actually each initial pixel value of the brightness Y of the image to be processed. In one embodiment, the extracting of brightness component data and component data other than the brightness component data of the image to be processed includes: if the image to be processed is in RGB color mode, converting the image to be processed to YUV color space, and extracting brightness component data and component data other than the brightness component data of the image to be processed in YUV color mode. Since image processing directly in RGB color space will cause image distortion, it is necessary to first convert the image to be processed from RGB color space to YUV color space. In this way, by converting the image to be processed in RGB color mode to YUV color mode, the change of image information can be conveniently and accurately controlled to ensure accurate improvement of the dynamic range of the image.

[0066] Step S202: performing brightness stretching processing on the brightness component data to obtain the brightness component data after brightness stretching processing;

[0067] Optionally, performing brightness stretching processing on the brightness component data to obtain the brightness component data after brightness stretching processing includes:

[0068] Filtering the brightness component data to obtain a background layer and a detail layer of the brightness component data;

[0069] Performing contrast-limited adaptive histogram equalization processing on the background layer to obtain a target background layer;

[0070] The detail layer and the target background layer are merged to obtain the brightness component data after brightness stretching processing.

[0071] It should be noted that, since the brightness stretching processing of the brightness component data is realized, it is sufficient to perform brightness stretching processing on the background layer of the brightness component data. Therefore, the background layer and detail layer of the brightness component data are obtained by filtering the brightness component data, that is, the Y channel data of the image to be processed is divided into the background layer and the detail layer. Among them, the detail layer contains the specific detail information of the image to be processed, and the background layer contains the background information of the image to be processed, such as light brightness, etc. In addition, by filtering the brightness component data, the image information can be retained as much as possible.

[0072] Here, performing contrast-limited adaptive histogram equalization processing on the background layer to obtain a target background layer may include:

[0073] Evenly dividing the background layer into blocks according to at least two different image segmentation methods, to obtain at least two background layers divided into blocks;

[0074] Performing histogram equalization processing on the at least two background layers after being divided into blocks, respectively, to obtain at least two background layers after being divided into blocks and subjected to histogram equalization processing;

[0075] The at least two background layers that have been segmented into blocks and processed with histogram equalization are weightedly fused to obtain a target background layer.

[0076] It can be understood that dividing the background layer into blocks according to at least two different image segmentation methods means dividing the background layer into blocks evenly and without overlap according to at least two different image segmentation methods, and each image segmentation method divides the background layer into blocks separately, so as to obtain at least two background layers divided into blocks, and each background layer is segmented by only one image segmentation method. For example, the image segmentation method includes a first image segmentation method for dividing the background layer into N1*N1 blocks and a second image segmentation method for dividing the background layer into N2*N2 blocks, and N1 and N2 are positive integers and N1 is less than N2. Assume that N1 is 2 and N2 is 4, see Figure 3 , is a schematic diagram of the background layer after being divided into blocks, where Figure 3 (a) is a schematic diagram of the background layer after being divided into N1*N1 blocks, each block is marked as 1, 2, 3, and 4 in sequence; Figure 3 (b) is a schematic diagram of the background layer after being divided into N2*N2 blocks, and each block is sequentially labeled 1, 2, 3, 4, ... 15, 16. Here, the sizes of N1 and N2 can be set according to actual needs, for example, N1 can be set to be less than or equal to 5, and N2 can be set to be greater than or equal to 4.

[0077] Here, the background layer is divided into blocks according to at least two different image segmentation methods, and at least two background layers after being divided into blocks are obtained, and the at least two background layers after being divided into blocks are subjected to histogram equalization, and at least two background layers after being divided into blocks and subjected to histogram equalization are obtained. A certain number of limited contrast histogram equalization CLAHE modules can be created, and each CLAHE module divides a background layer into a set number of blocks, and then the background layers after being divided into blocks are subjected to histogram equalization. The weighted fusion of the at least two background layers after being divided into blocks and subjected to histogram equalization to obtain the target background layer refers to weighted fusion of the values ​​of the same pixel points in the at least two background layers after being divided into blocks and subjected to histogram equalization, so as to obtain the target background layer. In one embodiment, the image segmentation method includes a first image segmentation method for segmenting the background layer into N1*N1 blocks and a second image segmentation method for segmenting the background layer into N2*N2 blocks, and N1 and N2 are positive integers and N1 is less than N2; the at least two background layers after segmentation into blocks and histogram equalization processing include a first background layer and a second background layer; the weighted fusion of the at least two background layers after segmentation into blocks and histogram equalization processing to obtain a target background layer includes:

[0078] Obtaining a first weight matrix of the first background layer and a second weight matrix of the second background layer, wherein each element in the first weight matrix represents a weight of a corresponding pixel point in the first background layer, and each element in the second weight matrix represents a weight of a corresponding pixel point in the second background layer;

[0079] The first background layer and the second background layer are weightedly fused according to the first weight matrix and the second weight matrix to obtain a target background layer.

[0080] Please refer again Figure 3 ,by Figure 3 The background layer represented by (a) is the first background layer. Figure 3(b) Take the second background layer as an example, if the number of pixels contained in the first background layer and the second background layer is recorded as X*Y, then the size of the first weight matrix and the second weight matrix is ​​also recorded as X*Y. Assuming that e is a pixel in the first background layer and the second background layer, e1 represents the value of the pixel in the first background layer, e2 represents the value of the pixel in the second background layer, f1 represents the weight of the pixel in the first weight matrix, and f2 represents the weight of the pixel in the second weight matrix, then when the first background layer and the second background layer are weighted and fused according to the first weight matrix and the second weight matrix, the value of the pixel can be expressed as e1*f1+e2*f2. In practical applications, the first weight matrix and the second weight matrix can be predefined, that is, default, for example, the weights of all pixels in the first weight matrix and the second weight matrix are 0.5 respectively by default, that is, the weight corresponding to the first background layer and the weight corresponding to the second background layer are both set to 0.5 by default. The first weight matrix and the second weight matrix can also be customized, such as setting the weights of all pixels in the first weight matrix to 0.4, setting the weights of all pixels in the second weight matrix to 0.6, etc. In addition, the user can also use the first background layer as the base and specify the weights of different blocks in the second background layer for weighted fusion. For example, the user can select the first block and the second block in the second background layer and set the weights of these two blocks to 0.6. At this time, the weights of the blocks in the first background layer and the remaining blocks in the second background layer can be the default values ​​respectively. In order to enhance the local brightness of the image, the weights of the blocks in the first background layer can all be set to 1.

[0081] In one implementation, the obtaining of a first weight matrix of the first background layer and a second weight matrix of the second background layer includes:

[0082] Displaying a setting interface including the first background layer and / or the second background layer to the user, wherein the setting interface includes a block selection option and a weight setting option;

[0083] Acquire a user's selection operation on the block selection option and a setting operation on the weight setting option in the setting interface;

[0084] A first weight matrix for the first background layer and / or a second weight matrix for the second background layer are correspondingly generated according to the selection operation and the setting operation of the user.

[0085] Take the second background layer as an example, where it is divided into 16 blocks and the default weight is 0.5. Figure 4 , which is a schematic diagram of the background layer setting interface, Figure 4(a) A schematic diagram showing the initial setting interface of the background layer. Figure 4 (b) A schematic diagram showing the background layer setting interface after setting, where the weight of each block is set to 0.5 by default and is represented by W1, W2, ..., W 15 , W 16 Indicates that the small circle in each block is used to identify whether the block is selected. The setting interface displays the second background layer after being divided into blocks and the default weight of each block. If the user needs to select a block, the block selection operation can be realized by clicking the block or the circle in the block. When any block in the background layer is selected, the small circle in the block will be filled with black for easy identification by the user. If the user needs to set the weight of a block, the weight to be set can be entered in the weight input rectangular box corresponding to the block to realize free setting of the weight. In this example, the user selects the first block and the second block, and sets the weights of the first block and the second block to 0.6 respectively. Then the small circles in the first block and the second block are filled with black and W1=0.6, W2=0.6. In this way, by setting the weights of the blocks in the background layer, that is, setting the first weight matrix of the first background layer and / or the second weight matrix of the second background layer, the image processing effect can be intelligently improved through different weighting methods, which is convenient and flexible to operate.

[0086] In another embodiment, if the user has pre-set the weight value of the first background layer and the weight value of the second background layer, the step of obtaining the first weight matrix of the first background layer and the second weight matrix of the second background layer may be to generate a first weight matrix based on the weight value of the first background layer and the number of pixels of the first background layer, and to generate a second weight matrix based on the weight value of the second background layer and the number of pixels of the second background layer, wherein the number of pixels of the first background layer is the same as the number of pixels of the second background layer.

[0087] Step S203: fusing the brightness component data after the brightness stretching processing with the component data other than the brightness component data to obtain a target image with overall dynamic brightness enhancement.

[0088] Specifically, the target image is obtained by fusing the brightness component data obtained in step S202 after brightness stretching processing and the component data other than the brightness component data obtained in step S201, thereby improving the overall dynamic range of the image.

[0089] In summary, in the image processing method provided in the above embodiment, by dynamically adjusting the brightness of a single frame image, the overall dynamic range of the image brightness is improved, including suppressing the brightness of the high-brightness area in the image and / or increasing the brightness of the low-brightness area, thereby improving the image quality, and there is no need to use multiple frames of exposure images to achieve dynamic range adjustment, the operation is convenient and fast, and the user experience is improved. In addition, it is possible to process images taken or obtained in various complex light environments.

[0090] In one embodiment, after dynamically adjusting the brightness of the image to be processed based on the brightness parameter to obtain a target image with overall dynamically improved brightness, the method further includes:

[0091] Extracting a region image corresponding to a target object from the target image;

[0092] Performing light effect enhancement processing on the area image corresponding to the target object to obtain the area image corresponding to the target object after the light effect enhancement processing;

[0093] The regional image corresponding to the target object after the light effect enhancement processing is weightedly superimposed on the target image to obtain the target image after regional enhancement.

[0094] It can be understood that the image after dynamic range stretching based on brightness stretching processing will have a significant improvement in the overall dynamic range. However, the regional image enhancement processing cannot be implemented for some target object areas such as face areas, which may be difficult to meet user requirements, thereby affecting the user experience. In addition, the effect of the target object may be the most expected by the user. If the effect of the target object presented on the image is not good, it will also directly affect the user experience. Therefore, the regional image corresponding to the target object in the target image can also be subjected to light effect enhancement processing so that it can adapt to various complex light environments to further improve the image quality and better capture the target object. Optionally, before extracting the regional image corresponding to the target object area from the target image, it also includes: detecting whether there is a target object in the target image; when it is determined that the target object exists in the target image, executing the step of extracting the regional image corresponding to the target object area from the target image. Specifically, detecting whether there is a target object in the target image, and when it is determined that the target object exists in the target image, executing the step of extracting the regional image corresponding to the target object area from the target image, otherwise stop processing. Here, only when it is detected that the target object exists in the target image, the step of extracting the regional image corresponding to the target object region from the target image is performed, thereby achieving accurate enhancement processing of the target object.

[0095] In one embodiment, the target object includes a human face; and extracting a region image corresponding to the target object from the target image includes:

[0096] Obtaining facial region coordinates and facial key point data in the target image;

[0097] Based on the key point data of the human face, a region growing method is used to determine the contour of the human face region;

[0098] A region image corresponding to the face is extracted from the target image according to the face region coordinates and the face region contour.

[0099] Here, the target image can be detected by an existing face recognition method to obtain the face area coordinates and key point data of the face in the target image when there is a face in the target image. The face area coordinates refer to the coordinate data within the rectangular frame surrounding the face area, so the face area coordinates contain data other than the face coordinates. In order to accurately determine the area image corresponding to the face, it is also necessary to determine the face area contour based on the key point data of the face by using the regional growth method, and then extract the area image corresponding to the face from the target image according to the face area coordinates and the face area contour. In this way, by accurately obtaining the area image corresponding to the face, the accuracy of the light effect enhancement processing of the area image corresponding to the face is ensured to avoid interference.

[0100] In one embodiment, performing light effect enhancement processing on the area image corresponding to the target object to obtain the area image corresponding to the target object after the light effect enhancement processing includes:

[0101] Performing light effect enhancement from strong to weak on the area image corresponding to the face from the center to the edge to obtain the area image corresponding to the face after light effect enhancement processing; or,

[0102] The regional image corresponding to the face is subjected to light effect enhancement from weak to strong from the edge to the center to obtain the regional image corresponding to the face after light effect enhancement processing.

[0103] Here, the center and edge of the regional image corresponding to the face can be known based on the regional image corresponding to the face, so that the regional image corresponding to the face can be enhanced from strong to weak in light effect from the center to the edge, or from weak to strong in light effect from the edge to the center, so as to achieve brightening of the face and gradually reduce the brightness of the face from the center to the edge, that is, obtain the regional image corresponding to the face after light effect enhancement. In this way, by performing light effect enhancement processing on the regional image corresponding to the face, it can adapt to various complex light environments, further improve image quality, and better collect faces.

[0104] In one embodiment, the target object includes the moon; and extracting a region image corresponding to the target object from the target image includes:

[0105] Detecting the position of the moon in the target image;

[0106] Based on the position of the moon, a segmentation algorithm is used to segment the area image corresponding to the moon from the target image.

[0107] Specifically, the position of the moon in the target image is detected, and based on the detected position of the moon in the target image, a segmentation algorithm is used to completely segment the moon in the target image, so as to segment the region image corresponding to the moon from the target image. Here, the segmentation algorithm may use an instance segmentation algorithm, etc. In this way, by accurately acquiring the region image corresponding to the moon, the accuracy of the light effect enhancement processing of the region image corresponding to the moon is ensured, and interference is avoided.

[0108] In one embodiment, performing light effect enhancement processing on the area image corresponding to the moon to obtain the area image corresponding to the moon after light effect enhancement processing includes:

[0109] A deblurring algorithm and a super-resolution algorithm are used to enhance the light effect of the image of the area corresponding to the moon, so as to obtain the image of the area corresponding to the moon after the light effect enhancement processing.

[0110] In this way, by enhancing the light effect of the image of the area corresponding to the moon, the clarity of the image of the area corresponding to the moon can be improved, and light diffusion can be effectively reduced.

[0111] It should be noted that the weighted superposition of the regional image corresponding to the target object after the light effect enhancement processing to the target image to obtain the target image after regional enhancement means weighted fusion of the regional image corresponding to the target object after the light effect enhancement processing and the value of the same pixel in the target image to obtain the target image after regional enhancement. It can also be understood as weighted fusion of the target image and the regional image corresponding to the target object according to the third weight matrix of the target image and the fourth weight matrix of the regional image corresponding to the target object to obtain the target image after regional enhancement. The size of the third weight matrix is ​​the same as the number of pixels contained in the target image, and the size of the fourth weight matrix is ​​the same as the number of pixels contained in the regional image corresponding to the target object. In practical applications, the third weight matrix and the fourth weight matrix can be predefined, that is, default, for example, the weights of all pixels in the third weight matrix and the fourth weight matrix are 0.5 respectively, that is, the weight corresponding to the target image and the weight corresponding to the regional image corresponding to the target object are both set to 0.5 by default. The third weight matrix and the fourth weight matrix may also be customized, for example, the weights of all pixels in the third weight matrix are set to 0.4, and the weights of all pixels in the fourth weight matrix are set to 0.6, etc. In this way, by weightedly superimposing the regional image corresponding to the target object after the light effect enhancement processing onto the target image, the regional enhancement processing of the target object is achieved, the image quality is improved, and the user experience is further enhanced.

[0112] The embodiment of the present invention provides an image processing device, such as Figure 5 As shown, the image processing device includes: a processor 110 and a memory 111 for storing a computer program that can be run on the processor 110; wherein, Figure 5 The processor 110 shown in the figure is not used to indicate that the number of the processor 110 is one, but is only used to indicate the positional relationship of the processor 110 relative to other devices. In actual applications, the number of the processor 110 may be one or more; similarly, Figure 5 The memory 111 shown in the figure has the same meaning, that is, it is only used to refer to the position relationship of the memory 111 relative to other devices. In practical applications, the number of memories 111 can be one or more. The processor 110 is used to implement the above image processing method when running the computer program.

[0113] The image processing device may further include: at least one network interface 112. The various components in the image processing device are coupled together via a bus system 113. It is understood that the bus system 113 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 113 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, the bus system 113 is not described in detail. Figure 5 Various buses are labeled as bus system 113 .

[0114] The memory 111 may be a volatile memory or a nonvolatile memory, or may include both volatile and nonvolatile memories. The nonvolatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a ferromagnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM); the magnetic surface memory may be a disk memory or a tape memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM, SyncLink Dynamic Random Access Memory), and direct RAM bus random access memory (DRRAM, Direct Rambus Random Access Memory).The memory 111 described in the embodiments of the present invention is intended to include but is not limited to these and any other suitable types of memories.

[0115] The memory 111 in the embodiment of the present invention is used to store various types of data to support the operation of the image processing device. Examples of these data include: any computer program used to operate on the image processing device, such as an operating system and an application; contact data; phone book data; messages; pictures; videos, etc. Among them, the operating system includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., which are used to implement various basic services and process hardware-based tasks. The application program may include various applications, such as a media player (Media Player), a browser (Browser), etc., which are used to implement various application services. Here, the program that implements the method of the embodiment of the present invention may be included in the application program.

[0116] Based on the same inventive concept as the above-mentioned embodiment, this embodiment further provides a computer storage medium, in which a computer program is stored. The computer storage medium may be a ferromagnetic random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM); or it may be various devices including one or any combination of the above-mentioned memories, such as a mobile phone, a computer, a tablet device, a personal digital assistant, etc. When the computer program stored in the computer storage medium is executed by the processor, the image processing method applied to the above-mentioned image processing device is implemented. For the specific steps implemented when the computer program is executed by the processor, please refer to Figure 1 or Figure 2 The description of the illustrated embodiment will not be repeated here.

[0117] Based on the same inventive concept as the above embodiments, this embodiment describes the technical solution of the above embodiments in detail through specific examples. The specific process of an image processing method provided by the embodiment of the present invention is as follows:

[0118] First, the image is converted to YUV color space and the Y brightness channel data is extracted for dynamic range stretching.

[0119] Next, use a filter to separate the Y channel into two layers, the background layer and the detail layer.

[0120] Here, the detail layer needs to be retained, while the background layer needs to be dynamically stretched.

[0121] Next, two threads are created to stretch the dynamic range of the background layer. In this embodiment, two parallel limited contrast histogram equalization CLAHE modules are used to perform corresponding processing; wherein,

[0122] The first CLAHE module is CLAHE2X2: the image is divided into 2*2 small blocks, and the image is histogram equalized to adjust the overall dynamic range of the image;

[0123] The second CLAHE module is CLAHE16X16: it divides the image into 16*16 small blocks, performs histogram equalization on the image, improves the local brightness of the image, and has a significant enhancement effect on the local area.

[0124] Next, the CLAHE2X2 result and the CLAHE16X16 result are weighted according to the weighted fusion method to obtain the processed background layer. The weighted fusion method is as follows:

[0125] Predefined mode: CLAHE2X2 result and CLAHE16X16 result are weighted fusion with half each;

[0126] Custom mode: Users can adjust the weighted fusion ratio of CLAHE2X2 results and CLAHE16X16 results according to their needs;

[0127] Smart mode: Based on the block-based weighted fusion solution, users can specify which blocks need to be weighted fused to achieve image enhancement. With CLAHE2X2 as the base, CLAHE16X16 blocks with different weights are used to intelligently improve the image effect.

[0128] Next, an image with a dynamic range stretched is obtained based on the processed background layer and detail layer.

[0129] Next, the image area is enhanced based on the target object detection. The image after dynamic range stretching will have a significant improvement in the overall dynamic range, but no special enhancement is made for certain specific areas, such as the face area, or the moon in night scene shooting, which may easily lead to light diffusion problems.

[0130] Taking the face as an example, first detect whether there is a face in the image, and if there is a face, perform special enhancement on the face area. Obtain the coordinates of the face area and multiple key point data of the face, and use the region growing method to detect the precise face area contour in the face frame based on the multiple key point data of the face, and then perform face light effect enhancement based on the face area coordinates. In this embodiment, the entire face area is enhanced from the center to the edge from strong to weak light effects, and different brightening methods can generate different face light effects.

[0131] Take shooting the moon at night as an example. Light diffusion is easy to occur during shooting, resulting in blurred edges and unclear details of the moon. First, the position of the moon in the night scene is detected, and the complete moon is segmented through the segmentation algorithm. The deblurring and super-resolution algorithms are used to enhance the moon, improve the clarity of the moon area and reduce light diffusion.

[0132] Finally, the enhanced target object area image is weighted and superimposed back to the original image to complete the single-frame image dynamic range stretching for the target object area enhancement. Figure 6 , which is a comparison diagram before and after image processing. Figure 6 (a) is a schematic diagram before image processing according to the above image processing method, Figure 6 (b) is a schematic diagram of an image after being processed according to the above image processing method. By comparison, it can be seen that the overall brightness and local brightness of the image processed according to the above image processing method are improved, and the image quality is improved.

[0133] Here, the weighted fusion method used to weight the enhanced target object area image back to the original image can be divided into the following two methods: one is the default setting of weighting with half the weight each; the other is to complete the weighting by manually setting the weight ratio by the user.

[0134] In summary, in the image processing method provided in the above embodiment, the overall brightness of the image and the local brightness are improved through weighted fusion, and the dynamic range of the single frame image is stretched. In addition, the special area in the image is detected and identified, and different light effects are processed for the area, which can achieve personalized or specialized processing.

[0135] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0136] In this document, the terms "comprises," "comprising," or any other variations thereof, are intended to cover a non-exclusive inclusion of elements other than those listed and may also include additional elements not expressly listed.

[0137] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. An image processing method, characterized in that: include: Get the brightness parameter of the image to be processed; Dynamically adjusting the brightness of the image to be processed based on the brightness parameter to obtain a target image with overall dynamically improved brightness; The step of obtaining the brightness parameter of the image to be processed includes: Extracting brightness component data and component data other than the brightness component data of the image to be processed; The step of dynamically adjusting the brightness of the image to be processed based on the brightness parameter to obtain a target image with overall dynamically improved brightness includes: Filtering the brightness component data to obtain a background layer and a detail layer of the brightness component data; Evenly dividing the background layer into blocks according to at least two different image segmentation methods, to obtain at least two background layers divided into blocks; Performing histogram equalization processing on the at least two background layers after being divided into blocks, respectively, to obtain at least two background layers after being divided into blocks and subjected to histogram equalization processing; Performing weighted fusion on the at least two background layers that have been segmented into blocks and processed with histogram equalization to obtain a target background layer; Merging the detail layer and the target background layer to obtain the brightness component data after brightness stretching processing; The brightness component data after the brightness stretching process and the component data other than the brightness component data are merged to obtain a target image with overall dynamic brightness enhancement.

2. The image processing method according to claim 1, characterized in that: The image segmentation method includes a first image segmentation method for segmenting the background layer into N1*N1 blocks and a second image segmentation method for segmenting the background layer into N2*N2 blocks, and N1 and N2 are positive integers and N1 is less than N2; the at least two background layers after segmentation into blocks and histogram equalization processing include a first background layer and a second background layer; the weighted fusion of the at least two background layers after segmentation into blocks and histogram equalization processing to obtain a target background layer includes: Obtaining a first weight matrix of the first background layer and a second weight matrix of the second background layer, wherein each element in the first weight matrix represents a weight of a corresponding pixel point in the first background layer, and each element in the second weight matrix represents a weight of a corresponding pixel point in the second background layer; The first background layer and the second background layer are weightedly fused according to the first weight matrix and the second weight matrix to obtain a target background layer.

3. The image processing method according to claim 2, characterized in that: The obtaining of a first weight matrix of the first background layer and a second weight matrix of the second background layer includes: Displaying a setting interface including the first background layer and the second background layer to a user, wherein the setting interface includes a block selection option and a weight setting option; Acquire a user's selection operation on the block selection option and a setting operation on the weight setting option in the setting interface; A first weight matrix for the first background layer and a second weight matrix for the second background layer are generated according to the selection operation and the setting operation of the user.

4. The image processing method according to any one of claims 1 to 3, characterized in that: After dynamically adjusting the brightness of the image to be processed based on the brightness parameter to obtain a target image with overall dynamically improved brightness, the method further includes: Extracting a region image corresponding to a target object from the target image; Performing light effect enhancement processing on the area image corresponding to the target object to obtain the area image corresponding to the target object after the light effect enhancement processing; The regional image corresponding to the target object after the light effect enhancement processing is weightedly superimposed on the target image to obtain the target image after regional enhancement.

5. The image processing method according to claim 4, characterized in that: The target object includes a human face; performing light effect enhancement processing on the area image corresponding to the target object to obtain the area image corresponding to the target object after the light effect enhancement processing includes: Performing light effect enhancement from strong to weak on the area image corresponding to the face from the center to the edge to obtain the area image corresponding to the face after light effect enhancement processing; or, The regional image corresponding to the face is subjected to light effect enhancement from weak to strong from the edge to the center to obtain the regional image corresponding to the face after light effect enhancement processing.

6. An image processing device, characterized in that: include: A processor and a memory for storing a computer program that can be run on the processor, wherein when the processor runs the computer program, the image processing method according to any one of claims 1 to 5 is implemented.

7. A computer storage medium, characterized in that: A computer program is stored, and when the computer program is executed by a processor, the image processing method according to any one of claims 1 to 5 is implemented.

Images