Image processing method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application disclose an image processing method and device, electronic equipment and storage medium. The method comprises: determining a local region in an original image to be subjected to brightness adjustment, the local region being an image region with a brightness value lower than a first threshold value or a brightness value higher than a second threshold value; detecting a first region and a second region in the local region, the first region being an image region in the local region with a brightness value lower than a third threshold value or a brightness value higher than a fourth threshold value, and the second region being an image region in the local region other than the first region; performing brightness adjustment processing on the first region, and synthesizing the processed first region and the second region to obtain a modified image; and fusing the modified image and the original image to obtain a target image. The image processing method, device, electronic equipment and storage medium described above can better preserve image details in the process of brightness adjustment of the image, and make the image present a better imaging effect.

Description

Technical Field

[0001] This application relates to the field of imaging technology, specifically to an image processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] In modern society, the transmission of images (such as pictures and videos) has permeated all aspects of people's lives. With continuous social progress, people are conveying more and more information, and the demands on this information carrier—images—are also increasing. The presentation quality of an image plays a crucial role in conveying information, especially the impact of image brightness. Poor image brightness can negatively affect various aspects such as contrast, sharpness, noise, and saturation, severely impacting the image's presentation. Therefore, how to adjust image brightness to achieve better image quality has become a key technical issue of concern in the industry. Summary of the Invention

[0003] This application discloses an image processing method, apparatus, electronic device, and storage medium, which can better preserve image details and produce better imaging effects during the process of adjusting the brightness of an image.

[0004] This application discloses an image processing method, including:

[0005] Determine the local region in the original image to be adjusted for brightness, wherein the local region is an image region with a brightness value lower than a first threshold or a brightness value higher than a second threshold;

[0006] The first region and the second region in the local region are detected. The first region is the image region in the local region whose brightness value is lower than a third threshold or whose brightness value is higher than a fourth threshold. The second region is the image region in the local region other than the first region.

[0007] The brightness of the first region is adjusted, and the processed first region is combined with the second region to obtain the modified image;

[0008] The modified image is fused with the original image to obtain the target image.

[0009] In one embodiment, determining the local region in the original image to be adjusted for brightness includes:

[0010] The brightness value distribution of each pixel in the original image is statistically analyzed to generate a first brightness histogram corresponding to the original image;

[0011] The local region in the original image to be adjusted for brightness is determined based on the first brightness histogram.

[0012] In one embodiment, the local region includes dark areas and / or bright areas; determining the local region in the original image to be brightness adjusted based on the first brightness histogram includes:

[0013] In the original image, the image region corresponding to the pixels distributed in the first low-brightness interval of the first brightness histogram is defined as a dark area, wherein the brightness value of the pixels distributed in the first low-brightness interval is lower than that of the pixels not distributed in the first low-brightness interval, and the number of pixels in the first low-brightness interval accounts for a first percentage of the original image, and the first threshold is the maximum brightness value of the first low-brightness interval; and / or,

[0014] In the original image, the image region corresponding to the pixels distributed in the first bright interval of the first brightness histogram is determined as the bright region, wherein the brightness value of the pixels distributed in the first bright interval is higher than that of the pixels not distributed in the first bright interval, and the number of pixels in the first bright interval accounts for a second percentage of the original image, and the second threshold is the minimum brightness value of the first bright interval.

[0015] In one embodiment, prior to detecting the first and second regions within the local region, the method includes:

[0016] Extract the local region from the original image to obtain a local image;

[0017] The detection of the first and second regions within the local region includes:

[0018] The brightness value distribution of each pixel in the local image is statistically analyzed to generate a second brightness histogram corresponding to the local image;

[0019] The first region and the second region in the local image are distinguished based on the second brightness histogram.

[0020] In one embodiment, the local image includes a dark area image and / or a bright area image; distinguishing the first region and the second region in the local image based on the second brightness histogram includes:

[0021] In the dark area image, the image region corresponding to the pixels distributed in the second low-brightness interval of the second brightness histogram corresponding to the dark area image is defined as the first region, and the other image regions in the dark area image besides the first region are defined as the second region. The brightness values ​​of pixels distributed in the second low-brightness interval are lower than those of pixels not distributed in the second low-brightness interval, and the number of pixels in the second low-brightness interval accounts for a third percentage of the dark area image. The third threshold is the maximum brightness value of the second low-brightness interval; and / or,

[0022] In the bright area image, the image region corresponding to the pixels distributed in the second bright interval of the second brightness histogram corresponding to the bright area image is determined as the first region, and the other image regions in the bright area image other than the first region are determined as the second region. The brightness value of the pixels distributed in the second bright interval is higher than that of the pixels not distributed in the second bright interval. The number of pixels in the second bright interval accounts for a fourth percentage of the bright area image, and the fourth threshold is the minimum brightness value of the second bright interval.

[0023] In one embodiment, the step of performing brightness adjustment processing on the first region and merging the processed first region with the second region to obtain a modified image includes:

[0024] The brightness of the first region is adjusted to obtain the processed first region;

[0025] Based on the unprocessed first region and the original image, determine the adjustment parameters corresponding to the second region;

[0026] The brightness of the second region is adjusted according to the adjustment parameters, and the processed second region is combined with the processed first region to obtain the modified image.

[0027] In one embodiment, determining the adjustment parameters corresponding to the second region based on the processed first region and the original image includes:

[0028] Calculate the brightness difference between the processed first region and the original image;

[0029] The adjustment parameters corresponding to the second region are determined based on the brightness difference.

[0030] This application discloses an image processing apparatus, including:

[0031] The region determination module is used to determine the local region in the original image to be adjusted for brightness, wherein the local region is an image region with a brightness value lower than a first threshold or a brightness value higher than a second threshold;

[0032] The detection module is used to detect a first region and a second region in the local region. The first region is an image region in the local region whose brightness value is lower than a third threshold or whose brightness value is higher than a fourth threshold. The second region is an image region in the local region other than the first region.

[0033] An adjustment module is used to perform brightness adjustment processing on the first region and combine the processed first region with the second region to obtain a modified image;

[0034] The fusion module is used to fuse the modified image with the original image to obtain the target image.

[0035] This application discloses an electronic device, including a memory and a processor. The memory stores a computer program, and when the computer program is executed by the processor, the processor causes the processor to implement the method described in any of the above embodiments.

[0036] This application discloses a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the method described in any of the above embodiments.

[0037] The image processing method, apparatus, electronic device, and storage medium disclosed in this application identify a local region in the original image to be adjusted for brightness. This local region is an image region whose brightness value is below a first threshold or above a second threshold. A first region and a second region are detected within this local region. The first region is an image region in the local region whose brightness value is below a third threshold or above a fourth threshold, and the second region is an image region in the local region excluding the first region. Brightness adjustment processing is performed on the first region, and the processed first region is combined with the second region to obtain a modified image. Then, the modified image is fused with the original image to obtain a target image. In this application embodiment, the local region to be adjusted for brightness in the original image is further distinguished into a first region and a second region, and the processed first region is combined with the second region to obtain a modified image. This allows for better preservation of image details during the brightness adjustment process, resulting in a better imaging effect. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 This is a block diagram of an image processing circuit in one embodiment;

[0040] Figure 2 This is a flowchart of an image processing method in one embodiment;

[0041] Figure 3 A flowchart of an image processing method in another embodiment;

[0042] Figure 4 This is a flowchart illustrating the process of obtaining a modified image in one embodiment;

[0043] Figure 5 This is a block diagram of an image processing apparatus in one embodiment;

[0044] Figure 6 This is a structural block diagram of an electronic device in one embodiment. Detailed Implementation

[0045] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0046] It should be noted that the terms "comprising" and "having," and any variations thereof, in the embodiments and accompanying drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the steps or units listed, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0047] It is understood that the terms "first," "second," etc., used in this application may be used herein to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish one element from another. For example, without departing from the scope of this application, a first region may be referred to as a second region, and similarly, a second region may be referred to as a first region. Both the first region and the second region are image regions within local regions, but they are not the same image region. The term "multiple" as used in this application refers to two or more.

[0048] This application provides an electronic device, which may include, but is not limited to, mobile phones, smart wearable devices, tablet computers, PCs (Personal Computers), vehicle terminals, digital cameras, etc., and this application does not limit the scope of the devices. The electronic device includes an image processing circuit, which can be implemented using hardware and / or software components and may include various processing units that define an ISP (Image Signal Processing) pipeline. Figure 1 This is a block diagram of an image processing circuit in one embodiment. For ease of explanation, Figure 1 Only aspects of the image processing techniques relevant to embodiments of this application are shown.

[0049] like Figure 1As shown, the image processing circuit includes an ISP processor 140 and a control logic unit 150. Image data captured by the imaging device 110 is first processed by the ISP processor 140, which analyzes the image data to capture image statistics that can be used to determine one or more control parameters of the imaging device 110. The imaging device 110 may include one or more lenses 112 and an image sensor 114. The image sensor 114 may include a color filter array (such as a Bayer filter), and can acquire light intensity and wavelength information captured by each imaging pixel, providing a set of raw image data that can be processed by the ISP processor 140. An attitude sensor 120 (such as a three-axis gyroscope, Hall sensor, accelerometer, etc.) can provide the acquired image processing parameters (such as image stabilization parameters) to the ISP processor 140 based on the attitude sensor 120 interface type. The attitude sensor 120 interface can be an SMIA (Standard Mobile Imaging Architecture) interface, other serial or parallel camera interfaces, or a combination of the above interfaces.

[0050] It should be noted that, although Figure 1 Only one imaging device 110 is shown in the illustration, but in this embodiment, at least two imaging devices 110 may be included. Each imaging device 110 may correspond to one image sensor 114, or multiple imaging devices 110 may correspond to one image sensor 114; this is not limited here. The operation of each imaging device 110 can be referred to the description above.

[0051] In addition, image sensor 114 can also send raw image data to attitude sensor 120. Attitude sensor 120 can provide raw image data to ISP processor 140 based on attitude sensor 120 interface type, or attitude sensor 120 can store raw image data in image memory 130.

[0052] The ISP processor 140 processes raw image data pixel by pixel in various formats. For example, each image pixel may have a bit depth of 8, 10, 12, or 14 bits. The ISP processor 140 may perform one or more image processing operations on the raw image data and collect statistical information about the image data. The image processing operations may be performed with the same or different bit depth precision.

[0053] The ISP processor 140 can also receive image data from the image memory 130. For example, the attitude sensor 120 interface sends raw image data to the image memory 130, and the raw image data in the image memory 130 is then provided to the ISP processor 140 for processing. The image memory 130 may be part of a memory device, a storage device, or a separate dedicated memory within an electronic device, and may include DMA (Direct Memory Access) features.

[0054] Upon receiving raw image data from image sensor 114 interface, attitude sensor 120 interface, or image memory 130, ISP processor 140 may perform one or more image processing operations, such as temporal filtering. The processed image data may be sent to image memory 130 for further processing before display. ISP processor 140 receives processed data from image memory 130 and performs image data processing on the processed data in the raw domain and in the RGB and YCbCr color spaces. The processed image data may be output to display 160 for user viewing and / or further processed by a graphics engine or GPU (Graphics Processing Unit). Furthermore, the output of ISP processor 140 may also be sent to image memory 130, and display 160 may read image data from image memory 130. In one embodiment, image memory 130 may be configured to implement one or more frame buffers.

[0055] The statistical data determined by the ISP processor 140 can be sent to the control logic unit 150. For example, the statistical data may include image sensor 114 statistics such as gyroscope vibration frequency, automatic exposure, automatic white balance, automatic focus, flicker detection, black level compensation, and lens 112 shading correction. The control logic unit 150 may include a processor and / or microcontroller executing one or more routines (such as firmware) that determine control parameters for the imaging device 110 and the ISP processor 140 based on the received statistical data. For example, the control parameters for the imaging device 110 may include attitude sensor 120 control parameters (e.g., gain, integral time for exposure control, image stabilization parameters, etc.), camera flash control parameters, camera image stabilization shift parameters, lens 112 control parameters (e.g., focal length for focusing or zooming), or combinations of these parameters. The ISP control parameters may include gain levels and color correction matrices for automatic white balance and color adjustment (e.g., during RGB processing), and lens 112 shading correction parameters.

[0056] For example, combined Figure 1The image processing circuit described herein illustrates the image processing method, apparatus, electronic device, and computer-readable storage medium provided in the embodiments of this application. The ISP processor 140 can receive an original image sent by the imaging device 110 or retrieve an original image from the image memory 130. The ISP processor 140 can determine a local region in the original image to be adjusted for brightness, wherein the local region is an image region with a brightness value lower than a first threshold or a brightness value higher than a second threshold. The ISP processor 140 can detect a first region and a second region within the local region, wherein the first region is an image region in the local region with a brightness value lower than a third threshold or a brightness value higher than a fourth threshold, and the second region is an image region in the local region other than the first region. The ISP processor 140 can perform brightness adjustment processing on the first region, and then combine the processed first region with the second region to obtain a modified image, and then fuse the modified image with the original image to obtain a target image. Optionally, the ISP processor 140 can send the target image to the image memory 130 for storage, or send the target image to the display 160 for display.

[0057] like Figure 2 As shown, in one embodiment, an image processing method is provided, which can be applied to the above-described electronic device. The method may include the following steps:

[0058] Step 210: Determine the local area in the original image to be adjusted for brightness. This local area is an image region where the brightness value is lower than the first threshold or higher than the second threshold.

[0059] In some embodiments, the original image may be an image captured in real time by an electronic device via a camera, or an image pre-stored in a memory, from which the original image can be read. After acquiring the original image, the electronic device can determine the local area in the original image that needs brightness adjustment. This local area may include dark areas and / or bright areas, wherein a dark area refers to an image area that is too dark and needs to have its brightness increased, and the dark area may be an image area in the original image with a brightness value lower than a first threshold; a bright area refers to an image area that is too bright and needs to have its brightness reduced, and the bright area may be an image area in the original image with a brightness value higher than a second threshold.

[0060] The first threshold mentioned above can be smaller than the second threshold. Optionally, the first and second thresholds can be preset fixed brightness values, which can be determined through multiple experiments. Alternatively, the first and second thresholds can be brightness values ​​that vary based on the overall brightness information of the original image. This overall brightness information reflects the brightness of the original image, and different overall brightness information of the original image corresponds to different first and second thresholds. For example, when the overall brightness of the original image is bright, the corresponding first and second thresholds can be larger; when the overall brightness of the original image is dark, the corresponding first and second thresholds can be smaller. Dynamically adjusting the first and second thresholds based on the overall brightness information of different original images can make the determined local areas to be adjusted for brightness more accurate, further improving the image processing effect.

[0061] Step 220: Detect the first region and the second region in the local region. The first region is the image region in the local region whose brightness value is lower than the third threshold or higher than the fourth threshold. The second region is the image region in the local region other than the first region.

[0062] An electronic device can extract a local region to be adjusted for brightness from an original image to obtain a local image, and detect and distinguish a first region and a second region within this local image. The first region refers to the image area in the local image that requires brightness adjustment, while the second region refers to the image area in the local image that does not require brightness adjustment. The first region can be an image area in the local image whose brightness value is below a third threshold or above a fourth threshold, and the second region is any image area in the local image other than the first region.

[0063] In some embodiments, the local region may include dark areas and / or bright areas. The first region of the local image corresponding to the dark area may refer to an image area within the dark area that needs to be brightened, and the second region may refer to an image area within the dark area that does not need to be brightened. Further, the first region of the local image corresponding to the dark area may be an image area in the local image whose brightness value is lower than a third threshold. Similarly, the first region of the local image corresponding to the bright area may refer to an image area within the bright area whose brightness needs to be reduced, and the second region may refer to an image area within the bright area whose brightness does not need to be reduced. Further, the first region of the local image corresponding to the bright area may be an image area in the local image whose brightness value is higher than a fourth threshold.

[0064] The aforementioned third threshold can be less than the fourth threshold. Further, the third threshold can be less than the first threshold, and the fourth threshold can be greater than the second threshold. Optionally, the third and fourth thresholds can be pre-set fixed brightness values, which can be determined through multiple experiments. Alternatively, the third and fourth thresholds can be brightness values ​​that vary based on the overall brightness information of the local image; different overall brightness information in the local image can correspond to different third and / or fourth thresholds.

[0065] Step 230: Adjust the brightness of the first region and combine the processed first region with the second region to obtain the modified image.

[0066] In some embodiments, if a local area is a bright area, the brightness value of the first region within that local area can be reduced; if a local area is a dark area, the brightness value of the first region within that local area can be increased. Optionally, the degree of brightness adjustment corresponding to the first region can be determined based on the brightness information of the first region. For example, if a local area is a bright area and the brightness of the first region within that local area is high (i.e., the first region is excessively bright), the corresponding degree of brightness adjustment can be large (i.e., the brightness reduction is large); if the brightness of the first region within that local area is low (i.e., the first region is less excessively bright), the corresponding degree of brightness adjustment can be small (i.e., the brightness reduction is small). Similarly, if a local area is a dark area and the brightness of the first region within that local area is low (i.e., the first region is excessively dark), the corresponding degree of brightness adjustment can be large (i.e., the brightness increase is large); if the brightness of the first region within that local area is high (i.e., the first region is less excessively dark), the corresponding degree of brightness adjustment can be small (i.e., the brightness increase is small).

[0067] Optionally, the brightness adjustment level of the first region can be determined based on the overall brightness information of the original image, so that the brightness information of the processed first region matches the overall brightness information of the original image. As a specific implementation, the difference between the average brightness of the first region and the average brightness of the original image can be calculated, and the brightness adjustment level of the first region can be determined based on this difference. The difference and the brightness adjustment level can be positively correlated; a larger difference indicates a larger brightness deviation between the first region and the original image, thus requiring a larger brightness adjustment level. Conversely, a smaller difference indicates a smaller brightness deviation, thus requiring a smaller brightness adjustment level. By flexibly adjusting the brightness adjustment level of the first region and performing brightness adjustment processing on the first region based on this adjustment level, the brightness adjustment effect of the first region can be improved, further enhancing the image quality of the subsequently obtained target image.

[0068] In one implementation, the second region may not be processed. The electronic device can directly combine the processed first region with the second region to obtain a modified image. By retaining some image areas in the bright and / or dark areas, image details can be better preserved, avoiding the loss of details after image processing, and further improving the image quality of the subsequent target image.

[0069] As another implementation, considering that the brightness of the second region may change during image compositing—for example, the second region in a bright area may become brighter, or the second region in a dark area may become darker—the electronic device can also perform brightness adjustment processing on the second region before compositing it with the processed first region to obtain a modified image. The degree of brightness adjustment for the second region can be less than that for the first region, preserving image details while making the subsequent target image more natural and further improving the image quality of the target image.

[0070] Step 240: Fuse the modified image with the original image to obtain the target image.

[0071] As one implementation method, a first weight value corresponding to the modified image and a second weight value corresponding to the original image can be set, and the sum of the first and second weight values ​​can be 1. The modified image and the original image can be fused according to these first and second weight values ​​to obtain the target image. For pixels at the same pixel location in both the modified and original images, the pixel value of the pixel in the modified image can be multiplied by the first weight value, and the pixel value of the pixel in the original image can be multiplied by the second weight value. The two multiplication results are then added together to obtain the pixel value corresponding to that pixel location in the target image. Using this fusion method results in a more natural effect and better imaging quality in the target image.

[0072] As another implementation, the image region in the original image that matches the modified image (i.e., the local region to be adjusted for brightness as determined above) can be directly replaced with the modified image to obtain the target image.

[0073] In this embodiment, the local area to be brightness adjusted in the original image is further divided into a first region and a second region, and the first region after brightness adjustment is combined with the second region to obtain a modified image. This can better preserve image details during the brightness adjustment process, resulting in a better imaging effect.

[0074] like Figure 3 As shown, in another embodiment, an image processing method is provided, which can be applied to the above-described electronic device. The method may include the following steps:

[0075] Step 302: Statistically analyze the brightness value distribution of each pixel in the original image to generate the first brightness histogram corresponding to the original image.

[0076] The first brightness histogram can be used to reflect the overall brightness information of the original image. The first brightness histogram can show the number of pixels in the original image at each brightness value, that is, the distribution of pixels in the original image at each brightness value.

[0077] Step 304: Determine the local region in the original image to be adjusted for brightness based on the first brightness histogram. This local region is an image region where the brightness value is lower than the first threshold or higher than the second threshold.

[0078] The electronic device can determine a first threshold and a second threshold based on a first brightness histogram, and identify the image regions in the original image whose brightness values ​​are lower than the first threshold or higher than the second threshold as local regions to be adjusted for brightness.

[0079] In some embodiments, the local region may include dark areas and / or bright areas. The image region corresponding to pixels distributed in a first low-brightness interval of a first brightness histogram in the original image can be defined as a dark area, wherein the brightness value of pixels distributed in the first low-brightness interval is lower than that of pixels not distributed in the first low-brightness interval, and the number of pixels in the first low-brightness interval accounts for a first percentage of the original image, with a first threshold being the maximum brightness value of the first low-brightness interval.

[0080] The electronic device can determine a first low-brightness region based on a first brightness histogram. Pixels distributed in the first low-brightness region are the pixels with the lowest brightness values ​​in the original image, representing a first percentage. For example, the pixels distributed in the first low-brightness region can be the 10% of pixels with the lowest brightness values ​​in the original image (brightness values ​​less than the remaining 90%). The first threshold can be the maximum brightness value of the first low-brightness region, and the number of pixels with brightness values ​​lower than the first threshold in the original image represents a first percentage. Based on the first brightness histogram, the image area corresponding to the first percentage of pixels with the lowest brightness values ​​in the original image can be determined as a dark area. For example, the image area corresponding to the 10% of pixels with the lowest brightness values ​​in the original image can be determined as a dark area.

[0081] In some embodiments, the image region corresponding to the pixels distributed in the first bright region of the first brightness histogram in the original image can be determined as a bright region, wherein the brightness value of the pixels distributed in the first bright region is higher than that of the pixels not distributed in the first bright region, and the number of pixels in the first bright region accounts for a second percentage of the original image, and the second threshold is the minimum brightness value of the first bright region.

[0082] The electronic device can determine a first bright region based on a first brightness histogram. Pixels distributed within the first bright region represent the second percentage of pixels with the highest brightness values ​​in the original image. For example, the pixels distributed within the first bright region can be the top 10% of pixels with the highest brightness values ​​in the original image (brightness values ​​greater than the remaining 90% of pixels). The second threshold can be the minimum brightness value of the first bright region. The number of pixels with brightness values ​​higher than the second threshold in the original image accounts for the first percentage. Based on the first brightness histogram, the image region corresponding to the second percentage of pixels with the highest brightness values ​​in the original image can be determined as a bright region. For example, the image region corresponding to the top 10% of pixels with the highest brightness values ​​in the original image can be determined as a bright region.

[0083] By using the first brightness histogram corresponding to the original image, the local areas in the original image that need brightness adjustment can be determined more accurately, thus improving the accuracy of image processing.

[0084] Step 306: Extract local regions from the original image to obtain a local image.

[0085] A local region may include dark areas and / or bright areas, and a local image may include a dark area image and / or a bright area image. Dark areas can be extracted from the original image to obtain a dark area image, and bright areas can be extracted from the original image to obtain a bright area image.

[0086] Step 308: Statistically analyze the brightness value distribution of each pixel in the local image to generate a second brightness histogram corresponding to the local image.

[0087] The second brightness histogram can be used to reflect the overall brightness information of a local image. The second brightness histogram can show the number of pixels in the local image at each brightness value, that is, it shows the distribution of pixels in the local image at each brightness value.

[0088] Step 310: Based on the second brightness histogram, distinguish the first region and the second region in the local image. The first region is the image region in the local image whose brightness value is lower than the third threshold or higher than the fourth threshold. The second region is the image region in the local image other than the first region.

[0089] The electronic device can determine a third threshold based on a second luminance histogram corresponding to the dark area image, and distinguish between a first region and a second region of the dark area image based on the third threshold. Similarly, it can determine a fourth threshold based on a second luminance histogram corresponding to the bright area image, and distinguish between a first region and a second region of the bright area image based on the fourth threshold.

[0090] In some embodiments, the image region corresponding to the pixels distributed in the second low-brightness interval of the second brightness histogram corresponding to the dark area image can be defined as the first region, and the other image regions in the dark area image other than the first region can be defined as the second region. The brightness value of the pixels distributed in the second low-brightness interval is lower than that of the pixels not distributed in the second low-brightness interval, the number of pixels in the second low-brightness interval accounts for a third percentage of the dark area image, and the third threshold is the maximum brightness value of the second low-brightness interval.

[0091] The electronic device can determine a second low-brightness region based on a second brightness histogram corresponding to the dark area image. Pixels distributed in the second low-brightness region represent the third percentage of pixels with the lowest brightness values ​​in the dark area image. For example, the pixels distributed in the second low-brightness region can be the 60% of pixels with the lowest brightness values ​​in the dark area image (brightness values ​​less than the remaining 40% of pixels). This third threshold can be the maximum brightness value of the second low-brightness region, and the number of pixels with brightness values ​​lower than the third threshold in the dark area image accounts for the third percentage. Based on the second brightness histogram corresponding to the dark area image, the image region corresponding to the third percentage of pixels with the lowest brightness values ​​in the dark area image can be determined as the first region, and the remaining region can be determined as the second region. For example, the image region corresponding to the 60% of pixels with the lowest brightness values ​​in the dark area image can be determined as the first region, and the image region corresponding to the remaining 40% of pixels can be determined as the second region.

[0092] In some embodiments, the image region corresponding to the pixels distributed in the second high-brightness interval of the second brightness histogram corresponding to the bright area image can be defined as the first region, and the other image regions in the bright area image other than the first region can be defined as the second region. The brightness value of the pixels distributed in the second high-brightness interval is higher than that of the pixels not distributed in the second high-brightness interval, the number of pixels in the second high-brightness interval accounts for a fourth percentage of the bright area image, and the fourth threshold is the minimum brightness value of the second high-brightness interval.

[0093] The electronic device can determine a second high-brightness region based on a second brightness histogram corresponding to the bright area image. Pixels distributed within this second high-brightness region represent the fourth percentage of pixels with the highest brightness values ​​in the bright area image. For example, the pixels distributed within the second high-brightness region can be the 45% of pixels with the highest brightness values ​​in the bright area image (brightness values ​​greater than the remaining 55% of pixels). This fourth threshold can be the minimum brightness value of the second high-brightness region, and the number of pixels with brightness values ​​higher than the fourth threshold in the bright area image accounts for the fourth percentage. Based on the second brightness histogram corresponding to the bright area image, the image region corresponding to the fourth percentage of pixels with the highest brightness values ​​in the bright area image can be determined as the first region, and the remaining regions can be determined as the second region. For example, the image region corresponding to the 45% of pixels with the highest brightness values ​​in the bright area image can be determined as the first region, and the image region corresponding to the remaining 55% of pixels can be determined as the second region.

[0094] By using the second brightness histogram corresponding to the local image, the first and second regions in the local image can be distinguished more accurately, thus improving the accuracy of image processing.

[0095] Step 312: Adjust the brightness of the first region and combine the processed first region with the second region to obtain the modified image.

[0096] like Figure 4 As shown, in one embodiment, the step of adjusting the brightness of the first region and merging the processed first region with the second region to obtain the modified image may include steps 402 to 404:

[0097] Step 402: Perform brightness adjustment processing on the first region to obtain the processed first region.

[0098] The description of step 402 can be found in the relevant descriptions in the above embodiments, and will not be repeated here.

[0099] Step 404: Determine the adjustment parameters corresponding to the second region based on the unprocessed first region and the original image.

[0100] In this embodiment of the application, since the brightness of the second region may change when the first region and the second region are combined, or when the modified image is combined with the original image, for example, the second region in the bright area becomes brighter, or the second region in the dark area becomes darker, the brightness of the second region of the local image can be adjusted, and then the processed second region can be combined with the processed first region to obtain the modified image.

[0101] The adjustment parameters corresponding to the second region can be determined based on the unprocessed first region and the original image. These adjustment parameters can be used to characterize the degree of brightness adjustment. The degree of brightness adjustment corresponding to the second region is less than that corresponding to the first region. Further, the brightness difference between the unprocessed first region and the original image can be calculated, and the adjustment parameters corresponding to the second region can be determined based on this brightness difference. Optionally, the brightness difference and the degree of brightness adjustment can be positively correlated; the larger the brightness difference, the larger the corresponding degree of brightness adjustment, and an adjustment parameter corresponding to a larger degree of brightness adjustment can be set; conversely, the smaller the brightness difference, the smaller the corresponding degree of brightness adjustment, and an adjustment parameter corresponding to a smaller degree of brightness adjustment can be set.

[0102] In some embodiments, the brightness difference may be the difference between the average brightness of the unprocessed first region and the average brightness of the original image, or the difference between the median brightness of the unprocessed first region and the median brightness of the original image, or the difference between the maximum brightness of the unprocessed first region and the maximum brightness of the original image, etc., without limitation.

[0103] Step 406: Adjust the brightness of the second region according to the adjustment parameters, and then combine the processed second region with the processed first region to obtain the modified image.

[0104] The electronic device can adjust the brightness of the second region according to the determined adjustment parameters, and then combine the processed second region with the processed first region to obtain the modified image.

[0105] It should be noted that the specific processing method for brightness adjustment involved in the embodiments of this application is not limited. For example, histogram equalization, Retinex algorithm, dehazing algorithm, gamma correction, etc., can be used for brightness adjustment. Taking gamma correction as an example, the adjustment parameter mentioned above can be the gamma coefficient in gamma correction.

[0106] Step 314: Fuse the modified image with the original image to obtain the target image.

[0107] The description of step 314 can be found in the relevant descriptions in the above embodiments, and will not be repeated here.

[0108] In this embodiment of the application, the local image to be processed for brightness can be accurately extracted from the original image by using the brightness histogram, and the first region and the second region in the local image can be further distinguished. Different degrees of brightness adjustment processing can be performed on the first region and the second region respectively, which can improve the accuracy of brightness adjustment and further improve the image effect of the target image.

[0109] like Figure 5As shown, in one embodiment, an image processing apparatus 500 is provided, which can be applied to the above-mentioned electronic device. The image processing apparatus 500 may include a region determination module 510, a detection module 520, an adjustment module 530, and a fusion module 540.

[0110] The region determination module 510 is used to determine the local region in the original image to be adjusted for brightness. The local region is an image region where the brightness value is lower than a first threshold or higher than a second threshold.

[0111] The detection module 520 is used to detect a first region and a second region in the local area. The first region is the image region in the local area whose brightness value is lower than a third threshold or whose brightness value is higher than a fourth threshold. The second region is the image region in the local area other than the first region.

[0112] The adjustment module 530 is used to perform brightness adjustment processing on the first region and combine the processed first region with the second region to obtain the modified image.

[0113] The fusion module 540 is used to fuse the modified image with the original image to obtain the target image.

[0114] In this embodiment, the local area to be brightness adjusted in the original image is further divided into a first region and a second region, and the first region after brightness adjustment is combined with the second region to obtain a modified image. This can better preserve image details during the brightness adjustment process, resulting in a better imaging effect.

[0115] In one embodiment, the region determination module 510 includes a first generation unit and a determination unit.

[0116] The first generation unit is used to statistically analyze the brightness value distribution of each pixel in the original image to generate the first brightness histogram corresponding to the original image.

[0117] The determining unit is used to determine the local area in the original image to be adjusted in brightness based on the first brightness histogram.

[0118] In one embodiment, the local area includes dark areas and / or bright areas.

[0119] The determining unit is further configured to determine the image region corresponding to the pixels distributed in the first low-brightness interval of the first brightness histogram in the original image as a dark area, wherein the brightness value of the pixels distributed in the first low-brightness interval is lower than that of the pixels not distributed in the first low-brightness interval, and the number of pixels in the first low-brightness interval accounts for a first percentage of the original image, and the first threshold is the maximum brightness value of the first low-brightness interval; and / or, further configured to determine the image region corresponding to the pixels distributed in the first brightness interval of the first brightness histogram in the original image as a bright area, wherein the brightness value of the pixels distributed in the first brightness interval is higher than that of the pixels not distributed in the first brightness interval, and the number of pixels in the first brightness interval accounts for a second percentage of the original image, and the second threshold is the minimum brightness value of the first brightness interval.

[0120] In one embodiment, the image processing apparatus 500 described above further includes an extraction module.

[0121] The extraction module is used to extract local regions from the original image to obtain a local image.

[0122] The detection module 520 includes a second generation unit and a differentiation unit.

[0123] The second generation unit is used to statistically analyze the brightness value distribution of each pixel in the local image to generate a second brightness histogram corresponding to the local image.

[0124] The distinguishing unit is used to distinguish the first region and the second region in the local image based on the second brightness histogram.

[0125] In one embodiment, the local image includes a dark area image and / or a bright area image.

[0126] The distinguishing unit is further configured to define the image region corresponding to the pixels in the second low-brightness interval of the second brightness histogram corresponding to the dark area image as the first region, and define the other image regions in the dark area image other than the first region as the second region, wherein the brightness value of the pixels distributed in the second low-brightness interval is lower than that of the pixels not distributed in the second low-brightness interval, the number of pixels in the second low-brightness interval accounts for a third percentage of the dark area image, and the third threshold is the maximum brightness value of the second low-brightness interval; and / or, further configured to define the image region corresponding to the pixels in the second high-brightness interval of the second brightness histogram corresponding to the bright area image as the first region, and define the other image regions in the bright area image other than the first region as the second region, wherein the brightness value of the pixels distributed in the second high-brightness interval is higher than that of the pixels not distributed in the second high-brightness interval, the number of pixels in the second high-brightness interval accounts for a fourth percentage of the bright area image, and the fourth threshold is the minimum brightness value of the second high-brightness interval.

[0127] In one embodiment, the adjustment module 530 includes an adjustment unit and a synthesis unit.

[0128] The adjustment unit is used to perform brightness adjustment processing on the first area to obtain the processed first area.

[0129] The adjustment unit is also used to determine the adjustment parameters corresponding to the second region based on the unprocessed first region and the original image, and to perform brightness adjustment processing on the second region according to the adjustment parameters.

[0130] In one embodiment, the adjustment unit is further configured to calculate the brightness difference between the processed first region and the original image; and determine the adjustment parameters corresponding to the second region based on the brightness difference.

[0131] The compositing unit is used to combine the processed second region with the processed first region to obtain the modified image.

[0132] In this embodiment of the application, the local image to be processed for brightness can be accurately extracted from the original image by using the brightness histogram, and the first region and the second region in the local image can be further distinguished. Different degrees of brightness adjustment processing can be performed on the first region and the second region respectively, which can improve the accuracy of brightness adjustment and further improve the image effect of the target image.

[0133] In one embodiment, an electronic device is provided, which can be a head-mounted display device, and its internal structure diagram can be as follows: Figure 6 As shown, the electronic device includes a processor, memory, and network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements the aforementioned method for acquiring facial expressions applied to the first head-mounted display device. The display device of the electronic device can be lenses, etc., and the input device can be buttons, a trackball, or a touchpad mounted on the device's casing, or an external keyboard, touchpad, or gamepad, etc.

[0134] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the electronic device to which the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0135] In one embodiment, the facial expression acquisition device for a first head-mounted display device provided in this application can be implemented as a computer program, and the computer program can be implemented as follows: Figure 6 The device operates on the electronic device shown. The memory of the electronic device can store various program modules that make up the facial expression acquisition device applied to the first head-mounted display device, for example, Figure 5 The illustrated modules are region determination module 510, detection module 520, adjustment module 530, and fusion module 540. The computer program comprised of these modules causes the processor to execute the methods of the various embodiments described in this specification.

[0136] This application discloses a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the methods described in the above embodiments.

[0137] This application discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program can be executed by a processor to implement the methods described in the above embodiments.

[0138] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, ROM, etc.

[0139] Any references to memory, storage, databases, or other media used herein may include non-volatile and / or volatile memory. Suitable non-volatile memory may include ROM, Programmable ROM (PROM), Erasable PROM (EPROM), Electrically Erasable PROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which is used as an external cache. By way of illustration and not limitation, RAM may take many forms, such as Static RAM (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), Rambus DRAM (RDRAM), and Direct Rambus DRAM (DRDRAM).

[0140] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Those skilled in the art should also recognize that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application. It should be noted that "multiple" in this application includes "two or more".

[0141] In the various embodiments of this application, it should be understood that the sequence number of each process does not necessarily imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0142] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they can be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0143] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0144] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0145] The foregoing has provided a detailed description of an image processing method, apparatus, electronic device, and storage medium disclosed in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. An image processing method, characterized by, The method comprises the following steps: determining a local region in an original image to be subjected to brightness adjustment, the local region comprising a dark region and / or a bright region, the dark region being an image region with a brightness value lower than a first threshold value, and the bright region being an image region with a brightness value higher than a second threshold value; detecting a first region and a second region in the local region, the first region in the dark region being an image region with a brightness value lower than a third threshold value in the dark region, the second region in the dark region being an image region other than the first region in the dark region, the third threshold value being lower than the first threshold value; the first region in the bright region being an image region with a brightness value higher than a fourth threshold value, the second region in the bright region being an image region other than the first region in the bright region, the fourth threshold value being higher than the second threshold value; subjecting the first region to brightness adjustment processing, and synthesizing the processed first region and the second region to obtain a modified image; fusing the modified image and the original image to obtain a target image.

2. The method of claim 1, wherein, The method comprises the following steps: statistically analyzing the brightness value distribution of each pixel point in the original image to generate a first brightness histogram corresponding to the original image; determining a local region in the original image to be subjected to brightness adjustment according to the first brightness histogram.

3. The method of claim 2, wherein, The method comprises the following steps: determining, in the original image, an image region corresponding to a pixel point distributed in a first low-brightness interval of the first brightness histogram as a dark region, wherein the brightness value of the pixel point distributed in the first low-brightness interval is lower than that of a pixel point not distributed in the first low-brightness interval, the number of pixel points in the first low-brightness interval accounts for a first percentage of the original image, and the first threshold value is the maximum brightness value of the first low-brightness interval; and / or determining, in the original image, an image region corresponding to a pixel point distributed in a first high-brightness interval of the first brightness histogram as a bright region, wherein the brightness value of the pixel point distributed in the first high-brightness interval is higher than that of a pixel point not distributed in the first high-brightness interval, the number of pixel points in the first high-brightness interval accounts for a second percentage of the original image, and the second threshold value is the minimum brightness value of the first high-brightness interval.

4. The method of claim 1, wherein, Before the step of detecting the first region and the second region in the local region, the method comprises the following steps: extracting the local region from the original image to obtain a local image; The method comprises the following steps: statistically analyzing the brightness value distribution of each pixel point in the local image to generate a second brightness histogram corresponding to the local image; distinguishing the first region and the second region in the local image according to the second brightness histogram.

5. The method of claim 4, wherein, The local image comprises a dark region image and / or a bright region image; the method comprises the following steps: determine a first region in the dark region image as an image region corresponding to a pixel point distributed in a second low-brightness interval of a second brightness histogram corresponding to the dark region image, and determine a second region in the dark region image as an image region other than the first region, wherein a brightness value of the pixel point distributed in the second low-brightness interval is lower than a pixel point not distributed in the second low-brightness interval, a number of the pixel points in the second low-brightness interval accounts for a third percentage of the dark region image, and the third threshold value is a maximum brightness value of the second low-brightness interval; and / or determine a first region in the bright region image as an image region corresponding to a pixel point distributed in a second high-brightness interval of a second brightness histogram corresponding to the bright region image, and determine a second region in the bright region image as an image region other than the first region, wherein a brightness value of the pixel point distributed in the second high-brightness interval is higher than a pixel point not distributed in the second high-brightness interval, a number of the pixel points in the second high-brightness interval accounts for a fourth percentage of the bright region image, and the fourth threshold value is a minimum brightness value of the second high-brightness interval.

6. The method according to any one of claims 1 to 5, characterized in that, the first region is subjected to a brightness adjustment process, and the processed first region is combined with the second region to obtain a modified image, including: the first region is subjected to a brightness adjustment process to obtain a processed first region; adjustment parameters corresponding to the second region are determined according to the unprocessed first region and the original image; the second region is subjected to a brightness adjustment process according to the adjustment parameters, and the processed second region is combined with the processed first region to obtain a modified image.

7. The method of claim 6, wherein, the adjustment parameters corresponding to the second region are determined according to the processed first region and the original image, including: a brightness difference value between the processed first region and the original image is calculated; the adjustment parameters corresponding to the second region are determined according to the brightness difference value.

8. An image processing apparatus characterized by comprising: including: a region determination module configured to determine a local region in an original image to be subjected to a brightness adjustment, the local region including a dark region and / or a bright region, the dark region being an image region with a brightness value lower than a first threshold value, and the bright region being an image region with a brightness value higher than a second threshold value; a detection module configured to detect a first region and a second region in the local region, the first region in the dark region being an image region with a brightness value lower than a third threshold value, the second region in the dark region being an image region other than the first region in the dark region, and the third threshold value being lower than the first threshold value; the first region in the bright region being an image region with a brightness value higher than a fourth threshold value, the second region in the bright region being an image region other than the first region in the bright region, and the fourth threshold value being higher than the second threshold value; an adjustment module configured to subject the first region to a brightness adjustment process, and combine the processed first region with the second region to obtain a modified image; a fusion module configured to fuse the modified image with the original image to obtain a target image.

9. An electronic device, comprising: A computer program product comprising a memory having stored therein a computer program, the computer program being executable by a processor to implement the method of any one of claims 1-7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executable by a processor to implement the method of any one of claims 1-7.

Images