Image processing method, electronic device, readable storage medium and program product

Abstract

The application provides an image processing method, an electronic device, a readable storage medium and a program product. The image processing method comprises: acquiring a to-be-processed image, and determining a current brightness of a target scene presented in the to-be-processed image; inputting the to-be-processed image into a pre-trained target model to obtain a predicted brightness of the target scene output by the target model, the target model being used for predicting the brightness of a scene shown in an image; in a case where the current brightness is less than the predicted brightness, brightening the brightness of the target scene according to the current brightness and the predicted brightness to obtain a processed target image; and in a case where the current brightness is greater than the predicted brightness, darkening the brightness of the target scene according to the current brightness and the predicted brightness to obtain a processed target image. Through the scheme, the brightness of the processed image is more consistent with the perception expectation of the human eye for the scene, and the overexposure or underexposure problem caused by traditional fixed parameter adjustment is avoided.

Description

Technical Field

[0001] This application relates to the field of image processing, and more specifically, to an image processing method, an electronic device, a readable storage medium, and a program product. Background Technology

[0002] In current terminal device photography systems, automatic exposure (AE) algorithms are typically used to determine the final image brightness. In the AE algorithm, it is assumed that the average reflectance of the shooting scene is approximately 18% (neutral gray), and based on this, exposure parameters are adjusted to make the overall image brightness statistics approach this target gray level. This assumption performs well in many scenes (such as mixed-tone environments under natural outdoor light), producing visually balanced images.

[0003] However, when the reflectance of the subject or scene deviates significantly from the average reflectance, it becomes impossible to accurately distinguish between white-plus and black-minus scenes. Consequently, the resulting image has a poor visual quality; for example, white-plus scenes appear darker and lighter colors appear grayish, while black-minus scenes appear brighter and the blacks are not pure enough or appear grayish. Summary of the Invention

[0004] This application addresses some shortcomings by providing an image processing method, electronic device, readable storage medium, and program product. In this image processing method, an image to be processed is acquired, the current brightness of the target scene is determined, a pre-trained target model is used to predict the predicted brightness of the target scene, and the actual brightness is compared with the ideal brightness predicted by the model to adaptively perform brightening or darkening operations. This results in a processed image brightness that better matches the human eye's perception of the scene, avoiding overexposure or underexposure problems caused by traditional fixed parameter adjustments.

[0005] In a first aspect, an image processing method is provided, comprising: acquiring an image to be processed and determining the current brightness of a target scene presented in the image to be processed; inputting the image to be processed into a pre-trained target model to obtain the predicted brightness of the target scene output by the target model, wherein the target model is used to predict the brightness of the scene shown in the image; when the current brightness is less than the predicted brightness, brightening the brightness of the target scene according to the current brightness and the predicted brightness to obtain a processed target image; when the current brightness is greater than the predicted brightness, darkening the brightness of the target scene according to the current brightness and the predicted brightness to obtain a processed target image.

[0006] This scheme utilizes a pre-trained target model to intelligently predict the brightness of the target scene in the image to be processed, and uses the predicted brightness as an ideal brightness reference value. In this image processing method, the current brightness of the target scene is determined by acquiring the image to be processed, and the pre-trained target model is used to predict the brightness of the target scene. The actual brightness is compared with the ideal brightness predicted by the model to adaptively perform brightening or darkening operations. This makes the brightness of the processed image more consistent with the human eye's perception of the scene, avoiding overexposure or underexposure problems caused by traditional fixed parameter adjustments. Furthermore, when the actual current brightness is lower than the predicted brightness, automatic brightening is performed; when the actual current brightness is higher than the predicted brightness, automatic darkening is performed, allowing the image brightness to adaptively adjust according to the scene content. Ultimately, this avoids the poor lighting effects caused by setting fixed parameters in traditional methods, effectively improving the intelligence of image processing and the accuracy of brightness adjustment.

[0007] In conjunction with the first aspect, in a possible implementation of the first aspect, the current brightness includes the current average brightness, and the predicted brightness includes the predicted average brightness. The method further includes: determining a brightness difference ratio based on the ratio of the predicted average brightness to the current average brightness; brightening the target scene based on the current brightness and the predicted brightness when the current brightness is less than the predicted brightness, including: brightening the target scene based on the brightness difference ratio when the brightness difference ratio is greater than 1; and darkening the target scene based on the current brightness and the predicted brightness when the current brightness is greater than the predicted brightness, including: darkening the target scene based on the brightness difference ratio when the brightness difference ratio is less than 1.

[0008] This scheme uses the ratio of the predicted average brightness to the current average brightness as the brightness difference ratio. When the ratio is greater than 1, it indicates that the image is too dark and needs to be brightened; when the ratio is less than 1, it indicates that the image is too bright and needs to be darkened. This makes the direction and magnitude of brightness adjustment more intuitive and precise, avoids the problem of mismatched adjustment intensity, and improves the robustness and consistency of brightness adjustment.

[0009] In conjunction with the first aspect, in a possible implementation of the first aspect, brightening the target scene based on the brightness difference ratio includes: determining a brightening weight factor based on whether the target scene meets preset conditions, wherein the brightening weight factor is equal to 1 if the target scene meets the preset conditions, and less than 1 if the target scene does not meet the preset conditions; determining a brightening intensity based on the brightness difference ratio and the brightening weight factor, and brightening the target scene based on the brightening intensity.

[0010] This method determines the brightening weight factor based on preset conditions, thereby preventing over-enhancement and overexposure of the image, resulting in a more natural and balanced brightening effect.

[0011] In conjunction with the first aspect, in a possible implementation of the first aspect, the method further includes: obtaining a luminance map of the image to be processed; determining a luminance histogram of the image to be processed based on the luminance map, and determining the position corresponding to the maximum value of the bright area in the luminance histogram as the concentrated bright area region in the image to be processed; determining a brightening weight factor based on whether the target scene meets preset conditions includes: determining the brightening weight factor based on whether the bright area features of the concentrated bright area meet preset conditions, wherein the bright area features include luminance and / or the bright area concentration ratio, the bright area concentration ratio is used to indicate the proportion of the concentrated bright area region to the total number of bright pixels in the luminance map, and when the bright area features of the concentrated bright area meet the preset conditions, the brightening weight factor is equal to 1, and when the bright area features of the concentrated bright area do not meet the preset conditions, the brightening weight factor is less than 1.

[0012] In conjunction with the first aspect, in possible implementations of the first aspect, the bright area feature includes the brightness, the preset condition includes the brightness of the concentrated bright area being less than or equal to a preset first brightness threshold, and the brightening weight factor includes a first brightening weight factor; and / or, the bright area feature includes the bright area concentration ratio, the preset condition includes the bright area concentration ratio of the concentrated bright area being greater than or equal to a preset first ratio threshold, and the brightening weight factor includes a second brightening weight factor.

[0013] This scheme dynamically determines the first brightening weight factor by comparing the brightness of concentrated bright areas with a first brightness threshold. Then, when the concentrated bright areas are already quite bright, the brightening weight is reduced; when the brightness of the concentrated bright areas is moderate, the brightening weight is maintained at a normal level. Ultimately, this effectively suppresses over-enhancement of bright areas during the brightening process, preventing overexposure and loss of detail in the image's highlights, resulting in a more natural and balanced brightening effect. Furthermore, a second brightening weight factor is determined by analyzing the proportion of concentrated bright areas to the total number of pixels and combining this with a preset proportion threshold. Then, when the proportion of concentrated bright areas is small, the brightening intensity is reduced to avoid over-enhancing small bright areas within a large dark area; when the proportion of concentrated bright areas is large, the brightening intensity is maintained at a normal level. Ultimately, this allows the brightening process to adaptively adjust according to the distribution range of bright areas in the image, avoiding the problem of overall image brightness imbalance caused by over-enhancing local bright areas.

[0014] In conjunction with the first aspect, in possible implementations of the first aspect, the preset condition includes that the target scene is not located in a low ambient brightness scene, and the brightening weight factor includes a third brightening weight factor.

[0015] This scheme identifies whether the target scene is in a low-light environment where the model's predictions are inaccurate, and then uses a third brightening weight factor less than 1 to reduce the brightness adjustment intensity in such environments. This effectively avoids amplifying image noise due to over-brightening in low-light conditions. Furthermore, it effectively adjusts for inaccurate brightness predictions by the model, improving the robustness and output quality of image processing in extreme lighting conditions.

[0016] In conjunction with the first aspect, in a possible implementation of the first aspect, brightening the target scene based on the brightening intensity includes: determining a brightening control point based on the brightening intensity, wherein the abscissa of the brightening control point is the abscissa position corresponding to the concentrated bright area, and the ordinate of the brightening control point is determined based on the abscissa position corresponding to the concentrated bright area and the brightening intensity; determining a brightening auxiliary control point based on the brightening control point, wherein the position of the brightening auxiliary control point is located around the brightening control point, and the brightening auxiliary control point is used to maintain the local contrast of the image to be processed; determining a dark area preservation point at the minimum brightness value of the brightness map; obtaining a first mapping curve based on the interpolation calculation of the brightening control point, the brightening auxiliary control point and the dark area preservation point, wherein the abscissa of the pixel in the first mapping curve is the original brightness of the pixel, and the ordinate of the pixel in the first mapping curve is the brightness of the pixel after brightening; and brightening the image to be processed based on the first mapping curve.

[0017] This scheme achieves precise focusing of bright areas by setting brightening control points at their locations. Furthermore, auxiliary brightening control points are placed around these control points to maintain local image contrast, preventing the graying of dark areas and contrast loss that can occur with global brightening. Finally, shadow preservation points are set at the minimum brightness levels to maintain the original brightness of dark areas.

[0018] In conjunction with the first aspect, in a possible implementation of the first aspect, the brightness of the target scene is darkened according to the brightness difference ratio, including: determining a darkening weight factor based on whether the target scene meets a preset condition, wherein the darkening weight factor is equal to 1 if the target scene meets the preset condition, and less than 1 if the target scene does not meet the preset condition; determining the darkening intensity based on the brightness difference ratio and the darkening weight factor, and darkening the brightness of the target scene according to the darkening intensity.

[0019] This scheme determines the darkening weight factor based on preset conditions, which can effectively suppress over-darkening, prevent the loss of detail and blackness in the image, and make the darkening effect more natural and balanced.

[0020] In conjunction with the first aspect, in a possible implementation of the first aspect, the method further includes: obtaining a luminance map of the image to be processed; determining a luminance histogram of the image to be processed based on the luminance map, and determining the position corresponding to the maximum value of the dark area in the luminance histogram as the dark area concentration region in the image to be processed; determining a darkening weight factor based on whether the target scene meets preset conditions includes: determining the darkening weight factor based on whether the dark area features of the dark area concentration region meet preset conditions, wherein the dark area features include luminance and / or dark area concentration ratio, the dark area concentration ratio is used to indicate the proportion of the dark area concentration region to the total number of dark pixels in the luminance map, and when the dark area features of the dark area concentration region meet the preset conditions, the darkening weight factor is equal to 1, and when the dark area features of the dark area concentration region do not meet the preset conditions, the darkening weight factor is less than 1.

[0021] In conjunction with the first aspect, in possible implementations of the first aspect, the dark area feature includes the brightness, the preset condition includes the brightness of the concentrated dark area being greater than or equal to a preset second brightness threshold, and the darkening weight factor includes a first darkening weight factor; and / or, the dark area feature includes the dark area concentration ratio, the preset condition includes the dark area concentration ratio of the concentrated dark area being greater than or equal to a preset second ratio threshold, and the darkening weight factor includes a second darkening weight factor.

[0022] This scheme dynamically determines the first darkening weight factor by comparing the brightness of the concentrated dark areas with a second brightness threshold. Then, when the concentrated dark areas are already quite dark, the darkening weight is reduced; when the brightness of the concentrated dark areas is moderate, the darkening weight is maintained at a normal level. Ultimately, this effectively suppresses over-darkening of dark areas during the darkening process, preventing loss of detail and completely black areas in the image, resulting in a more natural and balanced darkening effect. Furthermore, the second darkening weight factor is determined by analyzing the proportion of concentrated dark areas to the total number of pixels and combining this with a preset proportion threshold. Then, when the proportion of concentrated dark areas is small, the darkening intensity is reduced to avoid over-darkening small dark areas within a large bright area; when the proportion of concentrated dark areas is large, the darkening intensity is maintained at a normal level. Ultimately, this allows the darkening process to adaptively adjust according to the distribution range of dark areas in the image, avoiding the problem of overall image detail loss due to over-darkening of local dark areas.

[0023] In conjunction with the first aspect, in possible implementations of the first aspect, the preset condition includes that the target scene is not located in a low ambient brightness scene, and the darkening weight factor includes a third darkening weight factor.

[0024] This scheme identifies whether the target scene is in a low-light environment where the model's predictions are inaccurate, and then uses a third darkening weight factor less than 1 to reduce the brightness adjustment intensity in such environments. This effectively prevents further loss of shadow details due to excessive darkening in low-light conditions. Furthermore, it effectively adjusts for inaccurate brightness predictions by the model, improving the robustness and output quality of image processing in extreme lighting conditions.

[0025] In conjunction with the first aspect, in a possible implementation of the first aspect, darkening the brightness of the target scene based on the darkening intensity includes: determining a darkening control point based on the darkening intensity, wherein the abscissa of the darkening control point is the abscissa position corresponding to the concentrated dark area, and the ordinate of the darkening control point is determined based on the abscissa position corresponding to the concentrated dark area and the darkening intensity; determining a darkening auxiliary control point based on the darkening control point, wherein the position of the darkening auxiliary control point is located around the darkening control point, and the darkening auxiliary control point is used to maintain the local contrast of the image to be processed; determining a highlight preservation point at the minimum brightness value of the brightness map; obtaining a second mapping curve based on the interpolation calculation of the darkening control point, the darkening auxiliary control point and the highlight preservation point, wherein the abscissa of the pixel in the second mapping curve is the original brightness of the pixel, and the ordinate of the pixel in the second mapping curve is the brightness of the pixel after darkening; and darkening the brightness of the image to be processed based on the second mapping curve.

[0026] This scheme achieves precise focus on dark areas by setting darkening control points at locations with concentrated dark areas. Furthermore, auxiliary darkening control points are placed around these control points to maintain local image contrast, avoiding the darkening of bright areas and contrast loss caused by global darkening. Additionally, highlight preservation points are set at the locations of maximum brightness to maintain the original brightness of bright areas.

[0027] In conjunction with the first aspect, in a possible implementation of the first aspect, determining the brightness histogram of the image to be processed based on the brightness map includes: determining an initial brightness histogram of the brightness map based on the brightness map; and performing window summation on the initial brightness histogram to obtain the brightness histogram.

[0028] This scheme effectively smooths noise and local fluctuations in the initial brightness histogram by performing small-window summation, resulting in a more stable and reliable brightness distribution. Furthermore, it improves the accuracy of locating concentrated bright and dark areas, enhancing the stability of subsequent brightness adjustments.

[0029] In conjunction with the first aspect, in one of the possible implementations of the first aspect, the target model is constructed and trained based on a convolutional neural network.

[0030] This approach employs a target model built and trained based on a convolutional neural network for scene brightness prediction. It fully leverages the powerful image feature extraction capabilities of convolutional neural networks, learning the complex mapping relationship between scene content and ideal brightness from a large number of samples. This results in more accurate brightness predictions that align with human visual perception.

[0031] In a second aspect, an electronic device is provided, comprising: a memory for storing one or more programs; and a processor for executing the one or more programs to cause the electronic device to implement the method of any one of the first aspects.

[0032] Thirdly, a readable storage medium is provided, which includes one or more programs that, when executed on an electronic device, cause the electronic device to implement the method of any one of the first aspects.

[0033] Fourthly, a program product is provided that, when executed on an electronic device, causes the electronic device to implement any one of the methods in the first aspect.

[0034] For a description of the beneficial effects of aspects two through four, please refer to the description of aspect one. Attached Figure Description

[0035] Figure 1 A schematic flowchart of the image processing method provided in an embodiment of this application is shown; Figure 2 A schematic diagram illustrating the process of determining the parameters required for image processing provided in an embodiment of this application is shown; Figure 3 A schematic diagram of a histogram provided in an embodiment of this application is shown; Figure 4 A schematic diagram of the image processing procedure provided in an embodiment of this application is shown; Figure 5 This illustration shows a schematic diagram of a first mapping curve provided in an embodiment of this application; Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0036] The technical solutions in this application will now be described with reference to the accompanying drawings. The illustrative embodiments of this application include, but are not limited to, image processing methods, electronic devices, readable storage media, and program products.

[0037] It is understood that the terminal electronic device in the embodiments of this application may also be referred to as a terminal, terminal equipment, user equipment (UE), mobile station (MS), mobile terminal (MT), etc. Terminal equipment can be a mobile phone, smart TV, wearable device, tablet computer, computer with wireless transceiver function, virtual reality (VR) terminal device, augmented reality (AR) terminal device, wireless terminal in industrial control, wireless terminal in self-driving, wireless terminal in remote medical surgery, wireless terminal in smart grid, wireless terminal in transportation safety, wireless terminal in smart city, wireless terminal in smart home, etc.

[0038] The following section introduces the technical problems existing in the current technology.

[0039] As mentioned earlier, when the reflectance of the subject or scene deviates significantly from the average reflectance (18%), it will be impossible to accurately distinguish between white plus / black subtract scenes, resulting in a poor visual quality of the final image.

[0040] For example, in actual shooting, there are many high-reflectivity and low-reflectivity scenes. High-reflectivity scenes include snow, white walls, light-colored clothing, and bright skies, while low-reflectivity scenes include black clothing, dark vegetation, dark wood textures, and dark areas in night scenes. For instance, in high-reflectivity scenes, the object's surface reflects most of the incident light to the camera sensor, resulting in the metering module receiving a much higher luminous flux than the expected luminous flux value corresponding to 18% reflection. Consequently, the after-effect (AE) algorithm misjudges this as excessive ambient light and actively reduces the exposure to bring back the target grayscale. Ultimately, white objects in high-reflectivity scenes appear light gray, bright areas lack detail, and the overall image is dull and grayish. In low-reflectivity scenes, the light signal received by the sensor is weaker, and the AE algorithm tends to over-expose, attempting to adjust dark areas to mid-gray brightness. Ultimately, black objects in low-reflectivity scenes appear muddy and washed out, dark details are compressed, and contrast is insufficient. The above-mentioned inaccuracies of white becoming darker and black becoming brighter are not due to limitations of the sensor hardware, but rather to errors between the reference reflectance and the reflectance of the actual scene.

[0041] In conclusion, the AE algorithm cannot distinguish between high-reflectivity and low-reflectivity scenes, and cannot adapt to today's complex and ever-changing shooting environment, resulting in poor image quality.

[0042] To address the aforementioned problems, this application proposes an image processing method 100. Figure 1 A schematic flowchart of an image processing method 100 provided in an embodiment of this application is shown. Figure 1 As shown, method 100 includes steps S110 to S140 and can be applied to the aforementioned terminal electronic device. In method 100, the current brightness of the target scene is determined by acquiring the image to be processed, and a pre-trained target model is used to predict the predicted brightness of the target scene. The actual brightness is then compared with the ideal brightness predicted by the model to adaptively perform brightening or darkening operations. This makes the brightness of the processed image more consistent with the human eye's perception of the scene, avoiding overexposure or underexposure problems caused by traditional fixed parameter adjustments.

[0043] Step S110: Obtain the image to be processed and determine the current brightness of the target scene presented in the image to be processed.

[0044] The terminal electronic device can acquire the image to be processed. This image can be a preview image captured in real-time by the terminal electronic device through a camera, or an image stored locally by the terminal electronic device or received from other devices. Furthermore, the terminal electronic device determines the current brightness of the target scene presented in the image to be processed. This current brightness reflects the actual illumination level of the target scene in the image. For example, in embodiments of this application, the current brightness can be the average brightness of all pixels in the image to be processed, such as the current average brightness, or it can be the average brightness of pixels in a local area selected according to specific rules.

[0045] Step S120: Input the image to be processed into the pre-trained target model to obtain the predicted brightness of the target scene output by the target model.

[0046] In this embodiment, the target model can be a pre-trained model used to predict the ideal brightness of a scene shown in an image. For example, the target model can predict the brightness value that a scene in an input image should present under ideal exposure conditions based on content features such as object type, texture distribution, and light and shadow relationships in the input image. Furthermore, the predicted brightness output by the target model can be used as an ideal brightness reference value for the target scene to guide subsequent brightness adjustment operations.

[0047] Optionally, in some embodiments of this application, the target model may be stored in the local storage of the terminal electronic device or in the cloud, and then the terminal electronic device may call the target model in the cloud for inference.

[0048] Optionally, in some embodiments of this application, the target model can refer to an artificial intelligence (AI) model built and trained based on a convolutional neural network (CNN). For example, the CNN model can adopt a deep fully convolutional architecture, such as containing multiple convolutional layers, batch normalization layers, activation function layers, and pooling layers. If necessary, residual connections or attention mechanisms can also be introduced to enhance feature representation capabilities. Furthermore, the input end receives raw image data such as an RGB three-channel pixel matrix, and extracts spatial features from different abstract levels, such as edges, textures, and semantic objects, through successive convolution and pooling operations. The output end uses fully connected layers or global average pooling in conjunction with a regression head to map high-dimensional features to a single continuous value or pixel-by-pixel brightness adjustment parameters, thereby achieving accurate prediction of the brightness of an ideal scene.

[0049] Optionally, during the training of the target model, supervised learning can be performed based on a large number of sample images with real brightness annotations. To construct the training set, images under different lighting conditions and different scene types (such as indoor, outdoor, night scene, backlight, etc.) can be collected, and ideal brightness annotations by professional metering equipment or human color experts can be used as supervision signals. Furthermore, through learning from a large number of sample images, the target model can establish a complex mapping relationship from image content features to ideal scene brightness, thereby providing more accurate and human-perceived predicted brightness.

[0050] Step S130: If the current brightness is less than the predicted brightness, the brightness of the target scene is brightened according to the current brightness and the predicted brightness to obtain the processed target image.

[0051] Specifically, when the terminal electronic device determines that the current brightness is less than the predicted brightness, it indicates that the target scene in the image to be processed is too dark and needs to be brightened. Then, the terminal electronic device determines the brightening range or intensity based on the difference between the current brightness and the predicted brightness. This allows the brightness of the processed target image to more closely approximate the ideal state predicted by the target model, avoiding issues such as a grayish image and poor visual quality.

[0052] Step S140: If the current brightness is greater than the predicted brightness, the brightness of the target scene is darkened according to the current brightness and the predicted brightness to obtain the processed target image.

[0053] Specifically, when the terminal electronic device determines that the current brightness is greater than the predicted brightness, it indicates that the target scene in the image to be processed is too bright and needs to be darkened. Then, the terminal electronic device determines the darkening range or intensity based on the difference between the current brightness and the predicted brightness. This makes the brightness of the processed target image closer to the ideal state predicted by the target model, avoiding problems such as overly bright images and impure blacks.

[0054] The above method 100 achieves adaptive adjustment of image brightness by brightening when the current brightness is less than the predicted brightness and darkening when the current brightness is greater than the predicted brightness. This avoids overexposure or underexposure problems caused by traditional fixed parameter adjustments and effectively improves the intelligence of image processing and the accuracy of brightness adjustment.

[0055] Possible embodiments of each step in method 100 are described below.

[0056] Optionally, in steps S130 and S140, this embodiment of the application can determine the brightness difference ratio by calculating the ratio of the predicted average brightness to the current average brightness, and perform a brightening or darkening operation based on the ratio. Specifically, the current brightness includes the current average brightness, and the predicted brightness includes the predicted average brightness. The method further includes: determining the brightness difference ratio based on the ratio of the predicted average brightness to the current average brightness. When the current brightness is less than the predicted brightness, the brightness of the target scene is brightened based on the current brightness and the predicted brightness, including: when the brightness difference ratio is greater than 1, the brightness of the target scene is brightened based on the brightness difference ratio. When the current brightness is greater than the predicted brightness, the brightness of the target scene is darkened based on the current brightness and the predicted brightness, including: when the brightness difference ratio is less than 1, the brightness of the target scene is darkened based on the brightness difference ratio.

[0057] For example, if the predicted average brightness is targetLuma and the current average brightness is avgLuma, then the brightness difference ratio gain can refer to the ratio of the predicted average brightness to the current average brightness, such as gain = targetLuma / avgLuma. When gain is greater than 1, it indicates that the actual brightness of the target scene in the image to be processed is lower than the ideal brightness predicted by the model, belonging to a "white plus" scene, which needs to be brightened. When gain is less than 1, it indicates that the actual brightness of the target scene in the image to be processed is higher than the ideal brightness predicted by the model, belonging to a "black minus" scene, which needs to be darkened.

[0058] Optionally, in embodiments of this application, a concentrated bright area or a concentrated dark area can be determined by analyzing the luminance histogram of the image to be processed, and the brightening intensity can be dynamically adjusted based on the luminance characteristics of the concentrated bright area, or the darkening intensity can be dynamically adjusted based on the luminance characteristics of the concentrated dark area. Specifically, in a white-plus scenario, the method further includes: obtaining a luminance map of the image to be processed; determining a luminance histogram of the image to be processed based on the luminance map, and determining the position corresponding to the maximum value of the bright area in the luminance histogram as the concentrated bright area in the image to be processed. And, in a black-minimum scenario, the method further includes: obtaining a luminance map of the image to be processed; determining a luminance histogram of the image to be processed based on the luminance map, and determining the position corresponding to the maximum value of the dark area in the luminance histogram as the concentrated dark area in the image to be processed.

[0059] In the embodiments of this application, the brightness map can refer to image data output by a front-end algorithm, used to characterize the brightness distribution information of each pixel in the image to be processed. The brightness histogram can be obtained by statistically analyzing the number of pixels at each brightness level in the brightness map, reflecting the distribution pattern of different brightness values ​​in the target scene. Furthermore, the concentrated bright area can refer to the horizontal coordinate position corresponding to the bright area with the most pixels in the brightness histogram, used to indicate the main clustering location of bright pixels in the image. The concentrated dark area can refer to the horizontal coordinate position corresponding to the dark area with the most pixels in the brightness histogram, used to indicate the main clustering location of low-brightness pixels in the image.

[0060] Optionally, in embodiments of this application, the brightness histogram can be obtained by summing an initial brightness histogram using a small window. Specifically, determining the brightness histogram of the image to be processed based on the brightness map includes: determining an initial brightness histogram of the brightness map based on the brightness map; and summing the initial brightness histogram using a small window to obtain the brightness histogram. Furthermore, by summing the initial brightness histogram using a small window, noise and local fluctuations in the histogram can be effectively smoothed, making the brightness distribution characteristics more stable and reliable, and improving the accuracy of locating concentrated bright or dark areas.

[0061] For example, the terminal electronic device can first calculate the histogram of the brightness map output by the front-end algorithm to obtain an initial brightness histogram hist1. Then, by performing window summation on histogram hist1, a smoothed brightness histogram hist2 is obtained. Furthermore, the horizontal axis coordinate of the maximum value of the bright area on hist2 is used as the concentrated region of the bright area, or the horizontal axis coordinate of the maximum value of the dark area on hist2 is used as the concentrated region of the dark area.

[0062] Optionally, in embodiments of this application, a weighting factor, such as a brightening weighting factor or a darkening weighting factor, can be determined based on whether the target scene meets preset conditions. Specifically, if the target scene meets the preset conditions, the weighting factor (such as a brightening weighting factor or a darkening weighting factor) is equal to 1; if the target scene does not meet the preset conditions, the weighting factor (such as a brightening weighting factor or a darkening weighting factor) is less than 1. Furthermore, embodiments of this application can determine the brightening intensity or darkening intensity based on the brightness difference ratio and the weighting factor, and brighten the target scene based on the brightening intensity, or darken the target scene based on the darkening intensity.

[0063] Optionally, in embodiments of this application, determining whether a target scene meets preset conditions may include determining whether the bright area characteristics of the concentrated bright area or the dark area characteristics of the concentrated dark area meet preset conditions, or may include determining whether the target scene is not located in a low ambient brightness scene.

[0064] For example, the bright area feature includes brightness and / or bright area concentration ratio, which indicates the proportion of the bright area concentration region to the total number of bright pixels in the brightness map. If the bright area feature of the bright area concentration region meets a preset condition, the brightening weight factor is equal to 1; if the bright area feature of the bright area concentration region does not meet the preset condition, the brightening weight factor is less than 1. For example, the bright area feature includes brightness, the preset condition includes the brightness of the bright area concentration region being less than or equal to a preset first brightness threshold, and the brightening weight factor includes a first brightening weight factor; and / or, the bright area feature includes the bright area concentration ratio, the preset condition includes the bright area concentration ratio of the bright area concentration region being greater than or equal to a preset first ratio threshold, and the brightening weight factor includes a second brightening weight factor.

[0065] For example, the dark area feature includes brightness and / or dark area concentration ratio, which indicates the proportion of the dark area concentration region to the total number of pixels in the dark area of ​​the brightness map. If the dark area feature of the dark area concentration region meets a preset condition, the darkening weight factor is equal to 1; if the dark area feature of the dark area concentration region does not meet the preset condition, the darkening weight factor is less than 1. For example, the dark area feature includes brightness, the preset condition includes brightness of the dark area concentration region being greater than or equal to a preset second brightness threshold, and the darkening weight factor includes a first darkening weight factor; and / or, the dark area feature includes dark area concentration ratio, the preset condition includes dark area concentration ratio of the dark area concentration region being greater than or equal to a preset second ratio threshold, and the darkening weight factor includes a second darkening weight factor.

[0066] For another example, when the preset conditions include that the target scene is not in a low ambient brightness scene, the weighting factors include a third brightening weighting factor or a third darkening weighting factor.

[0067] The preset conditions mentioned above will be explained in detail below.

[0068] When the brightness of a concentrated bright area is already high, further brightening it at the normal intensity will lead to overexposure and loss of detail in the highlight areas. Optionally, to address this technical problem, in step S130, this embodiment can dynamically determine a first brightening weight factor based on a comparison between the brightness of the concentrated bright area and a preset first brightness threshold, to suppress excessive enhancement of the highlight areas. Specifically, brightening the target scene based on the current brightness and the predicted brightness includes: determining a first brightening weight factor based on a comparison between the brightness of the concentrated bright area and a preset first brightness threshold, wherein the first brightening weight factor is less than 1 when the brightness of the concentrated bright area is greater than the first brightness threshold, and equal to 1 when the brightness of the concentrated bright area is less than or equal to the first brightness threshold; determining the brightening intensity based on the ratio between the current brightness and the predicted brightness and the first brightening weight factor, and brightening the target scene based on the brightening intensity.

[0069] In this embodiment, the first brightness threshold can refer to a preset brightness reference value used to determine whether the concentrated bright area is in a high-brightness state. The first brightening weight factor is used to adjust the brightening amplitude. When the brightness of the concentrated bright area is greater than the first brightness threshold, it indicates that the area is already bright enough and does not need to be brightened further. In this case, the first brightening weight factor is less than 1, thereby reducing the brightening amplitude to avoid overexposure and blurring. When the brightness of the concentrated bright area is less than or equal to the first brightness threshold, the first brightening weight factor is equal to 1, maintaining a normal brightening amplitude.

[0070] Similarly, if the brightness of a concentrated dark area is already very low, further darkening at the normal intensity will result in completely black dark areas and loss of detail. Optionally, in step S140, this embodiment can also dynamically determine a first darkening weight factor based on a comparison between the brightness of the concentrated dark area and a preset second brightness threshold to suppress excessive darkening of the dark areas. Specifically, darkening the brightness of the target scene based on the current brightness and the predicted brightness includes: determining a first darkening weight factor based on a comparison between the brightness of the concentrated dark area and a preset second brightness threshold, wherein the first darkening weight factor is less than 1 when the brightness of the concentrated dark area is less than the second brightness threshold, and equal to 1 when the brightness of the concentrated dark area is greater than or equal to the second brightness threshold; determining the darkening intensity based on the ratio between the current brightness and the predicted brightness and the first darkening weight factor, and darkening the brightness of the target scene based on the darkening intensity.

[0071] In this embodiment, the second brightness threshold can refer to a preset brightness reference value used to determine whether the concentrated dark area is in a low-brightness state. In this embodiment, the second brightness threshold is less than the first brightness threshold. Furthermore, the first darkening weight factor is used to adjust the darkening amplitude. Therefore, when the brightness of the concentrated dark area is less than the second brightness threshold, it indicates that the area is already very dark and does not need to be darkened excessively. In this case, the first darkening weight factor is less than 1, thereby reducing the darkening amplitude to avoid pure blacks and excessive contrast. When the brightness of the concentrated dark area is greater than or equal to the second brightness threshold, the first darkening weight factor is equal to 1, maintaining a normal darkening amplitude.

[0072] When the concentrated bright area occupies a very small proportion of the entire image, it indicates that the grayscale distribution of the current scene is relatively uniform and it is not a typical white-plus scene. If the brightness is still increased according to the normal intensity, it will cause the brightness of the areas outside the concentrated bright area in the image to be unbalanced. Optionally, in order to solve the above technical problem, in step S130, the embodiments of this application can dynamically determine the second brightening weight factor based on the comparison between the concentrated bright area proportion and the preset first proportion threshold. Specifically, based on the current brightness and the predicted brightness, the brightness of the target scene is increased, including: determining the bright area concentration ratio based on the brightness histogram, the bright area concentration ratio indicating the proportion of the concentrated bright area to the total number of bright pixels in the brightness map; determining a second brightening weight factor based on a comparison between the bright area concentration ratio and a preset first ratio threshold, wherein the second brightening weight factor is less than 1 when the bright area concentration ratio is less than or equal to the ratio threshold, and equal to 1 when the bright area concentration ratio is greater than the ratio threshold; determining the brightening intensity based on the ratio between the current brightness and the predicted brightness and the second brightening weight factor, and increasing the brightness of the target scene based on the brightening intensity.

[0073] In this embodiment, the bright area concentration ratio can be used to indicate the proportion of the concentrated bright area to the total number of bright pixels in the brightness map. The first ratio threshold can refer to a preset ratio reference value used to determine whether the bright area concentration ratio is typical. The second brightening weight factor is used to adjust the brightening amplitude. Furthermore, when the bright area concentration ratio is less than or equal to the ratio threshold, it indicates that the grayscale distribution of the current scene is relatively uniform, belonging to an atypical white-plus scene. Therefore, the second brightening weight factor is less than 1, reducing the brightening amplitude to avoid over-brightness in areas other than the concentrated bright area. When the bright area concentration ratio is greater than the ratio threshold, the second brightening weight factor is equal to 1, maintaining a normal brightening amplitude.

[0074] Similarly, when the dark area concentration region occupies a very small proportion of the entire image, it indicates that the current scene's grayscale distribution is relatively uniform and it is not a typical black reduction scene. If it is still darkened with the normal intensity, it will lead to the loss of detail in areas other than the bright area concentration region in the image. Optionally, in order to solve the above problem, in step S140, this embodiment of the application can also dynamically determine a second darkening weight factor based on the comparison between the dark area concentration ratio and a preset ratio threshold. Specifically, darkening the brightness of the target scene based on the current brightness and the predicted brightness includes: determining the dark area concentration ratio based on the proportion of the dark area concentration region to the total number of pixels in the brightness map; determining a second darkening weight factor based on the comparison between the dark area concentration ratio and a preset second ratio threshold, wherein when the dark area concentration ratio is less than or equal to the second ratio threshold, the second darkening weight factor is less than 1, and when the dark area concentration ratio is greater than the second ratio threshold, the second darkening weight factor is equal to 1; and determining the darkening intensity based on the ratio between the current brightness and the predicted brightness and the second darkening weight factor.

[0075] The dark area concentration ratio indicates the proportion of the concentrated dark area to the total number of dark pixels in the brightness map. The second ratio threshold can refer to a preset ratio reference value used to determine whether the dark area concentration ratio is typical. In embodiments of this application, the second ratio threshold can be the same as the first ratio threshold. Furthermore, the second darkening weight factor is used to adjust the darkening amplitude. When the dark area concentration ratio is less than or equal to the ratio threshold, it indicates that the grayscale distribution of the current scene is relatively uniform, an atypical black-reduction scene, and the second darkening weight factor is less than 1, reducing the darkening amplitude. When the dark area concentration ratio is greater than the ratio threshold, the second darkening weight factor is equal to 1, maintaining a normal darkening amplitude.

[0076] Optionally, in the process of brightening the image to be processed, embodiments of this application can achieve brightening processing by setting brightening control points, brightening auxiliary control points, and shadow preservation points, and performing curve calculation based on interpolation calculation methods. Specifically, brightening the brightness of the target scene according to the brightening intensity includes: determining a brightening control point according to the brightening intensity, wherein the abscissa of the brightening control point is the abscissa position corresponding to the concentrated bright area, and the ordinate of the brightening control point is determined according to the abscissa position corresponding to the concentrated bright area and the brightening intensity; determining a brightening auxiliary control point according to the brightening control point, wherein the position of the brightening auxiliary control point is located around the brightening control point, and the brightening auxiliary control point is used to maintain the local contrast of the image to be processed; determining a shadow preservation point at the minimum brightness value of the brightness map; obtaining a first mapping curve based on the interpolation calculation of the brightening control point, the brightening auxiliary control point, and the shadow preservation point; and brightening the brightness of the image to be processed according to the first mapping curve.

[0077] In some embodiments of this application, the brightening control point can refer to a core reference point used to control the brightening operation. Its horizontal coordinate is the horizontal coordinate position corresponding to the concentrated bright area, and its vertical coordinate is calculated and determined based on the horizontal coordinate position and the brightening intensity. The brightening auxiliary control point can refer to auxiliary reference points located around the brightening control point, used to constrain the shape of the curve to maintain the local contrast of the image to be processed and avoid the image becoming hazy after brightening. The dark area preservation point can refer to a control point set at the minimum brightness value of the brightness map, used to keep the original brightness of the dark area unaffected by the brightening operation. In some other embodiments of this application, the first mapping curve can refer to a brightness mapping curve calculated based on piecewise cubic hermite (piecewise cubic hermite interpolating polynomial, PCHIP) interpolation. By applying this curve to the brightness map of the image to be processed, the brightness of the target scene is brightened.

[0078] Similarly, in the process of darkening the image to be processed, the embodiments of this application can achieve darkening processing by setting darkening control points, darkening auxiliary control points, and highlight preservation points, and performing curve calculation based on interpolation calculation methods. Specifically, darkening the brightness of the target scene according to the darkening intensity includes: determining darkening control points according to the darkening intensity, wherein the abscissa of the darkening control points is the abscissa position corresponding to the dark area concentration region, and the ordinate of the darkening control points is determined according to the abscissa position corresponding to the dark area concentration region and the darkening intensity; determining darkening auxiliary control points according to the darkening control points, wherein the positions of the darkening auxiliary control points are located around the darkening control points, and the darkening auxiliary control points are used to maintain the local contrast of the image to be processed; determining highlight preservation points at the brightness maximum value of the brightness map; obtaining a second mapping curve based on the interpolation calculation of the darkening control points, the darkening auxiliary control points, and the highlight preservation points; and darkening the brightness of the image to be processed according to the second mapping curve.

[0079] In this embodiment, the darkening control point can refer to a core reference point used to control the darkening operation. Its horizontal coordinate is the horizontal coordinate position corresponding to the concentrated dark area, and its vertical coordinate is calculated and determined based on the horizontal coordinate position and the darkening intensity. The darkening auxiliary control point can refer to auxiliary reference points located around the darkening control point, used to constrain the shape of the curve to maintain the local contrast of the image to be processed and avoid the image becoming blurry after darkening. The brightness preservation point can refer to a control point set at the maximum brightness value of the brightness map, used to keep the original brightness of the bright area unaffected by the darkening operation. The second mapping curve can refer to the brightness mapping curve calculated based on PCHIP interpolation. By applying this curve to the brightness map of the image to be processed, the brightness of the target scene is darkened.

[0080] Furthermore, in low-light environments such as night scenes, the prediction results of the target model may be inaccurate. If the brightness is still increased or decreased at the normal intensity, it will lead to increased image noise or loss of detail. Optionally, to solve the above problems, embodiments of this application can determine a third brightening weight factor or a third darkening weight factor based on a comparison between the current ambient brightness and a preset ambient brightness threshold, so as to reduce the brightness adjustment intensity. Specifically, the method further includes: determining whether the target scene is located in a preset low ambient brightness scene based on the current brightness and a preset ambient brightness threshold; if the target scene is located in the low ambient brightness scene, determining a third brightening weight factor or a third darkening weight factor, wherein the third brightening weight factor or the third darkening weight factor is less than 1; brightening the target scene based on the current brightness and the predicted brightness, including: brightening the target scene based on the ratio between the current brightness and the predicted brightness and the third brightening weight factor; or darkening the target scene based on the current brightness and the predicted brightness, including: darkening the target scene based on the ratio between the current brightness and the predicted brightness and the third darkening weight factor.

[0081] In the embodiments of this application, the low ambient brightness scene can refer to a scene where the current ambient brightness is lower than a preset ambient brightness threshold, such as a night scene or a low-light environment. The third brightening weight factor or the third darkening weight factor is used to reduce the intensity of brightness adjustment in low ambient brightness scenes. Furthermore, since the prediction of the AI ​​model may be inaccurate in low ambient brightness, by using a weight factor less than 1, it is possible to effectively avoid noise amplification caused by excessive brightening or loss of dark details caused by excessive darkening.

[0082] In summary, in the embodiments of this application, the brightening intensity of the image to be processed can be determined based on the ratio between the current brightness and the predicted brightness, and the product of at least one of the first brightening weight factor, the second brightening weight factor, and the third brightening weight factor. For example, the brightening intensity can be determined based on the product of the aforementioned gain with any one or more of the first, second, and third brightening weight factors, or it can be determined based on the total product of the aforementioned gain with the first, second, and third brightening weight factors. Similarly, the darkening intensity of the image to be processed can be determined based on the ratio between the current brightness and the predicted brightness, and the product of at least one of the first, second, and third darkening weight factors.

[0083] The following describes a detailed embodiment of method 100 with reference to the accompanying drawings.

[0084] Figure 2A schematic diagram illustrating the process of determining the parameters required for image processing provided in an embodiment of this application is shown. Figure 3 A schematic diagram of a histogram provided in an embodiment of this application is shown. Wherein, in Figure 2 In the process shown, the characteristic peaks and their proportions of the brightest and darkest areas are determined by analyzing the image brightness histogram.

[0085] like Figure 2 As shown, firstly, this embodiment of the application can perform histogram and brightness mean statistics on the pixels of the image to be processed based on the brightness map of the image to be processed, to obtain the brightness mean (i.e., the current brightness in step S110) and Figure 3 The square shown in (a) Figure 1 (That is, the aforementioned original brightness histogram). The mean brightness reflects the basic exposure level of the entire image to be processed, while the histogram... Figure 1 This records the exact number of pixels at each gray level from 0 to 255, and the histogram... Figure 1 This is the raw data without smoothing.

[0086] After that, as Figure 2 As shown, the embodiments of this application address the histogram. Figure 1 The bright and dark areas in the image are summed and filtered according to the size of the small window, resulting in the following: Figure 3 The histogram shown in (b) is as follows. Figure 2 (That is, the brightness histogram in step S110). For example, embodiments of this application can divide the grayscale values ​​of the image to be processed into bright areas according to a threshold (e.g., Figure 3 (a) shows 128~255) and dark areas (e.g. Figure 3 The diagram shows two regions (0-127) in (a), and a sliding window with a fixed size is used to sum the values ​​within these two regions. Furthermore, the histogram can be calculated... Figure 1 Smoothing and denoising effectively filter out stray peaks caused by image sensor noise or local micro-patterns, generating a smoothed histogram. Figure 2 .

[0087] Then, as Figure 3 As shown in (b) above, based on smooth histogram Figure 2This application embodiment can locate the extreme coordinates and quantity of bright and dark areas. For example, this application embodiment can find the coordinates WhiteMaxIndex where the histogram value is maximum in the bright area, and the number of pixels WhiteMax corresponding to that coordinate WhiteMaxIndex. Then, based on the coordinates WhiteMaxIndex and the corresponding number of pixels WhiteMax, the aforementioned concentrated bright area region can be determined. As another example, this application embodiment can find the coordinates BlackMaxIndex where the histogram value is maximum in the dark area, and the number of pixels BlackMax corresponding to that coordinate BlackMaxIndex. Then, based on the coordinates BlackMaxIndex and the corresponding number of pixels BlackMax, the aforementioned concentrated dark area region can be determined.

[0088] Finally, the brightness concentration ratio WhiteRatio is determined based on the proportion of pixels in the bright area (WhiteMax), and the dark area concentration ratio BlackRatio is determined based on the proportion of pixels in the dark area (BlackMax). These two ratios represent the concentration of the brightest and darkest pixels in the image, respectively. For example, a high WhiteRatio value indicates the presence of large, very bright highlight areas in the image.

[0089] In the embodiments of this application, after determining the above parameters for image processing, image processing can be performed based on the above parameters. Figure 4 A schematic diagram of the image processing process provided in an embodiment of this application is shown. Wherein, in Figure 4 In the process shown, this embodiment of the application first uses an AI model to predict the target brightness, and then combines... Figure 3 The parameters shown and the ambient brightness determine the precise weights for brightening or darkening. Finally, a global tone mapping curve (GTM) is generated and applied to the image to be processed. A monotonicity check is performed to ensure that the image does not have any problems.

[0090] Specifically, in this embodiment of the application, the image to be processed is input into the AI ​​model, and then the AI ​​model combines the scene semantics and lighting information in the image to obtain the predicted brightness output by the AI ​​model, which is the ideal brightness value of the scene in the image to be processed.

[0091] Afterwards, based on Figure 3The process shown uses the average brightness obtained and the predicted brightness output by the AI ​​model to determine the brightness difference ratio, gain, between the predicted brightness and the average brightness, i.e., gain = predicted brightness / average brightness. Furthermore, when gain > 1, it indicates that the current brightness of the image to be processed is less than the predicted brightness, requiring brightening (a "white plus" scene); when gain < 1, it indicates that the current brightness of the image to be processed is greater than the predicted brightness, requiring darkening (a "black minus" scene).

[0092] Then, in order to avoid the problem of overexposure in white scenes and underexposure in black scenes caused by inaccurate predicted brightness output by AI models, this application introduces three weighting factors to determine the brightening intensity or darkening intensity based on the weighting factors and gain.

[0093] For example, as mentioned earlier, if the brightness of a concentrated bright area is already high, further brightening it at the normal intensity will lead to overexposure and loss of detail in the highlight areas. Furthermore, if the brightness of a concentrated dark area is already low, further darkening it at the normal intensity will result in completely black shadows and loss of detail in the dark areas. Therefore, as... Figure 4 As shown, in the embodiments of this application, based on the aforementioned first brightness threshold and second brightness threshold, it can be determined whether the brightness of the area corresponding to WhiteMaxIndex is sufficiently high, or whether the brightness of the area corresponding to BlackMaxIndex is sufficiently low. Furthermore, based on the linear interpolation calculation method, it can be determined... Figure 4 The WeightLoc shown is used as the first brightening weight factor or the first darkening weight factor.

[0094] For example, assuming the bright area ranges from 128 to 256 and the dark area ranges from 0 to 127, the brightness value corresponding to the first brightness threshold is 210, and the brightness value corresponding to the second brightness threshold is 60. The WeightLoc corresponding to the first and second brightness thresholds is 1, which is a critical point. Furthermore, assuming that when the brightness is at its maximum value of 256 or close to the maximum value such as 250, there is no need to further brighten it (i.e., WeightLoc is 0), and when the brightness is at its minimum value of 0 or close to the minimum value such as 10, there is no need to further darken it (i.e., WeightLoc is 0). Therefore, based on the above coordinate points such as (10, 0), (60, 1), (210, 1), and (250, 0), a brightness weight interpolation curve corresponding to WeightLoc can be constructed.

[0095] Furthermore, in the WhiteMaxIndex scenario, the first brightening weight factor corresponding to the concentrated bright area is determined by comparing the brightness of the concentrated bright area with the first brightness threshold portion of the aforementioned brightness weight interpolation curve. For example, when the brightness of the concentrated bright area is 200 and less than the first brightness threshold of 210, it is said that the brightness of the concentrated bright area itself is not too bright. Therefore, WeightLoc is 1. As another example, when the brightness of the concentrated bright area is 220 and greater than the first brightness threshold of 210, it is said that the brightness of the concentrated bright area itself is too bright. Therefore, the value of WeightLoc can be determined based on the brightness weight interpolation curve, such as WeightLoc being 1×(250-220) / (250-210)=0.75.

[0096] Similarly, in black reduction scenarios, the first darkening weight factor corresponding to the concentrated dark area is determined by comparing the brightness of the concentrated dark area corresponding to BlackMaxIndex with the second brightness threshold portion of the aforementioned weighted interpolation curve. For example, when the brightness of the concentrated dark area is 70 and greater than the second brightness threshold of 60, it indicates that the brightness of the concentrated dark area itself is not too dark. Therefore, WeightLoc is 1. As another example, when the brightness of the concentrated dark area is 40 and less than the second brightness threshold of 60, it indicates that the brightness of the concentrated bright area itself is too dark. Therefore, the value of WeightLoc can be determined based on the brightness weighted interpolation curve, such as WeightLoc being 1×(40-10) / (60-10)=0.6.

[0097] Optionally, in embodiments of this application, if the weighting factors only include a first brightening weighting factor or a first darkening weighting factor, then embodiments of this application can directly determine the brightening intensity based on the product of gain and the first brightening weighting factor, or determine the darkening intensity based on the product of gain and the first darkening weighting factor. That is, brightening intensity / darkening intensity = 1 + (gain - 1) × WeightLoc.

[0098] For another example, as mentioned earlier, when the concentrated bright areas occupy a very small proportion of the entire image, it indicates that the current scene's grayscale distribution is relatively uniform, and it is not a typical "white plus" scene. If the image is still brightened at the normal intensity, it will lead to an imbalance in brightness in areas other than the concentrated bright areas. Furthermore, when the concentrated dark areas occupy a very small proportion of the entire image, it indicates that the current scene's grayscale distribution is relatively uniform, and it is not a typical "black minus" scene. If the image is still darkened at the normal intensity, it will lead to a loss of detail in areas other than the concentrated bright areas. Therefore, if... Figure 4As shown, in the embodiments of this application, it can be determined whether the brightness concentration ratio (WhiteRatio) is sufficiently high, or whether the dark area concentration ratio (BlackRatio) is sufficiently high, based on the aforementioned first and second ratio thresholds. Furthermore, based on a linear interpolation calculation method, it can be determined... Figure 4 The WeightRatio shown serves as the second brightening weight factor or the second darkening weight factor.

[0099] In some embodiments of this application, the first proportional threshold and the second proportional threshold can be equal. In other embodiments of this application, the first proportional threshold and the second proportional threshold can be different, such as the first proportional threshold being 30% and the second proportional threshold being 20%, thus the proportional weight interpolation curves corresponding to the first proportional threshold and the second proportional threshold are different, the only difference being the coordinate points of the bends in the curves.

[0100] Furthermore, interpolation calculations can be performed based on the first and second proportional thresholds. For example, assuming the first proportional threshold is 30% and the WeightRatio is 0 when the proportion is 5%, interpolation calculations can be performed based on coordinate points such as (0.05, 0), (0.3, 1), and (1, 1) to construct the proportional weight interpolation curve corresponding to WeightRatio. Then, in the white-plus scenario, based on the comparison between WhiteRatio and the first proportional threshold (e.g., 30%), the second brightening weight factor corresponding to the concentrated bright area is determined. For example, when WhiteRatio is 40% and greater than the first proportional threshold of 30%, it means the concentrated bright area occupies a sufficiently large proportion of the entire image. Therefore, WeightRatio is 1. As another example, when WhiteRatio is 20% and less than the first proportional threshold of 30%, it means the concentrated bright area occupies a very small proportion of the entire image. Furthermore, the value of WeightRatio can be determined based on the proportional weight interpolation curve, such as WeightRatio being 1×(20-5) / (30-5)=0.6.

[0101] Similarly, in black reduction scenarios, the second darkening weight factor corresponding to the concentrated dark areas is determined based on the comparison between BlackRatio and the second proportional threshold (e.g., 20%). For example, when BlackRatio is 35% and greater than the second proportional threshold of 20%, it indicates that the concentrated dark areas occupy a sufficiently large proportion of the entire image. Therefore, WeightRatio is 1. As another example, when BlackRatio is 10% and less than the second proportional threshold of 20%, it indicates that the concentrated dark areas occupy a very small proportion of the entire image. Therefore, the value of WeightRatio can be determined based on the proportional weight interpolation curve, such as WeightRatio being 1×(10-5) / (20-5)≈0.33.

[0102] Optionally, in embodiments of this application, if the weighting factors only include a second brightening weighting factor or a second darkening weighting factor, then embodiments of this application can directly determine the brightening intensity based on the product of gain and the second brightening weighting factor, or determine the darkening intensity based on the product of gain and the second darkening weighting factor. That is, brightening intensity / darkening intensity = 1 + (gain - 1) × WeightRatio. In other embodiments of this application, the weighting factors include a first brightening weighting factor and a second brightening weighting factor, or include a first darkening weighting factor and a second darkening weighting factor. In this case, brightening intensity / darkening intensity can be 1 + (gain - 1) × WeightLoc × WeightRatio.

[0103] For another example, as mentioned earlier, in low-light environments such as night scenes, the prediction results of the target model may be inaccurate. If the image is still brightened or darkened using the usual intensity, it will lead to increased image noise or loss of detail. Furthermore, as... Figure 4 As shown, embodiments of this application can determine whether a scene is in a low ambient light environment based on an ambient light threshold. Figure 4 The WeightLv shown. In the embodiments of this application, Figure 4 The ambient brightness shown can be the same as the average brightness.

[0104] Furthermore, interpolation calculations can be performed based on an ambient brightness threshold. For example, assuming the ambient brightness threshold is 30 and the corresponding WeightLv is 0 when the ambient brightness is 5, interpolation calculations can be performed based on coordinate points such as (5, 0), (30, 1), and (256, 1) to construct the environmental weight interpolation curve corresponding to WeightLv.

[0105] Furthermore, in scenarios where white is added or black is subtracted, a third brightening weight factor or a third darkening weight factor is determined based on the comparison between the ambient brightness and the ambient brightness threshold (e.g., 30). For example, when the ambient brightness is 40 and greater than the ambient brightness threshold of 30, it indicates that the target scene is not in a low ambient brightness scene. Therefore, WeightLv is 1. As another example, when the ambient brightness is 20 and less than the ambient brightness threshold of 30, it indicates that the target scene is in a low ambient brightness scene. Therefore, the value of WeightLv can be determined based on the ambient weight interpolation curve, such as WeightLv being 1×(20-5) / (30-5)=0.6.

[0106] Optionally, in embodiments of this application, if the weighting factors only include a third brightening weighting factor or a third darkening weighting factor, then embodiments of this application can directly determine the brightening intensity based on the product of gain and the third brightening weighting factor, or determine the darkening intensity based on the product of gain and the third darkening weighting factor. That is, brightening intensity / darkening intensity = 1 + (gain - 1) × WeightLv. In other embodiments of this application, if the weighting factors also include a first brightening weighting factor, a second brightening weighting factor, a first darkening weighting factor, and a second darkening weighting factor, then the brightening intensity / darkening intensity is 1 + (gain - 1) × WeightLoc × WeightRatio × WeightLv.

[0107] Continue to refer to Figure 4 After determining the brightening intensity, the embodiments of this application can determine the brightening control point, the brightening auxiliary control point, and the dark area retention point based on the brightening intensity, and calculate the first mapping curve based on the interpolation calculation method to achieve the brightening process. Figure 5 A schematic diagram of a first mapping curve provided in an embodiment of this application is shown.

[0108] like Figure 5 As shown, this highlight control point can refer to the core reference point used to control the highlighting operation, and its horizontal axis is... x 1 is Figure 3 The horizontal coordinate of the concentrated bright area shown is [blank], and its vertical coordinate is [blank]. y 1. Based on the x-axis x 1. The brightness and enhancement intensity are calculated and determined based on the horizontal axis position. x The product of 1 and the brightening intensity is obtained. The brightening auxiliary control point can be located around the brightening control point, such as to the left or right, to constrain the shape of the curve, maintaining the local contrast of the image to be processed and preventing the image from becoming hazy after brightening. The dark area preservation point can refer to a control point set at the minimum brightness value of the brightness map, used to maintain the original brightness of the dark area from being affected by the brightening operation. Furthermore, in the embodiments of this application, PCHIP interpolation calculations can be performed based on the above brightening control point, brightening auxiliary control point, and dark area preservation point to obtain the following... Figure 5 The first mapping curve is shown. The horizontal axis of this first mapping curve represents the original brightness of each pixel, and the vertical axis represents the adjusted brightness of each pixel. Finally, this first mapping curve can be applied to the brightness map of the image to be processed.

[0109] Similarly, after determining the darkening intensity, this embodiment of the application can achieve darkening processing by setting darkening control points, darkening auxiliary control points, and brightness preservation points, and performing curve calculation based on interpolation methods. The darkening control point can refer to a core reference point used to control the darkening operation; its horizontal coordinate is the horizontal coordinate position corresponding to the concentrated dark area, and its vertical coordinate is calculated based on the horizontal coordinate position and darkening intensity. The darkening auxiliary control point can refer to auxiliary reference points located around the darkening control point, used to constrain the shape of the curve to maintain the local contrast of the image to be processed and avoid the image becoming hazy after darkening. The brightness preservation point can refer to a control point set at the maximum brightness value of the brightness map, used to keep the original brightness of the bright area unaffected by the darkening operation. The second mapping curve can refer to the brightness mapping curve calculated based on PCHIP interpolation; by applying this curve to the brightness map of the image to be processed, the brightness of the target scene is darkened.

[0110] Continue to refer to Figure 4 After determining the first or second mapping curve, a monotonicity check is performed on the generated curve to determine if it exhibits wavy or non-monotonic behavior, thus preventing artifacts or color anomalies in localized areas of the processed image. Finally, if the check passes, the mapping curve is applied to the original brightness map, and the processed image is output. If the check fails, the process terminates directly, and the curve is not applied, ensuring that image quality is not affected by erroneous curves.

[0111] In the above embodiments, a first brightening weight factor is dynamically determined by comparing the brightness of the concentrated bright areas with a first brightness threshold. Furthermore, when the concentrated bright areas are already quite bright, the brightening weight is reduced; when the brightness of the concentrated bright areas is moderate, the brightening weight is maintained at a normal level. Ultimately, the brightening process effectively suppresses excessive enhancement of bright areas, preventing overexposure and loss of detail in the image's highlights, resulting in a more natural and balanced brightening effect. Similarly, a first darkening weight factor is dynamically determined by comparing the brightness of the concentrated dark areas with a second brightness threshold. Furthermore, when the concentrated dark areas are already quite dark, the darkening weight is reduced; when the brightness of the concentrated dark areas is moderate, the darkening weight is maintained at a normal level. Ultimately, the darkening process effectively suppresses excessive darkening of dark areas, preventing loss of detail and completely black areas in the image's dark regions, resulting in a more natural and balanced darkening effect.

[0112] Furthermore, by analyzing the proportion of concentrated bright areas to the total number of pixels and combining it with a preset proportion threshold, a second brightening weight factor is determined. Then, when the proportion of concentrated bright areas is small, the brightening intensity is reduced to avoid over-enhancing small bright areas within a large dark area; when the proportion of concentrated bright areas is large, the brightening intensity is maintained at a normal level. Ultimately, this allows the brightening process to adaptively adjust according to the distribution range of bright areas in the image, avoiding the problem of overall image brightness imbalance caused by over-brightening of local bright areas. Similarly, by analyzing the proportion of concentrated dark areas to the total number of pixels and combining it with a preset proportion threshold, a second darkening weight factor is determined. Then, when the proportion of concentrated dark areas is small, the darkening intensity is reduced to avoid over-darkening of small dark areas within a large bright area; when the proportion of concentrated dark areas is large, the darkening intensity is maintained at a normal level. Ultimately, this allows the darkening process to adaptively adjust according to the distribution range of dark areas in the image, avoiding the problem of overall image detail loss caused by over-darkening of local dark areas.

[0113] Furthermore, by identifying whether the target scene is in a low-light environment where the model's predictions are inaccurate, a third brightening or darkening weight of less than 1 is applied to reduce the brightness adjustment intensity in low-light environments. This effectively avoids amplifying image noise due to over-brightening or further losing shadow details due to over-darkening in low-light conditions. In addition, it effectively adjusts for inaccurate brightness predictions by the model, improving the robustness and output quality of image processing in extreme lighting conditions.

[0114] like Figure 6 As shown, the electronic device 600 may include a processor 610, an external memory interface 620, an internal memory 621, a universal serial bus (USB) interface 630, a charging management module 640, a power management module 641, a battery 642, an antenna 1, a wireless communication module 660, an audio module 670, a speaker 670A, a receiver 670B, a microphone 670C, a sensor module 680, buttons 690, a motor 691, an indicator 692, a camera 693, a display screen 694, etc. The sensor module 680 may include a pressure sensor 680A, a gyroscope sensor 680B, a barometric pressure sensor 680C, a magnetic sensor 680D, an accelerometer sensor 680E, a proximity sensor 680F, a proximity light sensor 680G, a fingerprint sensor 680H, a temperature sensor 680J, a touch sensor 680K, an ambient light sensor 680L, a bone conduction sensor 680M, etc.

[0115] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 600. In other embodiments of this application, the electronic device 600 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0116] The processor 610 may include one or more processing units, such as an application processor (AP), a modem processor, a GPU, an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and a neural network processing unit (NPU). These different processing units may be independent devices or integrated into one or more processors.

[0117] The controller can generate operation control signals based on the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0118] The processor 610 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 610 is a cache memory. This memory can store instructions or data that the processor 610 has just used or that are used repeatedly. If the processor 610 needs to use the instruction or data again, it can directly retrieve it from the aforementioned memory. This avoids repeated accesses, reduces the waiting time of the processor 610, and thus improves the efficiency of the system.

[0119] It should be noted that the structure of the electronic device 600 shown in the foregoing embodiment is only an example. In other embodiments, the electronic device may include more or fewer modules, or some modules may be combined or split, which is not limited here.

[0120] This application also provides a computer program product, which may be a software or program product including instructions, capable of running on a computing device or stored on any usable medium. When the computer program product is run on at least one computing device, the at least one computing device implements the ink generation method provided in this application.

[0121] This application also provides a computer-readable storage medium. The computer-readable storage medium can be any storage medium (e.g., magnetic medium, optical medium, semiconductor medium, etc.) capable of enabling a computing device to store and retrieve at least one type of data. The computer-readable storage medium includes instructions that direct a computing device to implement the ink generation method provided in this application.

[0122] In the accompanying drawings, some structural or methodological features may be shown in at least one arrangement and order. However, it should be understood that such arrangement or order may not be necessary. Rather, in some embodiments, these features may be arranged in at least one manner and order different from those shown in the illustrative drawings. Furthermore, the inclusion of structural or methodological features in the figures does not imply that such features are required in all embodiments, and in some embodiments, these features may be omitted or may be combined with other features.

[0123] It should be noted that in the examples and description of this application, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the term "comprising" or any other variations thereof is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0124] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present invention.

Claims

1. An image processing method, characterized in that, include: Acquire the image to be processed and determine the current brightness of the target scene presented in the image to be processed; The image to be processed is input into a pre-trained target model to obtain the predicted brightness of the target scene output by the target model. The target model is used to predict the brightness of the scene shown in the image. If the current brightness is less than the predicted brightness, the brightness of the target scene is increased based on the current brightness and the predicted brightness to obtain the processed target image. If the current brightness is greater than the predicted brightness, the brightness of the target scene is darkened according to the current brightness and the predicted brightness to obtain the processed target image.

2. The method according to claim 1, characterized in that, The current brightness includes the current average brightness, the predicted brightness includes the predicted average brightness, and the method further includes: The brightness difference ratio is determined based on the ratio of the predicted average brightness to the current average brightness. When the current brightness is less than the predicted brightness, brightening the target scene based on the current brightness and the predicted brightness includes: If the brightness difference ratio is greater than 1, the brightness of the target scene is increased according to the brightness difference ratio. When the current brightness is greater than the predicted brightness, the step of darkening the brightness of the target scene based on the current brightness and the predicted brightness includes: If the brightness difference ratio is less than 1, the brightness of the target scene is darkened according to the brightness difference ratio.

3. The method according to claim 2, characterized in that, The step of brightening the target scene according to the brightness difference ratio includes: A brightening weight factor is determined based on whether the target scene meets the preset conditions. When the target scene meets the preset conditions, the brightening weight factor is equal to 1, and when the target scene does not meet the preset conditions, the brightening weight factor is less than 1. The brightness enhancement intensity is determined based on the brightness difference ratio and the brightness enhancement weight factor, and the brightness of the target scene is enhanced based on the brightness enhancement intensity.

4. The method according to claim 3, characterized in that, The method further includes: Obtain the brightness map of the image to be processed; The brightness histogram of the image to be processed is determined based on the brightness map, and the position corresponding to the maximum value of the bright area in the brightness histogram is determined as the concentrated bright area region in the image to be processed. The step of determining the brightening weight factor based on whether the target scene meets preset conditions includes: The brightening weight factor is determined based on whether the bright area characteristics of the concentrated bright area meet preset conditions. The bright area characteristics include brightness and / or bright area concentration ratio. The bright area concentration ratio is used to indicate the proportion of the concentrated bright area to the total number of bright pixels in the brightness map. When the bright area characteristics of the concentrated bright area meet the preset conditions, the brightening weight factor is equal to 1. When the bright area characteristics of the concentrated bright area do not meet the preset conditions, the brightening weight factor is less than 1.

5. The method according to claim 4, characterized in that, The bright area feature includes the brightness, the preset condition includes the brightness of the concentrated bright area being less than or equal to a preset first brightness threshold, and the brightening weight factor includes a first brightening weight factor. And / or, The bright area feature includes the bright area concentration ratio, the preset condition includes the bright area concentration ratio of the bright area concentration region being greater than or equal to a preset first ratio threshold, and the brightening weight factor includes a second brightening weight factor.

6. The method according to claim 4, characterized in that, The preset conditions include that the target scene is not located in a low ambient brightness scene, and the brightening weight factor includes a third brightening weight factor.

7. The method according to any one of claims 4 to 6, characterized in that, The step of brightening the target scene according to the brightening intensity includes: The brightness control point is determined based on the brightness intensity. The horizontal coordinate of the brightness control point is the horizontal coordinate position corresponding to the concentrated bright area. The vertical coordinate of the brightness control point is determined based on the horizontal coordinate position corresponding to the concentrated bright area and the brightness intensity. A highlighting auxiliary control point is determined based on the highlighting control point. The highlighting auxiliary control point is located around the highlighting control point and is used to maintain the local contrast of the image to be processed. Determine the dark area preservation point at the minimum brightness value of the brightness map; Based on the interpolation calculation of the brightening control point, the brightening auxiliary control point and the dark area preservation point, a first mapping curve is obtained. The horizontal coordinate of the pixel in the first mapping curve is the original brightness of the pixel, and the vertical coordinate of the pixel in the first mapping curve is the brightness of the pixel after brightening. The brightness of the image to be processed is enhanced based on the first mapping curve.

8. The method according to claim 2, characterized in that, Based on the brightness difference ratio, the brightness of the target scene is darkened, including: A darkening weight factor is determined based on whether the target scene meets the preset conditions. When the target scene meets the preset conditions, the darkening weight factor is equal to 1, and when the target scene does not meet the preset conditions, the darkening weight factor is less than 1. The darkening intensity is determined based on the brightness difference ratio and the darkening weight factor, and the brightness of the target scene is darkened based on the darkening intensity.

9. The method according to claim 8, characterized in that, The method further includes: Obtain the brightness map of the image to be processed; The brightness histogram of the image to be processed is determined based on the brightness map, and the position corresponding to the maximum value of the dark area in the brightness histogram is determined as the concentrated dark area region in the image to be processed. The step of determining the darkening weight factor based on whether the target scene meets preset conditions includes: The darkening weight factor is determined based on whether the dark area features of the dark area concentration region meet preset conditions. The dark area features include brightness and / or dark area concentration ratio. The dark area concentration ratio is used to indicate the proportion of the dark area concentration region to the total number of dark pixels in the brightness map. When the dark area features of the dark area concentration region meet the preset conditions, the darkening weight factor is equal to 1. When the dark area features of the dark area concentration region do not meet the preset conditions, the darkening weight factor is less than 1.

10. The method according to claim 9, characterized in that, The dark area feature includes the brightness, the preset condition includes the brightness of the concentrated dark area being greater than or equal to a preset second brightness threshold, and the darkening weight factor includes a first darkening weight factor. And / or, The dark area feature includes the dark area concentration ratio, the preset condition includes the dark area concentration ratio of the dark area concentration region being greater than or equal to a preset second ratio threshold, and the darkening weight factor includes a second darkening weight factor.

11. The method according to claim 9, characterized in that, The preset conditions include that the target scene is not located in a low ambient brightness scene, and the darkening weight factor includes a third darkening weight factor.

12. The method according to any one of claims 9 to 11, characterized in that, The step of darkening the brightness of the target scene according to the darkening intensity includes: The darkening control point is determined based on the darkening intensity. The horizontal coordinate of the darkening control point is the horizontal coordinate position corresponding to the dark area concentration region. The vertical coordinate of the darkening control point is determined based on the horizontal coordinate position corresponding to the dark area concentration region and the darkening intensity. Darkening auxiliary control points are determined based on the darkening control points. The positions of the darkening auxiliary control points are located around the darkening control points, and the darkening auxiliary control points are used to maintain the local contrast of the image to be processed. Determine the point where the brightness is at its minimum value in the brightness map; A second mapping curve is obtained by interpolation calculation based on the darkening control point, the darkening auxiliary control point, and the bright part preservation point. The horizontal coordinate of the pixel in the second mapping curve is the original brightness of the pixel, and the vertical coordinate of the pixel in the second mapping curve is the brightness of the pixel after darkening. The brightness of the image to be processed is darkened according to the second mapping curve.

13. The method according to any one of claims 4, 5, 9, and 10, characterized in that, Determining the brightness histogram of the image to be processed based on the brightness map includes: Based on the brightness map, determine the initial brightness histogram of the brightness map; The initial brightness histogram is summed in a small window to obtain the brightness histogram.

14. The method according to any one of claims 1 to 6 and 8 to 11, characterized in that, The target model is constructed and trained based on a convolutional neural network.

15. An electronic device, characterized in that, include: Memory, used to store one or more programs; A processor for executing the one or more programs to cause the electronic device to perform the method of any one of claims 1 to 14.

16. A readable storage medium, characterized in that, The readable storage medium includes one or more programs that, when executed on an electronic device, cause the electronic device to perform the method according to any one of claims 1 to 14.

17. A program product, characterized in that, When the program product is executed on an electronic device, it causes the electronic device to perform the method described in any one of claims 1 to 14.

Images