Image enhancement method, electronic device and computer program product

Abstract

The invention provides an image enhancement method, electronic equipment and a computer program product. The image enhancement method comprises the steps of performing semantic segmentation on an image, and determining position information of various objects in the image; based on the position information, performing enhancement processing on image features of an area where the first type of object is located in the image to obtain a local enhanced image; performing enhancement processing on the image features of the areas where the various objects are located in the image to obtain a global enhanced image; based on the position information, distributing a weight value for each pixel position in the local enhanced image and / or the global enhanced image; and performing weighted fusion on the local enhanced image and the global enhanced image based on the weight value allocated to each pixel position to obtain an enhanced image.

Description

Technical Field

[0001] This disclosure relates to an image enhancement method, electronic device, and computer program product. Background Technology

[0002] In fields such as video conferencing and security monitoring, the demand for image quality is increasing.

[0003] In related technologies, the image is first enhanced globally to obtain a globally enhanced image, and then the image is enhanced locally to obtain a locally enhanced image. Finally, the globally enhanced image and the locally enhanced image are weighted and fused into the final enhanced image using a pre-configured fixed ratio.

[0004] However, this fixed ratio is difficult to adapt to different images, resulting in poor quality of the generated enhanced images. Summary of the Invention

[0005] This disclosure provides an image enhancement method, an electronic device, a readable storage medium, and a computer program product.

[0006] According to one aspect of this disclosure, an image enhancement method is provided, comprising: performing semantic segmentation on an image to determine the location information of multiple objects in the image; enhancing image features of a region containing a first type of object in the image based on the location information to obtain a locally enhanced image; enhancing image features of a region containing multiple objects in the image to obtain a globally enhanced image; assigning weight values ​​to each pixel position in the locally enhanced image and / or the globally enhanced image based on the location information; and performing weighted fusion of the locally enhanced image and the globally enhanced image based on the weight values ​​assigned to each pixel position to obtain an enhanced image.

[0007] According to one technical solution, since the weight values ​​of each pixel position used in the weighted fusion process of local enhanced image and global enhanced image are not fixed, but dynamically allocated based on the positions of multiple objects in the image, that is, the weight values ​​can change with the changes in the positions of multiple objects in the image, the local enhanced image and global enhanced image can be fused more accurately, thereby improving the quality of the enhanced image.

[0008] An image enhancement method according to at least one embodiment of the present disclosure, based on the location information, enhances the image features of the region where a first type of object is located in the image to obtain a locally enhanced image, including: dividing the image into a first image block and a second image block based on the location information, wherein the first image block is the image block corresponding to the region where the first type of object is located in the image, and the second image block is the image block that does not contain the region where the first type of object is located; enhancing the contrast and brightness of the first image block to obtain an enhanced first image block; and stitching the enhanced first image block and the second image block together to obtain the locally enhanced image.

[0009] According to the technical solution of this embodiment, the contrast and brightness are enhanced only in the area where the first type of object is located, so that the details of the first type of object are fully optimized and unnecessary processing of areas that do not need to be enhanced is avoided, thereby improving the accuracy and efficiency of local enhancement.

[0010] An image enhancement method according to at least one embodiment of the present disclosure enhances image features of regions containing multiple objects in an image to obtain a globally enhanced image, including enhancing the color and contrast of regions containing multiple objects in the image to obtain the globally enhanced image.

[0011] According to the technical solution of this embodiment, by enhancing the color and contrast of all object areas in the image, the overall visual effect of the image is effectively improved. This can solve problems such as uneven overall lighting and color distortion, providing a high-quality global enhanced image foundation for subsequent fusion steps, ensuring that the enhanced image obtained after fusion maintains overall consistency in appearance, and improving the quality of the enhanced image.

[0012] According to at least one embodiment of the image enhancement method of this disclosure, a weight value is assigned to each pixel position in the local enhanced image and / or the global enhanced image based on the location information, including: based on the mapping relationship between object type and weight value, using the weight value corresponding to the object type of each pixel position in the local enhanced image as the weight value of the corresponding pixel position in the local enhanced image, wherein the weight value corresponding to the first type of object is configured to be higher than the weight value corresponding to other objects, the other objects being objects other than the first type of object among the various objects; and for each pixel position, the value obtained by subtracting the weight value of the pixel position in the local enhanced image from the maximum weight threshold is used as the weight value of the pixel position in the global enhanced image.

[0013] According to the technical solution of this embodiment, a higher weight value is assigned to the first type of object (such as a face), so that it mainly retains the local enhancement result in the fusion result, while the global enhancement result participates in the weighted fusion through complementary weights. Thus, intelligent adaptive fusion based on semantic understanding is realized, so that the area where the first type of object is located is emphasized and enhanced, improving the display effect of the first type of object in the enhanced image, while ensuring the overall quality of the enhanced image.

[0014] An image enhancement method according to at least one embodiment of the present disclosure performs semantic segmentation on an image to determine the location information of multiple objects in the image, including: providing the image to a first processing unit of an image processing model, performing semantic segmentation on the image by the first processing unit to determine the location information of multiple objects in the image; and enhancing the image features of the region where a first type of object is located in the image based on the location information to obtain a locally enhanced image, including: providing the image and the location information of the first type of object in the image to a second processing unit of the image processing model, and enhancing the image features of the region where the first type of object is located in the image by the second processing unit to obtain the locally enhanced image; and enhancing the image features of the region where multiple objects are located in the image to obtain a globally enhanced image, including: providing the image to a second processing unit of the image processing model and the location information of the first type of object in the image, performing semantic segmentation on the image by the second processing unit to determine the location information of multiple objects in the image. The third processing unit of the image processing model provides the image, and enhances the image features of the regions where multiple objects are located in the image to obtain the global enhanced image; based on the location information, weight values ​​are assigned to each pixel position in the local enhanced image and / or the global enhanced image, including: providing the image and the location information of the multiple objects in the image to the fourth processing unit of the image processing model, and assigning weight values ​​to each pixel position in the local enhanced image through the fourth processing unit; based on the weight values ​​assigned to each pixel position, the local enhanced image and the global enhanced image are weighted and fused to obtain the enhanced image, including: the image processing model performs weighted fusion of the local enhanced image and the global enhanced image according to the weight values ​​assigned to each pixel position in the local enhanced image to obtain the enhanced image.

[0015] According to the technical solution of this embodiment, semantic segmentation, local enhancement, global enhancement, weight value allocation and weighted fusion are integrated into a unified image processing model. This enables an end-to-end image enhancement process, which not only improves the processing efficiency of image enhancement and reduces intermediate data transmission overhead, but also ensures the effective transmission of semantic information throughout the entire processing chain, making the final enhancement effect more accurate and consistent.

[0016] According to at least one embodiment of the image enhancement method of this disclosure, the image processing model is trained through the following steps: providing a preset image to the image processing model; processing the preset image through the image processing model to obtain predicted location information of multiple objects in the preset image and a predicted enhanced image; based on the predicted location information of the multiple objects in the preset image, determining the images of the regions where the multiple objects are located in the predicted enhanced image and the preset enhanced image corresponding to the preset image; for each of the multiple objects, determining the loss value corresponding to the region where the object is located in the predicted enhanced image and the image of the region where the object is located in the preset enhanced image; performing a weighted summation of the loss values ​​corresponding to the multiple objects, and using the resulting sum as the loss value of the image processing model; and adjusting the parameters of the image processing model based on the loss value of the image processing model to obtain a trained image processing model.

[0017] According to the technical solution of this embodiment, semantic information is incorporated into the loss value of the image processing model, which enriches the loss function of the image processing model. This allows the image processing model to be trained under the guidance of semantic information, enabling the image processing model to pay more attention to the actual enhancement effect of the region where each object is located during the training process. This improves the image enhancement effect of the trained image processing model on multiple objects and enhances the quality of the enhanced image output by the image processing model.

[0018] According to at least one embodiment of the image enhancement method of this disclosure, for each of the plurality of objects, a loss value corresponding to the region where the object is located is determined based on the image of the region where the object is located in the predicted enhanced image and the image of the region where the object is located in the preset enhanced image, including: in response to the object being a face, determining a first mean absolute error and a perceptual loss between the image of the region where the face is located in the predicted enhanced image and the image of the region where the face is located in the preset enhanced image; and taking the weighted sum of the first mean absolute error and the perceptual loss as the loss value corresponding to the region where the face is located.

[0019] According to the technical solution of this embodiment, a dual constraint mechanism combining the first mean absolute error and perceptual loss is designed for the face area. This not only ensures pixel-level enhancement accuracy, but also maintains the accuracy of high-level semantic features (such as expression recognition rate, lip clarity, and detail preservation) through perceptual loss. This significantly improves the visual quality of the enhanced face area, making it particularly suitable for application scenarios such as video conferencing that require accurate face presentation.

[0020] According to at least one embodiment of the image enhancement method of this disclosure, for each of the plurality of objects, a loss value corresponding to the region where the object is located is determined based on the image of the region where the object is located in the predicted enhanced image and the image of the region where the object is located in the preset enhanced image, including: in response to the object being a light source, determining a second mean absolute error between the image of the region where the light source is located in the predicted enhanced image and the image of the region where the light source is located in the preset enhanced image; determining overexposed pixels in the image of the region where the light source is located in the predicted enhanced image, wherein the overexposed pixels are pixels with a brightness value greater than or equal to a brightness value threshold; integrating the brightness values ​​of the overexposed pixels in the image of the region where the light source is located in the predicted enhanced image, and using the integrated value as an overexposed loss value; and using the weighted sum of the second mean absolute error and the overexposed loss value as the loss value corresponding to the region where the light source is located.

[0021] According to the technical solution of this embodiment, a dual constraint mechanism combining the second mean absolute error and overexposure loss is designed for the area where the light is located. This not only ensures the accuracy of the basic reconstruction, but also effectively suppresses the overexposure problem that is common in the enhancement process of the area where the light is located by identifying and penalizing overexposed pixels. This ensures that the details of the area where the light is located are preserved and that it transitions naturally with the surrounding environment, thereby improving the dynamic range and visual comfort of the enhanced image.

[0022] According to at least one embodiment of the image enhancement method of this disclosure, for each of the plurality of objects, a loss value corresponding to the region where the object is located is determined based on the image of the region where the object is located in the predicted enhanced image and the image of the region where the object is located in the preset enhanced image, including: in response to the object being the background, determining a third mean absolute error between the image of the background region in the predicted enhanced image and the image of the background region in the preset enhanced image; performing noise filtering on the image of the background region in the predicted enhanced image to obtain a noise-filtered image; determining a fourth mean absolute error between the image of the background region in the predicted enhanced image and the noise-filtered image; and using the weighted sum of the third mean absolute error and the fourth mean absolute error as the loss value corresponding to the background region.

[0023] According to the technical solution of this embodiment, a dual constraint mechanism combining the third mean absolute error and noise loss (i.e., the fourth mean absolute error) is designed for the background area. This not only ensures the accuracy of basic reconstruction, but also reduces noise amplification in the background area, maintains the cleanliness and naturalness of the background, avoids excessive noise due to unnecessary detail enhancement, and makes the background area of ​​the enhanced image smoother and more natural, thus improving the overall visual experience.

[0024] According to another aspect of this disclosure, an electronic device is provided, comprising: a memory storing a computer program; and a processor executing the computer program stored in the memory, such that the processor performs an image enhancement method according to any embodiment of this disclosure.

[0025] According to another aspect of this disclosure, a readable storage medium is provided, wherein a computer program is stored therein, which, when executed by a processor, is used to implement the image enhancement method of any embodiment of this disclosure.

[0026] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements an image enhancement method according to any embodiment of this disclosure. Attached Figure Description

[0027] The accompanying drawings illustrate exemplary embodiments of the present disclosure and, together with the description thereof, serve to explain the principles of the present disclosure. These drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this specification.

[0028] Figure 1 This is a schematic flowchart of an image enhancement method according to one embodiment of the present disclosure.

[0029] Figure 2 This is a schematic diagram of a local enhancement process according to one embodiment of the present disclosure.

[0030] Figure 3 This is a schematic diagram illustrating the process of allocating weight values ​​according to one embodiment of the present disclosure.

[0031] Figure 4 This is a schematic diagram illustrating the process of enhancing an image using an image processing model according to one embodiment of the present disclosure.

[0032] Figure 5 This is a schematic diagram illustrating the process of training an image processing model according to one embodiment of the present disclosure.

[0033] Figure 6 This is a schematic diagram illustrating the process of determining the loss value corresponding to the region where a face is located according to one embodiment of the present disclosure.

[0034] Figure 7 This is a schematic diagram illustrating the process of determining the loss value corresponding to the area where the light is located according to one embodiment of the present disclosure.

[0035] Figure 8 This is a schematic diagram illustrating the process of determining the loss value corresponding to the background area according to one embodiment of the present disclosure.

[0036] Figure 9 This is a schematic flowchart of an image enhancement method according to another embodiment of the present disclosure.

[0037] Figure 10 This is a schematic diagram of the process of training an image processing model according to another embodiment of the present disclosure.

[0038] Figure 11 This is a schematic structural block diagram of an image enhancement apparatus according to one embodiment of the present disclosure.

[0039] Figure 12 This is a schematic block diagram of an electronic device according to one embodiment of the present disclosure. Detailed Implementation

[0040] The present disclosure will now be described in further detail with reference to the accompanying drawings and examples. It should be understood that the specific examples described herein are for illustrative purposes only and are not intended to limit the scope of the disclosure. Furthermore, it should be noted that, for ease of description, only the parts relevant to the present disclosure are shown in the accompanying drawings.

[0041] It should be noted that, where there is no conflict, the embodiments and features described in this disclosure can be combined with each other. The technical solutions of this disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0042] If a fixed weighted fusion ratio is used to weight the local and global enhanced images, for example, if the weight value of each pixel in the global enhanced image is the same and the weight value of each pixel in the local enhanced image is the same, then regardless of the image content at a pixel location, the same weight value will be used for that pixel location every time the weighted fusion is performed. This will cause the fusion process to fail to adapt to the image content, making it impossible to guarantee the quality of the enhanced image obtained by weighted fusion under different image content. This may result in uneven lighting, backlighting of people, loss of facial details, and other issues.

[0043] To this end, the present disclosure proposes the following technical solution, wherein, in the weighted fusion process of local enhanced image and global enhanced image, the weight values ​​of each pixel position are not fixed, but dynamically allocated based on the positions of multiple objects in the image, that is, the weight values ​​can change with the changes in the positions of multiple objects in the image. Therefore, the local enhanced image and global enhanced image can be fused more accurately, thereby improving the quality of the enhanced image.

[0044] The image enhancement method disclosed herein can be used to enhance the displayed image during user video calls or video conferences, thereby improving the quality of the displayed image.

[0045] In this disclosure, electronic devices include, but are not limited to, mobile phones, tablets, laptops, personal computers, wearable devices, ATMs, etc.

[0046] Figure 1 A schematic diagram illustrating the overall flow of an image enhancement method according to one embodiment of this disclosure is shown. Figure 1 The method M100 shown includes steps S110 to S150. This method can be executed by electronic devices such as mobile phones and tablets.

[0047] In step S110, semantic segmentation is performed on the image to determine the location information of various objects in the image.

[0048] Images can be directly captured photographs or image frames extracted from videos. Objects can include various elements such as faces, lights, backgrounds, text, and plants.

[0049] For example, a pre-trained semantic segmentation neural network (such as DeepLabv3+ or U-Net architecture) can be invoked to perform semantic segmentation on an image, thereby determining the location information of various objects in the image.

[0050] The semantic segmentation result can be presented as a multi-channel mask of the same size as the input image. Different channels of the mask represent the location information of different types of objects in the image. For example, the first channel mask can represent the location information of a face in the image, and the second channel mask can represent the location information of a light source. The value of each pixel in each channel mask represents the probability that the pixel belongs to the corresponding type of object, ranging from 0 to 1. For example, a value of 0.1 for the first pixel in the first channel mask indicates a probability of 0.1 for a face; a value of 0.7 for the first pixel in the second channel mask indicates a probability of 0.7 for a light source. Furthermore, for each pixel location, by comparing its probability in all channel masks, the object type corresponding to the channel with the highest probability is determined as the object type of that pixel location. The distribution area formed by different pixel locations of the same object type represents the location of that type of object in the image. Thus, through semantic segmentation, the location information of various objects in the image is determined.

[0051] In step S120, based on the location information, the image features of the region where the first type of object is located in the image are enhanced to obtain a locally enhanced image.

[0052] The first type of object can be of only one type or multiple types. The specific type of the first type of object can be pre-defined according to the local enhancement requirements, without any limitation here. In one example, the first type of object is a face, which facilitates targeted enhancement of facial details in the image and prioritizes the enhancement quality of the facial region.

[0053] In step S130, the image features of the regions containing various objects in the image are enhanced to obtain a globally enhanced image.

[0054] For example, image features of the regions containing all objects in an image are enhanced to obtain a globally enhanced image.

[0055] In this disclosure, image features may include one or more of sharpness, contrast, brightness, color, noise, and texture. This allows for the enhancement of specific image features as needed, thereby improving the image enhancement effect and flexibility. The types of image features targeted by the enhancement processing during the generation of locally enhanced images and globally enhanced images may be the same or different.

[0056] In this disclosure, the enhancement process may include one or more of the following steps: improving sharpness to make blurred edges sharper; improving contrast to make dark areas less dark and bright areas less bright, and to make details more distinct; correcting color to eliminate color cast and restore true skin tones and object colors; reducing noise to remove random graininess or spots; and restoring lost details to recover submerged texture information under conditions such as low light. In this way, different types of steps can be configured as needed, thereby improving the image enhancement effect and flexibility. The enhancement process steps used in generating locally enhanced images and globally enhanced images may be the same or different.

[0057] In step S140, weight values ​​are assigned to each pixel position in the local enhanced image and / or the global enhanced image based on the location information.

[0058] A pixel location can be understood as a spatial coordinate point in an image coordinate system, such as (x, y). A pixel location may only represent its geometric position in the image, without representing the pixel value or image content information at that position.

[0059] In one example, based on location information, weight values ​​are assigned only to the pixel locations in the locally enhanced image. In this case, the weight values ​​for the pixel locations in the globally enhanced image during the subsequent weighted fusion process can be the default weight values, thus simplifying the weighted fusion logic. For example, when the weight range is [0,1], the default weight value can be configured to 1 or 0.5.

[0060] In another example, based on location information, weight values ​​are assigned only to the pixel locations in the globally enhanced image. In this case, the weight values ​​of the pixel locations in the local enhanced image during the subsequent weighted fusion process can be the default weight values, thereby simplifying the weighted fusion logic.

[0061] In another example, weight values ​​are assigned to the pixel locations in the local and global enhanced images based on location information, thereby more accurately controlling the weighted fusion process and improving the effect of weighted fusion.

[0062] In step S150, the local enhanced image and the global enhanced image are weighted and fused based on the weight values ​​assigned to each pixel position to obtain the enhanced image.

[0063] An image, a locally enhanced image, a globally enhanced image, and an enhanced image can have the same pixel size.

[0064] For example, pixel-level weighted fusion is performed on the local enhanced image I_local and the global enhanced image I_global. The pixel value I_fused(x,y) at any pixel position (x,y) in the enhanced image I_fused is I_fused(x,y) = W_local(x,y) × I_local(x,y) + W_global(x,y) × I_global(x,y), where W_local(x,y) is the weight value of that pixel position in the local enhanced image, I_local(x,y) is the pixel value of that pixel position in the local enhanced image, W_global(x,y) is the weight value of that pixel position in the global enhanced image, and I_global(x,y) is the pixel value of that pixel position in the global enhanced image.

[0065] For example, the weighted blending process can be performed in the YUV color space, and dynamic weighting is applied only to the Y (luminance) channel, while the U and V (chrominance) channels are blended using fixed weight values ​​(such as 0.7:0.3) to maintain color stability.

[0066] The image enhancement method of this disclosure uses a weighted fusion process of local and global enhanced images. The weight values ​​of each pixel position used in the process are not fixed, but dynamically allocated based on the positions of multiple objects in the image. That is, the weight values ​​can change with the position of multiple objects in the image. Therefore, it can more accurately fuse local and global enhanced images, thereby improving the quality of the enhanced image.

[0067] Regarding step S120, as one possible implementation, it may include, for example... Figure 2 Steps S121 to S123 are shown.

[0068] In step S121, based on location information, the image is divided into a first image block and a second image block. The first image block is the image block corresponding to the area where the first type of object is located in the image, and the second image block is the image block that does not contain the area where the first type of object is located.

[0069] In step S122, the contrast and brightness of the first image block are enhanced to obtain the enhanced first image block.

[0070] Since the first image block mainly contains objects of the first type, enhancing the contrast and brightness of the first image block can focus on enhancing different regions of the first type of objects (such as facial features like eyes, nose, and lips), thereby improving the enhancement effect of different regions of the first type of objects.

[0071] For example, an adaptive histogram equalization algorithm can be used to enhance contrast, which effectively mitigates the effects of uneven lighting without introducing excessive noise. Simultaneously, gamma correction can be applied to adjust brightness, selecting an appropriate gamma value (e.g., γ=0.8) to increase shadow detail and make Type I objects more clearly visible.

[0072] In addition, to further optimize the details of the first type of object, additional sharpening filters (such as Laplacian operators) can be applied to different regions of the first type of object.

[0073] During the process of enhancing the contrast and brightness of the first image block, the parameter settings can be dynamically adjusted according to the actual image content of the first image block. For example, the brightness gain coefficient can be appropriately increased in low light environments, while the focus is on fine adjustment of contrast in high light conditions, thereby ensuring the best visual effect.

[0074] In step S123, the enhanced first image block and the second image block are stitched together to obtain a locally enhanced image.

[0075] The image enhancement method described above only performs targeted contrast and brightness enhancement on the region where the first type of object is located, so that the details of the first type of object are fully optimized, and unnecessary processing of areas that do not need enhancement is avoided, thereby improving the accuracy and efficiency of local enhancement.

[0076] Regarding step S130, as one possible implementation, it may include: enhancing the color and contrast of the regions containing multiple objects in the image to obtain a globally enhanced image.

[0077] Since an image contains multiple objects, enhancing the color and contrast of the regions containing these objects allows for targeted image enhancement of different object regions, thereby improving the enhancement effect for each region.

[0078] For example, an image can be converted from the RGB color space to the CIELAB color space, and the luminance and chrominance channels can be processed separately. For the luminance channel, a multi-scale Retinex algorithm can be used for illumination correction, employing Gaussian filters of different scales to separate the illumination and reflection components, effectively addressing the problem of uneven illumination. For the chrominance channel, differentiated enhancement can be applied to different object regions based on the semantic segmentation results. For example, the warm tone of the face region can be slightly enhanced, the saturation of the background region can be moderately enhanced, and the original chrominance of the text region can be maintained to ensure readability. Contrast enhancement can employ an adaptive strategy, dynamically adjusting the enhancement intensity based on the local texture complexity, with stronger enhancement in smooth areas and weaker enhancement in textured areas to avoid noise amplification. After processing, the enhanced channels are recombined and converted back to the RGB color space, and global tone mapping is performed to ensure a reasonable distribution of the luminance range, thus obtaining a globally enhanced image.

[0079] The image enhancement method described above effectively improves the global visual effect of the image by enhancing the color and contrast of all object areas in the image. It can solve problems such as uneven overall illumination and color distortion, providing a high-quality global enhanced image foundation for subsequent fusion steps, ensuring that the enhanced image obtained after fusion maintains overall consistency in appearance, and improving the quality of the enhanced image.

[0080] Regarding step S140, as one possible implementation, it may include, for example... Figure 3 Steps S141 and S142 are shown.

[0081] In step S141, based on the mapping relationship between object type and weight value, the weight value corresponding to the object type at each pixel position in the local enhancement image is used as the weight value of the corresponding pixel position in the local enhancement image. The weight value corresponding to the first type of object is configured to be higher than the weight value corresponding to other objects. Other objects are objects other than the first type of object among a variety of objects.

[0082] In step S142, for each pixel location, the value obtained by subtracting the weight value of that pixel location in the local enhanced image from the maximum weight threshold is used as the weight value of that pixel location in the global enhanced image.

[0083] The maximum weight threshold can be set according to requirements, for example, it can be set to 1.

[0084] For example, the various objects include a face, lights, and a background, with the face being the first type of object. In the mapping relationship, the weight value corresponding to the face can be close to 1, for example, 0.8 or 0.9, the weight value corresponding to the lights can be close to 0, for example, 0.1 or 0.2, and the weight value corresponding to the background can be around 0.5, for example, 0.4, 0.5, or 0.6. That is, the weight value of the face region in the locally enhanced image can be close to 1, the weight value of the lights region in the locally enhanced image can be close to 0, and the weight value of the background region in the locally enhanced image can be around 0.5. Since the weight value of a pixel in the local enhanced image is subtracted from the maximum weight threshold and then used as the weight value of that pixel in the global enhanced image, the weight values ​​of the same pixel in the local and global enhanced images are complementary. Therefore, in the weighted fusion process with a maximum weight threshold of 1, the weight value of the face region in the local enhanced image is close to 1 while the weight value of the face region in the global enhanced image is close to 0; the weight value of the light region in the local enhanced image is close to 0 while the weight value of the light region in the global enhanced image is close to 1; and the weight value of the background region in the local enhanced image is similar to that in the global enhanced image. This ensures that during the weighted fusion process, the face region mainly uses the local enhancement result to improve the facial details in the enhanced image; the light region mainly uses the global enhancement result to avoid overexposure of the lights in the enhanced image; and the background region balances the local and global enhancement results.

[0085] The image enhancement method described above assigns a higher weight value to the first type of object (such as a face), so that it mainly retains the local enhancement result in the fusion result, while the global enhancement result participates in the weighted fusion through complementary weights. Thus, intelligent adaptive fusion based on semantic understanding is realized, so that the area where the first type of object is located is emphasized for enhancement, improving the display effect of the first type of object in the enhanced image, while ensuring the overall quality of the enhanced image.

[0086] Regarding steps S110 to S150, as another possible implementation, they may each include as follows: Figure 4 Steps S111' to S151' are shown.

[0087] In step S111', an image is provided to the first processing unit of the image processing model, and the first processing unit performs semantic segmentation on the image to determine the location information of various objects in the image.

[0088] In this disclosure, the image processing model is a model that obtains an enhanced image by performing semantic segmentation, local enhancement, global enhancement, weight value allocation, and weighted fusion on an input image. The image processing model may include a first processing unit, a second processing unit, a third processing unit, and a fourth processing unit, which are respectively capable of performing semantic segmentation, local enhancement, global enhancement, and weight value allocation.

[0089] The first processing unit can employ a lightweight semantic segmentation network, such as DeepLabv3+ or U-Net.

[0090] In step S121', the second processing unit of the image processing model is provided with the image and the location information of the first type of object in the image. The second processing unit performs image feature enhancement processing on the region where the first type of object is located in the image to obtain a locally enhanced image.

[0091] The second processing unit can adopt a U-Net structure, which can include an encoder and a decoder. The position information of the first type of object in the image is then incorporated into the skip connections as attention weights, enabling the second processing unit to focus on the region where the first type of object (such as a face) is located, thereby specifically enhancing the details of the first type of object in the image.

[0092] In step S131', an image is provided to the third processing unit of the image processing model, and the third processing unit performs image feature enhancement processing on the regions where multiple objects are located in the image to obtain a globally enhanced image.

[0093] The third processing unit can employ a 4-layer CNN network. This third processing unit primarily optimizes the overall color distribution and contrast, thereby obtaining a globally enhanced image.

[0094] The second and third processing units can execute the corresponding steps in parallel.

[0095] In step S141', the fourth processing unit of the image processing model is provided with the position information of the image and various objects in the image, and the fourth processing unit assigns weight values ​​to the position of each pixel in the locally enhanced image.

[0096] The fourth processing unit may include a stitching layer, a first convolutional layer, a second convolutional layer, and a sigmoid activation function. The stitching layer can stitch together the received image and the positional information of various objects in the image (i.e., semantic mask). The first and second convolutional layers can learn the correlation features between image content and semantic information in the stitched features. The learned features can be multi-channel features. After being compressed into single-channel features, a single-channel weight map can be generated using the sigmoid activation function. The size of the single-channel weight map is the same as the size of the local enhancement image, and the weight value of each pixel position is the weight value of that pixel position in the local enhancement image.

[0097] The first processing unit, the second processing unit, the third processing unit, and the fourth processing unit are independent processing units.

[0098] In step S151', the local enhancement image and the global enhancement image are weighted and fused according to the weight values ​​assigned to each pixel position in the local enhancement image by the image processing model to obtain the enhanced image.

[0099] After obtaining the weight values, local enhanced image, and global enhanced image, the image processing model can perform pixel-level weighted fusion operations on the local enhanced image and the global enhanced image based on the weighted fusion rules defined in the forward method, thereby obtaining and outputting the enhanced image.

[0100] The image enhancement method described above integrates semantic segmentation, local enhancement, global enhancement, weight allocation, and weighted fusion into a unified image processing model. This model enables an end-to-end image enhancement process, which not only improves the efficiency of image enhancement and reduces intermediate data transmission overhead, but also ensures the effective transmission of semantic information throughout the entire processing chain, resulting in a more accurate and consistent final enhancement effect.

[0101] In some embodiments of this disclosure, the image processing model can be implemented through, for example... Figure 5 The training is completed through steps S210 to S250 as shown.

[0102] In step S210, a preset image is provided to the image processing model, and the preset image is processed by the image processing model to obtain the predicted position information of various objects in the preset image and the predicted enhanced image.

[0103] During the training phase, the input to the image processing model can be a preset image, and the output can be the predicted location information of various objects in the preset image and the predicted enhanced image; during the application phase after training, the input to the image processing model can be an image, and the output can be the enhanced image.

[0104] In step S220, based on the predicted location information of multiple objects in the preset image, the images of the regions where multiple objects are located in the predicted enhanced image and the preset enhanced image corresponding to the preset image are determined respectively.

[0105] The predicted augmented image can be understood as the image before and after augmentation output by an image processing model that has not yet been trained. The image augmentation effect in the predicted augmented image may still differ from the training target.

[0106] The pre-defined enhanced image can be understood as a manually annotated enhanced image, and the image enhancement effect in the pre-defined enhanced image is the training target of the image processing model.

[0107] In step S230, for each of the multiple objects, the loss value corresponding to the region where the object is located is determined based on the image of the region where the object is located in the predicted enhanced image and the image of the region where the object is located in the preset enhanced image.

[0108] In step S240, the loss values ​​corresponding to multiple objects are weighted and summed, and the resulting sum is used as the loss value of the image processing model.

[0109] The weights used for the weighted summation of loss values ​​for multiple objects can be configured according to requirements.

[0110] In one example, multiple objects are included, such as faces, lights, and backgrounds. The loss value corresponding to the face has a weight of 0.6 in the weighted summation process, the loss value corresponding to the lights has a weight of 0.25 in the weighted summation process, and the loss value corresponding to the background has a weight of 0.15 in the weighted summation process. This makes the image processing model training focus more on face enhancement, thus improving the image processing model's face enhancement effect.

[0111] In step S250, the parameters of the image processing model are adjusted based on the loss value of the image processing model to obtain a trained image processing model.

[0112] For example, if the loss value of the image processing model has not converged, the parameters of the image processing model can be adjusted based on a preset step size, and steps S210 to S240 can be re-executed based on the adjusted image processing model until the loss value of the image processing model converges. If the loss value of the image processing model converges, training is stopped, and the image processing model at this point is taken as the trained image processing model.

[0113] The image enhancement method described above first calculates the loss value corresponding to the region where each object is located based on the location information (i.e., semantic information) of each object. Then, it uses the weighted sum of the loss values ​​corresponding to the regions where multiple objects are located as the loss value of the image processing model. Thus, semantic information is incorporated into the loss value of the image processing model, enriching the loss function of the image processing model. This allows the image processing model to be trained under the guidance of semantic information, making the image processing model pay more attention to the actual enhancement effect of the regions where each object is located during the training process. This improves the image enhancement effect of the trained image processing model on multiple objects and enhances the quality of the enhanced image output by the image processing model.

[0114] Regarding step S230, in some embodiments of this disclosure, it may include, for example... Figure 6 Steps S2301 and S2302 are shown.

[0115] In step S2301, in response to the object being a face, a first mean absolute error and perceptual loss are determined between the image of the face region in the predicted enhanced image and the image of the face region in the preset enhanced image.

[0116] The first mean absolute error is the average absolute error between the predicted image of the face region in the enhanced image and the preset image of the face region in the enhanced image.

[0117] The mean absolute error and perceived loss can be calculated using existing technical methods.

[0118] In step S2302, the weighted sum of the first average absolute error and the perceptual loss is used as the loss value corresponding to the region where the face is located.

[0119] The weight values ​​used in the weighted summation of the first mean absolute error and perceived loss can be configured according to requirements.

[0120] The image enhancement method described above employs a dual constraint mechanism that combines the first mean absolute error and perceptual loss for the face region. This mechanism not only ensures pixel-level enhancement accuracy but also maintains the accuracy of high-level semantic features (such as expression recognition rate, lip clarity, and detail preservation) through perceptual loss. This significantly improves the visual quality of the enhanced face region, making it particularly suitable for applications requiring precise face presentation, such as video conferencing.

[0121] In other implementations, the first mean absolute error can be used as the loss value corresponding to the face region, thereby simplifying the calculation logic of the loss value and improving the training speed of the model.

[0122] Regarding step S230, in some embodiments of this disclosure, it may also include, as follows: Figure 7 Steps S2303 to S2306 are shown.

[0123] In step S2303, in response to the object being a light, a second mean absolute error is determined between the image of the region where the light is located in the predicted enhanced image and the image of the region where the light is located in the preset enhanced image.

[0124] The second mean absolute error is the average absolute error between the image of the light-bearing area in the predicted enhanced image and the image of the light-bearing area in the preset enhanced image.

[0125] In step S2304, overexposed pixels in the area where the light is located in the predicted enhanced image are determined. Overexposed pixels are pixels whose brightness value is greater than or equal to a brightness value threshold.

[0126] The brightness threshold can be set according to requirements.

[0127] In step S2305, the brightness values ​​of overexposed pixels in the area where the light is located in the predicted enhanced image are integrated, and the integrated value is used as the overexposure loss value.

[0128] For example, integration can be achieved by summing, squaring, or weighting the brightness values ​​of overexposed pixels in the area where the light is located in the predicted enhanced image.

[0129] In step S2306, the weighted sum of the second average absolute error and the overexposure loss value is used as the loss value corresponding to the area where the light is located.

[0130] The weight values ​​used in the weighted summation of the second mean absolute error and the overexposure loss value can be configured according to requirements.

[0131] The image enhancement method described above employs a dual constraint mechanism that combines the second mean absolute error and overexposure loss for the area where the light is located. This mechanism not only ensures the accuracy of the basic reconstruction but also effectively suppresses the overexposure problem commonly encountered in the enhancement process of the area where the light is located by identifying and penalizing overexposed pixels. This ensures that the details of the area where the light is located are preserved and that the image transitions naturally with the surrounding environment, thereby improving the dynamic range and visual comfort of the enhanced image.

[0132] In other implementations, the second mean absolute error can be used as the loss value corresponding to the area where the light is located, thereby simplifying the calculation logic of the loss value and improving the training speed of the model.

[0133] Regarding step S230, in some embodiments of this disclosure, it may also include, as follows: Figure 8 Steps S2307 to S2310 are shown.

[0134] In step S2307, in response to the object being the background, a third mean absolute error is determined between the image of the background region in the predicted enhanced image and the image of the background region in the preset enhanced image.

[0135] The third mean absolute error is the average absolute error between the predicted image of the background region in the enhanced image and the image of the background region in the preset enhanced image.

[0136] In step S2308, noise filtering is performed on the image of the background region in the predicted enhanced image to obtain a noise-filtered image.

[0137] Noise filtering can be achieved using existing noise filtering methods.

[0138] In step S2309, the fourth mean absolute error between the image of the background region in the predicted enhanced image and the image after noise filtering is determined.

[0139] The fourth mean absolute error is the mean absolute error between the image of the background region in the enhanced image and the image after noise filtering.

[0140] In step S2310, the weighted sum of the third mean absolute error and the fourth mean absolute error is used as the loss value corresponding to the background region.

[0141] The weight values ​​used in the weighted summation of the third and fourth mean absolute errors can be configured according to requirements.

[0142] The image enhancement method described above employs a dual constraint mechanism that combines the third mean absolute error and noise loss (i.e., the fourth mean absolute error) for the background region. This mechanism not only ensures the accuracy of the basic reconstruction but also reduces noise amplification in the background region, maintaining the cleanliness and naturalness of the background. It avoids excessive noise caused by unnecessary detail enhancement, resulting in a smoother and more natural background region in the enhanced image and improving the overall visual experience.

[0143] In other implementations, the third mean absolute error can be used as the loss value corresponding to the background region, thereby simplifying the calculation logic of the loss value and improving the training speed of the model.

[0144] It should be noted that the specific values ​​mentioned above are only for illustrating the implementation of this disclosure in detail, and should not be construed as limiting the disclosure. In other examples, implementation methods, or embodiments, other values ​​may be selected based on this disclosure, and no specific limitations are made here.

[0145] Please combine Figure 9 In one example, the image enhancement method may include the following steps S301 to S306. The content related to steps S301 to S306 can be referred to the description of the above implementation method. For the sake of brevity, it will not be repeated here.

[0146] In step S301, video conference images are acquired in the conference room.

[0147] In step S302, a video conference image is provided to the lightweight semantic segmentation network of the image processing model. The lightweight semantic segmentation network performs semantic segmentation on the video conference image to obtain a three-channel semantic mask. The three-channel semantic mask represents the position information of three objects in the video conference image: face, light, and background.

[0148] In step S303, video conferencing images and facial semantic masks are provided to the U-Net network of the image processing model. The brightness and contrast of the facial area in the video conferencing image are enhanced by the U-Net network to obtain a locally enhanced image.

[0149] In step S304, a video conference image is provided to the 4-layer CNN network of the image processing model. The 4-layer CNN network enhances the color and contrast of the areas containing the three objects—face, light, and background—in the video conference image to obtain a globally enhanced image.

[0150] In step S305, a video conference image and a three-channel semantic mask are provided to the weight value generation unit (including the stitching layer, the first convolutional layer, the second convolutional layer, and the Sigmoid activation function) of the image processing model. A single-channel weight map is generated by the weight value generation unit, which represents the weight value of each pixel position in the locally enhanced image.

[0151] In step S306, the local enhancement image and the global enhancement image are weighted and fused according to the single-channel weight map by the image processing model to obtain the enhanced video conferencing image.

[0152] Please combine Figure 10 In one example, the training method for the image processing model may include the following steps S401 to S407. The content related to steps S401 to S407 can be referred to the description of the above implementation method. For the sake of brevity, it will not be repeated here.

[0153] In step S401, a preset video conference image is provided to the image processing model. The preset video conference image is processed by the image processing model to obtain a predicted three-channel semantic mask and a predicted enhanced video conference image. The predicted three-channel semantic mask is used to represent the position information of three objects—face, light, and background—in the preset video conference image.

[0154] In step S402, based on the predicted three-channel semantic mask, the images of the regions containing the three objects—face, light, and background—in the predicted enhanced video conferencing image and the preset enhanced video conferencing image are determined respectively.

[0155] In step S403, the loss value corresponding to the region where the face is located is calculated. For the specific calculation process, please refer to the description of steps S2301 and S2302.

[0156] In step S404, the loss value corresponding to the area where the light is located is calculated. For the specific calculation process, please refer to the description of steps S2303 to S2306.

[0157] In step S405, the loss value corresponding to the background region is calculated. For details of the calculation process, please refer to the descriptions of steps S2307 to S2310.

[0158] In step S406, the loss values ​​corresponding to the face region, the light region, and the background region are weighted and summed, and the sum is used as the loss value of the image processing model.

[0159] In step S407, the parameters of the image processing model are adjusted based on the loss value of the image processing model to obtain the trained image processing model.

[0160] The image processing model trained through steps S401 to S407 can significantly improve the quality of face region enhancement, increase the expression recognition rate by 25%, effectively control the overexposure problem in the lighting area, reduce the overexposed area by 50%, maintain the naturalness of the background area, and reduce noise by 40%; and improve the overall image quality subjective score by 35%.

[0161] Based on any of the above embodiments, this disclosure also provides an image enhancement apparatus.

[0162] Figure 11 This is a schematic block diagram of the structure of an image enhancement device according to one embodiment of the present disclosure.

[0163] like Figure 11 As shown, the image enhancement device includes: a semantic segmentation module 110, used to perform semantic segmentation on the image and determine the location information of various objects in the image; a local enhancement module 120, used to enhance the image features of the region where a first type of object is located in the image based on the location information to obtain a locally enhanced image; a global enhancement module 130, used to enhance the image features of the region where various objects are located in the image to obtain a globally enhanced image; an allocation module 140, used to allocate weight values ​​to each pixel position in the locally enhanced image and / or the globally enhanced image based on the location information; and a fusion module 150, used to perform weighted fusion of the locally enhanced image and the globally enhanced image based on the weight values ​​allocated to each pixel position to obtain an enhanced image.

[0164] The aforementioned image enhancement device can be in the form of computer software, and each module of the aforementioned image enhancement device can be implemented through computer software modules.

[0165] The specific implementation process of the functions and roles of each module in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0166] The image enhancement method in the specific embodiments of this disclosure can be implemented by electronic devices such as mobile phones and computers.

[0167] Therefore, based on any of the above embodiments, this disclosure also provides an electronic device that can perform the image enhancement method of any of the embodiments described above.

[0168] Figure 12 This is a schematic block diagram of an electronic device 1000 according to one embodiment of the present disclosure.

[0169] The hardware structure of electronic device 1000 can be implemented using a bus architecture. The bus architecture can include any number of interconnect buses and bridges, depending on the specific application and overall design constraints of the hardware. Bus 1100 connects various circuits including one or more processors 1200, memory 1300, and / or hardware modules. Bus 1100 can also connect various other circuits 1400 such as peripherals, voltage regulators, power management circuits, external antennas, etc. Bus 1100 can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Component (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, only one connection line is used in this figure, but this does not indicate that there is only one bus or one type of bus.

[0170] For ease of explanation, certain steps of the above method are described in relation to modules. It should be understood that the corresponding module performing one or more steps of the above method may be one or more hardware modules specifically configured to perform the corresponding step, or implemented by a processor configured to perform the corresponding step, or stored in a computer-readable medium for implementation by a processor, or implemented by some combination thereof.

[0171] This disclosure also provides a readable storage medium storing a computer program that, when executed by a processor, is used to implement the methods described above. A "readable storage medium" can be any means capable of containing, storing, communicating, propagating, or transmitting a program for use by or in conjunction with an instruction execution system, apparatus, or device. More specific examples of a readable storage medium include: an electrical connection with one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and portable read-only memory (CDROM), etc.

[0172] This disclosure also provides a computer program product, the methods of which can be implemented wholly or partially through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented wholly or partially as a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed, all or part of the processes or functions of this disclosure are performed.

[0173] Computer programs or instructions can be stored in a readable storage medium or transferred from one readable storage medium to another. For example, the computer program or instructions can be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The readable storage medium can be any available medium capable of access, or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium, such as a floppy disk, hard disk, or magnetic tape; an optical medium, such as a digital video optical disc; or a semiconductor medium, such as a solid-state drive. The computer-readable storage medium can be a volatile or non-volatile storage medium, or it can include both volatile and non-volatile types of storage media.

[0174] Those skilled in the art will understand that embodiments of this disclosure can be provided as methods, apparatus, or computer program products. Therefore, this disclosure can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this disclosure can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0175] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus, and computer program products according to this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0176] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0177] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0178] In the description of this specification, the references to terms such as "one embodiment / mode," "some embodiments / modes," "example," "specific example," or "some examples," etc., refer to specific features, structures, or characteristics described in connection with that embodiment / mode or example, which are included in at least one embodiment / mode or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment / mode or example. Moreover, the specific features, structures, or characteristics described may be combined in any suitable manner in one or more embodiments / modes or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments / modes or examples described in this specification, as well as the features of different embodiments / modes or examples.

[0179] Those skilled in the art should understand that the above embodiments are merely for illustrating the present disclosure and are not intended to limit the scope of the disclosure. Those skilled in the art can make other changes or modifications based on the above disclosure, and these changes or modifications still fall within the scope of the present disclosure.

Claims

1. An image enhancement method, characterized in that, include: Perform semantic segmentation on the image to determine the location information of various objects in the image; Based on the location information, the image features of the region where the first type of object is located in the image are enhanced to obtain a locally enhanced image; The image features of the regions containing multiple objects in the image are enhanced to obtain a globally enhanced image; Based on the location information, weight values ​​are assigned to each pixel position in the local enhanced image and / or the global enhanced image; as well as Based on the weight values ​​assigned to each pixel position, the local enhanced image and the global enhanced image are weighted and fused to obtain the enhanced image.

2. The image enhancement method according to claim 1, characterized in that, Based on the location information, the image features of the region where the first type of object is located in the image are enhanced to obtain a locally enhanced image, including: Based on the location information, the image is divided into a first image block and a second image block. The first image block is the image block corresponding to the area where the first type of object is located in the image, and the second image block is the image block that does not contain the area where the first type of object is located. The contrast and brightness of the first image block are enhanced to obtain an enhanced first image block; and The enhanced first image block and the second image block are stitched together to obtain the locally enhanced image.

3. The image enhancement method according to claim 1, characterized in that, The image features of regions containing multiple objects in the image are enhanced to obtain a globally enhanced image, including: The color and contrast of the regions containing various objects in the image are enhanced to obtain the globally enhanced image.

4. The image enhancement method according to claim 1, characterized in that, Based on the location information, weight values ​​are assigned to each pixel location in the local enhanced image and / or the global enhanced image, including: Based on the mapping relationship between object type and weight value, the weight value corresponding to the object type at each pixel location in the local enhancement image is used as the weight value at the corresponding pixel location in the local enhancement image. Specifically, the weight value corresponding to the first type of object is configured to be higher than the weight values ​​corresponding to other objects, where the other objects are objects other than the first type of object among the various types of objects. For each pixel location, the weight value of that pixel location in the local enhanced image is obtained by subtracting the weight value of that pixel location from the maximum weight threshold.

5. The image enhancement method according to claim 1, characterized in that, Semantic segmentation of the image is performed to determine the location information of various objects in the image, including: The image is provided to the first processing unit of the image processing model, and the first processing unit performs semantic segmentation on the image to determine the location information of various objects in the image; Based on the location information, the image features of the region where the first type of object is located in the image are enhanced to obtain a locally enhanced image, including: The image and the location information of the first type of object in the image are provided to the second processing unit of the image processing model. The second processing unit then performs image feature enhancement processing on the region where the first type of object is located in the image to obtain the locally enhanced image. The image features of regions containing multiple objects in the image are enhanced to obtain a globally enhanced image, including: The image is provided to the third processing unit of the image processing model, and the third processing unit performs image feature enhancement processing on the regions where multiple objects are located in the image to obtain the global enhanced image; Based on the location information, weight values ​​are assigned to each pixel location in the local enhanced image and / or the global enhanced image, including: The fourth processing unit of the image processing model is provided with the position information of the image and the various objects in the image, and the fourth processing unit assigns weight values ​​to each pixel position in the locally enhanced image. Based on the weight values ​​assigned to each pixel location, the local enhanced image and the global enhanced image are weighted and fused to obtain the enhanced image, including: The image processing model performs weighted fusion of the local enhanced image and the global enhanced image based on the weight values ​​assigned to each pixel position in the local enhanced image, thereby obtaining the enhanced image.

6. The image enhancement method according to claim 5, characterized in that, The image processing model is trained through the following steps: A preset image is provided to the image processing model, and the preset image is processed by the image processing model to obtain the predicted location information of various objects in the preset image and the predicted enhanced image; Based on the predicted location information of the multiple objects in the preset image, images of the regions where the multiple objects are located in the predicted enhanced image and the preset enhanced image corresponding to the preset image are determined respectively. For each of the various objects, the loss value corresponding to the region where the object is located is determined based on the image of the region where the object is located in the predicted enhanced image and the image of the region where the object is located in the preset enhanced image; The loss values ​​corresponding to the various objects are weighted and summed, and the sum is used as the loss value of the image processing model. as well as The parameters of the image processing model are adjusted based on the loss value of the image processing model to obtain a trained image processing model.

7. The image enhancement method according to claim 6, characterized in that, For each of the multiple objects, based on the image of the region where the object is located in the predicted enhanced image and the image of the region where the object is located in the preset enhanced image, the loss value corresponding to the region where the object is located is determined, including: In response to the object being a face, a first mean absolute error and perceptual loss are determined between the image of the face region in the predicted enhanced image and the image of the face region in the preset enhanced image; and The weighted sum of the first mean absolute error and the perceptual loss is used as the loss value corresponding to the region where the face is located.

8. The image enhancement method according to claim 6, characterized in that, For each of the multiple objects, based on the image of the region where the object is located in the predicted enhanced image and the image of the region where the object is located in the preset enhanced image, the loss value corresponding to the region where the object is located is determined, including: In response to the object being a light source, a second mean absolute error is determined between the image of the region where the light source is located in the predicted enhanced image and the image of the region where the light source is located in the preset enhanced image; Identify overexposed pixels in the area where the light is located in the predicted enhanced image, wherein the overexposed pixels are pixels with a brightness value greater than or equal to a brightness value threshold; Integrate the brightness values ​​of overexposed pixels in the area where the light source is located in the predicted enhanced image, and use the integrated value as the overexposure loss value; and The weighted sum of the second average absolute error and the overexposure loss value is taken as the loss value corresponding to the area where the light is located.

9. An electronic device, characterized in that, include: The memory stores computer programs; as well as A processor that executes a computer program stored in the memory, causing the processor to perform the image enhancement method according to any one of claims 1 to 8.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the image enhancement method according to any one of claims 1 to 8.

Images