Shadow area contrast enhancement method, model training method, device and medium

Abstract

The present disclosure provides a shadow area contrast enhancement method, a model training method, an apparatus and a medium. The contrast enhancement method comprises: inputting a first image into a pre-trained contrast enhancement model, the first image comprising a first shadow area; outputting a second image from the contrast enhancement model, the second image comprising a second shadow area corresponding to the first shadow area, the contrast of the second shadow area being higher than that of the first shadow area, wherein the contrast enhancement model generates a segmentation mask of the first shadow area, and performs local contrast enhancement on the first shadow area based on the segmentation mask while preserving the original contrast information of the non-shadow area. The present disclosure enhances the contrast of the shadow area in the image, thereby improving the image display effect and detail performance.

Description

TECHNICAL FIELD

[0001] The present disclosure belongs to the technical field of image processing, and particularly relates to a shadow region contrast enhancement method, a model training method, a device and a medium. BACKGROUND

[0002] With the continuous development of display technology, people's requirements for visual effects are constantly improving. Whether watching videos, playing games or engaging in other visual activities, people hope to obtain more realistic, clear and smooth picture effects. In many practical applications, images are often affected by uneven lighting, resulting in low contrast in shadow regions, loss of details and poor display effects. However, existing local contrast enhancement techniques mostly adopt global adaptive enhancement methods, which can improve the overall contrast of images to some extent, but have limited effect when processing specific local regions. These methods often cannot accurately identify and enhance shadow regions, resulting in unnecessary adjustment of the contrast of non-shadow regions, which further affects the naturalness and overall visual effect of the image. In addition, due to the lack of precise control over local regions, these techniques are prone to over-enhancement or insufficient enhancement when processing complex scenes, which cannot meet users' demand for high-quality images. SUMMARY

[0003] In view of the above problems, the present disclosure provides a shadow region contrast enhancement method, a model training method, a device and a medium, aiming to specifically enhance the contrast of shadow regions in images, thereby improving image display effects and detail performance.

[0004] According to a first aspect of the present disclosure, a shadow region contrast enhancement method is provided, comprising:

[0005] inputting a first image into a pre-trained contrast enhancement model, the first image including a first shadow region;

[0006] outputting a second image from the contrast enhancement model, the second image including a second shadow region corresponding to the first shadow region, the contrast of the second shadow region being higher than that of the first shadow region, wherein the contrast enhancement model generates a segmentation mask of the first shadow region, and performs local contrast enhancement on the first shadow region based on the segmentation mask while preserving the original contrast information of non-shadow regions.

[0007] Optionally, the contrast enhancement model includes a segmentation mask generation model, an image segmentation model and an image enhancement model, and the outputting of the second image from the contrast enhancement model comprises:

[0008] performing a binarization operation on the first image by the segmentation mask generation model to generate a segmentation mask of the first shadow region;

[0009] performing a dilation convolution operation on the segmentation mask of the first shadow area by the segmentation mask generation model to obtain a dilated segmentation mask of the first shadow area;

[0010] segmenting the first image based on the segmentation mask of the first shadow area by the image segmentation model to obtain a first segmentation map of the first shadow area;

[0011] segmenting the first image based on the dilated segmentation mask of the first shadow area by the image segmentation model to obtain a second segmentation map of the first shadow area;

[0012] performing image stitching on the first segmentation map, the second segmentation map and the first image by the image enhancement model to obtain the second image.

[0013] Optionally, the performing image stitching on the first segmentation map, the second segmentation map and the first image by the image enhancement model to obtain the second image comprises:

[0014] performing image enhancement operations on the first segmentation map, the second segmentation map and the first image respectively by the image enhancement model, and performing image stitching on the first segmentation map, the second segmentation map and the first image after the image enhancement operations;

[0015] performing image enhancement operations on the image obtained by the image stitching to obtain the second image.

[0016] According to a second aspect of the present disclosure, a training method of a shadow area contrast enhancement model is provided, comprising:

[0017] obtaining a training data set, wherein the training data set comprises a plurality of training samples and shadow area labels corresponding to the training samples;

[0018] inputting the training samples into a target contrast enhancement model, wherein the target contrast enhancement model comprises a target segmentation mask generation model, a target image segmentation model and a target image enhancement model;

[0019] adopting a secondary supervision mechanism to calculate loss functions of the target segmentation mask generation model and the target image enhancement model respectively;

[0020] performing reverse learning by using the loss functions to optimize model parameters of the target segmentation mask generation model, the target image segmentation model and the target image enhancement model.

[0021] Optionally, the adopting a secondary supervision mechanism to calculate loss functions of the target segmentation mask generation model and the target image enhancement model respectively comprises:

[0022] inputting the training sample into the target segmentation mask generation model, performing a binarization operation on the training sample by the target segmentation mask generation model, and generating a segmentation mask of a shadow region in the training sample;

[0023] calculating a loss function of the target segmentation mask generation model based on the segmentation mask of the shadow region in the training sample and the corresponding shadow region label of the training sample.

[0024] Optionally, the adopting a secondary supervision mechanism and calculating the loss functions of the target segmentation mask generation model and the target image enhancement model respectively further includes:

[0025] performing a dilated convolution operation on the segmentation mask of the shadow region in the training sample by the target segmentation mask generation model to obtain a dilated segmentation mask of the shadow region in the training sample;

[0026] segmenting the training sample based on the segmentation mask of the shadow region in the training sample by the target image segmentation model to obtain a first segmentation map of the shadow region in the training sample;

[0027] segmenting the training sample based on the dilated segmentation mask of the shadow region in the training sample by the target image segmentation model to obtain a second segmentation map of the shadow region in the training sample;

[0028] performing image stitching on the first segmentation map, the second segmentation map and the training sample by the target image enhancement model to obtain a target image of the training sample;

[0029] calculating a loss function of the target image enhancement model based on the target image of the training sample and the corresponding shadow region label of the training sample.

[0030] Optionally, the performing image stitching on the first segmentation map, the second segmentation map and the training sample by the target image enhancement model to obtain a target image of the training sample includes:

[0031] performing image enhancement operations on the first segmentation map, the second segmentation map and the training sample by the target image enhancement model respectively, and performing image stitching on the first segmentation map, the second segmentation map and the training sample after the image enhancement operations;

[0032] performing image enhancement operations on the stitched image to obtain a target image of the training sample.

[0033] According to a third aspect of the present disclosure, there is provided a device for contrast enhancement of a shadow region, comprising:

[0034] An image input unit is configured to input a first image into a pre-trained contrast enhancement model, the first image including a first shadow region;

[0035] A contrast enhancement unit is configured to output a second image from the contrast enhancement model, the second image including a second shadow region corresponding to the first shadow region, the second shadow region having a higher contrast than the first shadow region, wherein the contrast enhancement model generates a segmentation mask of the first shadow region, and performs local contrast enhancement on the first shadow region based on the segmentation mask while preserving original contrast information of a non-shadow region.

[0036] According to a fourth aspect of the present disclosure, a training device of a shadow region contrast enhancement model is provided, comprising:

[0037] A training data acquisition unit is configured to acquire a training data set, the training data set including a plurality of training samples and shadow region labels corresponding to the training samples;

[0038] A training sample input unit is configured to input the training samples into a target contrast enhancement model, the target contrast enhancement model including a target segmentation mask generation model, a target image segmentation model, and a target image enhancement model;

[0039] A loss function calculation unit is configured to calculate loss functions of the target segmentation mask generation model and the target image enhancement model respectively using a quadratic supervision mechanism;

[0040] A model parameter optimization unit is configured to perform reverse learning using the loss functions to optimize model parameters of the target segmentation mask generation model, the target image segmentation model, and the target image enhancement model.

[0041] According to a fifth aspect of the present disclosure, an electronic device is provided, comprising a processor, a memory, and a program stored on the memory and executable on the processor, the program being executed by the processor to implement the steps of the method as described above.

[0042] According to a sixth aspect of the present disclosure, a storage medium is provided, the storage medium storing a computer program or instructions, the computer program or instructions being executed by a processor to implement the steps of the method as described above.

[0043] According to a seventh aspect of the present disclosure, a chip is provided, comprising:

[0044] A contrast enhancement device as described above implements the method.

[0045] According to a seventh aspect of the present disclosure, a chip is provided, comprising:

[0046] The device is trained as described above to implement the method.

[0047] The present disclosure brings the following beneficial effects:

[0048] The contrast enhancement method of the shadow area provided by the present disclosure inputs the first image into the pre-trained contrast enhancement model, the contrast enhancement model generates a segmentation mask of the first shadow area of the first image, performs local contrast enhancement on the first shadow area based on the segmentation mask while preserving the original contrast information of the non-shadow area, and the second image output by the contrast enhancement model includes a second shadow area corresponding to the first shadow area, the contrast of the second shadow area is higher than that of the first shadow area, and the non-shadow area of the second image retains the contrast information of the corresponding non-shadow area of the first image. In this way, by accurately positioning the shadow area and performing contrast enhancement on the shadow area, the problem of unnecessary adjustment of the contrast of the non-shadow area caused by the traditional global self-adaptive enhancement algorithm is effectively solved, the image display effect and detail performance are improved, and the computational complexity is reduced.

[0049] The training method of the contrast enhancement model of the shadow area provided by the present disclosure, the training data set is obtained, the training data set includes a plurality of training samples and a shadow area label corresponding to the training sample, the training sample is input into the target contrast enhancement model, the target contrast enhancement model includes a target segmentation mask generation model, a target image segmentation model and a target image enhancement model, a secondary supervision mechanism is adopted, the loss functions of the target segmentation mask generation model and the target image enhancement model are calculated respectively, the loss functions are used for reverse learning, and the model parameters of the target segmentation mask generation model, the target image segmentation model and the target image enhancement model are optimized. In this way, by introducing the secondary supervision mechanism, the collaborative optimization of the segmentation mask generation and the image enhancement process is realized, and the model training efficiency and accuracy are improved.

[0050] Other features and advantages of the present disclosure will be described in the following description, and some will become apparent from the description, or will be learned from practice of the present disclosure. The purposes and other advantages of the present disclosure are achieved and obtained by the structures specifically pointed out in the description and the drawings.

[0051] In order to make the above-mentioned purposes, features and advantages of the present disclosure more obvious and easy to understand, the following preferred embodiments are specifically described, and the accompanying drawings are described in detail as follows. BRIEF DESCRIPTION OF DRAWINGS

[0052] The above and other purposes, features and advantages of the present disclosure will become more apparent from the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0053] Figure 1A flowchart of a contrast enhancement method for a shadow region according to an embodiment of the present disclosure is provided.

[0054] Figure 2A An algorithm block diagram of a contrast enhancement model according to an embodiment of the present disclosure is provided.

[0055] Figure 2B A loss function diagram of a target segmentation mask generation model according to an embodiment of the present disclosure is provided.

[0056] Figure 2C A loss function diagram of a target image enhancement model according to an embodiment of the present disclosure is provided.

[0057] Figure 3 A flowchart of a training method of a contrast enhancement model for a shadow region according to an embodiment of the present disclosure is provided.

[0058] Figure 4 A structural diagram of a contrast enhancement device for a shadow region according to an embodiment of the present disclosure is provided.

[0059] Figure 5 A structural diagram of a training device of a contrast enhancement model for a shadow region according to an embodiment of the present disclosure is provided.

[0060] Figure 6 A structural diagram of an electronic device according to an embodiment of the present disclosure is provided. DETAILED DESCRIPTION

[0061] Various embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. In the various drawings, the same elements are denoted by the same or similar reference numerals to indicate the same or similar elements. For the sake of clarity, each part in the drawings is not drawn to scale.

[0062] The following terms are used herein:

[0063] A shadow region refers to a region in which the local brightness is significantly lower than the surrounding region due to the blocking of a light source (such as the sun, light, etc.) by an object in the application scenario displayed by the display panel. The shadow region can include self-shadows (shadow regions formed by the blocking of light by the structure of the object itself) and projected shadows (shadows projected onto other surfaces by the object).

[0064] Contrast, refers to the brightness difference between adjacent regions or objects in an image, usually quantified by the ratio or difference between the maximum and minimum brightness values. In the field of digital image processing, contrast reflects the visibility and level of image details, higher contrast can make the edges, textures and structural features in the image more clear and distinguishable, while low contrast areas often appear dark and blurred. Especially in the shadow area, the brightness compression effect caused by light attenuation will significantly reduce the contrast value of the area, making it difficult to identify the details.

[0065] Figure 1 The flowchart of the contrast enhancement method of the shadow area according to an embodiment of the present disclosure is provided. As shown in Figure 1 The contrast enhancement method of the present embodiment includes:

[0066] In step S110, the first image is input into the pre-trained contrast enhancement model, and the first image includes a first shadow area.

[0067] In some embodiments, the first image can be a real-world photograph, or a two-dimensional image of a three-dimensional model in a virtual environment. It should be noted that the virtual environment is a virtual environment displayed (or provided) by an application running on a terminal. The virtual environment can be a three-dimensional virtual environment, or a two-dimensional virtual environment. The three-dimensional virtual environment can be a simulation environment of the real world, or a semi-simulation and semi-fictional environment, or a purely fictional environment. The first image includes a first shadow area. In real life, a certain object will form a shadow (the shadow area and the shadow area in the present disclosure are equivalent concepts) under sunlight.

[0068] In some embodiments, the contrast enhancement model is a pre-trained algorithm model for local contrast enhancement of the shadow area in the first image while preserving the original contrast information of the non-shadow area. In some embodiments, the first image is input into the pre-trained contrast enhancement model to facilitate the use of the contrast enhancement model to perform the contrast enhancement method of the present embodiment on the first image.

[0069] In step S120, the second image is output by the contrast enhancement model, the second image includes a second shadow area corresponding to the first shadow area, and the contrast of the second shadow area is higher than that of the first shadow area. The contrast enhancement model generates a segmentation mask of the first shadow area, and performs local contrast enhancement on the first shadow area based on the segmentation mask while preserving the original contrast information of the non-shadow area.

[0070] Figure 2A The algorithm block diagram of the contrast enhancement model according to an embodiment of the present disclosure is provided. The following will be described in combination withFigure 2A The contrast enhancement method of the embodiments of the present disclosure is described in detail. As shown in FIG. 1, the contrast enhancement method of the embodiments of the present disclosure comprises the following steps. Figure 2AAs shown, in some embodiments, the contrast enhancement model includes a segmentation mask generation model 210, an image segmentation model 220, and an image enhancement model 230. In some embodiments, the segmentation mask generation model 210 generates a segmentation mask of the first shadow region. The image segmentation model 220 cuts out the first shadow region from the first image based on the segmentation mask. The image enhancement model 230 performs local contrast enhancement on the first shadow region purposefully while preserving the original contrast information of the non-shadow region. In some embodiments, the segmentation mask generation model 210 includes a UNet deep neural network 211. The UNet deep neural network 211 builds a feature extraction layer by stacking multiple residual blocks (ResBlock) to perform feature extraction on the input image. These residual blocks adopt a skip connection structure, which enhances the learning ability of the network and at the same time alleviates the gradient vanishing problem in deep networks. In some embodiments, the first image is input into the UNet deep neural network 211, and the UNet deep neural network 211 performs a binarization operation on the first image to generate a segmentation mask (mask) of the first shadow region. The first image can be converted into a grayscale image for binarization processing, and a threshold binarization method is used to convert the grayscale image into a binary image (i.e., the segmentation mask of the first shadow region). The segmentation mask of the first shadow region can achieve the division of the shadow region and the non-shadow region in the first image, so that the shadow region can be clearly separated. For example, the gray value of the pixels in the shadow region in the first image is 1, and the gray value of the pixels in the non-shadow region is 0. In some embodiments, the segmentation mask generation model 210 further includes a dilated block 212. The segmentation mask of the first shadow region is input into the dilated block 212, and the dilated block 212 performs a dilated convolution operation on the segmentation mask of the first shadow region to obtain a dilated segmentation mask of the first shadow region. Through dilated convolution, the receptive field can be expanded to capture more extensive contextual information. The boundary range of the segmentation mask of the first shadow region is smaller than that of the dilated segmentation mask of the first shadow region, and the dilated segmentation mask of the first shadow region includes a peripheral associated region outside the segmentation mask of the first shadow region. In some embodiments, the image segmentation model 220 includes a first segmentation layer 221 and a second segmentation layer 222 built by, for example, multipliers. In some embodiments, the segmentation mask of the first shadow region and the first image are input into the first segmentation layer 221, and the first segmentation layer 221 segments the first image based on the segmentation mask of the first shadow region to obtain a first segmentation image of the first shadow region. The dilated segmentation mask of the first shadow region and the first image are input into the second segmentation layer 222, and the second segmentation layer 222 segments the first image based on the dilated segmentation mask of the first shadow region to obtain a second segmentation image of the first shadow region.In some embodiments, the first segmentation map, the second segmentation map and the first image are image-stitched by the image enhancement model 230 to obtain a second image. The second image includes a second shadow region corresponding to the first shadow region, and the contrast of the second shadow region is higher than that of the first shadow region. The image enhancement model 230 includes an original image enhancement layer 231, a segmentation mask image enhancement layer 232, an extended segmentation mask image enhancement layer 233, an image stitching layer 234 and a stitched image enhancement layer 235. The original image enhancement layer 231 is constructed by, for example, a residual block, and is used to perform image enhancement on the first image. The segmentation mask image enhancement layer 232 is constructed by, for example, a residual block, and is used to perform an image enhancement operation on the first segmentation map of the first shadow region of the first image. The extended segmentation mask image enhancement layer 233 is constructed by, for example, a residual block, and is used to perform an image enhancement operation on the second segmentation map of the first shadow region of the first image. The first segmentation map, the second segmentation map and the first image after the image enhancement operation are image-stitched by the image stitching layer 234. The image-stitched image is subjected to an image enhancement operation by the stitched image enhancement layer 235 to obtain the second image.

[0071] Figure 3 A flowchart of a training method of a shadow region contrast enhancement model according to an embodiment of the present disclosure is provided. As shown in Figure 3 , the training method of the embodiment of the present disclosure includes:

[0072] In step S310, a training data set is obtained, which includes a plurality of training samples and shadow region labels corresponding to the training samples.

[0073] In some embodiments, the training data set can be obtained from image samples collected from public data sets or actual application scenarios. The training samples include a shadow region bounding box and a pixel-level shadow region label marked by a professional marking tool. The label data can be marked in a RGB color space and a gray mask dual-channel manner, wherein the first channel records the color information of the original image, and the second channel marks the accurate position of the shadow region through binary processing.

[0074] In step S320, the training samples are input into a target contrast enhancement model, and the target contrast enhancement model includes a target segmentation mask generation model, a target image segmentation model and a target image enhancement model.

[0075] In some embodiments, the target contrast enhancement model can refer to the contrast enhancement model of Figure 2A . When the target contrast enhancement model reaches the training completion condition after training, the target contrast enhancement model is the contrast enhancement model of Figure 2A .

[0076] In step S330, a quadratic supervision mechanism is adopted to calculate the loss function of the target segmentation mask generation model and the target image enhancement model respectively.

[0077] In some embodiments, the training sample is input into the target segmentation mask generation model, and the target segmentation mask generation model performs a binarization operation on the training sample to generate a segmentation mask of the shadow region in the training sample. Figure 2B FIG. 2 shows a schematic diagram of the loss function of the target segmentation mask generation model according to an embodiment of the present disclosure. As shown in FIG. 2, the loss function Lmask of the target segmentation mask generation model is calculated based on the segmentation mask of the shadow region in the training sample and the corresponding shadow region label of the training sample. Figure 2B In some embodiments, the target segmentation mask generation model performs a dilated convolution operation on the segmentation mask of the shadow region in the training sample to obtain a dilated segmentation mask of the shadow region in the training sample. The target image segmentation model segments the training sample based on the segmentation mask of the shadow region in the training sample to obtain a first segmentation map of the shadow region in the training sample. The target image segmentation model segments the training sample based on the dilated segmentation mask of the shadow region in the training sample to obtain a second segmentation map of the shadow region in the training sample. The target image enhancement model performs image stitching on the first and second segmentation maps of the shadow region in the training sample and the training sample to obtain the target image of the training sample. In some embodiments, the target image enhancement model performs image enhancement operations on the first and second segmentation maps of the shadow region in the training sample and the training sample respectively, performs image stitching on the first and second segmentation maps of the shadow region in the training sample and the training sample after the image enhancement operations, and performs an image enhancement operation on the stitched image to obtain the target image of the training sample. Figure 2C FIG. 3 shows a schematic diagram of the loss function of the target image enhancement model according to an embodiment of the present disclosure. As shown in FIG. 3, in some embodiments, the loss function Limage of the target image enhancement model is calculated based on the target image of the training sample and the corresponding shadow region label of the training sample. Figure 2C

[0078] In step S340, the loss function is used for reverse learning to optimize the model parameters of the target segmentation mask generation model, the target image segmentation model, and the target image enhancement model.

[0079] ​In some embodiments, the loss function Lmask of the target segmentation mask generation model and the loss function Limage of the target image enhancement model are jointly optimized. For example, the total loss function Ltotal = a-Lmask + b-Limage can be constructed by weighted summation, where a and b are hyperparameters set according to task requirements. The gradient of the total loss function is propagated to the entire network architecture through the back propagation algorithm, and the Adam optimizer is used to update the UNet network parameters in the target segmentation mask generation model, the convolution kernel parameters of the dilated convolution block, and the parameters of each residual block in the target image enhancement model. During the training process, a dynamic learning rate adjustment strategy is set. When the validation set loss does not decrease significantly for three consecutive training periods, the learning rate is automatically multiplied by a decay factor of 0.8 until the preset upper limit of the training number or the loss function converges to within a threshold range.

[0080] It can be understood that after the model training is completed, the contrast enhancement method of the embodiments of the present disclosure realizes fine processing through a multi-scale feature fusion strategy, and improves the image enhancement effect. Specifically, the original segmentation mask output by the segmentation mask generation model and the dilated segmentation mask correspond to region features of different receptive field ranges: the original segmentation mask retains accurate shadow boundary information, and the dilated segmentation mask captures the light transition zone features around the shadow region by adjusting the dilated rate parameter (usually set to 2-4 times). The image segmentation model performs pixel-by-pixel multiplication operation on the two masks and the original image to obtain a double-scale segmentation map containing the core shadow area and the edge transition area. The image enhancement model integrates the feature vectors of the double-scale segmentation map and the original image through channel concatenation at the concatenation layer, so that the enhancement operation can more comprehensively consider the local and global information of the image, thereby improving the local contrast while maintaining the overall naturalness of the image.

[0081] Figure 4 A structural schematic diagram of a contrast enhancement device for a shadow area is shown according to one embodiment of the present disclosure. As Figure 4 The contrast enhancement device 400 for the shadow area includes an image input unit 410 and a contrast enhancement unit 420.

[0082] The image input unit 410 is configured to input a first image into a pre-trained contrast enhancement model, wherein the first image includes a first shadow area.

[0083] The contrast enhancement unit 420 outputs a second image including a second shadow region corresponding to the first shadow region, the contrast of the second shadow region being higher than the contrast of the first shadow region, wherein the contrast enhancement model generates a segmentation mask of the first shadow region, and performs local contrast enhancement on the first shadow region based on the segmentation mask while preserving original contrast information of a non-shadow region.

[0084] Since the specific process of enhancing the contrast of the shadow region has been described in detail above, it will not be repeated here.

[0085] Figure 5 A structural schematic diagram of a training device of a shadow region contrast enhancement model according to one embodiment of the present disclosure is shown. As Figure 5 The training device 500 of the shadow region contrast enhancement model includes a training data acquisition unit 510, a training sample input unit 520, a loss function calculation unit 530, and a model parameter optimization unit 540.

[0086] The training data acquisition unit 510 is configured to acquire a training data set, the training data set including a plurality of training samples and shadow region labels corresponding to the training samples.

[0087] The training sample input unit 520 is configured to input the training samples into a target contrast enhancement model, the target contrast enhancement model including a target segmentation mask generation model, a target image segmentation model, and a target image enhancement model.

[0088] The loss function calculation unit 530 is configured to calculate loss functions of the target segmentation mask generation model and the target image enhancement model respectively using a quadratic supervision mechanism.

[0089] The model parameter optimization unit 540 is configured to perform reverse learning using the loss functions to optimize model parameters of the target segmentation mask generation model, the target image segmentation model, and the target image enhancement model.

[0090] Since the specific process of training the contrast enhancement model has been described in detail above, it will not be repeated here.

[0091] The present disclosure also provides an electronic device 600, as Figure 6 As shown, the electronic device 600 includes a memory 620, a processor 610, a power component 630, a network interface 640, an input / output interface 650, and a program stored in the memory 620 and executable on the processor 610. When the program is executed by the processor 610, it can implement each process of each embodiment of the above method and achieve the same technical effects. To avoid repetition, it will not be repeated here.

[0092] The chip according to the embodiments of the present disclosure includes Figure 4 The contrast enhancement device 400 shown implements the steps of the method as described above. The chip here includes general-purpose processors (such as CPUs, GPUs), mobile device application processors (APs), and programmable logic chips (such as FPGAs) and application-specific integrated circuits (such as ASICs), etc. The beneficial effects that can be achieved by the method provided by the embodiments of the present disclosure can be achieved, which are described in detail in the foregoing embodiments and will not be repeated here.

[0093] The chip according to the embodiments of the present disclosure includes Figure 5 The training device 500 shown implements the steps of the method as described above. The chip here includes general-purpose processors (such as CPUs, GPUs), mobile device application processors (APs), and programmable logic chips (such as FPGAs) and application-specific integrated circuits (such as ASICs), etc. The beneficial effects that can be achieved by the method provided by the embodiments of the present disclosure can be achieved, which are described in detail in the foregoing embodiments and will not be repeated here.

[0094] Those skilled in the art can understand that all or part of the steps of the various methods of the above embodiments can be completed by instructions, or by instructions controlling relevant hardware, which can be stored in a computer-readable storage medium and loaded and executed by a processor. For this purpose, the embodiments of the present disclosure also provide a storage medium having a computer program or instructions stored thereon, which can implement the processes of each embodiment of the above method when executed by a processor.

[0095] Since the instructions stored in the storage medium can execute the steps in the method provided by the embodiments of the present disclosure, the beneficial effects that can be achieved by the method provided by the embodiments of the present disclosure can be achieved, which are described in detail in the foregoing embodiments and will not be repeated here. The specific implementation of each operation can be referred to the foregoing embodiments, which will not be repeated here.

[0096] In summary, according to the embodiments of the present disclosure, the first image is input into the pre-trained contrast enhancement model, the contrast enhancement model generates a segmentation mask of a first shadow region of the first image, performs local contrast enhancement on the first shadow region based on the segmentation mask while preserving the original contrast information of a non-shadow region, and the second image output by the contrast enhancement model includes a second shadow region corresponding to the first shadow region, the contrast of the second shadow region is higher than that of the first shadow region, and the non-shadow region of the second image preserves the contrast information of the corresponding non-shadow region of the first image. In this way, by accurately positioning the shadow region and performing contrast enhancement on the shadow region, the problem of unnecessary adjustment of the contrast of the non-shadow region caused by the traditional global self-adaptive enhancement algorithm is effectively solved, the image display effect and detail performance are improved, and the computational complexity is reduced.

[0097] According to an embodiment of the present disclosure, a training data set is obtained, the training data set including a plurality of training samples and a shadow area label corresponding to each training sample, the training sample is input into a target contrast enhancement model, the target contrast enhancement model including a target segmentation mask generation model, a target image segmentation model and a target image enhancement model, a secondary supervision mechanism is adopted, a loss function of the target segmentation mask generation model and the target image enhancement model is calculated respectively, and the model parameters of the target segmentation mask generation model, the target image segmentation model and the target image enhancement model are optimized through reverse learning based on the loss function. In this way, by introducing the secondary supervision mechanism, the collaborative optimization of the segmentation mask generation and the image enhancement process is realized, and the model training efficiency and accuracy are improved.

[0098] Finally, it should be noted that: Obviously, the above embodiments are only examples for clearly illustrating the present disclosure, and are not limitations on the embodiments. Based on the above description, other different forms of changes or variations can also be made by those of ordinary skill in the art. Here, it is not necessary and impossible to enumerate all the embodiments. The obvious changes or variations derived therefrom are still within the protection scope of the present disclosure.

Claims

1. A method for enhancing contrast of a shadow region, comprising: inputting a first image into a pre-trained contrast enhancement model, the first image comprising a first shadow region; outputting a second image from the contrast enhancement model, the second image comprising a second shadow region corresponding to the first shadow region, the second shadow region having a higher contrast than the first shadow region, and the second image retaining original contrast information of a non-shadow region, wherein the contrast enhancement model comprises a segmentation mask generation model, an image segmentation model, and an image enhancement model, the segmentation mask generation model performing a binarization operation on the first image to generate a segmentation mask of the first shadow region, the segmentation mask generation model performing a dilation convolution operation on the segmentation mask of the first shadow region to obtain a dilated segmentation mask of the first shadow region, the image segmentation model segmenting the first image based on the segmentation mask of the first shadow region to obtain a first segmentation map of the first shadow region, the image segmentation model segmenting the first image based on the dilated segmentation mask of the first shadow region to obtain a second segmentation map of the first shadow region, and the image enhancement model performing an image enhancement operation on the first segmentation map, the second segmentation map, and the first image respectively, and performing image stitching on the first segmentation map, the second segmentation map, and the first image after the image enhancement operation to obtain the second image.

2. The contrast enhancement method of claim 1, wherein, The contrast enhancement method further comprises: performing an image enhancement operation on the image obtained by image stitching to obtain the second image.

3. A training method for a contrast enhancement model of a shadow region, comprising: obtaining a training data set, the training data set comprising a plurality of training samples and shadow region labels corresponding to the training samples; inputting the training samples into a target contrast enhancement model, the target contrast enhancement model comprising a target segmentation mask generation model, a target image segmentation model, and a target image enhancement model; adopting a secondary supervision mechanism to calculate loss functions of the target segmentation mask generation model and the target image enhancement model respectively; performing backpropagation learning using the loss functions to optimize model parameters of the target segmentation mask generation model, the target image segmentation model, and the target image enhancement model, wherein a total loss function is constructed based on the loss function of the target segmentation mask generation model and the loss function of the target image enhancement model, the gradient of the total loss function is transmitted to the entire network architecture of the target contrast enhancement model through a backpropagation algorithm, the model parameters of the target segmentation mask generation model, the target image segmentation model, and the target image enhancement model are optimized until a preset upper limit of training rounds is reached or the total loss function converges to a threshold range.

4. The training method of claim 3, wherein, The adopting of the secondary supervision mechanism to calculate the loss functions of the target segmentation mask generation model and the target image enhancement model respectively comprises: inputting the training sample into the target segmentation mask generation model, performing a binarization operation on the training sample by the target segmentation mask generation model, and generating a segmentation mask of a shadow region in the training sample; calculating a loss function of the target segmentation mask generation model based on the segmentation mask of the shadow region in the training sample and a corresponding shadow region label of the training sample.

5. The training method of claim 4, wherein, The loss functions of the target segmentation mask generation model and the target image enhancement model are calculated respectively by using the secondary supervision mechanism, and the method further includes: performing a dilated convolution operation on the segmentation mask of the shadow region in the training sample by the target segmentation mask generation model to obtain a dilated segmentation mask of the shadow region in the training sample; segmenting the training sample based on the segmentation mask of the shadow region in the training sample by the target image segmentation model to obtain a first segmentation image of the shadow region in the training sample; segmenting the training sample based on the dilated segmentation mask of the shadow region in the training sample by the target image segmentation model to obtain a second segmentation image of the shadow region in the training sample; performing image stitching on the first segmentation image, the second segmentation image and the training sample by the target image enhancement model to obtain a target image of the training sample; calculating a loss function of the target image enhancement model based on the target image of the training sample and the corresponding shadow region label of the training sample.

6. The training method of claim 5, wherein, The target image of the training sample is obtained by performing image stitching on the first segmentation image, the second segmentation image and the training sample by the target image enhancement model, and the method includes: performing image enhancement operations on the first segmentation image, the second segmentation image and the training sample by the target image enhancement model respectively, and performing image stitching on the first segmentation image, the second segmentation image and the training sample after the image enhancement operations; performing image enhancement operations on the image obtained by image stitching to obtain the target image of the training sample.

7. A contrast enhancement device for a shadow region, comprising: an image input unit configured to input a first image into a pre-trained contrast enhancement model, the first image including a first shadow region; The contrast enhancement unit outputs a second image including a second shadow area corresponding to the first shadow area, the contrast of the second shadow area being higher than that of the first shadow area, and the second image retaining original contrast information of a non-shadow area, wherein the contrast enhancement model includes a segmentation mask generation model, an image segmentation model, and an image enhancement model, the segmentation mask generation model performs a binaryzation operation on the first image to generate a segmentation mask of the first shadow area, the segmentation mask generation model performs a dilation convolution operation on the segmentation mask of the first shadow area to obtain a dilated segmentation mask of the first shadow area, the image segmentation model segments the first image based on the segmentation mask of the first shadow area to obtain a first segmentation map of the first shadow area, the image segmentation model segments the first image based on the dilated segmentation mask of the first shadow area to obtain a second segmentation map of the first shadow area, and the image enhancement model performs an image enhancement operation on the first segmentation map, the second segmentation map, and the first image respectively, and image splicing is performed on the first segmentation map, the second segmentation map, and the first image after the image enhancement operation to obtain the second image.

8. A training device of a shadow area contrast enhancement model, comprising: a training data acquisition unit configured to acquire a training data set, the training data set comprising a plurality of training samples and shadow area labels corresponding to the training samples; a training sample input unit configured to input the training samples into a target contrast enhancement model, the target contrast enhancement model comprising a target segmentation mask generation model, a target image segmentation model, and a target image enhancement model; a loss function calculation unit configured to calculate loss functions of the target segmentation mask generation model and the target image enhancement model respectively using a secondary supervision mechanism; a model parameter optimization unit configured to perform reverse learning using the loss functions to optimize model parameters of the target segmentation mask generation model, the target image segmentation model, and the target image enhancement model, wherein a total loss function is constructed based on the loss function of the target segmentation mask generation model and the loss function of the target image enhancement model, a gradient of the total loss function is transmitted to an entire network architecture of the target contrast enhancement model through a back propagation algorithm, the model parameters of the target segmentation mask generation model, the target image segmentation model, and the target image enhancement model are optimized until a preset upper limit of training rounds is reached or the total loss function converges to a threshold range.

9. An electronic device comprising: A processor, a memory, and a program stored on the memory and executable on the processor, the program being executed by the processor to implement the steps of the method of any one of claims 1 to 6.

10. A storage medium having a computer program or instructions stored thereon, the computer program or instructions being executed by a processor to implement the steps of the method of any one of claims 1 to 6.

11. A chip comprising: The contrast enhancement device of claim 7, implementing the method of any of claims 1-2.

12. A chip comprising: The training device of claim 8, implementing the method of any of claims 3-6.

Images