Unsupervised shadow detection method, device and equipment based on double space fusion

Abstract

The application relates to a double-space fusion-based unsupervised shadow detection method, device and equipment. The method comprises the following steps: converting a shadow image into HSV and Lab color spaces, extracting intrinsic features representing light attenuation, performing threshold segmentation to obtain a coarse-grained shadow mask, using light enhancement to restore a reference non-shadow image as a condition, then converting the two color spaces, projecting pixel values into a two-dimensional brightness change coordinate system to obtain an initial data point set, modeling the non-shadow and shadow areas based on the point set using a multivariate Gaussian distribution mixture model, calculating data point membership posterior probability and adjusting the membership area, iteratively modeling to obtain two-space fine-grained shadow masks, fusing to obtain a precise shadow mask, realizing double-space fusion unsupervised shadow detection, and detecting image shadows with high quality and high efficiency.

Description

Technical Field

[0001] This application relates to the fields of intelligent segmentation of optical images and computer technology, and in particular to an unsupervised shadow detection method, apparatus and device based on dual-space fusion. Background Technology

[0002] Shadow detection is a crucial research area in computer vision and image processing, aiming to identify regions in images or videos formed by objects occluding light sources. Traditional methods typically rely on texture features (local binary patterns) or physical models (light-reflection models), segmenting shadow areas using variations in illumination and texture characteristics. However, these methods are prone to failure in complex lighting, low-contrast, or dynamic scenes. In recent years, deep learning methods have significantly improved detection accuracy. Convolutional Neural Network (CNN)-based models (such as U-Net and Mask R-CNN) can capture multi-scale shadow features through end-to-end learning, while Generative Adversarial Networks (GANs) can generate synthetic shadow data to enhance model generalization. With the increasing demand for long-range dependency modeling, recent research has begun exploring the Transformer architecture, which uses a self-attention mechanism to capture the global dependencies between shadows and objects / light sources in images, further improving shadow detection performance.

[0003] However, these methods suffer from two critical issues that urgently need to be addressed, preventing them from meeting the needs of fields such as autonomous driving and remote sensing image analysis: Firstly, the high coupling between the brightness-chromaticity gradient features of shadow edge regions and object texture and background noise leads to commonly blurred boundaries or jagged errors in the detection results. Secondly, these methods generally rely on paired data for training, limiting their application in real-world scenarios where high-quality data is scarce. Summary of the Invention

[0004] Therefore, it is necessary to provide an unsupervised shadow detection method, apparatus, and device based on dual-space fusion that can achieve high-quality and high-efficiency shadow detection in images, addressing the aforementioned technical problems.

[0005] An unsupervised shadow detection method based on dual-space fusion, the method comprising: Obtain the shadow image; After converting the shadow image to the HSV color space and the Lab color space respectively, the intrinsic features representing the light attenuation in the shadow area are extracted, and a coarse-grained shadow mask is obtained by threshold segmentation based on the intrinsic features. Using the coarse-grained shadow mask as a region condition, the shadow image is restored to a reference shadowless image using an illumination enhancement method; After converting the shadow image and the reference image without shadow to the HSV color space and the Lab color space, the pixel values ​​of the shadow image and the reference image without shadow are projected onto a two-dimensional brightness variation coordinate system in the HSV color space and the Lab color space, respectively, to obtain the initial data point sets corresponding to the two color spaces. Based on two initial data point sets, a multivariate Gaussian distribution mixture model is used to model the distribution of the shadowless region and the shadow region respectively. The posterior probability of each data point belonging to the shadowless region or the shadow region is calculated according to the Gaussian distribution modeling results. The region to which each data point belongs is adjusted according to the posterior probability. Then, Gaussian distribution modeling is re-performed according to the adjusted results. After multiple iterations, fine-grained shadow masks corresponding to the HSV color space and Lab color space are obtained respectively. By fusing the fine-grained shadow masks corresponding to the HSV color space and the Lab color space, the accurate shadow mask of the shadow image is obtained, realizing unsupervised shadow detection by dual-space fusion.

[0006] In one embodiment, extracting the intrinsic features characterizing the illumination attenuation in the shadow region includes: In the HSV color space, a brightness suppression factor that reflects the degree of deviation between the brightness value of each pixel and the global average brightness is calculated using global brightness statistics. In the Lab color space, the relative magnitude of the product of the chromaticity values ​​of each pixel in the a and b channels is used as the chromaticity enhancement factor. By fusing the luminance suppression factor and chromaticity enhancement factor using a composite function, an intrinsic characteristic representing the light attenuation in the shadow region is obtained.

[0007] In one embodiment, the brightness suppression factor is represented as: ; In the above formula, Represents the coordinate position of a pixel. This represents the brightness information of a specific pixel in the luminance channel. , These represent the mean and standard deviation of the luminance channel, respectively.

[0008] In one embodiment, the chromaticity enhancement factor is represented as: ; In the above formula, , These represent the coordinates of a certain position. The red-green hue and yellow-blue hue of the pixels, These represent the maximum values ​​of red-green and yellow-blue hues, respectively.

[0009] In one embodiment, the composite function is represented as: ; In the above formula, , These represent the coordinates of a certain position. The brightness suppression factor and color enhancement factor of the pixels.

[0010] In one embodiment, the pixel values ​​of the shadow image and the pixel values ​​of a reference image without shadow are projected onto a two-dimensional brightness variation coordinate system, wherein the pixel values ​​of the shadow image are used as... The axis is defined by the pixel values ​​of the reference shadowless image. The axes are constructed such that the two-dimensional brightness variation coordinate system represents the changes in brightness and chromaticity caused by shadows.

[0011] In one embodiment, during the process of performing multiple iterations to obtain fine-grained shadow masks corresponding to two color spaces: Latent variables are used to describe the regions to which data points belong in two color spaces. The value of the latent variable corresponding to each data point represents that it belongs to a certain Gaussian distribution. In each iteration, based on the distribution parameters of the data point sets in the two color spaces, the posterior probability of each data point belonging to each Gaussian distribution is calculated, and based on the calculated posterior probability, the latent variable of each data point is updated to the corresponding index with the highest probability. Based on the pre-defined correspondence between Gaussian distribution numbers and shadow / no-shadow areas, the coarse shadow mask is refined in two color spaces to obtain a fine-grained shadow mask with a precise correspondence.

[0012] In one embodiment, an adaptive weighted fusion strategy is used to fuse the fine-grained shadow masks corresponding to the HSV color space and the Lab color space.

[0013] This application also provides an unsupervised shadow detection device based on dual-space fusion, the device comprising: The shadow image acquisition module is used to acquire shadow images; The coarse-grained shadow mask generation module is used to convert the shadow image to the HSV color space and the Lab color space respectively, extract the intrinsic features that characterize the illumination attenuation of the shadow area, and obtain the coarse-grained shadow mask based on the intrinsic features through threshold segmentation. A reference shadowless image generation module is used to use the coarse-grained shadow mask as a region condition and employ an illumination enhancement method to restore the shadow image to a reference shadowless image. The initial data point set acquisition module is used to convert the shadow image and the reference image without shadow to the HSV color space and the Lab color space, respectively, and then project the pixel values ​​of the shadow image and the reference image without shadow to the two-dimensional brightness variation coordinate system in the HSV color space and the Lab color space to obtain the initial data point sets corresponding to the two color spaces. The dual-space shadow region adjustment module is used to model the distribution of the no-shadow region and the shadow region based on two initial data point sets using a multivariate Gaussian distribution mixture model. It calculates the posterior probability of each data point belonging to the no-shadow region or the shadow region based on the Gaussian distribution modeling results, and adjusts the region to which each data point belongs based on the posterior probability. Then, it re-models the Gaussian distribution based on the adjusted results. After multiple iterations, it obtains the fine-grained shadow mask corresponding to the HSV color space and the Lab color space, respectively. The precise shadow mask acquisition module is used to fuse the fine-grained shadow masks corresponding to the HSV color space and the Lab color space to obtain the precise shadow mask of the shadow image, thereby realizing unsupervised shadow detection by dual-space fusion.

[0014] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps in the above-described unsupervised shadow detection method based on dual-space fusion.

[0015] The aforementioned unsupervised shadow detection method, apparatus, and device based on dual-space fusion converts shadow images to both the HSV and Lab color spaces, extracts intrinsic features characterizing illumination attenuation in shadow regions, and obtains coarse-grained shadow masks through threshold segmentation based on these intrinsic features. Using these coarse-grained shadow masks as region conditions, an illumination enhancement method is employed to restore the shadow image to a reference shadowless image. After converting the shadow image and the reference shadowless image to the HSV and Lab color spaces respectively, the pixel values ​​of the shadow image and the reference shadowless image are projected onto a two-dimensional brightness variation coordinate system in the HSV and Lab color spaces, respectively, to obtain two corresponding color spaces. Based on two initial data point sets, a multivariate Gaussian distribution mixture model is used to model the distribution of shadowless and shadowed regions respectively. The posterior probability of each data point belonging to either a shadowless or shadowed region is calculated based on the Gaussian distribution modeling results. The region of belonging for each data point is adjusted according to the posterior probability, and Gaussian distribution modeling is re-performed based on the adjusted results. After multiple iterations, fine-grained shadow masks corresponding to the HSV and Lab color spaces are obtained. The fine-grained shadow masks corresponding to the HSV and Lab color spaces are fused to obtain the accurate shadow mask of the shadow image, achieving unsupervised shadow detection through dual-space fusion. This method enables high-quality and high-efficiency shadow detection in images. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating an unsupervised shadow detection method based on dual-space fusion in one embodiment; Figure 2 This is a schematic diagram illustrating the visualization results of this method on the ISTD dataset in an experiment; Figure 3 This is a schematic diagram illustrating the visualization results of the present invention on the SRD dataset in an experiment. Figure 4 This is a structural block diagram of an unsupervised shadow detection device based on dual-space fusion in one embodiment; Figure 5 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0018] To address the two problems of existing shadow image detection methods—namely, the high coupling between the brightness-chromaticity gradient features of the shadow edge region and object texture and background noise, leading to blurred boundaries or jagged errors in the detection results, and the reliance on paired data for training, which limits their application in real-world scenarios where high-quality data is scarce—this method fails to meet the needs of fields such as autonomous driving and remote sensing image analysis. This application provides an unsupervised shadow detection method based on dual-space fusion, specifically including the following steps: Step S100: Obtain the shadow image.

[0019] Step S110: After converting the shadow image to the HSV color space and the Lab color space respectively, the intrinsic features that characterize the illumination attenuation in the shadow area are extracted, and a coarse-grained shadow mask is obtained by threshold segmentation based on the intrinsic features.

[0020] Step S120: Using the coarse-grained shadow mask as a region condition, the shadow image is restored to the reference shadowless image using an illumination enhancement method.

[0021] Step S130: After converting the shadow image and the reference image without shadow to the HSV color space and the Lab color space, the pixel values ​​of the shadow image and the reference image without shadow are projected onto the two-dimensional brightness variation coordinate system in the HSV color space and the Lab color space, respectively, to obtain the initial data point set of the corresponding two color spaces.

[0022] Step S140: Based on two initial data point sets, the distribution of the no-shade region and the shaded region is modeled using a multivariate Gaussian distribution mixture model. The posterior probability of each data point belonging to the no-shade region or the shaded region is calculated according to the Gaussian distribution modeling results. The region to which each data point belongs is adjusted according to the posterior probability. Then, Gaussian distribution modeling is performed again according to the adjusted results. After multiple iterations, fine-grained shadow masks corresponding to the HSV color space and Lab color space are obtained respectively.

[0023] Step S150: After fusing the fine-grained shadow masks corresponding to the HSV color space and the Lab color space, the accurate shadow mask of the shadow image is obtained, realizing unsupervised shadow detection by dual-space fusion.

[0024] This application proposes an unsupervised shadow detection method based on HSV-Lab dual-space fusion. It calculates global luminance statistics through luminance channel analysis in the HSV color space and defines a luminance suppression factor to quantify the degree of pixel illumination attenuation. A chrominance enhancement factor is constructed using the chrominance channel in the Lab color space to characterize the chrominance system shift caused by shadows. Through the dual action of the luminance suppression and chrominance enhancement terms, feature layering of shadow and non-shadow regions is achieved. Simple threshold segmentation is then used to provide coarse shadow masks for subsequent modules. Furthermore, a two-dimensional Gaussian mixture model is constructed in both the HSV and Lab spaces to characterize the pixel luminance-chrominance variation distribution. The posterior probabilities of the variables representing luminance and chrominance are calculated iteratively through distribution fitting to update the shadow / non-shadow region division. The weights, mean, and covariance parameters of the Gaussian distribution are dynamically adjusted to achieve shadow-sensitive binarized output, significantly improving detection accuracy in complex scenes.

[0025] In step S100, the shadow image is an optical image in RGB space.

[0026] In step S110, the shadow image is converted to a color space, and intrinsic features representing the light attenuation of the shadow area are extracted in the luminance channel V of the HSV color space and the luminance channels a and b of the Lab color space, respectively. Then, the coarse extraction result of the shadow area is obtained by threshold segmentation.

[0027] In this embodiment, extracting the intrinsic features characterizing the illumination attenuation in shadow areas includes: in the HSV color space, calculating a luminance suppression factor that reflects the degree of deviation between the luminance value of each pixel and the global average luminance using global luminance statistics; and in the Lab color space, using the relative magnitude of the product of the chrominance values ​​of each pixel in channels a and b as a chrominance enhancement factor. Finally, fusing the luminance suppression factor and the chrominance enhancement factor using a composite function yields the intrinsic features characterizing the illumination attenuation in shadow areas.

[0028] In this example, H, S, and V in the HSV color space represent hue, saturation, and luminance information, respectively. Compared to images represented in RGB color space, this representation can effectively separate luminance and chromaticity information, providing a luminance prior for subsequent coarse extraction of shadow regions based on illumination attenuation.

[0029] Specifically, the input shadow image is first converted from RGB space to HSV space. In HSV space, the luminance channel V carries the most important shadow information. Since shadows are caused by the partial or complete blocking of light in the scene, the luminance values ​​of shadow areas are not only lower than those of non-shadow areas, but also exhibit a systematic shift in distribution across the entire image. To quantify this difference, this method calculates a global luminance statistic, including the mean of V for the luminance component. and standard deviation The calculation formula is as follows: ; In the above formula, The total number of pixels on the shaded image. For the first The brightness value of each pixel.

[0030] Furthermore, based on the aforementioned global brightness statistics, this method proposes a brightness suppression factor that reflects the degree of deviation between the brightness value of each pixel and the global average brightness. , represented as: ; In the above formula, Represents the coordinate position of a pixel. Represents the brightness information of a specific pixel. , These represent the mean and standard deviation of the global luminance component, respectively. Since shadow areas typically occupy only a small portion of the entire image, the global mean... The brightness value is closer to the average brightness of the shadowless area; when a pixel is in a shadow area, its brightness value... Significantly lower than the global mean The brightness suppression factor calculated from pixels in the shadowless region. Compared to the value of the shadow area, the value of the brightness suppression factor calculated for the pixel in the shadow area is larger, and the larger the value, the more significantly the pixel is affected by the shadow.

[0031] In the HSV color space, L, a, and b represent lightness, red-green hue, and yellow-blue hue, respectively. In the Lab color space, the specific variation patterns of the chroma channels provide crucial supplementary features for shadow detection. This invention experimentally reveals that in shadow regions, the product of the a and b channels... There is a systematic decreasing trend, which is directly related to the changes in spectral properties caused by shading.

[0032] In this embodiment, based on the aforementioned discovery of the HSV space, a chroma enhancement factor reflecting the relative magnitude of the product of the two channel chroma values ​​of each pixel is proposed, expressed as: ; In the above formula, , These represent the coordinates of a certain position. The red-green hue and yellow-blue hue of the pixels, These represent the maximum values ​​of red-green and yellow-blue hues, respectively.

[0033] Specifically, the design of the chroma enhancement factor has the following two advantages: First, the normalization of the denominator eliminates chroma differences between different images, improving the robustness of the algorithm; Second, The product form amplifies the changes in chromaticity features, enhancing the ability to detect weak shadows.

[0034] In this embodiment, the brightness suppression factor, which characterizes brightness information, is obtained. And chromaticity enhancement that characterizes chromaticity information Subsequently, this invention designs a composite function to fuse the two features, the composite function being expressed as: ; In the above formula, , These represent the coordinates of a certain position. The pixel's luminance suppression factor and chroma enhancement factor. This composite function primarily considers that shadow areas typically have lower luminance (leading to...). The composite function has a smaller value and lower chromaticity (resulting in a smaller exponent term), therefore... The value will be relatively small, a characteristic that allows the shadow area to be extracted through simple thresholding.

[0035] In this embodiment, after obtaining the intrinsic features of each pixel in the shadow image, a coarse-grained shadow mask can be obtained by initially segmenting the shadow and non-shadow regions in the shadow image using a preset threshold.

[0036] Although preliminary shadow region localization can be achieved through HSV-Lab dual-space feature extraction fusion and threshold segmentation via the above step S110, its accuracy is very limited. Therefore, in this method, the coarse-grained shadow mask is further refined through subsequent steps to obtain a fine-grained shadow mask.

[0037] In step S130, the pixel values ​​of the shadow image and the pixel values ​​of the reference shadowless image are projected onto a two-dimensional brightness variation coordinate system, wherein the pixel values ​​of the shadow image are used as... The axis is based on the pixel values ​​of the reference image without shadows. The axes are constructed to create a two-dimensional brightness variation coordinate system that represents the changes in brightness and chromaticity caused by shadows.

[0038] In this embodiment, each data point in the two-dimensional brightness variation coordinate system These all represent the changes in brightness and chromaticity caused by shadows. In this method, a Multi-Gaussian Distribution Model (MGDM) is used to model the distribution of data points in the coordinate system, and its joint probability density function is defined as: ; in: ; In the above formula, This represents the number of Gaussian distributions. , and They represent the first The weights, mean, and variance of each Gaussian distribution are calculated. Extensive data experiments revealed that the brightness variation in unshaded areas is relatively simple and can be fitted using a single Gaussian distribution. However, due to abrupt changes in brightness at the shadow edges, shadowed areas require a linear combination of more Gaussian distributions to accurately represent their distribution. Since shadowed and unshaded areas exhibit significant feature differences in both HSV and Lab spaces, this method statistically models both regions in HSV and Lab spaces respectively, iteratively optimizes them, and finally fuses the results from the two optimizations to achieve accurate segmentation of shadowed and unshaded areas.

[0039] In step S130, the reference shadowless image used is actually the coarse-grained shadow mask obtained in step S110 as the region condition. Traditional unsupervised illumination enhancement algorithms such as histogram equalization and Retinex model are used to restore the shadow image to a reference shadowless image (step S120). Then, the coarse-grained shadow mask is further optimized and solved.

[0040] In this embodiment, the image is converted from RGB space to HSV space and Lab space, and then projected onto a two-dimensional brightness variation coordinate system to obtain two sets of data points. as well as This will serve as the initial value for iterative optimization.

[0041] In step S140, during multiple iterations to obtain the fine-grained shadow mask corresponding to the two color spaces: latent variables are used to describe the regions to which the data points belong in the two color spaces. The value of the latent variable corresponding to each data point represents that it belongs to a certain Gaussian distribution. In each iteration, based on the distribution parameters of the data point set in the current two color spaces, the posterior probability of each data point belonging to each Gaussian distribution is calculated. Based on the calculated posterior probability, the latent variable of each data point is updated to the index of the corresponding maximum probability. According to the preset correspondence between the Gaussian distribution index and the shadow / no-shadow region, the coarse shadow mask is refined in the two color spaces respectively to obtain the fine-grained shadow mask.

[0042] Specifically, latent variables are used to facilitate formulaic expression. and Describe respectively and The region to which it belongs, express Belongs to the A Gaussian distribution, express Belongs to the There are several Gaussian distributions. The number of Gaussian distributions in the HSV space and the Lab space are counted separately. The first Gaussian distribution describes the unshaded region, and the remaining Gaussian distributions describe the shaded region. Therefore, if the latent variable value is 1, the pixel corresponding to that latent variable belongs to the unshaded region; otherwise, it belongs to the shaded region.

[0043] In this embodiment, in the first During round iteration, given the current set of HSV space coordinate points... Distribution parameters and Lab space coordinate point set Distribution parameters When the coordinates of a pixel point in a set of coordinate points are such that the posterior probability of the latent variable can be expressed in the following form: ; ; In the above formula, and They represent the first time. Coordinates of points in HSV space during round iteration Belongs to the The probability of a Gaussian distribution and the coordinates of the points in Lab space Belongs to the The probability of a Gaussian distribution.

[0044] Furthermore, after obtaining the posterior probability of the latent variable corresponding to each data point, this embodiment also proposes a statistical modeling algorithm to update the latent variables. The specific mathematical expression is as follows: ; .

[0045] After updating the latent variables, based on the previously defined correspondence between Gaussian distribution indices and shaded / unshaded regions, the coarse shadow mask is refined in both HSV and Lab spaces as follows, resulting in two more refined masks, as shown in the mathematical expressions below: ; In the above formula, and These represent the shadowless region and the shadowed region of the refined mask in HSV space, respectively. and These represent the unshaded and shaded regions of the refined mask in Lab space, respectively.

[0046] In this embodiment, in the first During round iteration, since the latent variables representing the category information of data points are first processed... as well as The data has been updated, therefore the Gaussian distribution parameters calculated using the data before the category update are used. , If it becomes impossible to continue fitting the distribution of data points after the category update, then the distribution parameters need to be updated in real time. This application uses the calculated posterior probability. , To update the distribution parameters, the specific mathematical formula is as follows: ; Updated distribution parameters , This will be used to calculate the posterior probability of the next iteration. , Further optimization of the shadow mask is performed until the distribution parameters converge iteratively, resulting in a refined shadow mask in both HSV and Lab spaces. , .

[0047] In step S150, an adaptive weighted fusion strategy is used to fuse the fine-grained shadow masks corresponding to the HSV color space and the Lab color space. By fusing the two color spaces using the adaptive weighted fusion strategy, complementary information optimization is effectively achieved, and a high-precision shadow mask is finally generated through dynamic thresholding.

[0048] In this embodiment, the fusion process is represented as follows: ; In the above formula, This is a weighted hyperparameter, and its value range is... Controlling and The percentage of information contribution to the final mask. Preferred weighted hyperparameters. Due to the obtained It is not binarized, so a thresholding operation is still needed to obtain a refined binary mask.

[0049] In this paper, the effectiveness of the proposed method is also demonstrated through experiments. Table 1 presents the quantitative experimental results of each method on the shadow detection dataset. It can be seen that the proposed method achieves balance error rates of 5.34 / 8.58 on the SBU and SRD datasets, respectively. Specifically, the balance error rate for shadow regions is 6.61 / 11.37, and the balance error rate for non-shadow regions is 1.59 / 3.49, significantly outperforming existing shadow detection methods. Figure 2 and Figure 3 The images show the visualization results of the proposed method on the ISRD and SRD datasets, respectively. It can be seen that the method accurately identifies the location of shadowed regions in the images, achieving satisfactory visualization results.

[0050] Table 1

[0051] The unsupervised shadow detection method based on dual-space fusion described above consists of two stages: coarse shadow region extraction based on color space transformation and fine-grained correction based on statistical modeling. In the first stage, the coarse shadow region extraction model based on color space transformation calculates a luminance suppression factor in HSV space using global luminance statistics and extracts a chrominance channel product to construct a chrominance enhancement factor in Lab space. Then, the two factors are fused using a feature fusion function to generate a feature map with significant shadow response. Finally, a coarse-grained shadow mask is obtained through threshold segmentation. In the second stage, pixel luminance-chrominance variations are mapped to two-dimensional luminance variation coordinate systems in HSV and Lab spaces, respectively. Then, a shadow-non-shadow distribution model based on a multivariate Gaussian mixture model (MGDM) is established. The posterior probability and distribution parameters of the pixel's membership region are iteratively calculated using a statistical modeling algorithm to dynamically optimize the shadow mask, ultimately obtaining a refined shadow detection result. Under unsupervised conditions, our method achieved balance error rates of 5.34 / 8.58 on the SBU and SRD datasets, respectively. The balance error rates for shaded regions were 6.61 / 11.37, and for unshaded regions were 1.59 / 3.49, which significantly outperformed existing shadow detection methods.

[0052] This method eliminates the dependence on training data and can still be used stably in real-world scenarios where high-quality data is scarce. It proposes a coarse shadow region extraction model framework based on color space transformation, fusing luminance information from the HSV space and chromaticity information from the Lab space to obtain coarse-grained shadow detection results. Furthermore, this method innovatively introduces a statistical modeling algorithm into the shadow detection problem, refining the coarse shadow mask from the perspective of probability distribution, resulting in a more accurate shadow mask compared to existing shadow detection algorithms.

[0053] It should be understood that, although Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0054] In one embodiment, such as Figure 4 As shown, an unsupervised shadow detection device based on dual-space fusion is provided, comprising: a shadow image acquisition module 200, a coarse-grained shadow mask generation module 210, a reference shadowless image generation module 220, an initial data point set acquisition module 230, a dual-space shadow region adjustment module 240, and a precise shadow mask acquisition module 250, wherein: The shadow image acquisition module 200 is used to acquire shadow images.

[0055] The coarse-grained shadow mask generation module 210 is used to convert the shadow image to the HSV color space and the Lab color space respectively, extract the intrinsic features that characterize the illumination attenuation of the shadow area, and obtain the coarse-grained shadow mask by threshold segmentation based on the intrinsic features.

[0056] The reference shadowless image generation module 220 is used to use the coarse-grained shadow mask as a region condition and employ an illumination enhancement method to restore the shadow image to the reference shadowless image.

[0057] The initial data point set acquisition module 230 is used to convert the shadow image and the reference shadowless image to the HSV color space and the Lab color space, respectively, and then project the pixel values ​​of the shadow image and the reference shadowless image onto the two-dimensional brightness variation coordinate system in the HSV color space and the Lab color space to obtain the initial data point sets corresponding to the two color spaces.

[0058] The dual-space shadow region adjustment module 240 is used to model the distribution of the no-shadow region and the shadow region respectively based on two initial data point sets using a multivariate Gaussian distribution mixture model. It calculates the posterior probability of each data point belonging to the no-shadow region or the shadow region based on the Gaussian distribution modeling results, and adjusts the region to which each data point belongs based on the posterior probability. Then, it re-models the Gaussian distribution based on the adjusted results. After multiple iterations, it obtains the fine-grained shadow mask corresponding to the HSV color space and the Lab color space respectively.

[0059] The accurate shadow mask acquisition module 250 is used to fuse the fine-grained shadow masks corresponding to the HSV color space and the Lab color space to obtain the accurate shadow mask of the shadow image, thereby realizing unsupervised shadow detection by dual-space fusion.

[0060] Specific limitations regarding the unsupervised shadow detection device based on dual-space fusion can be found in the limitations of the unsupervised shadow detection method based on dual-space fusion described above, and will not be repeated here. Each module in the aforementioned unsupervised shadow detection device based on dual-space fusion can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0061] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 5As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements an unsupervised shadow detection method based on dual-space fusion. The display screen can be a liquid crystal display (LCD) or an e-ink display. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0062] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0063] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps: Obtain the shadow image; After converting the shadow image to the HSV color space and the Lab color space respectively, the intrinsic features representing the light attenuation in the shadow area are extracted, and a coarse-grained shadow mask is obtained by threshold segmentation based on the intrinsic features. Using the coarse-grained shadow mask as a region condition, the shadow image is restored to a reference shadowless image using an illumination enhancement method; After converting the shadow image and the reference image without shadow to the HSV color space and the Lab color space, the pixel values ​​of the shadow image and the reference image without shadow are projected onto a two-dimensional brightness variation coordinate system in the HSV color space and the Lab color space, respectively, to obtain the initial data point sets corresponding to the two color spaces. Based on two initial data point sets, a multivariate Gaussian distribution mixture model is used to model the distribution of the shadowless region and the shadow region respectively. The posterior probability of each data point belonging to the shadowless region or the shadow region is calculated according to the Gaussian distribution modeling results. The region to which each data point belongs is adjusted according to the posterior probability. Then, Gaussian distribution modeling is re-performed according to the adjusted results. After multiple iterations, fine-grained shadow masks corresponding to the HSV color space and Lab color space are obtained respectively. By fusing the fine-grained shadow masks corresponding to the HSV color space and the Lab color space, the accurate shadow mask of the shadow image is obtained, realizing unsupervised shadow detection by dual-space fusion.

[0064] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor: Obtain the shadow image; After converting the shadow image to the HSV color space and the Lab color space respectively, the intrinsic features representing the light attenuation in the shadow area are extracted, and a coarse-grained shadow mask is obtained by threshold segmentation based on the intrinsic features. Using the coarse-grained shadow mask as a region condition, the shadow image is restored to a reference shadowless image using an illumination enhancement method; After converting the shadow image and the reference image without shadow to the HSV color space and the Lab color space, the pixel values ​​of the shadow image and the reference image without shadow are projected onto a two-dimensional brightness variation coordinate system in the HSV color space and the Lab color space, respectively, to obtain the initial data point sets corresponding to the two color spaces. Based on two initial data point sets, a multivariate Gaussian distribution mixture model is used to model the distribution of the shadowless region and the shadow region respectively. The posterior probability of each data point belonging to the shadowless region or the shadow region is calculated according to the Gaussian distribution modeling results. The region to which each data point belongs is adjusted according to the posterior probability. Then, Gaussian distribution modeling is re-performed according to the adjusted results. After multiple iterations, fine-grained shadow masks corresponding to the HSV color space and Lab color space are obtained respectively. By fusing the fine-grained shadow masks corresponding to the HSV color space and the Lab color space, the accurate shadow mask of the shadow image is obtained, realizing unsupervised shadow detection by dual-space fusion.

[0065] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0066] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0067] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An unsupervised shadow detection method based on dual-space fusion, characterized in that, The method includes: Obtain the shadow image; After converting the shadow image to the HSV color space and the Lab color space respectively, the intrinsic features representing the light attenuation in the shadow area are extracted, and a coarse-grained shadow mask is obtained by threshold segmentation based on the intrinsic features. Using the coarse-grained shadow mask as a region condition, the shadow image is restored to a reference shadowless image using an illumination enhancement method; After converting the shadow image and the reference image without shadow to the HSV color space and the Lab color space, the pixel values ​​of the shadow image and the reference image without shadow are projected onto a two-dimensional brightness variation coordinate system in the HSV color space and the Lab color space, respectively, to obtain the initial data point sets corresponding to the two color spaces. Based on two initial data point sets, a multivariate Gaussian distribution mixture model is used to model the distribution of the shadowless region and the shadow region respectively. The posterior probability of each data point belonging to the shadowless region or the shadow region is calculated according to the Gaussian distribution modeling results. The region to which each data point belongs is adjusted according to the posterior probability. Then, Gaussian distribution modeling is re-performed according to the adjusted results. After multiple iterations, fine-grained shadow masks corresponding to the HSV color space and Lab color space are obtained respectively. By fusing the fine-grained shadow masks corresponding to the HSV color space and the Lab color space, the accurate shadow mask of the shadow image is obtained, realizing unsupervised shadow detection by dual-space fusion.

2. The unsupervised shadow detection method based on dual-space fusion according to claim 1, characterized in that, Extracting the intrinsic features characterizing the illumination attenuation in the shadow region includes: In the HSV color space, a brightness suppression factor that reflects the degree of deviation between the brightness value of each pixel and the global average brightness is calculated using global brightness statistics. In the Lab color space, the relative magnitude of the product of the chromaticity values ​​of each pixel in the a and b channels is used as the chromaticity enhancement factor. By fusing the luminance suppression factor and chromaticity enhancement factor using a composite function, an intrinsic characteristic representing the light attenuation in the shadow region is obtained.

3. The unsupervised shadow detection method based on dual-space fusion according to claim 2, characterized in that, The brightness suppression factor is expressed as: In the above formula, Represents the coordinate position of a pixel. This represents the brightness information of a specific pixel in the luminance channel. , These represent the mean and standard deviation of the luminance channel, respectively.

4. The unsupervised shadow detection method based on dual-space fusion according to claim 2, characterized in that, The chromaticity enhancement factor is expressed as: In the above formula, , These represent the coordinates of a certain position. The red-green hue and yellow-blue hue of the pixels, These represent the maximum values ​​of red-green and yellow-blue hues, respectively.

5. The unsupervised shadow detection method based on dual-space fusion according to claim 2, characterized in that, The composite function is expressed as: In the above formula, , These represent the coordinates of a certain position. The brightness suppression factor and color enhancement factor of the pixels.

6. The unsupervised shadow detection method based on dual-space fusion according to any one of claims 1-5, characterized in that, The pixel values ​​of the shadow image and the pixel values ​​of the reference image without shadow are projected onto a two-dimensional brightness variation coordinate system, wherein the pixel values ​​of the shadow image are used as... The axis is defined by the pixel values ​​of the reference shadowless image. The axes are constructed such that the two-dimensional brightness variation coordinate system represents the changes in brightness and chromaticity caused by shadows.

7. The unsupervised shadow detection method based on dual-space fusion according to claim 6, characterized in that, In the process of obtaining fine-grained shadow masks corresponding to the two color spaces through multiple iterations: Latent variables are used to describe the regions to which data points belong in two color spaces. The value of the latent variable corresponding to each data point represents that it belongs to a certain Gaussian distribution. In each iteration, based on the distribution parameters of the data point sets in the two color spaces, the posterior probability of each data point belonging to each Gaussian distribution is calculated, and based on the calculated posterior probability, the latent variable of each data point is updated to the corresponding index with the highest probability. Based on the pre-defined correspondence between Gaussian distribution numbers and shadow / no-shadow areas, the coarse shadow mask is refined in two color spaces to obtain a fine-grained shadow mask with a precise correspondence.

8. The unsupervised shadow detection method based on dual-space fusion according to claim 7, characterized in that, An adaptive weighted fusion strategy is used to fuse the fine-grained shadow masks corresponding to the HSV color space and the Lab color space.

9. An unsupervised shadow detection device based on dual-space fusion, characterized in that, The device includes: The shadow image acquisition module is used to acquire shadow images; The coarse-grained shadow mask generation module is used to convert the shadow image to the HSV color space and the Lab color space respectively, extract the intrinsic features that characterize the illumination attenuation of the shadow area, and obtain the coarse-grained shadow mask based on the intrinsic features through threshold segmentation. A reference shadowless image generation module is used to use the coarse-grained shadow mask as a region condition and employ an illumination enhancement method to restore the shadow image to a reference shadowless image. The initial data point set acquisition module is used to convert the shadow image and the reference image without shadow to the HSV color space and the Lab color space, respectively, and then project the pixel values ​​of the shadow image and the reference image without shadow to the two-dimensional brightness variation coordinate system in the HSV color space and the Lab color space to obtain the initial data point sets corresponding to the two color spaces. The dual-space shadow region adjustment module is used to model the distribution of the no-shadow region and the shadow region based on two initial data point sets using a multivariate Gaussian distribution mixture model. It calculates the posterior probability of each data point belonging to the no-shadow region or the shadow region based on the Gaussian distribution modeling results, and adjusts the region to which each data point belongs based on the posterior probability. Then, it re-models the Gaussian distribution based on the adjusted results. After multiple iterations, it obtains the fine-grained shadow mask corresponding to the HSV color space and the Lab color space, respectively. The precise shadow mask acquisition module is used to fuse the fine-grained shadow masks corresponding to the HSV color space and the Lab color space to obtain the precise shadow mask of the shadow image, thereby realizing unsupervised shadow detection by dual-space fusion.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

Images