A shadow removal method, system, terminal and storage medium for urban building images

Abstract

The application relates to the technical field of image processing, and discloses a shadow removal method, system and terminal for urban building images and a storage medium, the method comprising the following steps: preparing a synthetic data set for a building scene, including multiple shadow-bearing images and shadow-free images; inputting the shadow-bearing images into a feature enhancement module, extracting a one-dimensional semantic vector, and generating a feature map according to the one-dimensional semantic vector and the shadow-bearing images; calculating feature consistency loss and absolute noise loss, and calculating a joint loss; optimizing the parameters of the feature enhancement module and a diffusion model according to the joint loss to obtain a trained target diffusion model; obtaining a to-be-removed shadow urban building image and inputting the to-be-removed shadow urban building image into the target diffusion model to output a shadow-free urban building image. The application aims to accurately detect and remove shadows in urban building images, restore real scene information blocked by the shadows, and significantly improve the accuracy of shadow area restoration under a complex light scene.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and particularly relates to a shadow removal method, system, terminal and computer readable storage medium for urban building images. BACKGROUND

[0002] In recent years, shadow removal technology has been widely concerned in the field of computer vision and image processing, and has become a key research direction for improving image quality. Shadows are ubiquitous in natural scene images, and when the light source is partially or completely blocked, shadows will significantly reduce the visual quality of the image and have a serious impact on subsequent visual tasks (such as target tracking, target detection, face recognition, etc.). Therefore, it is of great research significance and application value to recover a clean image from an image with shadows.

[0003] One of the main challenges of shadow removal is the uneven spatial distribution and diverse patterns of shadows, which make it difficult for mainstream models (such as convolutional neural networks and window-based Transformers) to effectively model shadows. This is mainly due to the inherent weight sharing characteristics of these models, which require a single parameter set to handle complex degrees of shadow degradation, which may result in suboptimal trade-offs.

[0004] To overcome this challenge, an intuitive solution is to choose an advanced model that can model spatially heterogeneous interactions. In other words, an ideal model should be able to take adaptive action based on the specific content of the shadow. A competitive candidate model is the classic visual Transformer, which can handle complex shadow distributions and lighting changes by utilizing global self-attention mechanisms. However, despite the excellent performance of these methods in certain scenarios, they still have limitations when dealing with shadows under complex outdoor lighting conditions, such as strong coupling between shadows and backgrounds, blurred edges of shadows, and interference of multiple shadows.

[0005] Therefore, the prior art still needs to be improved and developed. SUMMARY

[0006] The main purpose of the present application is to provide a shadow removal method, system, terminal and computer readable storage medium for urban building images, which aims to solve the problem that the existing technology cannot effectively handle complex shadow areas when removing shadows from urban building images under complex lighting conditions.

[0007] To achieve the above purpose, the present application provides a shadow removal method for urban building images, which comprises the following steps:

[0008] A synthetic dataset facing a building scene is made, which includes multiple shadowed images and multiple shadow-free images corresponding to each shadowed image one by one;

[0009] The shadowed image is input into a feature enhancement module, a one-dimensional semantic vector of the shadowed image is extracted through a CLIP visual encoder of the feature enhancement module, and a feature map is generated according to the one-dimensional semantic vector and the shadowed image through a feature enhancement network of the feature enhancement module;

[0010] A feature consistency loss is calculated according to the feature map and the shadow-free feature map, an absolute noise loss is calculated according to the feature map and the time step noise prediction, a joint loss is calculated according to the absolute noise loss and the feature consistency loss, and parameters of the feature enhancement module and the diffusion model are optimized according to the joint loss to obtain a trained target diffusion model;

[0011] A shadow-removed urban building image is obtained, and the shadow-removed urban building image is input into the target diffusion model to output a shadow-free urban building image.

[0012] Optionally, the shadow removal method for urban building images, wherein the synthetic dataset facing the building scene is made, specifically comprising:

[0013] The light source attribute parameters are adjusted by using Unreal Engine, and the light source attribute parameters include a solar elevation angle, a solar azimuth angle, a light intensity, a color temperature and a shadow softness;

[0014] A building scene is constructed in Unreal Engine, and the building scene includes a building model, an environment setting and a camera setting;

[0015] Multiple shadowed images are collected based on light conditions of three different time periods and diversified building scenes, a shadow-free image corresponding to each shadowed image is generated by disabling a shadow projection function in the scene, during the image collection process, all collected images are rendered at a preset pixel resolution, and all rendered images are saved in a preset format, and a synthetic dataset is obtained.

[0016] Optionally, the shadow removal method for urban building images, wherein the feature consistency loss is calculated according to the feature map and the shadow-free feature map, specifically comprising:

[0017] ;

[0018] wherein, the feature consistency loss is represented as Lconsistency, the target shadow-free image is represented as Igt, the shadowed image is represented as Igt, represents a one-dimensional semantic vector extracted by a CLIP visual encoder, represents a target shadow-free image obtained as input, represents a shadowed image obtained as input.

[0019] Optionally, the shadow removal method for urban building images, wherein the absolute noise loss is calculated according to the feature map and the time step noise prediction, specifically:

[0020] ;

[0021] wherein, represents the absolute noise loss, represents the expectation of a random variable, represents the actual noise, represents the intermediate state of the diffusion model at time step .

[0022] Optionally, the shadow removal method for urban building images, wherein the joint loss is calculated according to the absolute noise loss and the feature consistency loss, specifically:

[0023] ;

[0024] wherein, represents the joint loss, represents a weight parameter for balancing the contributions of the absolute noise loss and the feature consistency loss.

[0025] Optionally, the shadow removal method for urban building images, wherein the feature consistency loss is used to constrain the output of the feature enhancement network to ensure that the features of the shadowed image after encoding are consistent with the features of the target shadow-free image;

[0026] the absolute noise loss is used to combine the output of the feature enhancement module with the time step noise prediction to ensure that the shadow-free results generated by the diffusion model at different time steps meet the required quality;

[0027] the joint loss is used to enable the diffusion model to simultaneously learn the mapping relationship between the shadowed image and the target shadow-free image, as well as the semantic extraction ability of the feature enhancement network.

[0028] Optionally, the shadow removal method for urban building images, wherein the shadow removal method for urban building images further comprises:

[0029] The shadow-free urban building image output by the target diffusion model is taken as data support of an image processing task, and the field of the image processing task includes urban planning, automatic driving and remote sensing analysis.

[0030] In addition, to achieve the above object, the present application also provides a shadow removal system for urban building images, wherein the shadow removal system for urban building images comprises:

[0031] A data set making module is configured to make a synthetic data set for a building scene, and the synthetic data set comprises a plurality of shadowed images and a plurality of shadow-free images corresponding to each of the shadowed images;

[0032] A feature map generating module is configured to input the shadowed image into a feature enhancement module, extract a one-dimensional semantic vector of the shadowed image through a CLIP visual encoder of the feature enhancement module, and generate a feature map according to the one-dimensional semantic vector and the shadowed image through a feature enhancement network of the feature enhancement module;

[0033] A joint loss optimization module is configured to calculate a feature consistency loss according to the feature map and a shadow-free feature map, calculate an absolute noise loss according to the feature map and a time step noise prediction, calculate a joint loss according to the absolute noise loss and the feature consistency loss, and optimize parameters of the feature enhancement module and the diffusion model according to the joint loss to obtain a trained target diffusion model;

[0034] An image shadow removal module is configured to acquire a shadow-removal-targeted urban building image, input the shadow-removal-targeted urban building image into the target diffusion model, and output a shadow-free urban building image.

[0035] In addition, to achieve the above object, the present application also provides a terminal, wherein the terminal comprises a memory, a processor, and a shadow removal program for urban building images stored in the memory and executable on the processor, and the shadow removal program for urban building images implements the steps of the shadow removal method for urban building images when executed by the processor.

[0036] In addition, to achieve the above object, the present application also provides a computer readable storage medium, wherein the computer readable storage medium stores a shadow removal program for urban building images, and the shadow removal program for urban building images implements the steps of the shadow removal method for urban building images when executed by a processor.

[0037] In the present application, a synthetic dataset facing a building scene is made, which includes multiple shadow images and multiple shadow-free images corresponding to each shadow image; the shadow image is input into a feature enhancement module, a one-dimensional semantic vector of the shadow image is extracted through the CLIP visual encoder of the feature enhancement module, and a feature map is generated according to the one-dimensional semantic vector and the shadow image through the feature enhancement network of the feature enhancement module; the feature consistency loss is calculated according to the feature map and the shadow-free feature map, the absolute noise loss is calculated according to the feature map and the time step noise prediction, the joint loss is calculated according to the absolute noise loss and the feature consistency loss, and the parameters of the feature enhancement module and the diffusion model are optimized according to the joint loss to obtain a trained target diffusion model; a city building image to be removed from the shadow is obtained, the city building image to be removed from the shadow is input into the target diffusion model, and a shadow-free city building image is output. The present application aims to accurately detect and remove the shadow in the city building image, restore the real scene information blocked by the shadow, significantly improve the accuracy of the shadow area recovery under complex lighting scenes, and provide high-quality data support for image processing tasks. BRIEF DESCRIPTION OF DRAWINGS

[0038] Figure 1 is a flowchart of a preferred embodiment of the shadow removal method for city building images of the present application;

[0039] Figure 2 is a schematic diagram of training a diffusion model in a preferred embodiment of the shadow removal method for city building images of the present application;

[0040] Figure 3 is a structure diagram of a preferred embodiment of the shadow removal system for city building images of the present application;

[0041] Figure 4 is a structure diagram of a preferred embodiment of the terminal of the present application. DETAILED DESCRIPTION

[0042] To make the purpose, technical scheme and advantages of the present application clearer and more explicit, the present application is further described in detail below with reference to the drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0043] The shadow removal method for city building images according to the preferred embodiment of the present application, as shown in Figure 1 and Figure 2 The shadow removal method for city building images includes the following steps:

[0044] Step S10, a synthetic dataset for a building scene is made, the synthetic dataset includes multiple shadow images and multiple shadow-free images corresponding to each shadow image.

[0045] Specifically, as computer vision tasks increasingly require large-scale and high-quality datasets, synthetic data has gradually become an efficient way to obtain data. In particular, in the building scene, real data is difficult to collect and the annotation cost is high, which limits the coverage and diversity of light changes of existing datasets. Therefore, the present application uses Unreal Engine (UE) to make a synthetic dataset for a building scene, simulates the light conditions of three typical time periods, noon, afternoon and dusk, and generates building image pairs containing shadows and shadow-free images (for example, if there are five thousand pictures in the dataset, then there are two thousand five hundred shadow images and two thousand five hundred shadow-free images), ensuring the diversity and practicality of the dataset.

[0046] The light source attribute parameters are adjusted using Unreal Engine, including the solar elevation angle, the solar azimuth angle, the light intensity, the color temperature and the shadow softness; the building scene is constructed in Unreal Engine, including the building model, the environment setting and the camera setting; based on the light conditions of three different time periods and the diversified building scenes, multiple shadow images are collected, the shadow-free images corresponding to each shadow image are generated by disabling the shadow projection function in the scene, during the image collection process, all collected images are rendered at a preset pixel resolution, and all rendered images are saved in a preset format, to obtain the synthetic dataset.

[0047] Specifically, it is introduced as follows:

[0048] (1) Light condition simulation:

[0049] Shadows are an important part of the dataset. In order to realistically restore the light effects of different time periods and present the shadows of buildings under various light source directions and intensities, the following light source attribute parameters are adjusted in Unreal Engine:

[0050] A. Solar elevation angle: 90 degrees (direct sunlight) at noon, 45 degrees (medium height) in the afternoon, and 15 degrees (low angle) at dusk, to simulate the position of the sun at different times.

[0051] B. Solar azimuth angle: adjust according to the time period to ensure that the shadow direction conforms to the actual scene. The shadow is short and perpendicular in the noon, and gradually lengthens and deviates to one side in the afternoon and dusk.

[0052] C. Light intensity: Set to maximum at noon (simulating strong light), reduce to medium intensity in the afternoon, and further reduce to weak light intensity at dusk to simulate natural light attenuation.

[0053] D. Color temperature: 5500K at noon (cool white light, simulating daylight), 4500K in the afternoon (warm white light, simulating afternoon light), and 3000K at dusk (warm yellow light, simulating sunset light).

[0054] E. Shadow softness: Adjust the shadow map resolution and filtering parameters of the light source to control the sharpness of the shadow. The shadow is relatively sharp at noon, and gradually softens in the afternoon and at dusk to conform to the characteristics of natural light.

[0055] (2) Scene construction:

[0056] To ensure the diversity and authenticity of the dataset, a highly realistic building scene is constructed in Unreal Engine:

[0057] A. Building models: Use high-precision building models that cover a variety of architectural styles and structural complexities (such as high-rise buildings, low-rise buildings, and complex facade designs) to ensure the diversity of the dataset. Each building model is optimized to ensure clear details and high performance when rendered in UE.

[0058] B. Environment settings: To enhance the realism of the scene, add environmental elements such as skyboxes, terrain, vegetation (such as trees and shrubs), and roads. The skybox is adjusted according to the time period, with a bright blue sky at noon, a slightly warm sky in the afternoon, and an orange-red sunset sky at dusk.

[0059] C. Camera settings: Use multi-angle shooting, including overhead, eye-level, and oblique angles, to ensure that the images cover different parts of the building. The camera focal length and field of view range are fixed to ensure image consistency.

[0060] (3) Image acquisition:

[0061] During image acquisition, the invention renders all images at a resolution of 512x512 pixels, which balances computational efficiency while retaining sufficient detail, suitable for the input requirements of most deep learning models. The shadow-free image is generated by disabling the shadow projection function in the scene, ensuring that it is identical to the shadowed image in terms of scene content and lighting conditions, thereby providing accurate control samples for the shadow removal task. In addition, all images are saved in PNG format to ensure that high-quality image information is preserved, avoiding information loss due to the use of lossy compression formats such as JPEG. The dataset is stored according to lighting conditions and shadow state, with a clear directory structure for easy subsequent use and management.

[0062] The construction of the high-quality data set directly improves the stability of model training and the quality of generated images. The high-quality data set can reduce the sensitivity of model training to data noise, so that the model can better learn the rules of shadow removal, thereby generating more accurate shadow-free images. The high-quality data set constructed by the present application ensures data quality through artificial labeling, fine adjustment and quality control, and provides a reliable foundation for model training.

[0063] Step S20, inputting the shadow image into a feature enhancement module, extracting a one-dimensional semantic vector of the shadow image through a CLIP visual encoder of the feature enhancement module, and generating a feature map according to the one-dimensional semantic vector and the shadow image through a feature enhancement network of the feature enhancement module.

[0064] Specifically, in the shadow removal task based on the diffusion model, how to effectively introduce the conditional image information is the key. In order to maximize the utilization rate of the input image features, the present application proposes a feature enhancement module which runs separately from the diffusion model and has a similar network structure to the diffusion model. The main goal of the feature enhancement module is to extract high-quality semantic prior information from the shadow image, capture the potential semantic association and structural features in the image, and optimize the depth and accuracy of feature expression. The processing process of the feature enhancement module is shown in Table 1.

[0065] Table 1: Processing process of the feature enhancement module

[0066]

[0067] The CLIP semantic guidance enables the model to understand the overall semantics of the image during the generation process, thereby better restoring the reasonable colors and textures of the shadow area and avoiding color deviation and texture distortion. This ensures that the generated image is more accurate in both visual and semantic aspects. By extracting semantic features using CLIP and integrating them into the generation process of the diffusion model, the model can be guided to generate shadow-free images with more accurate semantic information.

[0068] Step S30, calculating a feature consistency loss according to the feature map and the shadow-free feature map, calculating an absolute noise loss according to the feature map and the time step noise prediction, calculating a joint loss according to the absolute noise loss and the feature consistency loss, and optimizing the parameters of the feature enhancement module and the diffusion model according to the joint loss to obtain a trained target diffusion model.

[0069] Specifically, in order to ensure that the feature map extracted by the feature enhancement network can effectively remove the shadow effect and has high semantic expression ability, the present application proposes a feature consistency loss, which is calculated as follows:

[0070] ;

[0071] wherein, denotes the feature consistency loss, denotes the target shadow-free image, denotes the shadowed image, denotes the one-dimensional semantic vector (i.e., semantic feature) extracted by the CLIP (Contrastive Language-Image Pre-training) visual encoder, denotes the shadow-free feature map obtained when the target shadow-free image is inputted into the feature enhancement network, denotes the feature map obtained when the shadowed image is inputted into the feature enhancement network.

[0072] The loss function ensures that the encoded features of the shadowed image are consistent with the features of the target shadow-free image by constraining the output of the feature enhancement network, thereby effectively extracting feature representations free of shadow interference and laying a foundation for subsequent generation processes.

[0073] In the training phase, the feature enhancement network (i.e., feature encoder) and the diffusion model adopt a joint optimization strategy to achieve the following goals:

[0074] (1) Compactness and semantic consistency of the feature space: the feature encoder is constrained by the loss function to extract a feature space with high semantic consistency. This compact feature space can reduce noise interference and strengthen the guiding effect of the diffusion model on the generation of shadow-free images.

[0075] (2) Multi-step noise prediction of the diffusion model: the feature map extracted by the feature encoder is concatenated to the input of the diffusion model to guide multi-time step noise prediction. Specifically, the noise prediction target of the diffusion model at time step is:

[0076] ;

[0077] wherein, denotes the absolute noise loss, denotes the expectation of a random variable, denotes the actual noise, denotes the intermediate state of the diffusion model at time step .

[0078] The loss function ensures that the shadow-free results generated by the diffusion model maintain high quality at different time steps by combining the output of the feature enhancement module with the time step noise prediction.

[0079] Joint optimization strategy: In actual training, the parameters of the feature enhancement module and the diffusion model are optimized jointly. The joint loss function is designed as:

[0080] ;

[0081] wherein, represents the joint loss, represents the weight parameter, used to balance the contribution of the absolute noise loss and the feature consistency loss.

[0082] Through joint optimization, the diffusion model can simultaneously learn the mapping relationship between the shadow image and the target shadow-free image, as well as the semantic extraction ability of the feature encoder.

[0083] The optimization of the joint loss function comprehensively considers the differences at the pixel level, the perception level, the adversarial level, and the semantic level, so that the model achieves a balance between color consistency, detail authenticity, and semantic accuracy, thereby obtaining better generation effect. For example, the pixel-level loss ensures that the generated image has small differences with the target image at the pixel level, the perception loss ensures that the generated image is more similar to the target image in perception, the adversarial loss improves the realism of the generated image, and the CLIP semantic loss ensures that the generated image is consistent with the target image in semantics. The comprehensive effect of these loss functions enables the model to generate high-quality shadow-free images.

[0084] Step S40, obtain a city building image to be removed from shadow, and input the city building image to be removed from shadow to the target diffusion model to output a shadow-free city building image.

[0085] Specifically, after the joint optimization process of step S30, a trained target diffusion model is obtained, and the target diffusion model can be directly used for shadow removal processing, i.e., obtaining a city building image to be removed from shadow, inputting the city building image to be removed from shadow to the target diffusion model, and directly outputting a shadow-free city building image.

[0086] Further, the shadow-free city building image output by the target diffusion model can be used as data support for image processing tasks, wherein the fields of the image processing tasks include city planning, autonomous driving, and remote sensing analysis.

[0087] The present application aims at the key problem of color difference in shadow area in the shadow removal task of urban building images, and proposes a systematic solution from data construction, feature design to optimization strategy. Unlike previous researches that simply rely on traditional methods or other generative models, the present application creates a high-quality dataset containing original images and shadow-free images, combines semantic features extracted by CLIP encoder to guide the diffusion model generation, and designs a joint loss system to improve color consistency and generation quality. The overall scheme realizes multiple breakthroughs in data, model and optimization, significantly improves the accuracy of shadow area recovery in complex lighting scenes, and shows the pioneering value in technical path and practical application scenarios. The method of the present application aims to accurately detect and remove shadows in images, recover the true scene information obscured by shadows, and provide high-quality data support for image processing tasks in urban planning, autonomous driving, remote sensing analysis and other fields.

[0088] The urban building image shadow removal scheme proposed by the present application realizes significant technical effects through innovation in data construction, model design and optimization strategy, and effectively solves the key problem of color difference in shadow area in the shadow removal task.

[0089] Firstly, high-quality shadow removal effect is the most direct manifestation. The scheme can effectively remove shadows from urban building images, restore the true colors and texture details of shadow areas, and make the overall image clearer and more natural. This not only improves the visual quality of the image, but also lays a foundation for subsequent image analysis and application.

[0090] Secondly, color consistency is another important advantage of the present application. Previous shadow removal methods often fail to ensure the color consistency of the image after removing shadows, and are prone to problems such as large color difference between shadow areas and non-shadow areas, leading to image distortion. The present application significantly improves the color consistency of the image after removing shadows through joint loss function optimization, especially the introduction of CLIP semantic loss. CLIP semantic loss can guide the model to consider the overall semantic information of the image during the generation process, thereby better restoring the color of the shadow area and avoiding color deviation.

[0091] In addition, the generation method based on diffusion model also greatly improves the generation quality. Traditional image processing methods are prone to problems such as blurring and distortion when dealing with shadow removal. However, the diffusion model used in the present application can generate high-quality, high-resolution shadow-free images that retain the details of the image, making the generated image more realistic and natural.

[0092] Generalization ability is an important indicator of a model's practicality. Through the training of high-quality datasets and the guidance of CLIP semantic information, the model's adaptability to different lighting conditions and different architectural styles is enhanced, thereby improving the model's generalization ability. This means that the model not only can handle shadow removal problems in specific scenarios, but also can be applied to more extensive practical scenarios.

[0093] Finally, semantic preservation is also an important advantage of the invention. In the shadow removal process, it is crucial to ensure that the semantic information of the image is not destroyed. The introduction of CLIP semantic features can ensure that the semantic information of the image is effectively preserved during the shadow removal process, avoiding semantic deviations. This makes the processed image not only more visually appealing, but also more accurate in terms of semantics.

[0094] In addition, in terms of dataset construction, data augmentation and simulated data can be performed on the dataset. Data augmentation: existing datasets can be augmented, such as rotation, scaling, translation, color transformation, etc., to increase the diversity of the dataset. Simulated data: 3D modeling software or game engines can be used to simulate the generation of urban building images containing different lighting conditions and architectural styles, and automatically generate corresponding shadow masks and shadow-free images.

[0095] In terms of model construction, diffusion model variants: other types of diffusion models can be tried, such as Score-Based Generative Models or Denoising Autoencoders, and corresponding adjustments can be made. CLIP alternatives: other pre-trained visual-linguistic models can be tried, such as ALIGN or FILIP, to extract semantic features of images. Semantic fusion methods: different ways of integrating semantic features into diffusion models can be tried, such as feature fusion through Transformer structure.

[0096] In terms of optimization strategies, loss function variants: other types of loss functions can be tried, such as style loss, edge loss, etc., to further improve the quality of generated images. Optimization algorithms: other optimization algorithms can be tried, such as AdamW, SGD, etc., and learning rate, batch size, etc. Hyperparameters can be adjusted. Adaptive weight adjustment: an adaptive weight adjustment strategy can be designed to dynamically adjust the weight coefficients of each loss function according to the feedback during training.

[0097] In terms of hardware acceleration: GPU is used for model training and inference acceleration. Distributed training strategy is adopted to speed up model training.

[0098] In the aspect of application scenario expansion, the present application is applied to other types of image shadow removal tasks, such as human face images, natural scenery images, etc. Or the present application is applied to the fields of image editing, image enhancement, etc.

[0099] Further, as Figure 3 shown, based on the above-mentioned shadow removal method for urban building images, the present application also correspondingly provides a shadow removal system for urban building images, wherein the shadow removal system for urban building images comprises:

[0100] a data set making module 51 for making a synthetic data set for building scenes, the synthetic data set comprising a plurality of shadowed images and a plurality of shadow-free images corresponding to each of the shadowed images;

[0101] a feature map generating module 52 for inputting the shadowed images into a feature enhancement module, extracting a one-dimensional semantic vector of the shadowed images through a CLIP visual encoder of the feature enhancement module, and generating a feature map according to the one-dimensional semantic vector and the shadowed images through a feature enhancement network of the feature enhancement module;

[0102] a joint loss optimization module 53 for calculating a feature consistency loss according to the feature map and a shadow-free feature map, calculating an absolute noise loss according to the feature map and a time step noise prediction, calculating a joint loss according to the absolute noise loss and the feature consistency loss, and optimizing parameters of the feature enhancement module and a diffusion model according to the joint loss to obtain a trained target diffusion model;

[0103] an image shadow removal module 54 for acquiring a shadow-removal-targeted urban building image, inputting the shadow-removal-targeted urban building image into the target diffusion model, and outputting a shadow-free urban building image.

[0104] Further, as Figure 4 shown, based on the above-mentioned shadow removal method for urban building images and the system, the present application also correspondingly provides a terminal, which comprises a processor 10, a memory 20 and a display 30. Figure 4 Only part of the components of the terminal are shown, but it should be understood that all the shown components are not required to be implemented, and more or fewer components can be alternatively implemented.

[0105] The memory 20 can be an internal storage unit of the terminal in some embodiments, such as a hard disk or a memory of the terminal. The memory 20 can also be an external storage device of the terminal in other embodiments, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. equipped on the terminal. Further, the memory 20 can include both the internal storage unit and the external storage device of the terminal. The memory 20 is used to store application software and various data installed on the terminal, such as program codes of the terminal, etc. The memory 20 can also be used to temporarily store data that has been output or will be output. In an embodiment, the memory 20 stores a shadow removal program for urban building images 40, which can be executed by the processor 10 to implement the shadow removal method for urban building images in the present application.

[0106] The processor 10 can be a Central Processing Unit (CPU), a microprocessor or other data processing chip in some embodiments, which is used to run program codes or process data stored in the memory 20, such as to execute the shadow removal method for urban building images, etc.

[0107] The display 30 can be an LED display, a liquid crystal display, a touch liquid crystal display, an OLED (Organic Light-Emitting Diode) touch, etc. in some embodiments. The display 30 is used to display information of the terminal and to display a visualized user interface. The processor 10, the memory 20 and the display 30 of the terminal communicate with each other through a system bus.

[0108] In an embodiment, the steps of the shadow removal method for urban building images as described above are implemented when the processor 10 executes the shadow removal program for urban building images 40 in the memory 20.

[0109] The present application also provides a computer readable storage medium, wherein the computer readable storage medium stores a shadow removal program for urban building images, which, when executed by a processor, implements the steps of the shadow removal method for urban building images as described above.

[0110] In summary, the present application provides a shadow removal method, system, terminal and computer readable storage medium for urban building images, the method comprising: making a synthetic dataset for building scenes, the synthetic dataset comprising multiple shadow images and multiple shadow-free images corresponding to each shadow image; inputting the shadow image into a feature enhancement module, extracting a one-dimensional semantic vector of the shadow image through the CLIP visual encoder of the feature enhancement module, and generating a feature map according to the one-dimensional semantic vector and the shadow image through the feature enhancement network of the feature enhancement module; calculating a feature consistency loss according to the feature map and the shadow-free feature map, calculating an absolute noise loss according to the feature map and the time step noise prediction, calculating a joint loss according to the absolute noise loss and the feature consistency loss, and optimizing the parameters of the feature enhancement module and the diffusion model according to the joint loss to obtain a trained target diffusion model; obtaining a shadow removal urban building image to be removed, inputting the shadow removal urban building image to be removed into the target diffusion model, and outputting a shadow-free urban building image. The present application aims to accurately detect and remove shadows in urban building images, restore the real scene information obscured by shadows, significantly improve the accuracy of shadow area recovery under complex lighting scenes, and provide high-quality data support for image processing tasks.

[0111] It should be noted that in this document, the terms "comprising", "including", or any other variant thereof are intended to cover a non-exclusive inclusion, such that processes, methods, articles, or terminals including a series of elements not only include those elements, but also include other elements not explicitly listed, or inherent to such processes, methods, articles, or terminals. Without more limitations, the element defined by the statement "including a" does not exclude the presence of additional identical elements in the process, method, article, or terminal including the element.

[0112] Of course, those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing relevant hardware (such as a processor, a controller, etc.) through a computer program, and the program can be stored in a computer readable storage medium readable by a computer, and the program can include the processes of the above-mentioned method embodiments when executed. The computer readable storage medium can be a memory, a disk, an optical disk, etc.

[0113] It should be understood that the application is not limited to the above examples, and those skilled in the art can make improvements or changes according to the above description, and all such improvements and changes shall fall within the scope of protection of the claims of the present application.

Claims

1. A method for shadow removal in urban architectural images, characterized in that, The shadow removal method for urban building images includes: Create a synthetic dataset for building scenes, the synthetic dataset including multiple shaded images and multiple unshaded images corresponding one-to-one with each shaded image; The shaded image is input into the feature enhancement module, and the CLIP visual encoder of the feature enhancement module extracts a one-dimensional semantic vector from the shaded image. The feature enhancement network of the feature enhancement module generates a feature map based on the one-dimensional semantic vector and the shaded image. The feature consistency loss is calculated based on the feature map and the shadowless feature map. The absolute noise loss is calculated based on the feature map and the time step noise prediction. The joint loss is calculated based on the absolute noise loss and the feature consistency loss. The parameters of the feature enhancement module and the diffusion model are optimized based on the joint loss to obtain the trained target diffusion model. Obtain an image of city buildings whose shadows need to be removed, input the image of city buildings whose shadows need to be removed into the target diffusion model, and output an image of city buildings without shadows; The calculation of feature consistency loss based on the feature map and the shadowless feature map is specifically as follows: ； in, This represents the feature consistency loss. This indicates an image of the target without shadows. Image with shadow, This represents a one-dimensional semantic vector extracted by the CLIP visual encoder. This indicates the target image without shadows. The shadowless feature map obtained as input This indicates that the image will be shaded. The feature map obtained when used as input; The calculation of absolute noise loss based on the feature map and time-step noise prediction specifically involves: ； in, Indicates absolute noise loss. It represents the expected value of a random variable. Indicates actual noise. This indicates that the diffusion model is at time step The intermediate state.

2. The shadow removal method for urban building images according to claim 1, characterized in that, The creation of the synthetic dataset for building scenes specifically includes: The Unreal Engine is used to adjust the light source attribute parameters, which include solar altitude angle, solar azimuth angle, light intensity, color temperature, and shadow softness. Build a building scene in Unreal Engine, which includes building models, environment settings, and camera settings; Based on lighting conditions at three different time periods and diverse building scenes, multiple images with shadows were acquired. By disabling shadow casting in the scene, a shadowless image corresponding to each image with shadows was generated. During the image acquisition process, all acquired images were rendered at a preset pixel resolution, and all rendered images were saved in a preset format to obtain a synthetic dataset.

3. The shadow removal method for urban building images according to claim 1, characterized in that, The calculation of the joint loss based on the absolute noise loss and the feature consistency loss is specifically as follows: ； in, Indicates joint loss, This represents a weighting parameter used to balance the contributions of absolute noise loss and the feature consistency loss.

4. The shadow removal method for urban building images according to claim 3, characterized in that, The feature consistency loss is used to constrain the output of the feature enhancement network to ensure that the features encoded in the shaded image are consistent with the features of the target unshaded image. The absolute noise loss is used to ensure that the shadowless results generated by the diffusion model maintain the required quality at different time steps by combining the output of the feature enhancement module with the time step noise prediction. The joint loss is used to enable the diffusion model to simultaneously learn the mapping relationship between the shaded image and the target unshaded image, as well as the semantic extraction capability of the feature enhancement network.

5. The shadow removal method for urban building images according to claim 1, characterized in that, The shadow removal method for urban building images also includes: The shadowless urban building images output by the target diffusion model are used as data support for image processing tasks, the fields of which include urban planning, autonomous driving, and remote sensing analysis.

6. A shadow removal system for urban architectural images, characterized in that, The shadow removal system for urban building images is used to implement the shadow removal method for urban building images according to any one of claims 1-5, wherein the shadow removal system for urban building images comprises: The dataset creation module is used to create a synthetic dataset for building scenes. The synthetic dataset includes multiple images with shadows and multiple images without shadows that correspond one-to-one with each image with shadows. The feature map generation module is used to input the shaded image into the feature enhancement module, extract a one-dimensional semantic vector from the shaded image through the CLIP visual encoder of the feature enhancement module, and generate a feature map based on the one-dimensional semantic vector and the shaded image through the feature enhancement network of the feature enhancement module. The joint loss optimization module is used to calculate the feature consistency loss based on the feature map and the shadowless feature map, calculate the absolute noise loss based on the feature map and the time step noise prediction, calculate the joint loss based on the absolute noise loss and the feature consistency loss, and optimize the parameters of the feature enhancement module and the diffusion model based on the joint loss to obtain the trained target diffusion model. The image shadow removal module is used to acquire city building images with shadows to be removed, input the city building images with shadows to be removed into the target diffusion model, and output city building images without shadows.

7. A terminal, characterized in that, The terminal includes: a memory, a processor, and a shadow removal program for urban building images stored in the memory and executable on the processor, wherein when the shadow removal program for urban building images is executed by the processor, it implements the steps of the shadow removal method for urban building images as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a shadow removal program for urban building images, which, when executed by a processor, implements the steps of the shadow removal method for urban building images as described in any one of claims 1-5.

Images