Urban building image-oriented shadow removal method and system, terminal and storage medium

Abstract

The invention relates to the technical field of image processing, and discloses an urban building image-oriented shadow removal method and system, a terminal and a storage medium, and the method comprises the steps: making a building scene-oriented synthetic data set which comprises a plurality of shadow images and shadow-free images; inputting the image with the shadow into a feature enhancement module, extracting a one-dimensional semantic vector, and generating a feature map according to the one-dimensional semantic vector and the image with the shadow; calculating feature consistency loss and absolute noise loss, calculating joint loss, and optimizing parameters of the feature enhancement module and the diffusion model according to the joint loss to obtain a trained target diffusion model; and obtaining an urban building image to be subjected to shadow removal, inputting the image to the target diffusion model, and outputting a non-shadow urban building image. The method aims at accurately detecting and removing the shadow in the urban building image, recovering real scene information shielded by the shadow, and remarkably improving the recovery accuracy of the shadow area in the complex illumination scene.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and particularly relates to a shadow removal method, system, terminal and computer readable storage medium for urban building images. BACKGROUND

[0002] In recent years, shadow removal technology has attracted extensive attention in the field of computer vision and image processing, and has become a key research direction for improving image quality. Shadows are ubiquitous in natural scene images, and when the light source is partially or completely blocked, shadows will significantly reduce the visual quality of the image and have a serious impact on subsequent visual tasks (such as target tracking, target detection, face recognition, etc.). Therefore, it is of great research significance and application value to recover a clean image from an image with shadows.

[0003] One of the main challenges of shadow removal is the uneven spatial distribution and diverse patterns of shadows, which make it difficult for mainstream models (such as convolutional neural networks and window-based Transformers) to effectively model shadows. This is mainly due to the inherent weight sharing characteristics of these models, which require a single parameter set to handle complex degrees of shadow degradation, which may result in suboptimal trade-offs.

[0004] To overcome this challenge, an intuitive solution is to choose an advanced model that can model spatially heterogeneous interactions. In other words, an ideal model should be able to take adaptive action based on the specific content of the shadow. A competitive candidate model is the classic visual Transformer, which can handle complex shadow distributions and lighting changes by utilizing global self-attention mechanisms. However, despite the excellent performance of these methods in certain scenarios, they still have limitations when dealing with shadows under complex outdoor lighting conditions, such as strong coupling between shadows and backgrounds, blurred shadow edges, and interference from multiple shadows.

[0005] Therefore, the prior art still needs to be improved and developed. SUMMARY

[0006] The main purpose of the present application is to provide a shadow removal method, system, terminal and computer readable storage medium for urban building images, which aims to solve the problem that the effect is not good when removing shadows from urban building images under complex lighting conditions in the prior art, and cannot effectively handle complex shadow areas.

[0007] To achieve the above purpose, the present application provides a shadow removal method for urban building images, which comprises the following steps: Producing a synthetic dataset for building scenes, wherein the synthetic dataset includes a plurality of shadowed images and a plurality of shadow-free images corresponding one-to-one to each shadowed image; Inputting the shadowed image into a feature enhancement module, extracting a one-dimensional semantic vector of the shadowed image through a CLIP visual encoder of the feature enhancement module, and generating a feature map according to the one-dimensional semantic vector and the shadowed image through a feature enhancement network of the feature enhancement module; Calculating a feature consistency loss based on the feature map and the unshadowed feature map, calculating an absolute noise loss based on the feature map and the time-step noise prediction, calculating a joint loss based on the absolute noise loss and the feature consistency loss, and optimizing parameters of the feature enhancement module and the diffusion model based on the joint loss to obtain a trained target diffusion model; An image of a city building with shadows to be removed is acquired, the image of the city building with shadows to be removed is input into the target diffusion model, and an image of a city building without shadows is output.

[0008] Optionally, in the shadow removal method for urban building images, the step of preparing a synthetic dataset for building scenes specifically includes: Using Unreal Engine to adjust light source attribute parameters, including sun altitude angle, sun azimuth angle, light intensity, color temperature, and shadow softness; Building a building scene in Unreal Engine, wherein the building scene includes a building model, environment settings, and camera settings; Based on the lighting conditions of three different time periods and diverse building scenes, multiple shadowed images are collected. By disabling the shadow casting function in the scene, a shadow-free image corresponding to each shadowed image is generated. During the image acquisition process, all collected images are rendered at a preset pixel resolution and saved in a preset format to obtain a synthetic dataset.

[0009] Optionally, in the shadow removal method for urban building images, the feature consistency loss is calculated based on the feature map and the shadow-free feature map, specifically: ; in, represents the feature consistency loss, represents the target shadow-free image, For a shadowed image, represents the one-dimensional semantic vector extracted by the CLIP visual encoder, Indicates that the target has no shadow image As the input, the shadow-free feature map is obtained. Indicates that the image will be shaded the feature map obtained as input.

[0010] Optionally, the shadow removal method for urban building images, wherein the absolute noise loss is calculated according to the feature map and the time step noise prediction, and specifically: ; wherein, the absolute noise loss is represented by, the expectation of a random variable is represented by, the actual noise is represented by, the intermediate state of the diffusion model at the time step is represented by.

[0011] Optionally, the shadow removal method for urban building images, wherein the joint loss is calculated according to the absolute noise loss and the feature consistency loss, and specifically: ; wherein, the joint loss is represented by, the weight parameter is used to balance the contribution of the absolute noise loss and the feature consistency loss.

[0012] Optionally, the shadow removal method for urban building images, wherein the feature consistency loss is used to constrain the output of the feature enhancement network to ensure that the features of the encoded shadow image are consistent with the features of the target shadow-free image; the absolute noise loss is used to combine the output of the feature enhancement module with the time step noise prediction to ensure that the shadow-free result generated by the diffusion model at different time steps meets the required quality; the joint loss is used to enable the diffusion model to simultaneously learn the mapping relationship between the shadow image and the target shadow-free image, as well as the semantic extraction ability of the feature enhancement network.

[0013] Optionally, the shadow removal method for urban building images, wherein the shadow removal method for urban building images further comprises: the shadow-free urban building image output by the target diffusion model is used as data support for image processing tasks, wherein the field of the image processing tasks includes urban planning, autonomous driving, and remote sensing analysis.

[0014] In addition, to achieve the above-mentioned purposes, the present application also provides a shadow removal system for urban building images, wherein the shadow removal system for urban building images comprises: A data set production module is used to produce a synthetic data set for a building scene, wherein the synthetic data set includes a plurality of shadowed images and a plurality of shadow-free images corresponding to each shadowed image; a feature map generation module, configured to input the shadowed image into a feature enhancement module, extract a one-dimensional semantic vector of the shadowed image through a CLIP visual encoder of the feature enhancement module, and generate a feature map based on the one-dimensional semantic vector and the shadowed image through a feature enhancement network of the feature enhancement module; a joint loss optimization module, configured to calculate a feature consistency loss based on the feature map and the shadow-free feature map, calculate an absolute noise loss based on the feature map and the time-step noise prediction, calculate a joint loss based on the absolute noise loss and the feature consistency loss, and optimize parameters of the feature enhancement module and the diffusion model based on the joint loss to obtain a trained target diffusion model; The image shadow removal module is used to obtain an image of a city building with shadows to be removed, input the image of the city building with shadows to be removed into the target diffusion model, and output an image of the city building without shadows.

[0015] In addition, to achieve the above-mentioned purpose, the present invention also provides a terminal, wherein the terminal includes: a memory, a processor, and a shadow removal program for urban building images stored in the memory and runnable on the processor, wherein the shadow removal program for urban building images, when executed by the processor, implements the steps of the shadow removal method for urban building images as described above.

[0016] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a shadow removal program for urban building images, and when the shadow removal program for urban building images is executed by a processor, the steps of the shadow removal method for urban building images as described above are implemented.

[0017] In the present application, a synthetic dataset facing a building scene is made, which includes multiple shadow images and multiple shadow-free images corresponding to each shadow image; the shadow image is input into a feature enhancement module, a one-dimensional semantic vector of the shadow image is extracted through the CLIP visual encoder of the feature enhancement module, and a feature map is generated according to the one-dimensional semantic vector and the shadow image through the feature enhancement network of the feature enhancement module; the feature consistency loss is calculated according to the feature map and the shadow-free feature map, the absolute noise loss is calculated according to the feature map and the time step noise prediction, the joint loss is calculated according to the absolute noise loss and the feature consistency loss, and the parameters of the feature enhancement module and the diffusion model are optimized according to the joint loss to obtain a trained target diffusion model; the shadow-removed urban building image is obtained, and the shadow-removed urban building image is input into the target diffusion model to output a shadow-free urban building image. The present application aims to accurately detect and remove the shadow in the urban building image, restore the real scene information blocked by the shadow, significantly improve the accuracy of the shadow area recovery under complex lighting scenes, and provide high-quality data support for image processing tasks. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 is a flowchart of a preferred embodiment of the shadow removal method for urban building images of the present application; Figure 2 is a schematic diagram of training a diffusion model in a preferred embodiment of the shadow removal method for urban building images of the present application; Figure 3 is a structure diagram of a preferred embodiment of the shadow removal system for urban building images of the present application; Figure 4 is a structure diagram of a preferred embodiment of the terminal of the present application. DETAILED DESCRIPTION

[0019] To make the purpose, technical scheme and advantages of the present application clearer and more explicit, the present application is further described in detail below with reference to the drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0020] The shadow removal method for urban building images according to the preferred embodiment of the present application, as shown in Figure 1 and Figure 2 , the shadow removal method for urban building images includes the following steps: Step S10, a synthetic dataset facing a building scene is made, which includes multiple shadow images and multiple shadow-free images corresponding to each shadow image.

[0021] Specifically, with the increasing requirements of computer vision tasks on the scale and quality of data sets, synthetic data has gradually become an efficient way of data acquisition. Especially in building scenes, real data is difficult to collect and the cost of annotation is high, which limits the coverage and diversity of light changes of existing data sets. Therefore, the application uses Unreal Engine (UE) to make a synthetic data set for building scenes, simulates the light conditions of three typical time periods of noon, afternoon and dusk, and combines diversified building scenes to generate building image pairs containing shadows and no shadows (for example, if the data set has five thousand pictures, then the five thousand pictures are two thousand five hundred shadow images and two thousand five hundred no shadow images), to ensure the diversity and practicality of the data set.

[0022] The light source attribute parameters include the solar elevation angle, the solar azimuth angle, the light intensity, the color temperature and the shadow softness. The building scene is constructed in the Unreal Engine, and the building scene includes a building model, an environment setting and a camera setting. Based on the light conditions of the three different time periods and the diversified building scenes, a plurality of shadow images are collected, and the shadow-free images corresponding to each shadow image are generated by disabling the shadow projection function in the scene. In the image collection process, all collected images are rendered at a preset pixel resolution, and all rendered images are saved in a preset format to obtain the synthetic data set.

[0023] Specifically, it is introduced as follows: (1) Light condition simulation: Shadows are an important part of the data set. In order to realistically restore the light effects of different time periods and present the shadows of buildings under various light source directions and intensities, the following light source attribute parameters are adjusted in the Unreal Engine: A. Solar elevation angle: 90 degrees (direct sunlight) at noon, 45 degrees (medium height) in the afternoon, and 15 degrees (low angle) at dusk to simulate the position of the sun at different times.

[0024] B. Solar azimuth angle: adjust according to the time period to ensure that the shadow direction conforms to the actual scene. The shadow is short and perpendicular in the direction of noon, and gradually lengthens and deviates to one side in the afternoon and dusk.

[0025] C. Light intensity: maximum value (simulating strong light) at noon, reduced to medium intensity in the afternoon, and further reduced to weak light intensity at dusk to simulate natural light attenuation.

[0026] D. Color Temperature: 5500K (cool white light, simulating daylight) at noon, 4500K (warm white light, simulating afternoon light) in the afternoon, and 3000K (warm yellow light, simulating sunset light) at dusk.

[0027] E. Shadow Softness: By adjusting the shadow map resolution and filtering parameters of the light source, the sharpness of the shadows is controlled. The noon shadows are relatively sharp, and the afternoon and dusk shadows gradually soften to conform to the natural lighting characteristics.

[0028] (2) Scene Construction: To ensure the diversity and authenticity of the dataset, a highly realistic building scene is constructed in Unreal Engine: A. Building Models: High-precision building models are used, covering various architectural styles and structural complexities (such as high-rise buildings, low-rise buildings, and complex facade designs) to ensure the diversity of the dataset. Each building model is optimized to ensure clear details and high performance when rendered in UE.

[0029] B. Environment Settings: To enhance the realism of the scene, environmental elements such as skyboxes, terrain, vegetation (such as trees and shrubs), and roads are added. The skybox is adjusted according to the time period, with a bright blue sky at noon, a slightly warm-toned sky in the afternoon, and an orange-red sunset sky at dusk.

[0030] C. Camera Settings: Multiple perspectives are used, including overhead, eye-level, and oblique angles, to ensure that different parts of the building are covered. The camera focal length and field of view range are fixed to ensure image consistency.

[0031] (3) Image Acquisition: During image acquisition, all images are rendered at a resolution of 512x512 pixels, which balances computational efficiency while retaining sufficient detail, suitable for the input requirements of most deep learning models. Shadow-free images are generated by disabling shadow projection in the scene, ensuring consistency with shadowed images in terms of scene content and lighting conditions, providing accurate control samples for shadow removal tasks. In addition, all images are saved in PNG format to preserve high-quality image information and avoid information loss due to the use of lossy compression formats such as JPEG. The dataset is stored by lighting conditions and shadow states, with a clear directory structure for easy subsequent use and management.

[0032] The construction of the high-quality data set directly improves the stability of model training and the quality of generated images. The high-quality data set can reduce the sensitivity of model training to data noise, enabling the model to better learn the rules of shadow removal and generate more accurate shadow-free images. The high-quality data set constructed by the present application ensures data quality through manual labeling, fine adjustment and quality control, providing a reliable foundation for model training.

[0033] Step S20, inputting the shadow image into a feature enhancement module, extracting a one-dimensional semantic vector of the shadow image through a CLIP visual encoder of the feature enhancement module, and generating a feature map according to the one-dimensional semantic vector and the shadow image through a feature enhancement network of the feature enhancement module.

[0034] Specifically, in the shadow removal task based on the diffusion model, how to effectively introduce the conditional image information is the key. In order to maximize the utilization rate of the input image features, the present application proposes a feature enhancement module which runs separately from the diffusion model and has a similar network structure to the diffusion model. The main goal of the feature enhancement module is to extract high-quality semantic prior information from the shadow image, capture the potential semantic association and structural features in the image, and optimize the depth and accuracy of feature expression. The processing process of the feature enhancement module is shown in Table 1.

[0035] Table 1: Processing process of the feature enhancement module

[0036] The CLIP semantic guidance enables the model to understand the overall semantics of the image during the generation process, thereby better restoring the reasonable colors and textures of the shadow area and avoiding color deviation and texture distortion. This ensures that the generated image is more accurate in both visual and semantic aspects. By extracting semantic features using CLIP and integrating them into the generation process of the diffusion model, the model can be guided to generate shadow-free images with more accurate semantic information.

[0037] Step S30, calculating a feature consistency loss according to the feature map and the shadow-free feature map, calculating an absolute noise loss according to the feature map and the time step noise prediction, calculating a joint loss according to the absolute noise loss and the feature consistency loss, and optimizing the parameters of the feature enhancement module and the diffusion model according to the joint loss to obtain a trained target diffusion model.

[0038] Specifically, in order to ensure that the feature map extracted by the feature enhancement network can effectively remove the shadow effect and has high semantic expression ability, the present application proposes a feature consistency loss, which is calculated as follows: ; wherein, represents the feature consistency loss, represents the target shadow-free image, For a shadowed image, represents the one-dimensional semantic vector (i.e., semantic feature) extracted by the CLIP (Contrastive Language-Image Pre-training) visual encoder, Indicates that the target has no shadow image As the input, the shadow-free feature map is obtained. Indicates that the image will be shaded The feature map obtained when used as input.

[0039] This loss function enhances the output of the network by constraining features, ensuring that the features of the encoded image with shadows are consistent with the features of the target image without shadows, thereby effectively extracting feature representations without shadow interference, laying the foundation for the subsequent generation process.

[0040] During the training phase, the feature enhancement network (i.e., feature encoder) and the diffusion model adopt a joint optimization strategy and are trained simultaneously to achieve the following goals: (1) Compactness and semantic consistency of feature space: The feature encoder is implemented through the loss function It is constrained to extract a feature space with high semantic consistency. This compact feature space can reduce noise interference and strengthen the guidance of the diffusion model for shadow-free image generation.

[0041] (2) Multi-step noise prediction of diffusion model: feature map extracted by feature encoder is spliced ​​into the input of the diffusion model to guide the noise prediction of multiple time steps. Specifically, the time step of the diffusion model The noise prediction target is: ; in, represents the absolute noise loss, represents the expectation of a random variable, represents the actual noise, Represents the diffusion model at time step intermediate state.

[0042] This loss function ensures that the shadow-free results generated by the diffusion model maintain high quality at different time steps by combining the output of the feature enhancement module with the time step noise prediction.

[0043] Joint optimization strategy: In actual training, the parameters of the feature enhancement module and the diffusion model are optimized jointly. The joint loss function is designed as: ; in, represents the joint loss, represents a weight parameter used to balance the contribution of absolute noise loss and the feature consistency loss.

[0044] Through joint optimization, the diffusion model can simultaneously learn the mapping relationship between shadowed images and target shadow-free images, as well as the semantic extraction capability of the feature encoder.

[0045] The optimization of the joint loss function comprehensively considers differences at the pixel level, perceptual level, adversarial level, and semantic level, enabling the model to strike a balance between color consistency, detail authenticity, and semantic accuracy, resulting in better generation results. For example, the pixel-level loss ensures that the generated image has minimal pixel-level differences from the target image, the perceptual loss ensures that the generated image is perceptually more similar to the target image, the adversarial loss improves the realism of the generated image, and the CLIP semantic loss ensures that the generated image is semantically consistent with the target image. The combined effect of these loss functions enables the model to generate high-quality, shadow-free images.

[0046] Step S40: Acquire an image of a city building with shadows to be removed, input the image of the city building with shadows to be removed into the target diffusion model, and output an image of a city building without shadows.

[0047] Specifically, after the joint optimization process of step S30, a trained target diffusion model is obtained, and then the target diffusion model can be directly used to perform shadow removal processing, that is, an image of a city building with shadows to be removed is obtained, and the image of the city building with shadows to be removed is input into the target diffusion model, and then a shadow-free city building image can be directly output.

[0048] Furthermore, the shadow-free urban building images output by the target diffusion model can be used as data support for image processing tasks, wherein the fields of the image processing tasks include urban planning, autonomous driving, and remote sensing analysis.

[0049] The present invention aims at the key problem of color difference in shadow areas in the shadow removal task of urban building images, and proposes a systematic solution from data construction, feature design to optimization strategy. Different from previous studies that simply rely on traditional methods or other generative models, the present invention produces a high-quality data set containing a set of original images and shadow-free images, combines the semantic features extracted by the CLIP encoder to guide the generation of the diffusion model, and designs a joint loss system to improve color consistency and generation quality. The overall solution achieves multiple breakthroughs in the three levels of data, model and optimization, significantly improving the accuracy of shadow area recovery in complex lighting scenes, and demonstrating pioneering value in technical paths and practical application scenarios. The method of the present invention aims to accurately detect and remove shadows in images, and restore real scene information obscured by shadows, thereby providing high-quality data support for image processing tasks in the fields of urban planning, autonomous driving, remote sensing analysis, etc.

[0050] The shadow removal solution for urban building images proposed in this paper achieves significant technical effects through innovations in three levels: data construction, model design, and optimization strategy, and effectively solves the key problem of color difference in shadow areas in shadow removal tasks.

[0051] First, high-quality shadow removal is the most direct manifestation. This solution effectively removes shadows from urban building images, restoring the true color and texture details of shadowed areas, making the overall image clearer and more natural. This not only improves the visual quality of the image but also lays the foundation for subsequent image analysis and applications.

[0052] Secondly, color consistency is another important advantage of the present invention. Previous shadow removal methods often find it difficult to ensure the color consistency of the image after shadow removal, and are prone to problems with large color differences between shadow areas and non-shadow areas, resulting in image distortion. The present invention significantly improves the color consistency of the image after shadow removal through joint loss function optimization, especially the introduction of CLIP semantic loss. CLIP semantic loss can guide the model to take into account the overall semantic information of the image during the generation process, thereby better restoring the color of the shadow area and avoiding color deviation.

[0053] Furthermore, the generation method based on the diffusion model significantly improves the generated image quality. Traditional image processing methods are prone to blurring and distortion when dealing with shadow removal. However, the diffusion model used in this invention can generate high-quality, high-resolution shadow-free images, preserving image details and making the generated images more realistic and natural.

[0054] Generalization ability is an important indicator of a model's practicality. Through the training of high-quality datasets and the guidance of CLIP semantic information, the model's adaptability to different lighting conditions and different architectural styles is enhanced, thereby improving the model's generalization ability. This means that the model not only can handle shadow removal problems in specific scenarios, but also can be applied to more extensive practical scenarios.

[0055] Finally, semantic preservation is also an important advantage of the invention. In the shadow removal process, it is crucial to ensure that the semantic information of the image is not destroyed. The introduction of CLIP semantic features can ensure that the semantic information of the image is effectively preserved during the shadow removal process, avoiding semantic deviations. This makes the processed image not only more visually appealing, but also more accurate in terms of semantics.

[0056] In addition, in terms of dataset construction, data augmentation and simulated data can be performed on the dataset. Data augmentation: existing datasets can be augmented, such as rotation, scaling, translation, color transformation, etc., to increase the diversity of the dataset. Simulated data: 3D modeling software or game engines can be used to simulate the generation of urban building images containing different lighting conditions and architectural styles, and automatically generate corresponding shadow masks and shadow-free images.

[0057] In terms of model construction, diffusion model variants: other types of diffusion models can be tried, such as Score-Based Generative Models or Denoising Autoencoders, and corresponding adjustments can be made. CLIP alternatives: other pre-trained visual-linguistic models can be tried, such as ALIGN or FILIP, to extract semantic features of images. Semantic fusion methods: different ways of integrating semantic features into diffusion models can be tried, such as feature fusion through Transformer structure.

[0058] In terms of optimization strategies, loss function variants: other types of loss functions can be tried, such as style loss, edge loss, etc., to further improve the quality of generated images. Optimization algorithms: other optimization algorithms can be tried, such as AdamW, SGD, etc., and learning rate, batch size, etc. Hyperparameters can be adjusted. Adaptive weight adjustment: an adaptive weight adjustment strategy can be designed to dynamically adjust the weight coefficients of each loss function according to the feedback during training.

[0059] In terms of hardware acceleration: GPU can be used for model training and inference acceleration. Distributed training strategy can be adopted to speed up model training.

[0060] In the aspect of application scenario expansion, the present application is applied to other types of image shadow removal tasks, such as human face images, natural scenery images, etc. Or the present application is applied to the fields of image editing, image enhancement, etc.

[0061] Further, as shown in Figure 3 based on the above-mentioned shadow removal method for urban building images, the present application also correspondingly provides a shadow removal system for urban building images, wherein the shadow removal system for urban building images comprises: a data set making module 51 for making a synthetic data set for building scenes, the synthetic data set comprising a plurality of shadowed images and a plurality of shadow-free images corresponding to each shadowed image one by one; a feature map generation module 52 for inputting the shadowed image into a feature enhancement module, extracting a one-dimensional semantic vector of the shadowed image through a CLIP visual encoder of the feature enhancement module, and generating a feature map according to the one-dimensional semantic vector and the shadowed image through a feature enhancement network of the feature enhancement module; a joint loss optimization module 53 for calculating a feature consistency loss according to the feature map and a shadow-free feature map, calculating an absolute noise loss according to the feature map and a time step noise prediction, calculating a joint loss according to the absolute noise loss and the feature consistency loss, and optimizing parameters of the feature enhancement module and a diffusion model according to the joint loss to obtain a trained target diffusion model; an image shadow removal module 54 for acquiring a shadow-removal-targeted urban building image, inputting the shadow-removal-targeted urban building image into the target diffusion model, and outputting a shadow-free urban building image.

[0062] Further, as shown in Figure 4 based on the above-mentioned shadow removal method for urban building images and system, the present application also correspondingly provides a terminal, which comprises a processor 10, a memory 20 and a display 30. Figure 4 Only part of the components of the terminal are shown, but it should be understood that all the shown components are not required to be implemented, and more or less components can be alternatively implemented.

[0063] The memory 20 can be an internal storage unit of the terminal in some embodiments, such as a hard disk or a memory of the terminal. The memory 20 can also be an external storage device of the terminal in other embodiments, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. equipped on the terminal. Further, the memory 20 can include both the internal storage unit and the external storage device of the terminal. The memory 20 is used to store application software and various data installed on the terminal, such as program codes of the terminal, etc. The memory 20 can also be used to temporarily store data that has been output or will be output. In an embodiment, the memory 20 stores a shadow removal program for urban building images 40, which can be executed by the processor 10 to implement the shadow removal method for urban building images in the present application.

[0064] The processor 10 can be a Central Processing Unit (CPU), a microprocessor or other data processing chip in some embodiments, which is used to run program codes or process data stored in the memory 20, such as to execute the shadow removal method for urban building images, etc.

[0065] The display 30 can be an LED display, a liquid crystal display, a touch liquid crystal display, an OLED (Organic Light-Emitting Diode) touch, etc. in some embodiments. The display 30 is used to display information of the terminal and to display visualized user interfaces. The processor 10, the memory 20 and the display 30 of the terminal communicate with each other through a system bus.

[0066] In an embodiment, the steps of the shadow removal method for urban building images as described above are implemented when the processor 10 executes the shadow removal program for urban building images 40 in the memory 20.

[0067] The present application also provides a computer readable storage medium, wherein the computer readable storage medium stores a shadow removal program for urban building images, which, when executed by a processor, implements the steps of the shadow removal method for urban building images as described above.

[0068] ​In summary, the present application provides a shadow removal method, system, terminal and computer readable storage medium for urban building images, the method comprising: making a synthetic dataset for building scenes, the synthetic dataset comprising multiple shadow images and multiple shadow-free images corresponding to each shadow image; inputting the shadow image into a feature enhancement module, extracting a one-dimensional semantic vector of the shadow image through the CLIP visual encoder of the feature enhancement module, and generating a feature map according to the one-dimensional semantic vector and the shadow image through the feature enhancement network of the feature enhancement module; calculating a feature consistency loss according to the feature map and the shadow-free feature map, calculating an absolute noise loss according to the feature map and the time step noise prediction, calculating a joint loss according to the absolute noise loss and the feature consistency loss, and optimizing the parameters of the feature enhancement module and the diffusion model according to the joint loss to obtain a trained target diffusion model; obtaining a shadow removal urban building image to be removed, inputting the shadow removal urban building image to be removed into the target diffusion model, and outputting a shadow-free urban building image. The present application aims to accurately detect and remove shadows in urban building images, restore the real scene information obscured by shadows, significantly improve the accuracy of shadow area recovery under complex lighting scenes, and provide high-quality data support for image processing tasks.

[0069] It should be noted that in this document, the terms "comprising", "including", or any other variant thereof are intended to cover a non-exclusive inclusion, such that processes, methods, articles, or terminals including a series of elements not only include those elements, but also include other elements not explicitly listed, or inherent to such processes, methods, articles, or terminals. Without more limitations, the element defined by the statement "including a" does not exclude the presence of additional identical elements in the process, method, article, or terminal including the element.

[0070] Of course, those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing relevant hardware (such as a processor, a controller, etc.) through a computer program, and the program can be stored in a computer readable storage medium readable by a computer, and the program can include the processes of the above-mentioned method embodiments when executed. The computer readable storage medium can be a memory, a disk, an optical disk, etc.

[0071] It should be understood that the application is not limited to the above examples, and those skilled in the art can make improvements or changes according to the above description, and all such improvements and changes shall fall within the scope of protection of the claims of the present application.

Claims

1. A shadow removal method for urban building images, characterized by, The shadow removal method for urban building images comprises the following steps: A synthetic dataset for building scenes is made, which comprises a plurality of shadowed images and a plurality of shadow-free images corresponding to each shadowed image; The shadowed image is input into a feature enhancement module, a one-dimensional semantic vector of the shadowed image is extracted by a CLIP visual encoder of the feature enhancement module, and a feature map is generated by a feature enhancement network of the feature enhancement module according to the one-dimensional semantic vector and the shadowed image; A feature consistency loss is calculated according to the feature map and a shadow-free feature map, an absolute noise loss is calculated according to the feature map and a time step noise prediction, a joint loss is calculated according to the absolute noise loss and the feature consistency loss, and parameters of the feature enhancement module and a diffusion model are optimized according to the joint loss to obtain a trained target diffusion model; A shadow-removed urban building image is obtained, and the shadow-removed urban building image is input into the target diffusion model to output a shadow-free urban building image.

2. The shadow removal method for urban building images according to claim 1, wherein, The synthetic dataset for building scenes is made in detail as follows: Light source attribute parameters are adjusted by Unreal Engine, the light source attribute parameters comprising a solar elevation angle, a solar azimuth angle, a light intensity, a color temperature and a shadow softness; A building scene is constructed in Unreal Engine, the building scene comprising a building model, an environment setting and a camera setting; A plurality of shadowed images are collected based on light conditions of three different time periods and diversified building scenes, a shadow-free image corresponding to each shadowed image is generated by disabling a shadow projection function in the scene, all collected images are rendered at a preset pixel resolution during image collection, and all rendered images are saved in a preset format to obtain the synthetic dataset. 3.The urban building image-oriented shadow removal method according to claim 1, wherein, The feature consistency loss is calculated according to the feature map and a shadow-free feature map in detail as follows: ； wherein, denotes a feature consistency loss, denotes a target shadow-free image, is a shadowed image, denotes a one-dimensional semantic vector extracted by a CLIP visual encoder, denotes a shadow-free feature map obtained when the target shadow-free image is input, denotes a feature map obtained when the shadowed image is input. 4.The city building image oriented shadow removal method according to claim 3, wherein, The absolute noise loss is calculated according to the feature map and a time step noise prediction in detail as follows: ； wherein, denotes the absolute noise loss, denotes the expectation over the random variable, denotes the actual noise, denotes the intermediate state of the diffusion model at time step of the sequence. 5.The city building image oriented shadow removal method according to claim 4, wherein, The joint loss is calculated according to the absolute noise loss and the feature consistency loss in detail as follows: ； wherein, denotes the joint loss, denotes a weight parameter balancing the contribution of the absolute noise loss and the feature consistency loss. 6.The city building image oriented shadow removal method according to claim 5, wherein, The feature consistency loss is used to constrain the output of the feature enhancement network to ensure that the features of the shadowed image after encoding are consistent with the features of the target shadow-free image; The absolute noise loss is used to combine the output of the feature enhancement module with the time step noise prediction to ensure that the shadow-free result generated by the diffusion model meets the required quality at different time steps; The joint loss is used to enable the diffusion model to simultaneously learn the mapping relationship between the shadowed image and the target shadow-free image, and the semantic extraction capability of the feature enhancement network. 7.The city building image oriented shadow removal method according to claim 1, wherein, The shadow removal method for urban building images further comprises the following steps: The shadow-free urban building image output by the target diffusion model is used as data support for an image processing task, wherein the field of the image processing task comprises urban planning, automatic driving and remote sensing analysis.

8. A shadow removal system for urban building images, characterized by, The shadow removal system for urban building images comprises the following steps: A dataset making module is configured to make a synthetic dataset for a building scene, the synthetic dataset including a plurality of shadowed images and a plurality of shadowless images corresponding to each of the shadowed images; A feature map generating module is configured to input the shadowed images into a feature enhancement module, extract one-dimensional semantic vectors of the shadowed images through a CLIP visual encoder of the feature enhancement module, and generate feature maps according to the one-dimensional semantic vectors and the shadowed images through a feature enhancement network of the feature enhancement module; A joint loss optimization module is configured to calculate a feature consistency loss according to the feature maps and shadowless feature maps, calculate an absolute noise loss according to the feature maps and time step noise prediction, calculate a joint loss according to the absolute noise loss and the feature consistency loss, and optimize parameters of the feature enhancement module and the diffusion model according to the joint loss to obtain a trained target diffusion model; An image shadow removal module is configured to obtain a to-be-shadow-removed urban building image, input the to-be-shadow-removed urban building image into the target diffusion model, and output a shadowless urban building image.

9. A terminal, characterized by comprising: The terminal includes a memory, a processor, and a shadow removal program for urban building images stored on the memory and executable on the processor, and the shadow removal program for urban building images, when executed by the processor, implements the steps of the shadow removal method for urban building images according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a shadow removal program for urban building images, and the shadow removal program for urban building images, when executed by the processor, implements the steps of the shadow removal method for urban building images according to any one of claims 1-7.

Images