A method for extracting urban water bodies taking into account shadows from high-resolution remote sensing images

Abstract

The present invention discloses a method for extracting urban water bodies taking into account the shadows of high-resolution remote sensing images, including: a high-resolution urban water body remote sensing image data set; a shadow object feature correction model based on radiation transmission and geometric optics, introducing K-M equation, bright channel prior, guided filtering method and building density data to estimate and enhance the environmental scattering of the shadow area, and obtaining a shadow object feature enhancement model; adding an attention mechanism and a loss function to the network model for optimization to obtain an optimized network model; inputting the high-resolution urban water body remote sensing image data set into the shadow enhancement algorithm model and the optimized network model to obtain a water body remote sensing enhanced image, and complete the extraction of urban water body information. The present invention not only retains the overall hierarchy of the remote sensing image, but also retains the details of the objects in the shadow, and realizes the accurate extraction of urban water body information.

Description

Technical Field

[0001] The invention belongs to the technical field of water body image extraction, and in particular relates to a method for extracting urban water bodies taking into account shadows of high-resolution remote sensing images. Background Art

[0002] Water bodies are an indispensable part of urban ecosystems and urban natural landscapes, and are one of the main driving forces for sustainable urban development. Small water bodies are like "capillaries" in cities. They are large in number, widely distributed, and easily polluted, and have a significant impact on the urban ecological environment. By extracting small urban water bodies, we can understand their distribution and area changes, which is of great significance for monitoring and protecting urban water bodies. Currently, low- and medium-resolution multispectral remote sensing images are used to extract urban water body information. However, due to their limited spatial resolution, they cannot accurately identify spatial details such as the edges and textures of water targets, and cannot distinguish water targets from other features, making it difficult to extract small water bodies such as ponds and narrow rivers, which is not conducive to the subsequent analysis and monitoring work, and brings difficulties to the accurate delineation and quantitative analysis of water bodies.

[0003] With the continuous improvement of spatial and spectral resolution of remote sensing images, the ability to display details of ground objects has been significantly improved, providing a reliable data source for the identification and extraction of urban water bodies, and greatly improving the accuracy of urban water body extraction. However, although optical high-resolution remote sensing images provide rich and effective information for surface water extraction, such as spectral features and texture features, they still face some challenges: First, the spectral information of high-resolution remote sensing images is limited, and it is impossible to fully utilize additional bands (such as short-wave infrared bands) to extract water body information, resulting in the image features of water bodies and low-reflectivity objects such as shadows being very similar, which is easy to cause confusion and the phenomenon of same-spectrum but different-objects; second, the difference in sand content and aquatic plant density in water bodies will cause spectral changes in water bodies in remote sensing images, resulting in the phenomenon of same-object but different-spectrum; third, the extraction of small targets in remote sensing images is a difficult problem, and small water bodies are easily affected by the surrounding environment; fourth, in urban areas with dense buildings, the interference of building shadows on urban water extraction is the most serious. Tiny water bodies are distributed in every corner of the city, intertwined with the buildings in the city, forming a complex scene. The shadows of the buildings often block part of the water, resulting in unclear boundaries of the water body, or even being mistaken for part of the building. In addition, the spectral and texture characteristics of the shadows are very similar to those of the water body, and there is a serious phenomenon of the same spectrum and different objects. Especially for tiny water bodies with small areas and irregular shapes, this mixed ground object greatly affects the accuracy of extraction.

[0004] Regarding the shadow problem in remote sensing images, current research focuses on shadow detection (or extraction) and shadow compensation (or restoration), mainly feature-based and deep learning-based methods. However, both traditional feature-based methods and deep learning-based methods must accurately detect shadows in remote sensing images in order to accurately extract shadows. In the field of shadow detection, existing methods usually take a lot of time and effort. In addition to direct manual shadow extraction, even if deep learning is used to automatically detect shadows, it is necessary to manually create a shadow sample set for model training. Based on the radiation transfer theory, Wen et al. proposed a simple shadow restoration method (Reflectance Equality Relationship-Based Method, RERB), which accurately restores the reflectance and spectral shape information of shadow features by establishing a radiation transfer equation. However, it has the following two major defects: First, the extraction of shadow areas by manual visual interpretation is a pain point that is difficult to solve in this method. In practice, the manual visual interpretation process is cumbersome and time-consuming, and may be affected by subjective factors and introduce errors; second, the existence of shadows is a basic requirement for perceiving the hierarchical information of objects through remote sensing images. Completely removing shadows may lead to a lack of hierarchy in remote sensing images. Summary of the invention

[0005] The purpose of the present invention is to provide a method for extracting urban water bodies by taking into account the shadows of high-resolution remote sensing images, which can automatically estimate and enhance the ambient scattered radiance of the shadow area, weaken the shadow, and highlight the objects in the shadow. It not only retains the overall hierarchy of the remote sensing image, but also retains the details of the objects in the shadow, thereby realizing the accurate extraction of urban water body information.

[0006] In order to solve the above technical problems, the technical solution of the present invention is: a method for extracting urban water bodies taking into account shadows of high-resolution remote sensing images, comprising the following steps:

[0007] S1. Based on high-resolution remote sensing images, image segmentation and data processing are performed on the existing urban water body dataset to obtain a high-resolution urban water body remote sensing image dataset; data processing includes data cleaning, sample balancing, data normalization, data enhancement and dataset division;

[0008] S2. Construct a shadow feature correction model based on radiation transfer and geometric optics, introduce the KM equation, bright channel prior theory, guided filtering method and building density data to estimate and improve the environmental scattering of the shadow area, enhance the brightness characteristics of the shadow-covered objects, reduce the shadow interference of tall buildings and terrain, and obtain a shadow enhancement algorithm model to restore the features of the objects inside the shadow;

[0009] S3. Add attention mechanism and loss function to the network model for optimization to obtain an optimized network model to enhance the attention to tiny water bodies;

[0010] S4. The high-resolution urban water body remote sensing image dataset is successively input into the shadow enhancement algorithm model and the optimization network model to obtain the water body remote sensing enhanced image and complete the urban water body information extraction.

[0011] The working principle of the shadow feature correction model based on radiation transfer and geometric optics in S2 is:

[0012] Based on the radiation transfer principle and the Lambertian surface assumption, the surface reflectance of the illuminated area and the shadow area is inverted;

[0013] Assuming that the illuminated area and the shadow area contain objects with the same characteristics, the surface reflectance of the shadow area of ​​the object under Lambertian surface conditions is considered to be equal to the surface reflectance of the illuminated area;

[0014] To reduce the impact of errors, radiance is used to recover shadow information.

[0015] The process of introducing the KM equation in S2 to estimate the ambient scattering in the shadow area is:

[0016] The KM equation is used to describe the interaction of absorption and scattering processes in the material spectrum;

[0017] In the illuminated area, because the direct sunlight is strong, the influence of scattered radiation on the ground objects can be ignored. Therefore, the direct sunlight flux in the four-flux KM equations is not considered, and only the two-parameter two-stream approximation is used to solve the radiation transfer process;

[0018] The KM equation is solved to optimize the recovery of shadow information in the shadow feature correction model, and the local shadow density is used to estimate the intensity of the surrounding environment scattering.

[0019] The method of introducing building density data in S2 to estimate the local environmental scattering in the shadow area of ​​the shadow feature correction model is as follows:

[0020] The shadow area density is estimated by calculating the building density data using the vector data of the urban building outlines. At the same time, the negative correlation between the local environmental scattering intensity and the shadow density of the area is taken into account. The atmospheric path radiation is calculated by dark object subtraction.

[0021] The method of introducing the bright channel prior theory in S2 to estimate and improve the local environmental scattering in the shadow area of ​​the shadow feature correction model is as follows:

[0022] According to the bright channel prior theory, the brightness characteristics of shadow-shaded objects in the shadow object feature correction model are enhanced to reduce the shadow interference of tall buildings and terrain. The value at a certain pixel tends to the intensity of the atmospheric light, which is expressed as:

[0023]

[0024] in, is the atmospheric light value obtained according to the bright channel prior theory:

[0025]

[0026] Among them, c is a color channel, r, g, and b correspond to the red band, green band, and blue band channels respectively. is the brightness value of the c channel corresponding to the y position in image J.

[0027] It also includes the introduction of dark channel prior theory to estimate and enhance the local environmental scattering in the shadow area in the shadow feature correction model. Specifically, the dark channel prior theory and the bright channel prior theory are combined to form a hybrid channel model to estimate the atmospheric light value more comprehensively.

[0028] Double-window fitting is used to calculate the dark channel image and the bright channel image. The erosion and dilation operations in image morphological operations are used to weaken the influence of objects with too high or too low reflectivity and noise. A larger convolution template is used to weaken the interference of objects with too high or too low reflectivity, and a smaller convolution template is used to eliminate high-brightness noise. The calculation results of the two convolution templates are fused. The window sizes of the larger convolution template and the smaller convolution template are a×a and b×b respectively, and a>b.

[0029] The specific steps of using double window fitting to calculate the dark channel image and the bright channel image and fuse them to estimate the atmospheric light value are as follows:

[0030] By using convolution templates with window sizes of a×a and b×b, the shadow image is expanded to obtain a bright channel image, and the dark channel image is eroded to obtain a dark channel image;

[0031] Perform linear fitting on the two calculation result graphs obtained by the corrosion and expansion operation to obtain two corresponding fitting graphs after denoising;

[0032] The dark channel image and the bright channel image are obtained by calculating the two fitting images respectively;

[0033] The pixels in the calculated dark channel image and bright channel image are sorted according to their brightness, and the average value of the top 0.1% pixels with the highest brightness is taken as the atmospheric light value;

[0034] The final atmospheric light value is calculated using the mixed channel model, and the maximum global atmospheric light value is introduced to prevent the atmospheric light from exceeding the normal value.

[0035] The method of introducing guided filtering in S2 to enhance the local environmental scattering of the shadow area in the shadow feature correction model is as follows:

[0036] According to the local linear optimization of guided filtering, the image For the input image Smoothing the edges;

[0037] Defines a linear filter that takes one pixel The linear filter at is expressed as:

[0038]

[0039] in, represents the guide image, represents the input image, Indicates the filtering result, i.e. the output image. is the kernel function of the filter, i.e., the filter weight; and All are pixel coordinates; Represents pixel coordinates The input image of

[0040] The core assumption of guided filtering is that the guided image With the output image There is a linear relationship, so the guided filtering can be regarded as a local linear filtering, and its filtering process is expressed as:

[0041]

[0042] in, Pixel The center is set to A window of and Both represent windows The constant coefficient within ; is the pixel coordinate The guiding image of

[0043] Taking the derivative of both sides of equation (4) we get:

[0044]

[0045] Where a is the linear function coefficient.

[0046] To achieve edge-preserving smoothing, the parameters and It should change with the image position, so that As large as possible to ensure the gradient, let As small as possible to ensure that the input image and output image The difference is the smallest, in order to solve the linear coefficient and , establish the cost function under window minimization, and set the linear coefficient and The solution is transformed into the problem of solving the minimum value of the cost function, which is expressed as:

[0047]

[0048] In the formula, For Window The constant coefficients in are and The cost function when is a pixel in the window, is the regularization parameter used to avoid Too large;

[0049] The optimal solution is obtained according to the principle of least squares method, which is expressed as:

[0050]

[0051] in, and The guide images In the filter window The mean and variance of ; is the filter window The number of pixels in ; For the input image The average value in the filter window; where the regularization parameter and There is a linear relationship between The more hours The larger the value, the clearer the image edge information; otherwise, The bigger The smaller it is, the image edge features are partially lost;

[0052] Apply the linear model to all local windows of the entire image, and average the linear function values ​​under multiple windows, expressed as:

[0053]

[0054] in, Indicates the application of guided filtering to the pixel point in the process of transmittance refinement Transmittance value after refinement; Pixel The slope of the fitted linear function at ; Pixel The intercept of the fitted linear function at .

[0055] The way to optimize the network model is to add attention mechanism and loss function to the network model for optimization to obtain the optimized network model. The construction process of the optimized network model in S3 is:

[0056] S3.1. In the encoder part of the network model, the remote sensing image first undergoes two 3×3 convolution operations to obtain the first feature layer. After the first convolution, the size of the first feature layer remains unchanged and the number of channels increases to 64; the second convolution is only used for feature fusion, and the size and number of channels of the first feature layer remain unchanged;

[0057] S3.2, the first feature layer is scaled down to 128×128 by a 2×2 maximum pooling operation;

[0058] S3.3, repeat S3.1 and S3.2 5 times to obtain a feature layer with a scale of 16×16×1024;

[0059] S3.4, 2×2 upsampling is performed in the decoder part to restore the feature scale extracted in the encoding stage to its original size. During the upsampling process, the number of feature channels is halved;

[0060] S3.5. Add an attention mechanism CBAM module in the decoder part. In the skip connection stage, a dense connection strategy is used to increase the depth and width of the network and integrate image features at different levels.

[0061] A computer-readable storage medium is also provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps of any of the above methods are implemented.

[0062] Compared with the prior art, the present invention has the following beneficial effects:

[0063] The present invention utilizes the limited information of high-resolution remote sensing images to construct an improved radiation transfer model, and solves the model using the bright channel prior theory and the guided filtering algorithm, automatically estimating and enhancing the ambient scattered radiance in the shadow area, weakening the shadow and highlighting the objects in the shadow. It not only retains the overall hierarchy of the remote sensing image, but also retains the details of the objects in the shadow. Combined with deep learning, it realizes the accurate extraction of urban water body information, which is particularly suitable for the extraction of information on small and micro water bodies in cities. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 A schematic diagram of a flow chart of an embodiment of the present invention;

[0065] Figure 2 This is an example diagram of a high-resolution urban water remote sensing image dataset in an embodiment of the present invention;

[0066] Figure 3This is a schematic diagram of the main radiation transmission process in an embodiment of the present invention;

[0067] Figure 4 It is a schematic diagram of KM equation assumption in an embodiment of the present invention;

[0068] Figure 5 It is a double-window fitting diagram in an embodiment of the present invention;

[0069] FIG6 (a) is a rough transmittance diagram of atmospheric light in an embodiment of the present invention; FIG6 (b) is a refined transmittance diagram of atmospheric light in an embodiment of the present invention;

[0070] Figure 7 This is a schematic diagram of the AM-UNet++ network structure in an embodiment of the present invention;

[0071] Figure 8 are three groups of images involved in qualitative analysis in the embodiment of the present invention;

[0072] Fig. 9 The satellite remote sensing image shadow enhancement result image in the embodiment of the present invention;

[0073] Fig.10 An image obtained by extracting texture features of a satellite remote sensing image based on an LBP algorithm in an embodiment of the present invention;

[0074] Fig.11 The shadow enhancement result image of the natural lighting image in the embodiment of the present invention;

[0075] Fig.12 is a low-light image shadow enhancement result image in an embodiment of the present invention;

[0076] Fig.13 A comparison diagram of shadow enhancement details of a low-light image in an embodiment of the present invention;

[0077] Fig.14 Schematic diagram of the detection result of the Candy operator for a natural light image in an embodiment of the present invention;

[0078] Fig.15 Schematic diagram of the detection result of the Candy operator in a low-light image in an embodiment of the present invention;

[0079] Fig.16 A comparison diagram of edge detection details of low-light images in an embodiment of the present invention;

[0080] Fig.17 A schematic diagram comparing the extraction effects of different shadow enhancement algorithms in an embodiment of the present invention;

[0081] Fig.18 Schematic diagram comparing the extraction effects of different shadow enhancement algorithms in typical scenes according to an embodiment of the present invention. DETAILED DESCRIPTION

[0082] The technical solution of the present invention is: a method for extracting urban water bodies taking into account shadows of high-resolution remote sensing images, such as Figure 1 As shown, the following steps are included:

[0083] S1. Based on high-resolution remote sensing images, image segmentation, data cleaning, sample balancing, data normalization, data enhancement and data set division are performed on the existing urban water body data sets (XX, YY, ZZ, etc.) to obtain a high-resolution urban water body remote sensing image data set;

[0084] S2. Construct a shadow feature correction model based on radiation transfer and geometric optics, introduce the KM equation, bright channel prior and guided filtering method to estimate and improve the environmental scattering in the shadow area, enhance the brightness characteristics of the shadow-covered objects, and reduce the shadow interference of tall buildings and terrain. Finally, restore the features of the objects inside the shadow;

[0085] S3. Add attention mechanism and loss function to the UNet++ network model for optimization to obtain the AM-UNet++ network model to enhance the attention to tiny water bodies.

[0086] S4. The high-resolution urban water remote sensing image dataset is input into the shadow enhancement algorithm model and the AM-UNet++ network model successively to obtain the water remote sensing enhanced image and complete the extraction of urban small water body information.

[0087] In order to solve the problem of difficulty in extracting small water bodies in urban areas of high-resolution remote sensing images and serious interference from complex backgrounds, firstly, a shadow enhancement algorithm / model is implemented to analyze the formation of shadows and the process of shadow radiation transmission from a physical perspective, and to combine the geometric optical model with the radiation transmission model from a mathematical level to find an optimization solution. The KM equation is introduced to improve the traditional shadow radiation transmission model, and then the local environmental scattering in the shadow area is estimated and enhanced using bright channel priors, guided filtering and building density data to restore the internal features of the shadow, while avoiding manual intervention and improving work efficiency. Secondly, the shadow enhancement algorithm is used to enhance the shadows of the remote sensing dataset of urban small water bodies. At the same time, the attention mechanism and new loss function are added to the UNet++ network for optimization to enhance the focus on tiny water bodies. The obtained model is used to extract small water bodies in cities from high-resolution remote sensing images to achieve more accurate urban water body information extraction.

[0088] 1. Construction of a high-resolution remote sensing image water dataset that takes into account urban shadows

[0089] Among the currently available remote sensing image datasets, there are almost no datasets specifically targeting water bodies, especially urban water bodies, in the field of image semantic segmentation. In view of the shortcomings of the existing datasets, a remote sensing image water body dataset (HRRS-UWID) that takes into account urban shadows is constructed. Figure 2 As shown in the figure, based on high-resolution remote sensing images, the HRRS-UWID dataset is constructed through image segmentation, data cleaning, sample balancing, data normalization, data enhancement and dataset partitioning, providing a basis for the accurate extraction of urban water body information.

[0090] 2. Shadow feature correction model based on radiation transfer and geometric optics

[0091] In order to solve the problem of interference from building shadows in the process of urban water body extraction in remote sensing images, the formation of shadows and the process of shadow radiation transmission are analyzed from a physical perspective. The geometric optics model and the radiation transfer model are combined from a mathematical level, and the KM equation is introduced to improve the traditional shadow radiation transfer model. Subsequently, the local environmental scattering in the shadow area is estimated and enhanced by using bright channel prior, guided filtering and building density data to restore the characteristics of the objects inside the shadow.

[0092] In optical remote sensing imaging, the sun is considered as the only light source. According to the radiation transfer theory, it is assumed that the underlying surface of the shadow is located on a flat terrain and exhibits a Lambertian surface. Figure 3 When solar radiation passes through the atmosphere and reaches the surface, it is mainly composed of direct radiation and scattered radiation (scattered radiation includes atmospheric scattering and ambient light scattering , this method is collectively referred to as environmental scattering). In the process of sunlight transmitting downward, due to the obstruction of clouds or surface features, direct sunlight is rarely or cannot reach the shadow area. Therefore, the surface irradiance in the shadow area is Mainly determined by environmental scattering. The surface irradiance received by the illuminated area and the shadow area can be expressed by the following formula:

[0093] in, Direct solar radiation; is the ambient scattering, indicating and sum; It is the part of radiation transmitted downward to the surface due to atmospheric scattering; It is a combination of various radiations produced by the scattering of ambient light formed by various light sources through scattering, reflection, diffusion, etc.

[0094] Satellite sensors record the spectral radiation brightness of surface objects and present it in the form of digital (DN) on remote sensing images. Surface radiation must pass through the atmosphere to be received by satellite sensors. At the same time, part of the radiation that does not reach the surface is directly reflected by the atmosphere, namely the atmospheric path radiation ( ), is also detected by the sensor. Therefore, the sensor finally measures the total radiation brightness of the ground target The radiance of the ground target after atmospheric attenuation and the radiation of the atmosphere itself It consists of two parts, such as Figure 3 As shown,

[0095]

[0096] Based on the radiation transfer principle and the Lambertian surface assumption, the surface reflectivity of the non-shadow area and the shadow area is inverted by the following formula:

[0097]

[0098] in, is the surface reflectance of the non-shadow area, is the surface reflectance of the shadow area; and Represents the radiance of remote sensing images in non-shadow and shadow areas; is the atmospheric transmittance in the sensor's viewing direction.

[0099] Then, substituting equation (4) into equation (5), we get the new radiation transfer equation:

[0100]

[0101] Assume that the non-shadow area and the shadow area contain the same type of objects (denoted as ), it can be reasonably assumed that the surface reflectivity of the shadow area of ​​the feature under Lambertian surface conditions is equal to that of the non-shadow area, that is, Therefore, formula (6) is simplified to:

[0102]

[0103] in, and The same features The radiance in non-shadow and shadow areas.

[0104] Substituting formula (7) into formula (5), we can get:

[0105]

[0106] in, is the shadow restoration factor. In formula (8), the estimated parameter and The uncertainty will propagate through the equations, resulting in larger errors in the final answer.

[0107] To reduce the impact of errors, the radiance is used instead of the surface reflectance to restore the shadow information. Therefore, equation (8) is rewritten as:

[0108]

[0109] The left side of equation (10) expresses a hypothetical process: the surface irradiance received by the non-shadow area ( ) is applied to shadow areas (which do not exist in the actual scene) and is reflected by the surface covered by the shadow ( ), light passes through the atmosphere ( ), which is finally received by the satellite sensor, and the received radiation brightness is . The actual meaning is the radiation brightness received by the sensor from the corrected shadow area, that is, the radiation brightness restored from the shadow surface. At the same time, according to formula (3), the total radiation brightness received by the satellite sensor end is Atmospheric radiation should also be included ( )part:

[0110]

[0111] Shadows are important features of objects in remote sensing images. Their existence provides important information about the layers of objects on the surface. Completely eliminating shadows will lose these key spatial details, leading to difficulties in identification and analysis in applications such as image interpretation and object classification. This method focuses on the deep learning framework, and through appropriate shadow enhancement, maintains the model's effective learning ability for scene context, thereby improving the effect of water body extraction. In order to avoid the use of manual visual interpretation to extract the shadow range and calculate L shw Based on the shadow radiation transfer model constructed based on RERB, the KM equation is introduced to convert L shw The approximate solution is obtained by mapping to the KM model space, and a shadow spectrum correction model is constructed to enhance the local scattering in the shadow area, improve the characteristic expression of the ground objects under the shadow cover, and reduce the interference of shadows on multispectral data imaging.

[0112] The KM equation is a description of the diffuse reflection mechanism and is a dual-flux analytical model used to describe the combined effects of the absorption and scattering processes of a material spectrum. Figure 4 As shown, it can be approximated by a set of differential equations:

[0113]

[0114] in, is the optical thickness of the scattering layer; equation (12-1) describes the change of diffuse scattering flux, and is the diffuse scattering light flux, representing the upward and downward fluxes propagating vertically and in the plane medium, respectively. is the absorption coefficient; is the scattering coefficient; equation (12-2) describes the change of direct light flux, and is the direct solar flux, and represents the upward and downward reflected flux, is the absorption coefficient, and Scattering coefficients that account for diffusely scattered flux in the forward and backward directions for the incident beam.

[0115] Remote sensing images can be divided into non-shadow areas and shadow areas, which are mainly composed of direct radiation and environmental scattering. By adjusting the intensity of environmental scattering, the overall brightness difference between the shadow area and the non-shadow area can be reduced, the details of the objects in the shadow can be enhanced, and the shadow effect can be weakened. In the non-shadow area, because the direct sunlight is strong, the effect of scattered radiation on the ground objects can be ignored. Therefore, the direct sunlight flux in the four-flux KM equation group is not considered, and only the two-parameter two-stream approximation is used to solve the radiation transfer process. The formula is as follows:

[0116]

[0117] The concepts of Shadow Density (SD) and Shadow Area Volume Density (SAVD) are proposed for the one-dimensional shadow radiation transfer model. Shadow density represents the area of ​​all surface shadows per unit area, which refers to the degree of shadow coverage within a certain range in the remote sensing image, and is a measure of the degree of shadow covering the surface.

[0118] Solving based on KM equation The process is essentially to use the local shadow density to estimate the intensity of the surrounding environment scattering. From this, we can get formula (14):

[0119]

[0120] in, is the height, Indicates at height The shadow area density;

[0121] To derive equation (13), it is necessary to define the transition variable :

[0122]

[0123] get:

[0124]

[0125] Assuming transition variables , find the partial derivative and insert it into equation (16), then we have:

[0126]

[0127] Analyzing the above equation, equation (17) can be simplified to:

[0128]

[0129] Integrating equation (18), we get two boundary quantities: When the background reflectivity of the non-shadow area is ; When , it means the thickness is The surface reflectance of the shadow area is . At the same time, the shadow thickness relationship is obtained:

[0130]

[0131] when When , the thickness tends to infinity, and the background reflectivity of the surface in the non-shadow area tends to 0, that is, , the shaded area The reflectivity can be expressed as:

[0132]

[0133] You can get:

[0134] (twenty one)

[0135] in, is the KM formula surface reflectance of the infinite thickness shadow area, which is recombined into the well-known KM equation:

[0136]

[0137] At this time, for the limit case , that is, there is a completely non-absorbing feature, and equation (13) can be written as:

[0138]

[0139] Also define the transition variable , combined with formula (22), we can get:

[0140]

[0141] in, is the scattering coefficient. According to the exponential decay law, It is expressed as:

[0142]

[0143] in, represents the wavelength, represents the attenuation coefficient corresponding to the corresponding wavelength, and d represents the distance from the sensor to the ground object;

[0144] Substitute equation (25) into equation (24):

[0145]

[0146] Rearrange formula (26):

[0147]

[0148] in, Represents a remote sensing image where the ambient scattering intensity in the shadow area is close to the regional limit; is a remote sensing image containing shadows collected by satellite sensors. Equation (27) is essentially a restoration of the radiation brightness of the shadow area.

[0149] Using the vector data of urban building outlines (including floor heights), we can calculate the building density to estimate the shadow area density.

[0150]

[0151] Substituting equations (27) and (28) into equation (11), taking into account that the local environmental scattering intensity is negatively correlated with the shadow density of the area, and the atmospheric path radiation is not considered here The influence of will be as follows:

[0152]

[0153] Here It does not represent the total radiation brightness received by the satellite sensor, but also needs to take into account the energy attenuation during the transmission process and the influence of air light.

[0154] Substitute equation (29) into the commonly used image degradation model formula:

[0155]

[0156] in, represents the image before degradation, that is, the image after shadow correction; represents the degraded image, i.e., the input image; Represents reflectivity;

[0157] get,

[0158]

[0159] Finally, consider the atmospheric path radiation In order to understand the influence of , we substitute formula (30) into formula (3):

[0160]

[0161] in, It represents the remote sensing image after appropriately enhancing the environmental scattering in the shadow, that is, the result after the shadow weakening processing; This is the case where the ambient scattering intensity in the shadow area constrained by the building density is close to the maximum limit; is the observed remote sensing image with shadows; is the local building density coefficient; is the global atmospheric light, i.e., local environmental scattering; , represents the transmittance related to the scattering coefficient, that is, the degree of enhancement of local environmental scattering; is the atmospheric path radiation.

[0162] Atmospheric path radiation can be identified by dark-object subtraction, which is also the minimum brightness value on the remote sensing image, as shown in the following formula:

[0163]

[0164] The bright channel prior theory is introduced into the field of optical remote sensing. The bright channel is used to guide the model to focus on areas with weaker lighting. By enhancing environmental scattering, shadow objects are revealed without losing the shadows, thereby retaining the layered information of the image.

[0165] Observe the improved shadow radiation transfer model, that is, Equation (31), which only requires the atmospheric light A, transmittance t(x) and atmospheric path radiation Three parameters can be used to derive the image with weakened shadows .

[0166] Since remote sensing images of urban areas cover a wide range, the atmospheric light values ​​calculated by only a single dark channel map or a single bright channel map have certain deviations. In order to obtain the atmospheric light values ​​more accurately, this method proposes a model that comprehensively considers the dark channel and bright channel information for shadow images, namely the mixed channel model.

[0167] According to the bright channel prior theory, the value of the bright channel at a certain pixel in a natural image tends to the intensity of atmospheric light:

[0168]

[0169] in, is the atmospheric light value obtained by bright channel prior.

[0170]

[0171] In order to obtain more accurate atmospheric light values, this method considers combining dark channel priors and bright channel priors to estimate atmospheric light values ​​more comprehensively. The specific expression of the mixed channel model is as follows:

[0172]

[0173] in, are the atmospheric light values ​​solved a priori for the bright channel and the dark channel respectively; and To adjust the parameters, the constraints are and .

[0174] Different from the traditional atmospheric light value solution method, this method uses double window fitting to calculate the dark and bright channel map based on the mixed channel model, and uses the dilation operation in the image morphology operation to reduce the influence of low reflectivity objects (dark pixels) and noise. A larger convolution template is used to reduce the interference of low reflectivity objects, and a smaller convolution template is used to eliminate high brightness noise. In this method, convolution templates with window sizes of 15×15 and 3×3 are selected for dilation operation, and then the dilation results of the two are fused through a linear fitting function. The specific steps are as follows:

[0175] 1) Dilate the shadow image to calculate the dark channel map by using convolution masks with window sizes of 15×15 and 3×3 Bright channel image ;

[0176] 2) Two result graphs obtained by the expansion operation of formula (36) and Perform linear fitting to obtain the denoised fitting graph In this formula, Represents the fitting factor, which is selected through experiments ;

[0177]

[0178] Among them, I15 represents the image after the convolution template with a window size of 15x15 is used for calculation, and I3 represents the image after the convolution template with a window size of 3x3 is used for calculation;

[0179] 3) For the fitting graphs Calculate dark channel map Bright channel image ;

[0180] 4) Sort the pixels of the two channel images separately and map the brightest 0.1% pixels to the fitting image The corresponding position of , and then select the average value of these pixels as the atmospheric light value. and ;

[0181] 5) Finally, the final atmospheric light value is calculated using the mixed channel model through equation (36): , and add a parameter, the maximum global atmospheric light value , to prevent the atmospheric light from exceeding the normal value, see equation (37). After a large number of comparative experiments, the parameters in equation (35) are set to and The initial transmittance map works best when .

[0182]

[0183] like Figure 5 As shown in the figure, (a) is the original remote sensing image, (b) is the fitting image generated after the original image is expanded, (c) is the dark channel image of the original image, (d) is the bright channel image of the original image, (e) and (f) are the dark channel image and bright channel image of the expansion fitting.

[0184] The transmittance map calculated using global atmospheric light is a rough transmittance map. There will be errors when it is used to enhance the ambient scattered radiance. Guided filtering is used to further refine the preliminary transmittance to obtain a more reasonable enhancement effect.

[0185] Guided filtering uses local linear optimization to guide the image For the input image First, define a general linear filter: represents the guide image, represents the input image, Indicates the filtering result, i.e. the output image. The filter at can be expressed as:

[0186]

[0187] in, is the kernel function of the filter, i.e., the filter weight; and are pixel coordinates; Represents pixel coordinates The input image.

[0188] Equation (38) defines the guided filter and kernel. The core assumption of the guided filter algorithm is that the guided image With the output image There is a linear relationship, so the guided filtering can be regarded as a local linear filtering, and its filtering process is expressed as follows:

[0189]

[0190] in, Pixel The center is set to A window of and Display Window The constant coefficient within can ensure that the relative nature of the image gradient remains unchanged; is the pixel coordinate The guiding image of ; taking the derivative of both sides of equation (39) yields:

[0191]

[0192] Where a is the linear function coefficient.

[0193] Equation (40) shows that the output image gradient is proportional to the guidance map gradient, that is, the guidance image and the output image have the same edges, and the guidance image map makes the input image smooth and maintains the edges a priori.

[0194] To achieve edge-preserving smoothing, the parameters and It should change with the image position, so that As large as possible to ensure the gradient, let As small as possible to ensure that the input image and output image To solve the linear coefficient, a cost function is established under window minimization. and Solving the linear coefficient is transformed into solving the problem of minimizing the cost function:

[0195]

[0196] In the formula, For Window The constant coefficients in are and The cost function when is a pixel in the window, is the regularization parameter used to avoid Too large. According to the principle of least squares, the optimal solution can be obtained:

[0197]

[0198] in, and The guide images In the filter window The mean and variance of ; is the filter window The number of pixels in ; For the input image The average value in the filter window. Regularization parameter and There is a linear relationship between The more hours The larger the value, the clearer the image edge information; otherwise, The bigger The smaller it is, the image edge features are lost. and When it is equal to the above expression, the linear model maintains the edge features of the image while smoothing and denoising.

[0199] Finally, the linear model needs to be applied to all local windows of the entire image as shown in Figure 6 (a) and Figure 6 (b). However, when calculating the linear coefficient of each window, the same pixel is included in multiple windows. A simple strategy is to average the linear function values ​​under the action of multiple windows:

[0200]

[0201] In the formula, Indicates the application of guided filtering to the pixel point in the process of transmittance refinement Transmittance value after refinement; Pixel The slope of the fitted linear function at ; Pixel The intercept of the fitted linear function at .

[0202] 3. AM-UNet++ network model combining convolutional attention mechanism and hybrid loss function

[0203] In order to solve the problems of difficulty in extracting tiny water bodies in urban areas and severe interference from complex backgrounds, the shadow enhancement algorithm proposed above is combined to correct the shadows of urban water samples. Then, the improved UNet++ network is used for training, and the obtained model is used for the task of extracting urban water bodies from high-resolution remote sensing images.

[0204] Based on the Unet++ network with dense skip connection encoder-decoder architecture and CBAM attention mechanism, this method designs an optimized network model for high-resolution remote sensing image segmentation for urban water extraction tasks. Figure 7 As shown in the figure, in the encoder part, the remote sensing image is first subjected to two 3×3 convolution operations to obtain the first feature layer. The size of the first convolution feature layer remains unchanged, and the number of channels increases to 64; the second convolution is only used for feature fusion, and the size and number of channels remain unchanged; then the first feature layer is scaled down to 128×128 by a 2×2 maximum pooling operation. Repeat the above operation 5 times to obtain a feature layer with a scale of 16×16×1024. Since the AM-Unet++ network is downsampled by 2×2 maximum pooling, the scale of each downsampled feature layer is doubled, so 2×2 upsampling is required in the decoder stage to restore the feature scale extracted in the encoding stage to its original size. In the upsampling process, the number of feature channels is halved. In order to obtain more useful global information, the design adds a CBAM module in the decoder stage of the network. In the skip connection stage, a dense connection strategy is used to increase the depth and width of the network and integrate image features at different levels. The attention mechanism can obtain water characteristics of different channels and focus on the spatial position of the water body, thereby increasing attention to complex water body information and suppressing irrelevant background. Therefore, the CBAM module is added to the skip connection stage of the network.

[0205] Key points in the implementation plan

[0206] 1. Construct a shadow feature correction model based on radiation transfer and geometric optics

[0207] Aiming at the problem of building shadow interference in the task of urban water extraction, this method starts with restoring the feature information of the objects blocked by shadows, and constructs a correction model of shadow object features based on radiation transfer and geometric optics. The KM equation is introduced, and the improved shadow radiation transfer model is constructed through detailed mathematical derivation. The model is solved by using the bright channel prior theory and the guided filtering algorithm, and the local environmental scattering in the shadow area is accurately estimated and appropriately enhanced, so as to achieve the purpose of restoring the features of the objects inside the shadow. The experimental results show that the algorithm proposed in this method shows the best performance in color balance, texture enhancement and restoration, and robustness. It can enhance the radiation brightness of the shadow area, restore the details of the objects inside the shadow, and keep the natural color tone of the non-shadow area, avoiding oversaturation of the bright objects. Compared with traditional shadow detection, this method not only helps to reduce shadow interference and restore the lost details of the objects, but also does not need to use additional means to extract the shadow range, which greatly enhances the scope of application. In addition, it can also make the remote sensing image maintain a certain sense of hierarchy, which has certain value in the fields of building height estimation.

[0208] 2. AM-UNet++ network model combining convolutional attention mechanism and hybrid loss function

[0209] Aiming at the problem that high-resolution remote sensing images contain abundant water features, urban micro-water bodies are difficult to extract, and complex background interference is serious, this method proposes an AM-UNet++ network model that combines a convolutional attention mechanism and a hybrid loss function. This model is based on the UNet++ network framework, and adds CBAM modules based on channel attention and spatial attention in the jump connection and decoder stages. The addition of the attention mechanism enables the network to connect pixels in different channels and different spatial positions during training, and makes the model pay more attention to the key features of complex and tiny water bodies. At the same time, the task of urban water body extraction is analyzed. In order to solve the problem of unbalanced urban water body samples, this method constructs an F-FT Loss loss function to optimize the original loss function. Compared with the original UNet++ network, the optimized network proposed by this method has an F1 value of 3% in the test area covering large water bodies, and an F1 value of 7% in areas without large water bodies. The experimental results show that the network model can better capture water body targets, accurately identify various types of water bodies, and the segmentation results are more refined.

[0210] 3. Constructing a water dataset that takes urban shadows into account

[0211] In the context that the existing high-resolution remote sensing image datasets cannot meet the problem of urban water extraction, the GF-2 remote sensing images are preprocessed, and the characteristic differences within urban water bodies and between urban water bodies and other landforms are deeply analyzed. According to the proposed annotation strategy, a high-resolution remote sensing image water dataset (HRRS-UWID) that takes into account urban shadows is constructed. HRRS-UWID undergoes strict data cleaning, deletes images containing invalid values, and screens samples with high similarity to maintain the balance of positive and negative samples.

[0212] Effect after implementation

[0213] 1. Evaluation of the effect of shadow enhancement method

[0214] A comprehensive evaluation of the shadow enhancement algorithm is conducted from both subjective and objective perspectives. The evaluation images are divided into three groups: satellite remote sensing images, natural light images, and low-light images to verify the effectiveness and robustness of the proposed method. Qualitative analysis is mainly evaluated from four aspects: (1) whether the color is natural; (2) whether the texture is clear; (3) whether it has cross-domain applicability; (4) introducing texture features, through LBP texture extraction and Candy operator edge extraction, to detect whether each algorithm can reduce the impact of shadows while enhancing the characteristics of the internal objects in the shadow; quantitative analysis sets full reference evaluation indicators and no reference evaluation indicators, and introduces the gray level co-occurrence matrix to conduct an objective and in-depth comparison of the algorithms, such as Figure 8 As shown, Figures (1) to (3) are satellite remote sensing images, Figures (4) and (5) are natural light images, and Figures (6) and (7) are images under low light conditions.

[0215] 2. Qualitative analysis

[0216] Using satellite remote sensing images, natural light images, and low-light images, three different types of shadow image data from different fields are used to compare and analyze this method with histogram-based HE and CLAHE, and Retinex-based MSR, MSRCP, and AutoMSRCR.

[0217] like Fig. 9 As shown in the figure, in terms of image color, the CLAHE, MSR and MSRCP algorithms show obvious color cast. CLAHE has darker colors, MSR has warmer colors, and MSRCP tends to be colder. These three algorithms cannot effectively correct the color difference of the image, which is a serious defect for remote sensing images. HE, AutoMSRCR and the algorithm proposed in this paper perform better in terms of color and can basically maintain the original color of the objects. However, HE has the problem of oversaturation on high-brightness objects, such as the white buildings in Figures (1) and (3); AutoMSRCR has the problem of inconsistent tones between different images, such as Figures (1) and (2) have normal tones, but Figure (3) has a gray-green river, which causes the tones of AutoMSRCR to be green. In contrast, the algorithm proposed in this paper maintains a consistent tones in the three images, which is close to the original color of the objects, and at the same time achieves a good balance in enhancing dark pixels and keeping high-brightness objects from oversaturation. In terms of local details, the CLAHE, MSR and MSRCP algorithms are difficult to accurately identify the attributes of objects in the shadows due to the color difference. In comparison, HE, AutoMSRCR and the proposed algorithm can clearly identify the white vehicle and blue house in the local shadow of Figure (1), as well as the pond hidden in the shadow of Figure (2). However, for the more complex overlapping shadows in Figure (3), the processing effect of HE and AutoMSRCR algorithms is obviously not as good as that of the text algorithm. HE and AutoMSRCR cannot identify the features and attributes of the objects in the shadow on the left side of Figure (3), but the proposed algorithm can clearly identify some white foundations and a large amount of yellow bare soil. Through careful comparison, the proposed algorithm performs better in improving the clarity of the features of the objects in the shadow, making the shadow lighter and achieving the effect of transparent shadows and clear objects.

[0218] like Fig.10As shown in the figure, in terms of texture, from a global perspective, although the texture features of the three algorithms HE, CLAHE, and MSR are all improved compared to the original image, the texture inside the shadow is still sparser than that in the unshadowed area. MSRCP and AutoMSRCR algorithms are slightly better than the first three in terms of enriching texture information, and the texture inside the shadow is denser, but in comparison, the proposed algorithm shows richer texture information, highlighting the texture details while retaining the boundary of the object. From a local analysis, the local texture information carried by Figure (1) and Figure (2) is gradually enriched (from left to right in the order of display), while Figure (3) more intuitively shows the advantages of the proposed algorithm. In the red box, the texture inside the shadow of the proposed algorithm is almost completely enriched and clearly visible. By comparison, the proposed algorithm performs well in improving the features of objects in the shadow, can significantly enhance the texture information of the image, and make the texture level of the object clearer, which is helpful for the subsequent deep neural network model to learn different features of the object.

[0219] refer to Figure 9-12 ,Comparison shows that AutoMSRCR and the proposed algorithm perform best in natural light and low light image comparison experiments.,Both achieve remarkable results in shadow enhancement and texture rendering, as shown in Figure (1) and Figure (6),,which can clearly show the grass and manhole cover under the shadow, and even show the details of each grass and stone brick on the grass.

[0220] by Fig.13 As an example, a detailed comparative analysis is conducted on the HE, AutoMSRCR and the proposed algorithm with better effects. All three algorithms can enhance the shadow area, but HE and AutoMSRCR are lacking in image color balance, the sky is grayish white, especially the overall image of the AutoMSRCR algorithm tends to be cold, and the visual effect is poor. In contrast, the proposed algorithm can maintain color balance, the sky is blue as it should be, the lawn is yellow-green, the overall color tone is coordinated, and it is more visually comfortable.

[0221] analyze Fig.14 and Fig.15The comparative test results show that due to the overexposure of the MSN algorithm, the edge details of most objects in the image are lost, making it difficult to be detected by the Canny operator, and the image information is even inferior to the original image. The HE algorithm does not perform well in the shadow area. Although the detected contour is almost the same as the original image, it does not significantly improve the shadow part. The CLAHE algorithm detects a richer object contour in Figure (4), but it does not mean that CLAHE has an excellent effect on shadow enhancement. In Figure (5), the CLAHE algorithm still fails to effectively restore the texture information of the manhole cover in the shadow, indicating that it has certain limitations in processing shadows. MSRCR, AutoMSRCR and the algorithm in this paper show relatively excellent performance in this group of comparative tests, and can effectively restore the object feature information in the shadow area. However, MSRCR will cause information loss in non-shadow areas, such as the grass in the upper right corner of Figure (4) and the stone brick in the lower right corner of Figure (5), where the edge detected contours are missing compared to the original image.

[0222] pass Fig.16 By comparing CLAHE, AutoMSRCR and the proposed algorithm, it can be clearly observed that AutoMSRCR and the proposed algorithm detect richer information than CLAHE, and perform better in recovering information in shadow areas, such as the red bricks of the house.

[0223] The above experiments verify the effectiveness of the algorithm in shadow enhancement of natural light and low light images in the field of computer vision. The algorithm proposed in this paper performs better in all aspects, followed by AutoMSRCR.

[0224] The results of comprehensive comparison show that the proposed algorithm performs best in all aspects, followed by AutoMSRCR and CLAHE, while HE and MSRCP perform worse than the former, and MSR is poor in all aspects.

[0225] 3. Indicator analysis

[0226] Since it is impossible to obtain true shadow-free images for remote sensing images and low-light images, only no-reference indicators can be used for comparison. In contrast, natural light images use the popular ISTD shadow dataset, which contains images with and without shadows. Therefore, natural light images introduce full-reference indicators as the standard for evaluating the quality of the algorithm. The specific comparative tests and the indicators involved are shown in Table 1, where ○ indicates that the corresponding evaluation indicator is used in this group of scenes, and × indicates that the corresponding indicator is not used.

[0227]

[0228] The reference-free index does not require the original shadow-free image, and is applicable to situations where it is difficult to obtain the original image in practical applications. This section uses the classic reference-free indexes average gradient (AG) and spatial frequency (SF). At the same time, in order to explore the algorithm's restoration effect on the characteristics of objects in the shadow, the gray-level co-occurrence matrix (GLCM) is introduced to deeply analyze the texture and structural characteristics of the image, including energy, entropy, contrast, and homogeneity. In the experiment, the average value of each index of each algorithm on all results is calculated. In the table, ↑ indicates that the image quality is better as the value of the index increases, and ↓ indicates that the smaller the value, the better the effect of the index and the better the image quality.

[0229]

[0230]

[0231]

[0232] Tables 2, 3 and 4 are the comparison of no-reference evaluation index results for remote sensing images, low-light images and natural light images, and the quantitative analysis of no-reference indexes for the three data sets. It can be clearly seen that the algorithm proposed in this paper achieves the best results in most indicators of each data set. The images processed by the algorithm in this paper show richer details, and the texture is clearer, more obvious and more delicate, which shows that the algorithm in this paper can effectively restore the feature information of the ground objects or objects blocked by shadows, and at the same time, it can also maintain the visible information of the original image in the non-shadow area, showing that it has an absolute advantage in comprehensive performance. In the remote sensing image dataset, the performance of the algorithm in this paper is better, which is mainly due to the fact that the algorithm is designed for remote sensing images under complex lighting conditions. Compared with other simpler images, it has high adaptability and strong robustness when processing other simpler images. Through comparative experiments, it can be found that the algorithm in this paper can not only effectively reduce the impact of shadows on the image, but also restore the ground features in the shadow area more clearly, which further verifies the correctness of the above qualitative analysis.

[0233]

[0234] Table 5 is a comparison of the full reference evaluation index results of natural light images. Analyzing Table 5, the performance of the proposed algorithm is the best in all indicators. Compared with other algorithms, the effective information restored by the image processed by the proposed algorithm increases, while avoiding the expansion and interference of noise, which is consistent with the subjective evaluation results mentioned above. The NRMSE and PSNR indicators reflect that the image processed by the proposed algorithm is very similar to the original image in terms of pixel-level features, indicating that the image is almost undistorted, but the perceptual characteristics of the human visual system are ignored. The four indicators of SSIM, MS-SSIM, FSIM and FSIMc make up for this defect, and verify from the perspective of human vision that the shadow reduction results of the proposed algorithm are very similar to the original shadow-free image, which means that the shadow reduction effect is excellent.

[0235] 4. Evaluation of the overall water extraction method

[0236] Using the same original remote sensing image data, CLAHE, AutoMSRSR and the above shadow enhancement algorithms were used for water extraction comparison experiments. The specific experimental process is as follows: First, the original data set is preprocessed by different shadow algorithms to obtain a shadow weakening data set, and then the processed data set is trained with the original U-Net++ network. The experimental area and results are shown in Figure 2. Fig.17 As shown in the figure, (a) is a remote sensing image, (b) is the produced true value label, (c) is the network model trained with unprocessed data, (d) is the network model trained with the AutoMSRCR algorithm, (e) is the network model trained with the CLACH algorithm, and (f) is the network model trained with the algorithm in this paper.

[0237] Fig.17The effects of three shadow enhancement algorithms and original data on the deep learning water extraction task are shown. Groups 1 and 2 in the figure show water bodies with smaller areas, and groups 3, 4, and 5 show shadows in building areas. It can be clearly observed that the model trained on the data (c) that has not been processed for shadow fading confuses water bodies and shadows, and some shadows are mistakenly identified as water bodies. In addition, it is difficult to accurately identify water bodies with a small area and surrounded by many trees. The data processed by the CLACH algorithm is similar to the data processed on the original remote sensing image. The trained model has the same problems as the unprocessed data, but it is better in water body identification and shadow resistance. The model trained on the data set processed by the algorithm proposed in this paper has better performance. The main reason is that the difference between water bodies and shadows in color and texture features is increased, so that the deep learning model can learn more abundant water body features without being disturbed by shadows. Therefore, it shows more accurate target positioning in the prediction results, and has a certain resistance to dark pixel objects such as shadows. The next best performer is AutoMSRCR, which is consistent with the analysis in Chapter 3. However, the color tone of the data processed by the AutoMSRCR algorithm has some deviations, which affects the accurate identification of water bodies.

[0238] like Fig.18 As shown, Fig.18 for Fig.17 The sixth analysis point ( Fig.17 The image shows a landscape pool in a residential area, which is a typical scene where shadows block water (yellow solid frame). After being processed by the algorithm, the trained models can accurately identify the water inside the shadow, but the algorithm proposed in this paper has the best effect, and the recognized water body has a complete shape and clear boundaries. The red and purple circles in the figure are shadow areas. Except for (f), the other three groups of models are more or less disturbed by shadows.

[0239] The test area of ​​the comparative test is mainly a large area of ​​densely populated building shadows. Except for a few small water bodies in the urban area, the Yangtze River in the upper left corner is a concentrated large water body. For large water bodies with basically no shadow interference around them, due to the obvious characteristics of the water bodies, the accuracy of model recognition is very high regardless of whether the data set has been pre-processed by the algorithm; at the same time, due to the huge proportion of large water bodies, the extraction effect of difficult samples (small water bodies) is difficult to be intuitively reflected in the indicators. In order to reflect the fairness of the experiment and the objectivity of the evaluation, the large water body in the upper left corner of the experimental area was removed, and the results are as follows:

[0240]

[0241] Table 6 shows the evaluation index results of different shadow enhancement algorithms (excluding large water bodies). The experimental results show that the model trained by the shadow enhancement algorithm proposed in this method shows excellent performance in all indicators. It can not only accurately identify water bodies to a great extent, but also keep the shape intact. For the extraction of tiny water bodies, the accuracy and recall of this method algorithm are improved by about 57.4% and 32% respectively compared with the unprocessed method, and the F1 value of the balance index is improved by 52%. The substantial improvement in accuracy shows that the data set processed by this method can greatly weaken the influence of shadows, while maintaining the feature details of non-shadow areas and restoring the feature information lost in shadow areas, providing an effective solution for the extraction of urban water bodies from high-resolution remote sensing images.

[0242] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for extracting urban water bodies taking into account shadows from high-resolution remote sensing images, characterized in that: The following steps are involved: S1. Based on high-resolution remote sensing images, image segmentation and data processing are performed on the existing urban water body dataset to obtain a high-resolution urban water body remote sensing image dataset; S2. Construct a shadow feature correction model based on radiation transfer and geometric optics, introduce the KM equation, bright channel prior theory, guided filtering method and building density data to estimate and improve the environmental scattering of the shadow area, enhance the brightness characteristics of the shadow-covered objects, reduce the shadow interference of tall buildings and terrain, and obtain a shadow enhancement algorithm model to restore the features of the objects inside the shadow; S3. Add attention mechanism and loss function to the network model for optimization to obtain an optimized network model to enhance the attention to tiny water bodies; S4, inputting the high-resolution urban water remote sensing image data set into the shadow enhancement algorithm model and the optimization network model in turn, obtaining the water remote sensing enhanced image, and completing the urban water information extraction; The process of introducing the KM equation in S2 to estimate the ambient scattering in the shadow area is: The KM equation is used to describe the interaction of absorption and scattering processes in the material spectrum; In the illuminated area, because the direct sunlight is strong, the influence of scattered radiation on the ground objects can be ignored. Therefore, the direct sunlight flux in the four-flux KM equations is not considered, and only the two-parameter two-stream approximation is used to solve the radiation transfer process; Based on solving the KM equation to optimize the restoration of shadow information in the shadow feature correction model, the local shadow density is used to estimate the intensity of the surrounding environment scattering. The method of introducing building density data in S2 to estimate the local environmental scattering in the shadow area of ​​the shadow feature correction model is as follows: The shadow area density is estimated by calculating the building density data using the vector data of the urban building outlines. At the same time, the negative correlation between the local environmental scattering intensity and the shadow density of the area is taken into account. The atmospheric path radiation is calculated by dark object subtraction.

2. The method for extracting urban water bodies taking into account shadows of high-resolution remote sensing images according to claim 1, characterized in that: The working principle of the shadow feature correction model based on radiation transfer and geometric optics in S2 is: Based on the radiation transfer principle and the Lambertian surface assumption, the surface reflectance of the illuminated area and the shadow area is inverted; Assuming that the illuminated area and the shadow area contain objects with the same characteristics, the surface reflectance of the shadow area of ​​the object under Lambertian surface conditions is considered to be equal to the surface reflectance of the illuminated area; To reduce the impact of errors, radiance is used to recover shadow information.

3. A method for extracting urban water bodies taking into account shadows of high-resolution remote sensing images according to claim 1 or 2, characterized in that: The method of introducing the bright channel prior theory in S2 to estimate and improve the local environmental scattering in the shadow area of ​​the shadow feature correction model is as follows: According to the bright channel prior theory, the brightness characteristics of shadow-shaded objects in the shadow object feature correction model are enhanced to reduce the shadow interference of tall buildings and terrain. brig h t The value of (x) at a certain pixel tends to the intensity of atmospheric light, which is expressed as: J brig h t (x)→A brig h t (x)(1) Among them, A brig h t (x) is the atmospheric light value obtained according to the bright channel prior theory: Among them, c is a color channel; r, g, and b correspond to the red band, green band, and blue band channels respectively. c (y) is the brightness value of the c channel corresponding to the y position in image J.

4. The method for extracting urban water bodies taking into account shadows of high-resolution remote sensing images according to claim 3, characterized in that: It also includes the introduction of dark channel prior theory to estimate and enhance the local environmental scattering in the shadow area in the shadow feature correction model. Specifically, the dark channel prior theory and the bright channel prior theory are combined to form a hybrid channel model to estimate the atmospheric light value more comprehensively. Double-window fitting is used to calculate the dark channel image and the bright channel image. The erosion and dilation operations in image morphological operations are used to weaken the influence of objects with too high or too low reflectivity and noise. The first convolution template is used to weaken the interference of objects with too high or too low reflectivity, and the second convolution template is used to eliminate high-brightness noise. The calculation results of the two convolution templates are fused; wherein the window sizes of the first convolution template and the second convolution template are a×a and b×b respectively, and a>b.

5. The method for extracting urban water bodies taking into account shadows of high-resolution remote sensing images according to claim 4, characterized in that: The specific steps of using double window fitting to calculate the dark channel image and the bright channel image and fuse them to estimate the atmospheric light value are as follows: By using convolution templates with window sizes of a×a and b×b, the shadow image is expanded to obtain a bright channel image, and the dark channel image is eroded to obtain a dark channel image; Perform linear fitting on the two calculation result graphs obtained by the corrosion and expansion operation to obtain two corresponding fitting graphs after denoising; The dark channel image and the bright channel image are obtained by calculating the two fitting images respectively; The pixels in the calculated dark channel image and bright channel image are sorted according to their brightness, and the average value of the top 0.1% pixels with the highest brightness is taken as the atmospheric light value; The final atmospheric light value is calculated using the mixed channel model, and the maximum global atmospheric light value is introduced to prevent the atmospheric light from exceeding the normal value.

6. A method for extracting urban water bodies taking into account shadows of high-resolution remote sensing images according to claim 1 or 2, characterized in that: The method of introducing guided filtering in S2 to enhance the local environmental scattering of the shadow area in the shadow feature correction model is as follows: The edges of the input image p are smoothed by the guided image I according to the local linear optimization of the guided filter; Define a linear filter. The linear filter at a pixel i is expressed as: q i =∑ j W ij (I)p j (3) Among them, I represents the guide image, p represents the input image, q represents the filtering result, and W ij is the kernel function of the filter, i.e., the filter weight; i and j are both pixel coordinates; p j represents the input image at pixel coordinate j; The core assumption of guided filtering is that the guided image I is linearly related to the output image q. Therefore, guided filtering can be regarded as a local linear filtering, and its filtering process is expressed as: Among them, ω k is a window centered at pixel k with a radius set to r; a k and b k Both represent window ω k The constant coefficient within I i is the guidance image of pixel coordinate i; Taking the derivative of both sides of equation (4) we get: Where a is the linear function coefficient; To achieve edge-preserving smoothing, parameter a k and b k It should change with the image position, and let a k As large as possible to ensure the gradient, let b k As small as possible to ensure that the difference between the input image p and the output image q is minimal. To solve the linear coefficient a k and b k , establish the cost function under window minimization, and set the linear coefficient a k and b k The solution is transformed into the problem of solving the minimum value of the cost function, which is expressed as: In the formula, E(a k ,b k ) is the window ω k The constant coefficients in are a k and b k The cost function when p i is a pixel in the window, ε is the regularization parameter used to avoid a k Too large; The optimal solution is obtained according to the principle of least squares method, which is expressed as: Among them, μ k and are respectively the guide image I in the filter window ω k The mean and variance in |ω| is the filter window ω k The number of pixels in ; is the average value of the input image p in the filter window; where the regularization parameter ε is related to a k There is a linear relationship between them. The smaller ε is, the k The larger the value, the clearer the image edge information; on the contrary, the .... k The smaller it is, the image edge features are partially lost; Apply the linear model to all local windows of the entire image, and average the linear function values ​​under multiple windows, expressed as: Among them, q i It represents the transmittance value of pixel i after the transmittance is refined by applying guided filtering; is the slope of the fitted linear function at pixel i; is the intercept of the fitted linear function at pixel i.

7. The method for extracting urban water bodies taking into account shadows of high-resolution remote sensing images according to claim 1, characterized in that: The way to optimize the network model is to add attention mechanism and loss function to the network model for optimization to obtain the optimized network model. The construction process of the optimized network model in S3 is: S3.

1. In the encoder part of the network model, the remote sensing image first undergoes two 3×3 convolution operations to obtain the first feature layer. After the first convolution, the size of the first feature layer remains unchanged and the number of channels increases to 64; the second convolution is only used for feature fusion, and the size and number of channels of the first feature layer remain unchanged; S3.2, the first feature layer is scaled down to 128×128 by a 2×2 maximum pooling operation; S3.3, repeat S3.1 and S3.2 5 times to obtain a feature layer with a scale of 16×16×1024; S3.4, 2×2 upsampling is performed in the decoder part to restore the feature scale extracted in the encoding stage to its original size. During the upsampling process, the number of feature channels is halved; S3.

5. Add an attention mechanism CBAM module in the decoder part. In the skip connection stage, a dense connection strategy is used to increase the depth and width of the network and integrate image features at different levels.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images