Method and System for Identifying Vegetation Coverage in Urban Remote Sensing Images Based on Deep Learning

Abstract

The embodiments of the present application relate to the field of artificial intelligence technology, and disclose a method and system for identifying vegetation coverage in urban remote sensing images based on deep learning. The method first obtains the thermal remote sensing image and visible light remote sensing image of the urban target area, and uses the features of the visible light image to accurately segment the thermal image to distinguish the high radiation area (buildings or roads) from the low radiation area (potential vegetation area); then, through the analysis of multi-temporal thermal radiation changes, further screen out specific thermal areas with significant dynamic changes in thermal radiation from the low radiation area. Finally, the coverage range of the specific thermal area is matched and verified with the building and water area coverage information in the urban model network, so as to accurately determine the vegetation coverage area. In this way, through multi-source data complementarity, dynamic feature mining and spatial constraint verification, the problem of low accuracy of single data source judgment and recognition in the prior art is effectively solved, and the accuracy of vegetation coverage recognition is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of artificial intelligence, and particularly to a method and system for identifying vegetation coverage in urban remote sensing images based on deep learning. Background Art

[0002] Urban vegetation coverage monitoring is an important basis for urban ecological assessment, heat island effect analysis, and sustainable development planning. Existing urban vegetation coverage detection methods usually identify based on the color features of visible light remote sensing images, but face multiple challenges in complex urban scenes: for example, the widely used green building materials in the city (such as artificial turf, painted roofs) are highly similar to natural vegetation in terms of color and texture features in the visible light band, resulting in a high misjudgment rate of traditional image segmentation algorithms; and the shadows cast by dense building clusters will change the true color of the vegetation surface, and the difference in the solar altitude angle at different times leads to a drastic fluctuation in the light reflection characteristics of the same vegetation area. Therefore, the accuracy of urban vegetation coverage detection methods in the prior art is relatively low. Summary of the Invention

[0003] The main object of the present invention is to provide a method and system for identifying vegetation coverage in urban remote sensing images based on deep learning, aiming to solve the technical problem of relatively low accuracy of urban vegetation coverage detection methods in the prior art.

[0004] To achieve the above object, in a first aspect, the embodiments of the present application provide a method for identifying vegetation coverage in urban remote sensing images based on deep learning, the method comprising:

[0005] Obtaining a thermal remote sensing image and a visible light remote sensing image of a urban target area respectively to obtain a first target remote sensing image and a second target remote sensing image;

[0006] Performing image segmentation on the first target remote sensing image according to the second target remote sensing image to obtain a first thermal area and a second thermal area, wherein the thermal radiation intensity of the first thermal area is greater than that of the second thermal area;

[0007] Analyzing the change range of the thermal radiation intensity of the second thermal area at different time periods, and performing image segmentation on the second thermal area according to the change range of the thermal radiation intensity to obtain a second main thermal area and a second secondary thermal area, wherein the change range of the thermal radiation intensity of the second main thermal area is greater than that of the second secondary thermal area;

[0008] Inputting the coverage range of the second main thermal area into a pre-established urban model network for area matching verification, the urban model network including building and water area coverage information;

[0009] Determining the second main thermal area as a vegetation coverage area when the area matching verification is passed.

[0010] In a possible implementation, the image segmentation of the first target remote sensing image according to the second target remote sensing image to obtain the first thermal area and the second thermal area includes:

[0011] Performing image pre-segmentation on the first target remote sensing image to obtain a first area to be segmented and a second area to be segmented;

[0012] Correcting the boundary lines of the first area to be segmented and the second area to be segmented according to the texture features of the second target remote sensing image to obtain the first thermal area and the second thermal area.

[0013] In a possible implementation, the image pre-segmentation uses a deep learning model based on U-Net, and the deep learning model is trained with historical remote sensing image data marked with the boundaries of thermal areas.

[0014] In a possible implementation, the correcting the boundary lines of the first area to be segmented and the second area to be segmented according to the texture features of the second target remote sensing image to obtain the first thermal area and the second thermal area includes:

[0015] Obtaining the change gradient of the texture features at the boundary lines of the first area to be segmented and the second area to be segmented;

[0016] When the change gradient of the texture features is less than the gradient threshold, moving and correcting the boundary lines of the first area to be segmented and the second area to be segmented in the direction towards the second area to be segmented to obtain the first thermal area and the second thermal area.

[0017] In a possible implementation, the moving and correcting the boundary lines of the first area to be segmented and the second area to be segmented in the direction towards the second area to be segmented to obtain the first thermal area and the second thermal area includes:

[0018] During the process of moving the boundary lines, the change gradient of the texture features at the current boundary lines is detected in real time;

[0019] When the change gradient of the texture features at the current boundary lines is equal to a preset value or meets a preset condition, stop moving the boundary lines and use the current latest boundary lines as the boundaries of the first thermal area and the second thermal area.

[0020] In a possible implementation, before determining that the second main thermal area is a vegetation-covered area, it further includes:

[0021] Performing sub-pixel decomposition on the mixed pixels in the second main thermal area, and extracting the proportion of each component using an endmember spectral library;

[0022] If the proportion of the vegetation component exceeds a preset ratio, it is retained as a vegetation-covered area, otherwise it is marked as an area to be verified;

[0023] Perform a secondary determination on the area to be verified using the second target remote sensing image.

[0024] In a possible implementation, the performing a secondary determination on the area to be verified using the second target remote sensing image includes:

[0025] Perform multispectral analysis on the visible light remote sensing image of the area to be verified, and extract the red band reflectance R red and the near-infrared band reflectance R nir ;

[0026] Input the red band reflectance R red and the near-infrared band reflectance R nir into the vegetation index model to obtain the vegetation index value N; where the vegetation index model satisfies the following expression:

[0027]

[0028] If the vegetation index value N ≥ T, mark it as a candidate vegetation area, otherwise mark it as a non-vegetation area, where T is a dynamic threshold, set to 0.4 in summer and 0.3 in winter.

[0029] In a possible implementation, the inputting the coverage range of the second main thermal area into the pre-established urban model network for area matching verification includes:

[0030] Calculate the overlapping area between the coverage range of the second main thermal area and the building and water area coverage ranges in the urban model network;

[0031] Determine the confidence level based on the overlapping area and the coverage range area of the second main thermal area;

[0032] In the case where the confidence level is greater than or equal to the first confidence threshold, determine that the area matching verification passes, and determine the second main thermal area as a vegetation-covered area.

[0033] In a possible implementation, after determining the confidence level based on the overlapping area and the coverage range area of the second main thermal area, it further includes:

[0034] In the case where the confidence level is greater than or equal to the second confidence threshold and less than the first confidence threshold, determine that the area matching verification fails, and determine the second main thermal area as a low-confidence vegetation-covered area, where the second confidence threshold is less than the first confidence threshold;

[0035] Trigger an artificial verification signal for the low-confidence vegetation-covered area, and the artificial verification signal is used to remind the user to perform artificial detection and identification.

[0036] In a second aspect, an embodiment of the present application further provides a vegetation coverage recognition system, including: a memory and a processor, where the memory is used to store program code; the processor is used to call the program code to execute the method described in the first aspect.

[0037] Different from the prior art, the method for recognizing vegetation coverage in urban remote sensing images based on deep learning provided by the embodiment of the present application first obtains the thermal remote sensing image and visible light remote sensing image of the urban target area, and uses the features of the visible light image to accurately segment the thermal image to distinguish the high-radiation area (buildings or roads) from the low-radiation area (potential vegetation area); then, through multi-temporal thermal radiation change analysis, further screen out specific thermal areas with significant dynamic changes in thermal radiation from the low-radiation area. Finally, perform area matching verification on the coverage range of the specific thermal area with the building and water area coverage information in the urban model network, so as to accurately determine the vegetation coverage area. In this way, through multi-source data complementarity, dynamic feature mining, and spatial constraint verification, the problem of low accuracy of single data source judgment and recognition in the prior art is effectively solved, the accuracy of vegetation coverage recognition is improved, and strong support is provided for urban ecological environment monitoring and management. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on the structures shown in these drawings.

[0039] Figure 1 It is a schematic flowchart of the method for recognizing vegetation coverage in urban remote sensing images based on deep learning in some embodiments of the present application;

[0040] Figure 2 It is a schematic flowchart of step S200 of the method for recognizing vegetation coverage in urban remote sensing images based on deep learning in some embodiments of the present application;

[0041] Figure 3 It is a schematic flowchart of step S220 of the method for recognizing vegetation coverage in urban remote sensing images based on deep learning in some embodiments of the present application;

[0042] Figure 4 It is a schematic diagram of the segmentation of the second target remote sensing image in some embodiments of the present application;

[0043] Figure 5 It is a schematic diagram of the hardware structure of the vegetation coverage recognition system in some embodiments of the present application.

[0044] The realization, functional features and advantages of the present invention will be further described with reference to the embodiments and the accompanying drawings. Detailed implementation manners

[0045] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0046] It should be noted that all directional indications (such as up, down, left, right, front, back...) in the embodiments of the present invention are only used to explain the relative positional relationship and movement conditions between components in a specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indications will also change accordingly.

[0047] In addition, the descriptions involving "first", "second", etc. in the present invention are only for descriptive purposes, and cannot be understood as indicating or implying their relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one of such features. In addition, "and / or" throughout the text includes three solutions. Taking A and / or B as an example, it includes the technical solution of A, the technical solution of B, and the technical solution that A and B are satisfied at the same time. In addition, the technical solutions between the various embodiments can be combined with each other, but it must be based on the fact that those of ordinary skill in the art can implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the protection scope required by the present invention.

[0048] Urban vegetation cover monitoring is an important basis for urban ecological assessment, heat island effect analysis and sustainable development planning. Existing urban vegetation cover detection methods usually identify based on the color features of visible light remote sensing images, but face multiple challenges in complex urban scenes: for example, the green building materials widely used in cities (such as artificial turf, painted roofs) are highly similar to natural vegetation in terms of color and texture features in the visible light band, resulting in a high misjudgment rate of traditional image segmentation algorithms; and the shadows cast by dense building clusters will change the true color of the vegetation surface, and the difference in the solar altitude angle at different times causes the light reflection characteristics of the same vegetation area to fluctuate violently. Therefore, the accuracy of urban vegetation cover detection methods in the prior art is relatively low.

[0049] Such as Figures 1 - 3As shown below, the vegetation cover recognition method based on deep learning is performed by a vegetation cover recognition system as an example. It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here. Please refer to the appendix Figure 1 , the method includes the following steps S100 - step S500:

[0050] Step S100, respectively obtain the thermal remote sensing image and visible light remote sensing image of the urban target area to obtain the first target remote sensing image and the second target remote sensing image;

[0051] Specifically, there are many ways to obtain the thermal remote sensing image and visible light remote sensing image of the urban target area. For example, when obtaining the thermal remote sensing image of the urban target area, a remote sensing platform equipped with a thermal infrared sensor (such as Landsat 8 TIRS, ASTER or drone thermal imaging module) can be called to scan over the target area to obtain surface thermal radiation data; then the thermal radiation energy data is converted into a digital signal by the sensor to generate a thermal image. When obtaining the visible light remote sensing image of the urban target area, a remote sensing platform equipped with an optical sensor (such as a high-resolution satellite or a drone) can be called to obtain the multi-spectral image of the target area, covering red, green, blue and near-infrared bands, etc. And in the embodiments of the present application, the acquisition time difference between the thermal remote sensing image and the visible light remote sensing image ≤ 1 hour to reduce the impact of temporal differences on subsequent analysis.

[0052] In one embodiment, after obtaining the target remote sensing image, a terrain correction step can be added to further improve the accuracy of the target remote sensing image. For example, the digital elevation model (DEM) of the target area can be obtained first, and the slope, aspect and terrain occlusion coefficient are calculated; then a regression model of the thermal radiation intensity and terrain factors is established to eliminate the radiation difference between the sunny side and the shady side in mountainous cities; finally, the corrected thermal radiation intensity is normalized to generate a thermal distribution remote sensing image independent of terrain for subsequent segmentation. In this way, the radiation distortion problem caused by the slope can be solved.

[0053] Step S200, perform image segmentation on the first target remote sensing image according to the second target remote sensing image to obtain a first thermal area and a second thermal area, wherein the thermal radiation intensity of the first thermal area is greater than that of the second thermal area;

[0054] It can be understood that the first target remote sensing image contains the thermal radiation intensity information of different regions, and the second target remote sensing image contains the color, texture and other optical information of different regions.

[0055] It should be noted that the thermal radiation in related areas such as buildings or roads is greatly affected by light, and the thermal radiation intensity is relatively high; while the thermal radiation in related areas such as vegetation or water areas is less affected by light, and the thermal radiation intensity is relatively low. Therefore, the first target remote sensing image can be segmented into a first thermal area and a second thermal area with obvious differences in thermal radiation intensity through the thermal radiation intensity information; for example, the thermal radiation intensity of the first thermal area is more than twice that of the second thermal area. In this way, buildings or roads, vegetation or water areas can be accurately distinguished.

[0056] In this way, the first target remote sensing image is segmented into a first thermal area and a second thermal area in the embodiment of the present application, and the high-radiation area (buildings or roads) and the low-radiation area (potential vegetation area) can be distinguished.

[0057] In order to further improve the accuracy of the image segmentation of the first target remote sensing image, in one embodiment, the step S200: segment the first target remote sensing image into a first thermal area and a second thermal area according to the second target remote sensing image, includes:

[0058] S210. Perform image pre-segmentation on the target remote sensing image to obtain a first area to be segmented and a second area to be segmented;

[0059] S220. Correct the boundary lines of the first area to be segmented and the second area to be segmented according to the texture features of the second target remote sensing image to obtain a first thermal area and a second thermal area.

[0060] Specifically, first, a deep learning model based on U-Net can be used to perform image pre-segmentation on the remote sensing image (the first target remote sensing image), and the deep learning model is trained with historical remote sensing image data marked with the boundaries of the thermal areas. Then, the boundary lines of the first area to be segmented and the second area to be segmented are corrected according to the texture features of the second target remote sensing image to obtain a first thermal area and a second thermal area.

[0061] Using a deep learning model based on U-Net for image pre-segmentation and correcting the boundary lines of the pre-segmentation results according to the texture features of the second target remote sensing image has a significant effect in improving the accuracy of remote sensing image segmentation. This method can not only automatically generate high-precision segmentation results, but also adapt to complex scenes and improve the robustness and reliability of segmentation.

[0062] Exemplarily, first, the preprocessed thermal image is input into the trained U-Net model to output a probability map (the probability that each pixel belongs to the high-temperature area). A binary mask is generated through threshold segmentation (the threshold is 0.5), and it is initially divided into a first area to be segmented (high-temperature area) and a second area to be segmented (low-temperature area).

[0063] It can be understood that according to the Stefan-Boltzmann law, the radiant energy of an object is proportional to the fourth power of its temperature and emissivity. Emissivity is an indicator of the radiant ability of the object's surface, ranging from 0 to 1. The higher the emissivity, the stronger the radiant energy. The emissivity of glass curtain walls is usually low (lower than that of vegetation). This means that at the same true temperature, the radiant energy of glass will be lower than that of vegetation, resulting in a lower radiant value being displayed in the thermal infrared image. Therefore, glass structures such as building curtain walls may display lower radiant values in thermal infrared images, similar to the temperature characteristics of vegetation. Therefore, to address the misjudgment problem caused by the similar thermal radiation characteristics of glass curtain walls and vegetation, the embodiments of the present application introduce visible light remote sensing images to correct the boundary line between the first area to be segmented and the second area to be segmented, thereby further improving the accuracy of the image segmentation of the first target remote sensing image.

[0064] In one embodiment, the step S220: correcting the boundary line between the first area to be segmented and the second area to be segmented according to the texture feature of the second target remote sensing image to obtain a first thermal area and a second thermal area includes:

[0065] S221. Obtain the change gradient of the texture feature at the boundary line between the first area to be segmented and the second area to be segmented;

[0066] S222. When the change gradient of the texture feature is less than the gradient threshold, move and correct the boundary line between the first area to be segmented and the second area to be segmented in the direction towards the second area to be segmented to obtain a first thermal area and a second thermal area.

[0067] Specifically, first obtain the change gradient of the texture feature at the boundary line between the first area to be segmented and the second area to be segmented. When the change gradient of the texture feature at the boundary line is less than the gradient threshold, it indicates that this boundary line is a glass feature (because the texture features of the glass surface are the same, so the change gradient of the texture feature is small). At this time, move the boundary line between the first area to be segmented and the second area to be segmented in the direction towards the second area to be segmented, that is, reduce the range of the second area to be segmented (low-temperature area), so as to demarcate the curtain wall glass from the potential vegetation area. In this way, the accuracy of the image segmentation of the first target remote sensing image is improved. And during the process of moving the boundary line, the change gradient of the texture feature at the current boundary line is detected in real time; when the change gradient of the texture feature at the current boundary line is equal to the preset value or meets the preset conditions (such as the change gradient of the texture feature at the current boundary line is close to the preset value), stop moving the boundary line and use the current latest boundary line as the boundary between the first thermal area and the second thermal area.

[0068] Exemplarily, such as Figure 4As shown in the figure, S1 in the figure is the first area to be segmented, and S2 is the second area to be segmented. The preliminary pre-segmentation boundary line is L1. At this time, if it is determined through visible light texture features that the boundary line L1 is the curtain wall glass feature, that is, the second area to be segmented S2 includes the curtain wall glass area, then the curtain wall glass area needs to be removed from the second area to be segmented S2, that is, the demarcation line is moved in the direction towards the second area to be segmented. For example, after the boundary is moved to the L2 position, the visible light texture features at the boundary line meet the predetermined requirements, and the boundary line at this time is the relatively accurate segmentation boundary line, so as to ensure that the second area to be segmented only includes vegetation or water areas.

[0069] Step S300: Analyze the change range of the thermal radiation intensity of the second thermal area in different time periods, and perform image segmentation on the second thermal area according to the change range of the thermal radiation intensity to obtain a second main thermal area and a second secondary thermal area, where the change range of the thermal radiation intensity of the second main thermal area is greater than the change range of the thermal radiation intensity of the second secondary thermal area;

[0070] It can be understood that after the first thermal area and the second thermal area are segmented, since the thermal radiation intensities of the vegetation area and the water area are both relatively low, therefore, the second thermal area may be a vegetation area or a water area. Based on this, in the embodiment of the present application, the second thermal area is further subjected to image segmentation according to the change range of the thermal radiation intensity to obtain a second main thermal area and a second secondary thermal area.

[0071] It should be noted that since the specific heat capacity of water is relatively large and the temperature change is relatively slow, that is, the change range of the thermal radiation intensity of water is relatively small compared to vegetation. Therefore, in the embodiment of the present application, the second thermal area is subjected to image segmentation according to the change range of the thermal radiation intensity to obtain a second main thermal area and a second secondary thermal area. The second main thermal area with a larger change range of the thermal radiation intensity is the vegetation area, and the second secondary thermal area with a smaller change range of the thermal radiation intensity is the water area.

[0072] Specifically, in the embodiment of the present application, the thermal radiation intensity data of the second thermal area in different time periods (such as multiple time points in a day or the same time period for several consecutive days) can be collected first. These data can be obtained by calling the historical data or real-time data of the remote sensing platform. Then, analyze the change range of the thermal radiation intensity of the second thermal area within these time periods. The change range can be measured by calculating the difference or percentage change of the thermal radiation intensity between adjacent time periods. For example, the average value of the thermal radiation intensity of each time period can be calculated, and then the difference or percentage change between the average values of adjacent time periods can be calculated. Then, image segmentation of the second thermal area is performed according to the change range of the thermal radiation intensity, which can be achieved by setting a change range threshold. The area with a thermal radiation intensity change range greater than the threshold is divided into the second main thermal area, and the area with a change range less than or equal to the threshold is divided into the second secondary thermal area.

[0073] Thus, through the above steps, the second thermal zone can be effectively segmented according to the change range of the thermal radiation intensity, so as to more accurately identify the vegetation-covered area.

[0074] Step S400: Input the coverage range of the second main thermal zone into a pre-established urban model network for area matching verification. The urban model network includes building and water area coverage information.

[0075] Step S500: When the area matching verification is passed, determine the second main thermal zone as the vegetation-covered area.

[0076] In one embodiment, step S400: Input the coverage range of the second main thermal zone into a pre-established urban model network for area matching verification, including: calculating the overlapping area between the coverage range of the second main thermal zone and the building and water area coverage ranges in the urban model network; determining the confidence level according to the overlapping area and the coverage area of the second main thermal zone; when the confidence level is greater than or equal to the first confidence threshold, determine that the area matching verification is passed and determine the second main thermal zone as the vegetation-covered area.

[0077] Before performing the area matching verification, a urban model network including building and water area coverage information can be established in advance. This model network can be comprehensively obtained from multiple data sources such as remote sensing images, geographic information system (GIS) data, and urban planning data. Ensure that the spatial resolution and coordinate system of the urban model network match the coverage range of the second main thermal zone for accurate area matching verification. Among them, the urban model network at least includes building and water area coverage information.

[0078] During the area matching verification, the coverage range of the second main thermal area is spatially superimposed and analyzed with the urban model network. First, calculate the overlapping area between the second main thermal area and the building and water coverage in the urban model network, which can be achieved through spatial analysis tools in GIS software, such as intersection, union, etc. operations. Then, according to the overlapping area and the coverage area of the second main thermal area, calculate the confidence level. The confidence level is defined as a certain function of the ratio of the overlapping area to the coverage area of the second main thermal area, or other appropriate statistical indicators are used to measure. And set a first confidence threshold for determining whether the area matching verification passes. This threshold can be set according to actual needs and historical data experience. If the calculated confidence level is greater than or equal to the first confidence threshold, it is determined that the area matching verification passes. In this case, the second main thermal area is determined to be a vegetation-covered area. This is because the overlapping area between the second main thermal area and the building and water coverage in the urban model network is small, indicating that this area is more likely to be a vegetation-covered area. If the confidence level is less than the first confidence threshold, the area matching verification fails. In this case, the second main thermal area can be further inspected and analyzed, such as collecting more remote sensing data, adjusting the urban model network, or triggering an artificial verification signal for the low-confidence vegetation-covered area, and the artificial verification signal is used to remind the user to perform artificial detection and identification to more accurately determine the coverage type of this area.

[0079] In other embodiments, the urban model network can be dynamically updated in the following ways: accessing the real-time change data stream of the urban geographic information system (GIS); when the geographic information system (GIS) is updated, the urban model network is automatically updated in real time to automatically mark the newly built building / water area; in addition, when the conflict area between the model network and the remote sensing image exceeds the threshold, trigger the update request of the urban model network to prevent the problem of automatic update failure. In this way, the efficiency of area matching verification can be improved.

[0080] In another embodiment, to further improve the accuracy of determining the vegetation-covered area. Before determining the second main thermal area as a vegetation-covered area, it further includes: performing sub-pixel decomposition on the mixed pixels in the second main thermal area, and using the endmember spectral library to extract the proportion of each component; if the proportion of the vegetation component exceeds the preset ratio, it is retained as a vegetation-covered area, otherwise it is marked as an area to be verified; perform a secondary determination on the area to be verified through the second target remote sensing image.

[0081] It can be understood that the second main thermal area is represented as an area with a relatively large change range of thermal radiation intensity, and this feature also exists in the vegetation area and the bare soil area (the bare area without vegetation). Therefore, in the embodiment of the present application, before determining the second main thermal area as a vegetation-covered area, first perform sub-pixel decomposition on the mixed pixels in the second main thermal area, and use the endmember spectral library to extract the proportion of each component; if the proportion of the vegetation component exceeds the preset ratio, it is retained as a vegetation-covered area, otherwise it is marked as an area to be verified; then use the second target remote sensing image to perform a secondary determination on the area to be verified.

[0082] Specifically, first perform sub-pixel decomposition on the mixed pixels (i.e., pixels that simultaneously contain multiple surface cover types) in the second main thermal area. Sub-pixel decomposition is a technique used to decompose mixed pixels into smaller sub-pixels that represent a single surface cover type. Then use the endmember spectral library for sub-pixel decomposition. The endmember spectral library contains the spectral characteristics of various surface cover types (such as vegetation, water bodies, soil, buildings, etc.). By comparing the spectral characteristics of the mixed pixels with the spectral characteristics in the endmember spectral library, the proportion of each component in the mixed pixels can be estimated. Based on the sub-pixel decomposition, extract the proportion of the vegetation component. This can be achieved by calculating the contribution of the vegetation spectral characteristics in the mixed pixel spectrum. Set a preset ratio to determine the dominant position of the vegetation component in the mixed pixels. This preset ratio can be set according to actual needs and historical data experience. If the proportion of the vegetation component exceeds the preset ratio, retain this area as a vegetation-covered area. This is because the main surface cover type in this area is vegetation. If the proportion of the vegetation component does not exceed the preset ratio, mark this area as an area to be verified. This is because this area may contain other surface cover types, or the vegetation cover is not dense enough, and further determination is required. Finally, use the second target remote sensing image to perform a secondary determination on the area to be verified. The second target remote sensing image can be a data source different from the initial remote sensing image, obtained at a different time, or in a different spectral band. By analyzing information such as spectral characteristics and texture characteristics in the second target remote sensing image, further determine the surface cover type of the area to be verified. According to the results of the secondary determination, perform a final classification on the area to be verified and determine it as a vegetation-covered area or other surface cover types.

[0083] In this way, through the above steps, the vegetation-covered area in the second main thermal area can be determined more accurately. The use of sub-pixel decomposition and the endmember spectral library improves the accuracy of mixed pixel decomposition, while the secondary determination further reduces the possibility of misjudgment, thereby improving the accuracy and reliability of vegetation cover identification.

[0084] In one embodiment, performing a secondary determination on the area to be verified through the second target remote sensing image includes: performing multi-spectral analysis on the visible light remote sensing image of the area to be verified, and extracting the red light band reflectance Rred and the reflectance R in the near-infrared band nir ; the reflectance R in the red light band red and the reflectance R in the near-infrared band nir are input into the vegetation index model to obtain the vegetation index value N; wherein, the vegetation index model satisfies the following expression:

[0085]

[0086] If the vegetation index value N≥T, it is marked as a candidate vegetation area, otherwise it is marked as a non-vegetation area, where T is a dynamic threshold, set to 0.4 in summer and 0.3 in winter.

[0087] It can be understood that due to the strong absorption of chlorophyll in vegetation in the red light band (Rred) and the high reflectance of the mesophyll cell structure in the near-infrared band (Rnir), the reflectance difference of vegetation in these two bands is significant. Through the design of the normalized vegetation index model, the vegetation coverage can be quantitatively characterized. The larger the vegetation index value (N), the higher the possibility of vegetation coverage.

[0088] It should be noted that the growth of vegetation is affected by seasons. In summer, the vegetation is lush, and the difference in red light absorption and near-infrared reflection is more obvious, so a higher threshold (0.4) is required for screening; in winter, the vegetation withers, the spectral characteristics weaken, and the threshold is reduced (0.3) to adapt to seasonal changes and improve the recognition accuracy. In other seasons, such as spring and autumn, the threshold can be set according to the actual situation.

[0089] Based on this, the method for identifying vegetation coverage in urban remote sensing images based on deep learning provided by the embodiments of the present application first obtains the thermal remote sensing image and visible light remote sensing image of the urban target area, and uses the features of the visible light image to accurately segment the thermal image to distinguish the high-radiation area (buildings or roads) from the low-radiation area (potential vegetation area); then through the analysis of multi-temporal thermal radiation changes, further screen out specific thermal areas with significant dynamic changes in thermal radiation from the low-radiation area. Finally, the coverage range of the specific thermal area is matched and verified with the building and water area coverage information in the urban model network, so as to accurately determine the vegetation coverage area. In this way, through multi-source data complementarity, dynamic feature mining and spatial constraint verification, the problem of low recognition accuracy in the prior art due to a single data source is effectively solved, the accuracy of vegetation coverage recognition is improved, and strong support is provided for urban ecological environment monitoring and management.

[0090] Such as Figure 5 shown Figure 5This is a schematic diagram of the hardware structure of the vegetation coverage recognition system in some embodiments of the present application. The vegetation coverage recognition system provided by the embodiments of the present application includes a memory 1000 and a processor 2000. Among them, the memory 1000 is used to store computer-readable instructions, and the processor 2000 is used to call the computer-readable instructions to execute the method for recognizing vegetation coverage in urban remote sensing images based on deep learning as described above.

[0091] Among them, the processor 2000 is used to provide computing and control capabilities to control the vegetation coverage recognition system to perform corresponding tasks. For example, it controls the vegetation coverage recognition system to execute the method for recognizing vegetation coverage in urban remote sensing images based on deep learning in any of the above method embodiments. The method includes: respectively obtaining a thermal remote sensing image and a visible light remote sensing image of an urban target area to obtain a first target remote sensing image and a second target remote sensing image; performing image segmentation on the first target remote sensing image according to the second target remote sensing image to obtain a first thermal area and a second thermal area, where the thermal radiation intensity of the first thermal area is greater than that of the second thermal area; analyzing the change range of the thermal radiation intensity of the second thermal area at different time periods, and performing image segmentation on the second thermal area according to the change range of the thermal radiation intensity to obtain a second main thermal area and a second secondary thermal area, where the change range of the thermal radiation intensity of the second main thermal area is greater than that of the second secondary thermal area; inputting the coverage range of the second main thermal area into a pre-established urban model network for area matching verification, and the urban model network includes building and water area coverage information; in the case where the area matching verification is passed, determining the second main thermal area as a vegetation coverage area.

[0092] The processor 2000 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), a hardware chip, or any combination thereof; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The above PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0093] The memory 1000, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the method for identifying vegetation coverage in urban remote sensing images based on deep learning in the embodiments of the present application. By running the non-transitory software programs, instructions, and modules stored in the memory 1000, the processor 2000 can implement the method for identifying vegetation coverage in urban remote sensing images based on deep learning in any of the above method embodiments.

[0094] Specifically, the memory 1000 may include volatile memory (VM), such as random access memory (RAM); the memory 1000 may also include non-volatile memory (NVM), such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD), or other non-transitory solid-state storage devices; the memory 1000 may further include a combination of the above types of memories.

[0095] In summary, the vegetation coverage identification system of the present application adopts the technical solution of any one of the above embodiments of the method for identifying vegetation coverage in urban remote sensing images based on deep learning. Therefore, it has at least the beneficial effects brought by the technical solutions of the above embodiments, which will not be elaborated here one by one.

[0096] The embodiments of the present application also provide a computer-readable storage medium, such as a memory including program codes, and the above program codes can be executed by a processor to complete the method for identifying vegetation coverage in urban remote sensing images based on deep learning in the above embodiments. For example, the computer-readable storage medium may be read-only memory (ROM), random access memory (RAM), compact disc read-only memory (CDROM), magnetic tape, floppy disk, and optical data storage devices, etc.

[0097] The embodiments of the present application also provide a computer program product, which includes one or more program codes, and the program codes are stored in a computer-readable storage medium. The processor of the vegetation coverage identification system reads the program codes from the computer-readable storage medium, and the processor executes the program codes to complete the steps of the method for identifying vegetation coverage in urban remote sensing images based on deep learning provided in the above embodiments.

[0098] Those of ordinary skill in the art can understand that all or part of the steps to implement the above embodiments can be completed by hardware, or can be completed by hardware related to program code. This program can be stored in a computer-readable storage medium. The storage medium mentioned above can be a read-only memory, a magnetic disk, an optical disc, etc.

[0099] It should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0100] Through the description of the above embodiments, those of ordinary skill in the art can clearly understand that each embodiment can be implemented by means of software plus a general hardware platform, and of course, it can also be implemented by hardware. Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing related hardware through a computer program. The program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. Among them, the storage medium can be a magnetic disk, an optical disc, a read-only memory (ROM), or a random access memory (RAM), etc.

[0101] The above are only the preferred embodiments of the present invention, and do not limit the patent scope of the present invention accordingly. Any equivalent structural transformation made by using the specification and drawings of the present invention under the inventive concept of the present invention, or direct / indirect application in other related technical fields is included in the patent protection scope of the present invention.

Claims

1. A method for identifying vegetation cover in urban remote sensing images based on deep learning, characterized in that: The method comprises: Acquire thermal remote sensing images and visible light remote sensing images of the urban target area to obtain a first target remote sensing image and a second target remote sensing image; Performing image segmentation on the first target remote sensing image according to the second target remote sensing image to obtain a first thermal zone and a second thermal zone, wherein the thermal radiation intensity of the first thermal zone is greater than the thermal radiation intensity of the second thermal zone; Analyze the variation range of the thermal radiation intensity of the second thermal zone in different time periods, and perform image segmentation on the second thermal zone according to the variation range of the thermal radiation intensity to obtain a second main thermal zone and a secondary thermal zone, wherein the variation range of the thermal radiation intensity of the second main thermal zone is greater than the variation range of the thermal radiation intensity of the secondary thermal zone; Inputting the coverage of the second main thermal zone into a pre-established urban model network for area matching verification, wherein the urban model network includes building and water coverage information; If the area matching check passes, the second main thermal zone is determined to be a vegetation covered area; The step of performing image segmentation on the first target remote sensing image according to the second target remote sensing image to obtain the first thermal zone and the second thermal zone includes: Performing image pre-segmentation on the first target remote sensing image to obtain a first area to be segmented and a second area to be segmented; Correcting the boundary lines of the first area to be segmented and the second area to be segmented according to the texture features of the second target remote sensing image to obtain a first thermal zone and a second thermal zone; The step of correcting the boundary lines of the first area to be segmented and the second area to be segmented according to the texture features of the second target remote sensing image to obtain the first thermal zone and the second thermal zone includes: Acquire a texture feature change gradient at a boundary line between the first area to be segmented and the second area to be segmented; When the gradient of the texture feature change is less than the gradient threshold, the boundary line between the first area to be segmented and the second area to be segmented is moved and corrected toward the second area to be segmented to obtain a first thermal zone and a second thermal zone.

2. The urban remote sensing image vegetation coverage identification method based on deep learning according to claim 1, characterized in that: The image pre-segmentation adopts a U-Net-based deep learning model, and the deep learning model is trained by historical remote sensing image data marked with thermal zone boundaries.

3. The urban remote sensing image vegetation coverage identification method based on deep learning as claimed in claim 1, characterized in that: The step of moving and correcting the boundary line between the first area to be segmented and the second area to be segmented toward the second area to be segmented to obtain the first thermal zone and the second thermal zone includes: In the process of moving the boundary line, the gradient of texture feature change at the current boundary line is detected in real time; When the texture feature change gradient at the current boundary line is equal to a preset value or meets a preset condition, the boundary line movement is stopped and the current latest boundary line is used as the boundary between the first thermal zone and the second thermal zone.

4. The urban remote sensing image vegetation coverage identification method based on deep learning as claimed in claim 1, characterized in that: Before determining the second main thermal zone as a vegetation covered area, the method further includes: Performing sub-pixel decomposition on the mixed pixels in the second main thermal zone, and extracting the proportion of each component using the endmember spectral library; If the proportion of vegetation components exceeds the preset ratio, it will be retained as a vegetation-covered area, otherwise it will be marked as an area to be verified; A secondary determination is made on the area to be verified using the second target remote sensing image.

5. The urban remote sensing image vegetation coverage identification method based on deep learning as claimed in claim 4, characterized in that: The second determination of the area to be verified by using the second target remote sensing image includes: Perform multispectral analysis on the visible light remote sensing image of the area to be verified and extract the red light band reflectance R red and near-infrared reflectivity R nir ; The red light band reflectivity R red and near-infrared reflectivity R nir The vegetation index model is input to obtain the vegetation index value N; wherein the vegetation index model satisfies the following expression: If the vegetation index value N ≥ T, it is marked as a candidate vegetation area, otherwise it is marked as a non-vegetation area, where T is a dynamic threshold, which is set to 0.4 in summer and 0.3 in winter.

6. The urban remote sensing image vegetation coverage identification method based on deep learning as claimed in claim 1, characterized in that: The step of inputting the coverage of the second main thermal zone into a pre-established urban model network for area matching verification comprises: Calculate the overlapping area between the coverage of the second main thermal zone and the coverage of buildings and water areas in the urban model network; Determining a confidence level based on the overlapping area and the coverage area of ​​the second main thermal zone; When the confidence level is greater than or equal to the first confidence level threshold, it is determined that the area matching check has passed, and the second main thermal zone is determined to be a vegetation covered area.

7. The urban remote sensing image vegetation coverage identification method based on deep learning according to claim 6, characterized in that: After determining the confidence level according to the overlapping area and the coverage area of ​​the second main thermal zone, the method further includes: When the confidence is greater than or equal to the second confidence threshold and less than the first confidence threshold, determining that the area matching check fails, and determining the second main thermal zone as a low-confidence vegetation coverage area, wherein the second confidence threshold is less than the first confidence threshold; A manual verification signal is triggered for the low-credible vegetation coverage area, and the manual verification signal is used to remind the user to perform manual detection and identification.

8. A vegetation cover identification system, characterized in that: include: A memory and a processor, wherein the memory is used to store program codes; The processor is used to call the program code to execute the method according to any one of claims 1 to 7.

Images