Urban remote sensing image vegetation coverage identification method and system based on deep learning

Abstract

The embodiment of the invention relates to the technical field of artificial intelligence, and discloses an urban remote sensing image vegetation coverage identification method and system based on deep learning, and the method comprises the steps: firstly obtaining a thermal remote sensing image and a visible light remote sensing image of an urban target region, and carrying out the precise segmentation of the thermal image through the features of the visible light image, high-radiation areas (buildings or roads) and low-radiation areas (potential vegetation areas) are distinguished; and then, through multi-temporal thermal radiation change analysis, a specific thermal area with remarkable thermal radiation dynamic change is further screened out from the low-radiation area. And finally, carrying out area matching verification on the coverage range of the specific thermal area and the building and water area coverage information in the urban model network so as to accurately judge the vegetation coverage area. Therefore, through multi-source data complementation, dynamic feature mining and spatial constraint verification, the problem of low accuracy of judgment and recognition of a single data source in the prior art is effectively solved, and the accuracy of vegetation coverage recognition is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method and system for identifying vegetation cover in urban remote sensing images based on deep learning. Background Art

[0002] Urban vegetation coverage monitoring is an important foundation for urban ecological assessment, urban heat island effect analysis and sustainable development planning. Existing urban vegetation coverage detection methods are usually based on the color characteristics of visible light remote sensing images for identification, but they face multiple challenges in complex urban scenes: for example, the color and texture characteristics of green building materials (such as artificial turf and painted roofs) widely used in cities are highly similar to those of natural vegetation in the visible light band, resulting in a high misjudgment rate of traditional image segmentation algorithms; and the shadows cast by dense buildings will change the true color of the vegetation surface, and the difference in solar altitude angles at different times will cause the light reflection characteristics of the same vegetation area to fluctuate dramatically. Therefore, the accuracy of urban vegetation coverage detection methods in the existing technology is low. Summary of the invention

[0003] The main purpose of the present invention is to provide a method and system for identifying vegetation cover in urban remote sensing images based on deep learning, aiming to solve the technical problem of low accuracy of urban vegetation cover detection methods in the prior art.

[0004] To achieve the above objectives, in a first aspect, an embodiment of the present application provides a method for identifying vegetation cover in urban remote sensing images based on deep learning, the method comprising: Acquire thermal remote sensing images and visible light remote sensing images of the urban target area to obtain a first target remote sensing image and a second target remote sensing image; Performing image segmentation on the first target remote sensing image according to the second target remote sensing image to obtain a first thermal zone and a second thermal zone, wherein the thermal radiation intensity of the first thermal zone is greater than the thermal radiation intensity of the second thermal zone; Analyze the variation range of the thermal radiation intensity of the second thermal zone in different time periods, and perform image segmentation on the second thermal zone according to the variation range of the thermal radiation intensity to obtain a second main thermal zone and a secondary thermal zone, wherein the variation range of the thermal radiation intensity of the second main thermal zone is greater than the variation range of the thermal radiation intensity of the secondary thermal zone; Inputting the coverage of the second main thermal zone into a pre-established urban model network for area matching verification, wherein the urban model network includes building and water coverage information; When the area matching check passes, the second main thermal zone is determined to be a vegetation covered area.

[0005] In a possible implementation, performing image segmentation on the first target remote sensing image according to the second target remote sensing image to obtain the first thermal zone and the second thermal zone includes: Performing image pre-segmentation on the first target remote sensing image to obtain a first area to be segmented and a second area to be segmented; According to the texture features of the second target remote sensing image, the boundary lines of the first area to be segmented and the second area to be segmented are corrected to obtain the first thermal zone and the second thermal zone.

[0006] In a possible implementation, the image pre-segmentation uses a U-Net-based deep learning model, and the deep learning model is trained by using historical remote sensing image data labeled with thermal zone boundaries.

[0007] In a possible implementation, the step of correcting the boundary lines of the first area to be segmented and the second area to be segmented according to the texture features of the second target remote sensing image to obtain the first thermal zone and the second thermal zone includes: Acquire a texture feature change gradient at a boundary line between the first area to be segmented and the second area to be segmented; When the gradient of the texture feature change is less than the gradient threshold, the boundary line between the first area to be segmented and the second area to be segmented is moved and corrected toward the second area to be segmented to obtain a first thermal zone and a second thermal zone.

[0008] In a possible implementation, the step of moving and correcting the boundary line between the first area to be segmented and the second area to be segmented toward the second area to be segmented to obtain the first thermal zone and the second thermal zone includes: In the process of moving the boundary line, the gradient of texture feature change at the current boundary line is detected in real time; When the texture feature change gradient at the current boundary line is equal to a preset value or meets a preset condition, the boundary line movement is stopped and the current latest boundary line is used as the boundary between the first thermal zone and the second thermal zone.

[0009] In a possible implementation manner, before determining the second main thermal zone as a vegetation covered area, the method further includes: Performing sub-pixel decomposition on the mixed pixels in the second main thermal zone, and extracting the proportion of each component using the endmember spectral library; If the proportion of vegetation components exceeds the preset ratio, it will be retained as a vegetation-covered area, otherwise it will be marked as an area to be verified; A secondary determination is made on the area to be verified using the second target remote sensing image.

[0010] In a possible implementation, the second determination of the to-be-verified area by using the second target remote sensing image includes: Perform multispectral analysis on the visible light remote sensing image of the area to be verified and extract the red light band reflectance R red and near-infrared reflectivity R nir ; The red light band reflectivity R red and near-infrared reflectivity R nir The vegetation index model is input to obtain the vegetation index value N; wherein the vegetation index model satisfies the following expression: If the vegetation index value N ≥ T, it is marked as a candidate vegetation area, otherwise it is marked as a non-vegetation area, where T is a dynamic threshold, which is set to 0.4 in summer and 0.3 in winter.

[0011] In a possible implementation, inputting the coverage of the second main thermal zone into a pre-established city model network for area matching verification includes: Calculate the overlapping area between the coverage of the second main thermal zone and the coverage of buildings and water areas in the urban model network; Determining a confidence level based on the overlapping area and the coverage area of ​​the second main thermal zone; When the confidence level is greater than or equal to the first confidence level threshold, it is determined that the area matching check has passed, and the second main thermal zone is determined to be a vegetation covered area.

[0012] In a possible implementation, after the confidence level is determined according to the overlapping area and the coverage area of ​​the second main thermal zone, the method further includes: When the confidence is greater than or equal to the second confidence threshold and less than the first confidence threshold, determining that the area matching check fails, and determining the second main thermal zone as a low-confidence vegetation coverage area, wherein the second confidence threshold is less than the first confidence threshold; A manual verification signal is triggered for the low-credible vegetation coverage area, and the manual verification signal is used to remind the user to perform manual detection and identification.

[0013] In a second aspect, an embodiment of the present application further provides a vegetation cover identification system, comprising: a memory and a processor, wherein the memory is used to store program code; and the processor is used to call the program code to execute the method described in the first aspect.

[0014] Different from the prior art, the urban remote sensing image vegetation coverage identification method based on deep learning provided in the embodiment of the present application first obtains the thermal remote sensing image and the visible light remote sensing image of the urban target area, and uses the characteristics of the visible light image to accurately segment the thermal image to distinguish the high radiation area (buildings or roads) from the low radiation area (potential vegetation area); and then further screens out the specific thermal areas with significant dynamic changes in thermal radiation from the low radiation area through multi-phase thermal radiation change analysis. Finally, the coverage range of the specific thermal area is matched and checked with the building and water coverage information in the urban model network, so as to accurately determine the vegetation coverage area. In this way, the problem of low accuracy of single data source judgment and identification in the prior art is effectively solved through multi-source data complementation, dynamic feature mining and spatial constraint verification, and the accuracy of vegetation coverage identification is improved, providing strong support for urban ecological environment monitoring and management. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0016] Figure 1 A schematic diagram of the flow of a method for identifying vegetation coverage in urban remote sensing images based on deep learning in some embodiments of the present application; Figure 2 This is a flowchart of step S200 of the urban remote sensing image vegetation coverage identification method based on deep learning in some embodiments of the present application; Figure 3 This is a flowchart of step S220 of the method for identifying vegetation coverage in urban remote sensing images based on deep learning in some embodiments of the present application; Figure 4 A schematic diagram of segmentation of a second target remote sensing image in some embodiments of the present application; Figure 5 This is a schematic diagram of the hardware structure of the vegetation cover identification system in some embodiments of the present application.

[0017] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0020] In addition, the descriptions of "first", "second", etc. in the present invention are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, "and / or" in the full text includes three solutions. Taking A and / or B as an example, it includes technical solution A, technical solution B, and technical solution that satisfies both A and B. In addition, the technical solutions between the various embodiments can be combined with each other, but it must be based on the ability of ordinary technicians in this field to implement. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0021] Urban vegetation coverage monitoring is an important foundation for urban ecological assessment, urban heat island effect analysis and sustainable development planning. Existing urban vegetation coverage detection methods are usually based on the color characteristics of visible light remote sensing images for identification, but they face multiple challenges in complex urban scenes: for example, the color and texture characteristics of green building materials (such as artificial turf and painted roofs) widely used in cities are highly similar to those of natural vegetation in the visible light band, resulting in a high misjudgment rate of traditional image segmentation algorithms; and the shadows cast by dense buildings will change the true color of the vegetation surface, and the difference in solar altitude angles at different times will cause the light reflection characteristics of the same vegetation area to fluctuate dramatically. Therefore, the accuracy of urban vegetation coverage detection methods in the existing technology is low.

[0022] like Figure 1-Figure 3 As shown, the following takes the vegetation cover recognition system executing the urban remote sensing image vegetation cover recognition method based on deep learning as an example. It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described can be performed in a different order from here. Figure 1 The method comprises the following steps S100 to S500: Step S100, respectively acquiring a thermal remote sensing image and a visible light remote sensing image of a target urban area to obtain a first target remote sensing image and a second target remote sensing image; Specifically, there are many ways to obtain thermal remote sensing images and visible light remote sensing images of urban target areas. For example, when obtaining thermal remote sensing images of urban target areas, a remote sensing platform equipped with a thermal infrared sensor (such as Landsat 8 TIRS, ASTER or drone thermal imaging module) can be called to scan over the target area to obtain surface thermal radiation data; then the thermal radiation energy data is converted into a digital signal through the sensor to generate a thermal image. When obtaining visible light remote sensing images of urban target areas, a remote sensing platform equipped with an optical sensor (such as a high-resolution satellite or drone) can be called to obtain multispectral images of the target area, covering red light, green light, blue light and near-infrared bands, etc. In addition, in an embodiment of the present application, the acquisition time difference of thermal remote sensing images and visible light remote sensing images is ≤1 hour to reduce the impact of phase differences on subsequent analysis.

[0023] In one embodiment, after acquiring the target remote sensing image, a terrain correction step can be added to further improve the accuracy of the target remote sensing image. For example, the digital elevation model (DEM) of the target area can be first obtained to calculate the slope, slope aspect and terrain shielding coefficient; then a regression model of thermal radiation intensity and terrain factors can be established to eliminate the radiation difference between the sunny side and the shady side in mountainous cities; finally, the corrected thermal radiation intensity is normalized to generate a thermal distribution remote sensing image map that is independent of the terrain for subsequent segmentation. In this way, the radiation distortion problem caused by the slope can be solved.

[0024] Step S200, performing image segmentation on the first target remote sensing image according to the second target remote sensing image to obtain a first thermal zone and a second thermal zone, wherein the thermal radiation intensity of the first thermal zone is greater than the thermal radiation intensity of the second thermal zone; It can be understood that the first target remote sensing image contains the thermal radiation intensity information of different regions, and the second target remote sensing image contains the color, texture and other optical information of different regions.

[0025] It should be noted that the thermal radiation of buildings or roads and other related areas is greatly affected by light, and the thermal radiation intensity is relatively high; while the thermal radiation of vegetation or water areas and other related areas is less affected by light, and the thermal radiation intensity is relatively low. Therefore, the first target remote sensing image can be segmented into a first thermal zone and a second thermal zone with obvious differences in thermal radiation intensity through thermal radiation intensity information; for example, the thermal radiation intensity of the first thermal zone is more than twice that of the second thermal zone, so that buildings or roads, vegetation or water areas can be accurately distinguished.

[0026] In this way, the embodiment of the present application performs image segmentation on the first target remote sensing image to obtain the first thermal zone and the second thermal zone, and can distinguish between high radiation areas (buildings or roads) and low radiation areas (potential vegetation areas).

[0027] To further improve the accuracy of the image segmentation of the first target remote sensing image, in one embodiment, the step S200: performing image segmentation of the first target remote sensing image according to the second target remote sensing image to obtain the first thermal zone and the second thermal zone includes: S210, performing image pre-segmentation on the target remote sensing image to obtain a first area to be segmented and a second area to be segmented; S220. Correct the boundary lines of the first area to be segmented and the second area to be segmented according to the texture features of the second target remote sensing image to obtain a first thermal zone and a second thermal zone.

[0028] Specifically, firstly, a U-Net-based deep learning model can be used to pre-segment the remote sensing image (first target remote sensing image), and the deep learning model is trained by using historical remote sensing image data with thermal zone boundaries marked. Then, according to the texture features of the second target remote sensing image, the boundary lines of the first area to be segmented and the second area to be segmented are corrected to obtain the first thermal zone and the second thermal zone.

[0029] The U-Net-based deep learning model is used for image pre-segmentation, and the boundary line of the pre-segmentation result is corrected according to the texture features of the second target remote sensing image. This method has a significant effect in improving the accuracy of remote sensing image segmentation. This method can not only automatically generate high-precision segmentation results, but also adapt to complex scenes and improve the robustness and reliability of segmentation.

[0030] For example, the preprocessed thermal image is first input into the trained U-Net model to output a probability map (the probability that each pixel belongs to the high temperature zone). A binary mask is generated through threshold segmentation (the threshold is 0.5), which is initially divided into the first area to be segmented (high temperature area) and the second area to be segmented (low temperature area).

[0031] It can be understood that according to the Stefan-Boltzmann law, the radiation energy of an object is proportional to its temperature and the fourth power of its emissivity. Emissivity is an indicator of the radiation capacity of the surface of an object, ranging from 0 to 1. The higher the emissivity, the stronger the radiation energy. The emissivity of glass curtain walls is usually low (lower than the emissivity of vegetation). This means that at the same real temperature, the radiation energy of glass will be lower than that of vegetation, resulting in lower radiation values ​​displayed in thermal infrared images. Therefore, glass structures such as building curtain walls may show lower radiation values ​​in thermal infrared images, similar to the temperature characteristics of vegetation. Therefore, in response to the problem of misjudgment caused by the similarity of thermal radiation characteristics of glass curtain walls to vegetation, the embodiment of the present application introduces visible light remote sensing images to correct the boundary lines of the first area to be segmented and the second area to be segmented, thereby further improving the accuracy of image segmentation of the first target remote sensing image.

[0032] In one embodiment, the step S220: correcting the boundary lines of the first area to be segmented and the second area to be segmented according to the texture features of the second target remote sensing image to obtain the first thermal zone and the second thermal zone, comprises: S221, obtaining a texture feature change gradient at a boundary line between the first area to be segmented and the second area to be segmented; S222. When the gradient of the texture feature change is less than a gradient threshold, the boundary line between the first area to be segmented and the second area to be segmented is moved and corrected toward the direction of the second area to be segmented to obtain a first thermal zone and a second thermal zone.

[0033] Specifically, firstly, the texture feature change gradient at the boundary line of the first area to be segmented and the second area to be segmented is obtained. When the texture feature change gradient at the boundary line is less than the gradient threshold, it means that the boundary line is a glass feature (since the texture features of the glass surface are the same, the texture feature change gradient is small). At this time, the boundary line of the first area to be segmented and the second area to be segmented is moved toward the second area to be segmented, that is, the range of the second area to be segmented (low temperature area) is reduced, so that the curtain wall glass is separated from the potential vegetation area, thus improving the accuracy of the first target remote sensing image segmentation. In addition, in the process of moving the boundary line, the texture feature change gradient at the current boundary line is detected in real time; when the texture feature change gradient at the current boundary line is equal to the preset value or meets the preset condition (such as the texture feature change gradient at the current boundary line is close to the preset value), the boundary line movement is stopped and the current latest boundary line is used as the boundary of the first thermal zone and the second thermal zone.

[0034] For example, Figure 4As shown in the figure, S1 is the first area to be segmented, and S2 is the second area to be segmented. The initial pre-segmented boundary line is L1. At this time, if the boundary line L1 is judged to be a curtain wall glass feature through the visible light texture feature, that is, the second area to be segmented S2 contains the curtain wall glass area, then the curtain wall glass area needs to be removed from the second area to be segmented S2, that is, the boundary line is moved toward the direction of the second area to be segmented. If the visible light texture feature at the boundary line meets the predetermined requirements after the boundary moves to the L2 position, the boundary line at this time is a more accurate segmentation boundary line, thereby ensuring that the second area to be segmented only contains vegetation or water areas.

[0035] Step S300, analyzing the variation range of the thermal radiation intensity of the second thermal zone in different time periods, and performing image segmentation on the second thermal zone according to the variation range of the thermal radiation intensity to obtain a second main thermal zone and a secondary thermal zone, wherein the variation range of the thermal radiation intensity of the second main thermal zone is greater than the variation range of the thermal radiation intensity of the secondary thermal zone; It can be understood that after the first thermal zone and the second thermal zone are segmented, since the thermal radiation intensity of the vegetation area and the water area is relatively low, the second thermal zone may be a vegetation area or a water area. Based on this, in the embodiment of the present application, the second thermal zone is further segmented according to the variation range of the thermal radiation intensity to obtain the second main thermal zone and the secondary thermal zone.

[0036] It should be noted that, due to the large specific heat capacity of water, the temperature changes slowly, that is, the variation range of the thermal radiation intensity of water is relatively small compared to that of vegetation. Therefore, in the embodiment of the present application, the second thermal zone is image segmented according to the variation range of the thermal radiation intensity to obtain the second main thermal zone and the secondary thermal zone. The second main thermal zone with a large variation range of the thermal radiation intensity is the vegetation zone, and the secondary thermal zone with a small variation range of the thermal radiation intensity is the water zone.

[0037] Specifically, the embodiment of the present application can first collect the thermal radiation intensity data of the second thermal zone in different time periods (such as multiple time points in a day or the same time period for several consecutive days). These data can be obtained by calling the historical data or real-time data of the remote sensing platform. Then, the variation range of the thermal radiation intensity of the second thermal zone in these time periods is analyzed. The variation range can be measured by calculating the difference or percentage change of the thermal radiation intensity between adjacent time periods. For example, the average value of the thermal radiation intensity of each time period can be calculated, and then the difference or percentage change between the average values ​​of adjacent time periods can be calculated. Then, the image segmentation of the second thermal zone can be performed according to the variation range of the thermal radiation intensity, which can be achieved by setting a variation range threshold. The area where the variation range of the thermal radiation intensity is greater than the threshold is divided into the second main thermal zone, and the area where the variation range is less than or equal to the threshold is divided into the second secondary thermal zone.

[0038] Thus, through the above steps, the image of the second thermal zone can be effectively segmented according to the variation range of the thermal radiation intensity, so as to more accurately identify the vegetation coverage area.

[0039] Step S400: input the coverage of the second main thermal zone into a pre-established urban model network for area matching verification, wherein the urban model network includes building and water coverage information; Step S500: When the area matching check passes, the second main thermal zone is determined to be a vegetation covered area.

[0040] In one embodiment, the step S400: inputs the coverage of the second main thermal zone into a pre-established urban model network for area matching verification, including: calculating the overlapping area between the coverage of the second main thermal zone and the coverage of buildings and water areas in the urban model network; determining the confidence level based on the overlapping area and the coverage area of ​​the second main thermal zone; when the confidence level is greater than or equal to a first confidence threshold, determining that the area matching verification has passed, and determining the second main thermal zone as a vegetation coverage area.

[0041] Before the area matching check, a city model network containing building and water coverage information can be pre-established. This model network can be obtained through a variety of data sources such as remote sensing images, geographic information system (GIS) data, and urban planning data. Ensure that the spatial resolution and coordinate system of the city model network match the coverage of the second main thermal zone in order to perform accurate area matching verification. Among them, the city model network at least includes building and water coverage information.

[0042] During the area matching check, the coverage of the second main thermal zone is spatially superimposed with the urban model network. First, the overlapping area between the second main thermal zone and the building and water coverage in the urban model network is calculated. This can be achieved through spatial analysis tools in GIS software, such as intersection and union operations. Then, the confidence is calculated based on the overlapping area and the coverage area of ​​the second main thermal zone. The confidence is defined as a function of the ratio of the overlapping area to the coverage area of ​​the second main thermal zone, or measured by other appropriate statistical indicators. And a first confidence threshold is set to determine whether the area matching check is passed. This threshold can be set according to actual needs and historical data experience. If the calculated confidence is greater than or equal to the first confidence threshold, it is determined that the area matching check is passed. In this case, the second main thermal zone is determined to be a vegetation covered area. This is because the overlapping area between the second main thermal zone and the building and water coverage in the urban model network is small, indicating that the area is more likely to be a vegetation covered area. If the confidence is less than the first confidence threshold, the area matching check fails. In this case, further inspection and analysis can be carried out on the second main thermal zone, such as collecting more remote sensing data, adjusting the urban model network, or triggering manual verification signals for areas with low-confidence vegetation coverage. The manual verification signals are used to remind users to perform manual detection and identification to more accurately determine the coverage type of the area.

[0043] In other embodiments, the city model network can be dynamically updated by: accessing the real-time change data stream of the city geographic information system (GIS); the city model network is automatically updated in real time when the geographic information system (GIS) is updated to automatically mark the newly built buildings / water areas; in addition, when the conflicting area between the model network and the remote sensing image exceeds a threshold, an update request for the city model network is triggered to prevent the problem of automatic update failure. In this way, the efficiency of area matching verification can be improved.

[0044] In another embodiment, in order to further improve the accuracy of the vegetation coverage area determination, before determining the second main thermal zone as the vegetation coverage area, it also includes: performing sub-pixel decomposition on the mixed pixels in the second main thermal zone, and extracting the proportion of each component using the end member spectral library; if the proportion of the vegetation component exceeds the preset ratio, it is retained as a vegetation coverage area, otherwise it is marked as a to-be-verified area; and performing a secondary determination on the to-be-verified area through the second target remote sensing image.

[0045] It can be understood that the second main thermal zone is represented by an area with a large variation in thermal radiation intensity, and this feature also exists in vegetation areas and open space soil areas (open space areas not covered with vegetation). Therefore, in the embodiment of the present application, before determining the second main thermal zone as a vegetation covered area, the mixed pixels in the second main thermal zone are firstly subjected to sub-pixel decomposition, and the proportion of each component is extracted using the end member spectral library; if the proportion of the vegetation component exceeds the preset ratio, it is retained as a vegetation covered area, otherwise it is marked as an area to be verified; and then the second target remote sensing image is used to perform a secondary determination on the area to be verified.

[0046] Specifically, firstly, sub-pixel decomposition is performed for mixed pixels (i.e., pixels containing multiple types of land cover at the same time) in the second main thermal zone. Sub-pixel decomposition is a technique used to decompose mixed pixels into smaller sub-pixels representing a single type of land cover. Then, sub-pixel decomposition is performed using the endmember spectral library. The endmember spectral library contains the spectral characteristics of various types of land cover (such as vegetation, water bodies, soil, buildings, etc.). By comparing the spectral characteristics of the mixed pixel with the spectral characteristics in the endmember spectral library, the proportion of each component in the mixed pixel can be estimated. Based on the sub-pixel decomposition, the proportion of vegetation components is extracted. This can be achieved by calculating the contribution of vegetation spectral characteristics in the spectrum of the mixed pixel. A preset ratio is set to determine the dominant position of vegetation components in the mixed pixel. This preset ratio can be set according to actual needs and historical data experience. If the proportion of vegetation components exceeds the preset ratio, the area is retained as a vegetation-covered area. This is because the main land cover type in the area is vegetation. If the proportion of vegetation components does not exceed the preset ratio, the area is marked as a pending verification area. This is because the area may contain other types of surface cover, or the vegetation cover is not dense enough, and further judgment is needed. Finally, the second target remote sensing image is used to make a second judgment on the area to be verified. The second target remote sensing image can be a different data source, acquired at a different time, or a different spectral band from the initial remote sensing image. By analyzing the spectral characteristics, texture characteristics and other information in the second target remote sensing image, the surface cover type of the area to be verified is further determined. According to the results of the second judgment, the area to be verified is finally classified and determined as a vegetation covered area or other surface cover type.

[0047] Thus, through the above steps, the vegetation coverage area in the second main thermal zone can be determined more accurately. The use of sub-pixel decomposition and end-member spectral library improves the accuracy of mixed pixel decomposition, and the secondary determination further reduces the possibility of misjudgment, thereby improving the accuracy and reliability of vegetation coverage identification.

[0048] In one embodiment, the second target remote sensing image is used to perform a secondary determination on the area to be verified, including: performing multispectral analysis on the visible light remote sensing image of the area to be verified, extracting the red light band reflectance Rred and near-infrared reflectivity R nir ; The red light band reflectivity R red and near-infrared reflectivity R nir The vegetation index model is input to obtain the vegetation index value N; wherein the vegetation index model satisfies the following expression: If the vegetation index value N ≥ T, it is marked as a candidate vegetation area, otherwise it is marked as a non-vegetation area, where T is a dynamic threshold, which is set to 0.4 in summer and 0.3 in winter.

[0049] It can be understood that vegetation has a significant difference in reflectivity in these two bands due to the strong absorption of red light band (Rred) by chlorophyll and high reflectivity of near-infrared band (Rnir) by mesophyll cell structure. Through the design of normalized vegetation index model, the vegetation coverage can be quantitatively characterized. The larger the vegetation index value (N), the higher the possibility of vegetation coverage.

[0050] It should be noted that vegetation growth is affected by seasons. In summer, vegetation is lush, and the difference between red light absorption and near-infrared reflection is more obvious, so a higher threshold (0.4) is required for screening. In winter, vegetation withers and spectral characteristics weaken, so the threshold (0.3) is lowered to adapt to seasonal changes and improve recognition accuracy. In other seasons, such as spring and autumn, the threshold can be set according to actual conditions.

[0051] Based on this, the urban remote sensing image vegetation coverage identification method based on deep learning provided in the embodiment of the present application first obtains the thermal remote sensing image and the visible light remote sensing image of the urban target area, and uses the characteristics of the visible light image to accurately segment the thermal image to distinguish the high radiation area (buildings or roads) from the low radiation area (potential vegetation area); and then further screens out the specific thermal areas with significant dynamic changes in thermal radiation from the low radiation area through multi-phase thermal radiation change analysis. Finally, the coverage range of the specific thermal area is matched and verified with the building and water coverage information in the urban model network, so as to accurately determine the vegetation coverage area. In this way, the problem of low accuracy of single data source judgment and recognition in the prior art is effectively solved through multi-source data complementation, dynamic feature mining and spatial constraint verification, and the accuracy of vegetation coverage recognition is improved, providing strong support for urban ecological environment monitoring and management.

[0052] like Figure 5 As shown, Figure 5This is a hardware structure diagram of a vegetation cover identification system in some embodiments of the present application. The vegetation cover identification system provided in the embodiments of the present application includes a memory 1000 and a processor 2000, wherein the memory 1000 is used to store computer-readable instructions, and the processor 2000 is used to call the computer-readable instructions to execute the urban remote sensing image vegetation cover identification method based on deep learning as described above.

[0053] Among them, the processor 2000 is used to provide computing and control capabilities to control the vegetation cover recognition system to perform corresponding tasks, for example, to control the vegetation cover recognition system to perform the urban remote sensing image vegetation cover recognition method based on deep learning in any of the above method embodiments, the method comprising: respectively obtaining a thermal remote sensing image and a visible light remote sensing image of an urban target area to obtain a first target remote sensing image and a second target remote sensing image; performing image segmentation on the first target remote sensing image according to the second target remote sensing image to obtain a first thermal zone and a second thermal zone, wherein the thermal radiation intensity of the first thermal zone is greater than the thermal radiation intensity of the second thermal zone; analyzing the variation range of the thermal radiation intensity of the second thermal zone in different time periods, and performing image segmentation on the second thermal zone according to the variation range of the thermal radiation intensity to obtain a second main thermal zone and a secondary thermal zone, wherein the variation range of the thermal radiation intensity of the second main thermal zone is greater than the variation range of the thermal radiation intensity of the second thermal zone; inputting the coverage range of the second main thermal zone into a pre-established urban model network for area matching verification, the urban model network including building and water coverage information; if the area matching verification passes, the second main thermal zone is determined as a vegetation coverage area.

[0054] The processor 2000 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), a hardware chip, or any combination thereof; it may also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a programmable logic device (PLD), or any combination thereof. The above-mentioned PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0055] The memory 1000, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as the program instructions / modules corresponding to the urban remote sensing image vegetation cover recognition method based on deep learning in the embodiment of the present application. The processor 2000 can implement the urban remote sensing image vegetation cover recognition method based on deep learning in any of the above method embodiments by running the non-transitory software programs, instructions and modules stored in the memory 1000.

[0056] Specifically, the memory 1000 may include a volatile memory (VM), such as a random access memory (RAM); the memory 1000 may also include a non-volatile memory (NVM), such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD) or other non-transitory solid-state storage device; the memory 1000 may also include a combination of the above-mentioned types of memory.

[0057] To sum up, the vegetation cover identification system of the present application adopts the technical solution of any one of the above-mentioned urban remote sensing image vegetation cover identification method embodiments based on deep learning. Therefore, it has at least the beneficial effects brought by the technical solutions of the above-mentioned embodiments, which will not be repeated here one by one.

[0058] The present application also provides a computer-readable storage medium, such as a memory including a program code, and the program code can be executed by a processor to complete the urban remote sensing image vegetation coverage identification method based on deep learning in the above embodiment. For example, the computer-readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CDROM), a magnetic tape, a floppy disk, and an optical data storage device.

[0059] The present application also provides a computer program product, which includes one or more program codes stored in a computer-readable storage medium. The processor of the vegetation cover recognition system reads the program code from the computer-readable storage medium, and the processor executes the program code to complete the steps of the urban remote sensing image vegetation cover recognition method based on deep learning provided in the above embodiment.

[0060] A person skilled in the art will appreciate that all or part of the steps for implementing the above embodiments may be accomplished by hardware or by hardware associated with a program code, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0061] It should be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0062] Through the description of the above embodiments, it can be clearly understood by those skilled in the art that each embodiment can be implemented by means of software plus a general hardware platform, or by hardware. It can be understood by those skilled in the art that all or part of the processes in the above embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0063] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. All equivalent structural changes made by using the contents of the present invention specification and drawings under the inventive concept of the present invention, or directly / indirectly applied in other related technical fields are included in the patent protection scope of the present invention.

Claims

1. A method for identifying vegetation cover in urban remote sensing images based on deep learning, characterized in that: The method comprises: Acquire thermal remote sensing images and visible light remote sensing images of the urban target area to obtain a first target remote sensing image and a second target remote sensing image; Performing image segmentation on the first target remote sensing image according to the second target remote sensing image to obtain a first thermal zone and a second thermal zone, wherein the thermal radiation intensity of the first thermal zone is greater than the thermal radiation intensity of the second thermal zone; Analyze the variation range of the thermal radiation intensity of the second thermal zone in different time periods, and perform image segmentation on the second thermal zone according to the variation range of the thermal radiation intensity to obtain a second main thermal zone and a secondary thermal zone, wherein the variation range of the thermal radiation intensity of the second main thermal zone is greater than the variation range of the thermal radiation intensity of the secondary thermal zone; Inputting the coverage of the second main thermal zone into a pre-established urban model network for area matching verification, wherein the urban model network includes building and water coverage information; When the area matching check passes, the second main thermal zone is determined to be a vegetation covered area.

2. The urban remote sensing image vegetation coverage identification method based on deep learning according to claim 1, characterized in that: The step of performing image segmentation on the first target remote sensing image according to the second target remote sensing image to obtain the first thermal zone and the second thermal zone includes: Performing image pre-segmentation on the first target remote sensing image to obtain a first area to be segmented and a second area to be segmented; According to the texture features of the second target remote sensing image, the boundary lines of the first area to be segmented and the second area to be segmented are corrected to obtain the first thermal zone and the second thermal zone.

3. The urban remote sensing image vegetation coverage identification method based on deep learning as claimed in claim 2, characterized in that: The image pre-segmentation adopts a U-Net-based deep learning model, and the deep learning model is trained by historical remote sensing image data marked with thermal zone boundaries.

4. The urban remote sensing image vegetation coverage identification method based on deep learning as claimed in claim 2, characterized in that: The step of correcting the boundary lines of the first area to be segmented and the second area to be segmented according to the texture features of the second target remote sensing image to obtain the first thermal zone and the second thermal zone includes: Acquire a texture feature change gradient at a boundary line between the first area to be segmented and the second area to be segmented; When the gradient of the texture feature change is less than the gradient threshold, the boundary line between the first area to be segmented and the second area to be segmented is moved and corrected toward the second area to be segmented to obtain a first thermal zone and a second thermal zone.

5. The urban remote sensing image vegetation coverage identification method based on deep learning as claimed in claim 4, characterized in that: The step of moving and correcting the boundary line between the first area to be segmented and the second area to be segmented toward the second area to be segmented to obtain the first thermal zone and the second thermal zone includes: In the process of moving the boundary line, the gradient of texture feature change at the current boundary line is detected in real time; When the texture feature change gradient at the current boundary line is equal to a preset value or meets a preset condition, the boundary line movement is stopped and the current latest boundary line is used as the boundary between the first thermal zone and the second thermal zone.

6. The urban remote sensing image vegetation coverage identification method based on deep learning as claimed in claim 1, characterized in that: Before determining the second main thermal zone as a vegetation covered area, the method further includes: Performing sub-pixel decomposition on the mixed pixels in the second main thermal zone, and extracting the proportion of each component using the endmember spectral library; If the proportion of vegetation components exceeds the preset ratio, it will be retained as a vegetation-covered area, otherwise it will be marked as an area to be verified; A secondary determination is made on the area to be verified using the second target remote sensing image.

7. The urban remote sensing image vegetation coverage identification method based on deep learning according to claim 6, characterized in that: The second determination of the area to be verified by using the second target remote sensing image includes: Perform multispectral analysis on the visible light remote sensing image of the area to be verified and extract the red light band reflectance R red and near-infrared reflectivity R nir ; The red light band reflectivity R red and near-infrared reflectivity R nir The vegetation index model is input to obtain the vegetation index value N; wherein the vegetation index model satisfies the following expression: If the vegetation index value N ≥ T, it is marked as a candidate vegetation area, otherwise it is marked as a non-vegetation area, where T is a dynamic threshold, which is set to 0.4 in summer and 0.3 in winter.

8. The urban remote sensing image vegetation coverage identification method based on deep learning as claimed in claim 1, characterized in that: The step of inputting the coverage of the second main thermal zone into a pre-established urban model network for area matching verification comprises: Calculate the overlapping area between the coverage of the second main thermal zone and the coverage of buildings and water areas in the urban model network; Determining a confidence level based on the overlapping area and the coverage area of ​​the second main thermal zone; When the confidence level is greater than or equal to the first confidence level threshold, it is determined that the area matching check has passed, and the second main thermal zone is determined to be a vegetation covered area.

9. The urban remote sensing image vegetation coverage identification method based on deep learning as claimed in claim 8, characterized in that: After determining the confidence level according to the overlapping area and the coverage area of ​​the second main thermal zone, the method further includes: When the confidence is greater than or equal to the second confidence threshold and less than the first confidence threshold, determining that the area matching check fails, and determining the second main thermal zone as a low-confidence vegetation coverage area, wherein the second confidence threshold is less than the first confidence threshold; A manual verification signal is triggered for the low-credible vegetation coverage area, and the manual verification signal is used to remind the user to perform manual detection and identification.

10. A vegetation cover identification system, characterized in that: include: A memory and a processor, wherein the memory is used to store program codes; The processor is used to call the program code to execute the method according to any one of claims 1 to 9.

Images