Hilly and mountainous area remote sensing image segmentation method based on CABR-ResUNet model

Abstract

The invention discloses a hilly and mountainous area remote sensing image segmentation method based on a CABR-ResUNet model, and relates to the technical field of image segmentation, and the method comprises the steps: obtaining a hilly and mountainous area electronic distribution map through employing a GIS and surveying and mapping data, and carrying out the multi-angle image collection through employing an unmanned plane carrying multiple sensors; preprocessing the image data, and generating a high-precision digital surface model based on LiDAR point cloud; carrying out ground feature labeling and time series data enhancement in combination with multi-source data, and using a CABR-ResUNet model to improve the perception ability for complex terrains and multi-scale targets; calculating a vegetation coverage influence coefficient and a crack influence coefficient, comparing thresholds, judging the influence on image segmentation, optimizing model weight and enhancing feature extraction; residual connection, a double attention mechanism, multi-scale feature extraction and boundary refinement technologies are combined, the perception ability of the model to a complex terrain and the adaptability to a multi-scale target are improved, and the edge segmentation precision is optimized; and outputting a high-precision image segmentation result.

Description

Technical Field

[0001] The present invention relates to the technical field of image segmentation, and specifically to a method for segmenting remote sensing images of hilly and mountainous areas based on the CABR-ResUNet model. Background Art

[0002] In the task of land object segmentation of remote sensing images driven by deep learning, the performance of the model directly depends on the quality and scale of the training data. However, in hilly and mountainous areas, the terrain is complex, the types of land objects are numerous, and the vegetation cover and geomorphic features are intertwined, which restricts the acquisition of remote sensing images. For example, there are limitations in single data sources, resolution limitations, insufficient terrain perception ability, lack of crack and slope information, lack of multi-temporal hero correction, and the influence of vegetation cover. As a result, the precise segmentation of remote sensing images has become a very challenging task.

[0003] Traditional image segmentation methods, such as threshold segmentation, edge detection, region growing, etc., usually rely on manually setting parameters and cannot fully adapt to the complex and changeable geomorphic features of hilly and mountainous areas. Especially under the influence of vegetation cover and mountain cracks, conventional methods often have difficulty maintaining a high segmentation accuracy. In addition, existing deep learning methods, such as FCN, U-Net, etc., are prone to problems such as detail loss and blurred boundaries when processing remote sensing images of complex terrains, resulting in inaccurate identification of target areas and even mis-segmentation or missed-segmentation phenomena. Summary of the Invention

[0004] Aiming at the deficiencies of the prior art, the present invention provides a method for segmenting remote sensing images of hilly and mountainous areas based on the CABR-ResUNet model to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention is realized through the following technical solutions: A method for segmenting remote sensing images of hilly and mountainous areas based on the CABR-ResUNet model, including the following steps:

[0006] Step 1: Use GIS technology and surveying and mapping data to obtain an electronic distribution map of hilly and mountainous areas, and divide it into several monitoring areas; use a drone equipped with an RGB visible light camera, a multi-spectral camera, a hyperspectral camera, a thermal infrared camera, and a LiDAR sensor to collect multi-angle images, and obtain images from top-down, oblique, and low-altitude angles respectively, and identify basic land objects, terrain, hillsides, cracks, vegetation cover, and shadow information to complete multi-temporal image collection;

[0007] Step 2: Perform spatio-temporal filtering on remotely sensed images obtained at different time points to eliminate noise and perform atmospheric correction, shadow removal, and illumination normalization; then crop the images to remove irrelevant backgrounds and retain the image data within the study area; next, use point cloud image registration and multi-view image registration techniques to perform geometric correction and elevation matching to generate a high-precision digital surface model DSM; finally, use sparse representation fusion technology to perform spectral fusion and feature extraction on multi-spectral and hyperspectral images to obtain the final preprocessed image data;

[0008] Step 3: Based on the preprocessed image data and multi-source data including visible light, multi-spectral, LiDAR, and thermal infrared, perform ground object annotation, and use semi-automatic label-assisted technology to construct a multi-layer label system covering basic classification, enhanced features, and temporal changes;

[0009] Step 4: Use multi-scale cropping to match basic ground objects, terrain, hillslopes, cracks, and vegetation features, and introduce temporal data augmentation to combine the robustness of different time image enhancement models to illumination changes;

[0010] Step 5: Build a CABR-ResUNet model, combine residual connections, CBAM dual attention mechanisms, ASPP multi-scale feature extraction, and BRN boundary refinement techniques to enhance the model's perception ability of complex terrain, adaptability to multi-scale targets, and optimize edge segmentation accuracy;

[0011] Step 6: Use multi-spectral and thermal infrared remote sensing technologies to monitor vegetation coverage, moisture content, and surface temperature, calculate the vegetation coverage influence coefficient ZBX, and compare it with the preset threshold Q1 to determine whether vegetation coverage has an impact on image segmentation. If there is an impact, use CBAM to strengthen the features of the vegetation area;

[0012] Step 7: Use LiDAR point cloud data and digital surface model DSM, combine histogram of oriented gradients HOG and Gabor texture analysis techniques to monitor the impact of cracks on image segmentation in real time, calculate the crack influence coefficient LFX, and compare it with the preset threshold Q2 to determine whether cracks have an impact on image segmentation. If there is an impact, activate the BoundaryLoss weight, increase the weighted values of HOG and Gabor, strengthen the analysis of slope changes and texture features, and improve crack detection accuracy;

[0013] Step 8: Optimize the CABR-ResUNet model, combine an adaptive loss function to dynamically adjust weights, use cross-entropy loss, Dice loss, and Focal loss to improve segmentation accuracy; set activation functions, optimizers, and multi-task loss functions to perform model training, and perform accuracy evaluation through evaluation metrics such as confusion matrices; finally, use MSF multi-scale fusion and IEP ignore edge prediction methods to output the final image segmentation result.

[0014] Preferably, step one includes:

[0015] S11. Using GIS geographic information system technology and related surveying and mapping data, obtain an electronic distribution map of the hilly mountainous area; divide the hilly mountainous area into several monitoring areas, and mark the several monitoring areas on the electronic distribution map of the hilly mountainous area as: Jcq1, Jcq2, ..., Jcq n ; n represents the number of monitoring areas.

[0016] Preferably, step one further includes:

[0017] S12. Use a drone equipped with multi-sensor remote sensing equipment including an RGB visible light camera, a multi-spectral camera, a hyperspectral camera, a thermal infrared camera, and a LiDAR sensor to collect multi-angle images of the monitoring area; the first round of image collection uses a standard overhead angle of 90° to obtain vertically high-resolution remote sensing images and identify the basic ground features of the monitoring area; the second round of image collection uses an inclined angle of 45°, adjust the attitude of the drone, collect oblique images, and perform three-dimensional perception of the terrain, hillside, and cracks in the monitoring area; the third round of image collection uses a low-altitude flight at a distance of 15m - 30m to collect the mountain slope and vegetation coverage area of the monitoring area and identify the mountain coverage area; and use multi-temporal image collection to perform image recognition on the shadows of the hilly mountainous area in the monitoring area at different time periods including early morning, noon, and dusk.

[0018] Preferably, step two includes:

[0019] S21. For the remote sensing images obtained at different time points, use spatio-temporal filtering technology to eliminate temporal noise, perform atmospheric correction and shadow removal, and perform illumination normalization;

[0020] S22. Crop the remote sensing image data after atmospheric correction and shadow removal, remove irrelevant background areas, and retain the image data within the research area;

[0021] S23. Adopt point cloud image registration technology to perform geometric correction and elevation information matching on the optical remote sensing image and the LiDAR point cloud image to obtain a high-precision digital surface model DSM. Use multi-view image registration technology to perform geometric correction and radiometric equalization on the remote sensing images obtained at different angles. Adopt sparse representation fusion technology to perform spectral information fusion on multi-spectral and hyperspectral images, extract high-dimensional spectral features, and remove redundant information to finally obtain preprocessed image data.

[0022] Preferably, step three includes:

[0023] S31. Based on the preprocessed image data, combined with multi-source data including visible light, multispectral, LiDAR, and thermal infrared, conduct ground object annotation;

[0024] S32. Adopt semi-automatic label assistance technology to construct a multi-layer label system, including basic classification, enhanced features, and temporal changes.

[0025] Preferably, step four includes:

[0026] S41. Adopt multi-scale cropping including 256*256, 512*512, and 1024*1024 to match the features of basic ground objects, terrain, hillsides, cracks, and vegetation-covered ground objects;

[0027] S42. Introduce temporal data augmentation, pair different time images including early morning, noon, and dusk, and enhance the robustness of the model to light changes.

[0028] Preferably, step five includes:

[0029] S51. Construct a CABR-ResUNet model: Based on the U-Net network, combine the residual connection ResidualConnection to improve the deep feature transfer ability, adopt the CBAM dual attention mechanism, enhance the attention to important feature regions through channel attention ChannelAttention and spatial attention SpatialAttention, and improve the model's perception ability of complex terrain; Introduce the ASPP multi-scale feature extraction module, use dilated convolutions with different dilation rates to capture ground object information at different scales, and enhance the model's adaptability to multi-scale targets in hilly and mountainous areas; Adopt the BRN boundary refinement technology to optimize the edge information in the decoding stage, and improve the accuracy of the segmentation result through feature enhancement and edge constraint, reducing edge blur and misjudgment.

[0030] Preferably, step six includes:

[0031] S61. Through multi-spectral remote sensing technology and thermal infrared remote sensing technology, monitor the vegetation coverage degree, vegetation water content, terrain slope, and surface temperature in real time. After dimensionless processing, calculate and obtain the vegetation coverage influence coefficient ZBX, and the formula is as follows:

[0032] ZBX = w1*Fndvi + w2*Fsavi + w3*Fθ + w4*Flst + w5*Fnir + w6*Fswir;

[0033] In the formula, Fndvi represents the normalized index, Fsavi represents the soil-adjusted vegetation index, Fθ represents the influence factor of slope terrain on vegetation growth, Flst represents the land surface temperature, Fnir represents the near-infrared reflectance, reflecting the vegetation density, Fswir represents the short-wave infrared reflectance, reflecting the vegetation water status, and w1, w2, w3, w4, w5, and w6 represent the weight coefficients;

[0034]

[0035] In the formula, NIR represents the reflectance value in the near-infrared band, and Red represents the reflectance value in the red band;

[0036]

[0037] In the formula, L represents the adjustment factor, where 0 < L < 1;

[0038]

[0039] In the formula, h represents the elevation, x represents the horizontal component of the slope, and y represents the vertical component of the slope;

[0040] S62. By presetting the first threshold Q1 in advance and comparing and analyzing the vegetation coverage influence coefficient ZBX with the first threshold Q1, the first evaluation result is obtained, including:

[0041] When the vegetation coverage influence coefficient ZBX < the first threshold Q1, it means that the vegetation coverage has no influence on the image segmentation, and continuous monitoring is carried out;

[0042] When the vegetation coverage influence coefficient ZBX ≥ the first threshold Q1, it means that the vegetation coverage has an influence on the image segmentation, triggering the first warning instruction and generating the first strategy: adjusting the image contrast and using CBAM to further strengthen the features of the vegetation coverage area.

[0043] Preferably, step seven includes:

[0044] S71. By combining LiDAR point cloud data and digital surface model DSM, using the histogram of oriented gradients HOG and Gabor texture analysis technology, the influence of cracks on image segmentation is monitored in real time. After dimensionless processing, the crack influence coefficient LFX is calculated and obtained. The formula is as follows:

[0045]

[0046] In the formula, represents the gradient change of the slope in the x direction, It represents the gradient change of the slope in the y direction. HOG represents the eigenvalue extracted from the histogram of gradient directions. Gabor represents the texture feature extracted by the Gabor filter. Boundary represents the boundary enhancement loss, which optimizes the boundary segmentation accuracy of the crack area. a1, a2, a3, and a4 represent weight coefficients;

[0047] S72. By presetting the second threshold Q2 in advance and comparing and analyzing the crack influence coefficient LFX with the second threshold Q2, the second evaluation result is obtained, including:

[0048] When the crack influence coefficient LFX < the second threshold Q2, it indicates that the crack has no influence on the image segmentation, and continuous monitoring is carried out;

[0049] When the crack influence coefficient LFX ≥ the second threshold Q2, it indicates that the crack has an influence on the image segmentation, triggering the second warning instruction and generating the second strategy: activating the weight of BoundaryLoss, increasing the weighted values of HOG and Gabor, strengthening the analysis of slope change and texture features, further enhancing the accuracy of crack detection, and ensuring the accurate identification of crack features.

[0050] Preferably, step eight includes:

[0051] S81. Further optimize the performance of the CABR-ResUNet model, combine the adaptive loss function mechanism, dynamically adjust the weights, adopt the classification cross-entropy loss, Dice loss, and Focal loss to improve the class balance and overall segmentation accuracy; on this basis, set the training parameters through the activation function, optimizer, and multi-task loss function of the segmentation task, conduct model training, and evaluate the model accuracy through the confusion matrix and other evaluation metrics; finally, use the MSF multi-scale fusion and IEP ignore edge prediction methods to predict the segmentation result of the high-resolution remote sensing image, and adjust the model according to the accuracy requirements to output the final segmentation result.

[0052] The present invention provides a method for segmenting remote sensing images in hilly and mountainous areas based on the CABR-ResUNet model. It has the following

[0053] Beneficial effects:

[0054] (1) For the method for segmenting remote sensing images in hilly and mountainous areas based on the CABR-ResUNet model, by combining the CABR-ResUNet model with the CBAM attention mechanism, ASPP multi-scale feature extraction, and BRN boundary refinement technology, it enhances the model's perception ability of complex ground object features in hilly and mountainous areas, improves the adaptability to multi-scale targets, optimizes the segmentation accuracy of edge details at the same time, and reduces the phenomenon of mis-segmentation and missed-segmentation in the target area.

[0055] (2) The remote sensing image segmentation method for hilly and mountainous areas based on the CABR-ResUNet model combines multi-temporal image acquisition, spatio-temporal filtering, atmospheric correction, shadow removal, and illumination normalization processing to reduce the impact of illumination changes, shadow interference, and atmospheric conditions on remote sensing image segmentation. At the same time, temporal data augmentation is adopted to improve the adaptability of the model to remote sensing data at different time periods and enhance the stability and consistency of image segmentation.

[0056] (3) The remote sensing image segmentation method for hilly and mountainous areas based on the CABR-ResUNet model calculates the vegetation coverage influence coefficient (ZBX) and the crack influence coefficient (LFX) to evaluate the impact of vegetation and cracks on image segmentation in real time, and adopts the method of CBAM to strengthen vegetation features or activate BoundaryLoss to improve the crack detection accuracy, ensuring the accurate segmentation of vegetation and crack areas and optimizing the detection effect of specific ground objects.

[0057] (4) The remote sensing image segmentation method for hilly and mountainous areas based on the CABR-ResUNet model adopts an adaptive loss function mechanism, combines categorical cross-entropy loss, Dice loss, and Focal loss to achieve category balance optimization and segmentation accuracy improvement. At the same time, through the MSF multi-scale fusion and IEP ignore edge prediction methods, the quality of the segmentation results in the inference stage is improved, the consumption of computing resources is reduced, and the model inference speed is accelerated. Description of the Drawings

[0058] Figure 1 It is a schematic diagram of the steps of a remote sensing image segmentation method for hilly and mountainous areas based on the CABR-ResUNet model of the present invention. Detailed Embodiments

[0059] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0060] Embodiment 1

[0061] Please refer to Figure 1 , the present invention provides a remote sensing image segmentation method for hilly and mountainous areas based on the CABR-ResUNet model, including the following steps:

[0062] Step 1: Use GIS technology and surveying data to obtain an electronic distribution map of hilly and mountainous areas. The electronic distribution map is generated based on GIS technology and surveying data, and is a digital map containing topographic features, road networks, ground object distributions, and slope aspect and gradient spatial information. It is used to divide the hilly and mountainous areas into several monitoring areas before remote sensing image acquisition to enable the orderly acquisition of multi-source images by drones. Use drones equipped with RGB visible light cameras, multi-spectral cameras, hyperspectral cameras, thermal infrared cameras, and LiDAR sensors to collect multi-angle images, obtaining images from top-down, oblique, and low-altitude angles respectively, and identifying basic ground objects, terrain, hillsides, cracks, vegetation cover, and shadow information to complete multi-temporal image acquisition;

[0063] Step 2: Perform spatio-temporal filtering on the remote sensing images obtained at different time points to eliminate noise and perform atmospheric correction, shadow removal, and illumination normalization; then crop the images to remove irrelevant backgrounds and retain the image data within the study area; next, use point cloud image registration and multi-view image registration techniques to perform geometric correction and elevation matching to generate a high-precision digital surface model DSM; finally, use sparse representation fusion technology to perform spectral fusion and feature extraction on multi-spectral and hyperspectral images to obtain the final preprocessed image data;

[0064] Step 3: Based on the preprocessed image data and multi-source data including visible light, multi-spectral, LiDAR, and thermal infrared, perform ground object annotation, and use semi-automatic label assistance technology to construct a multi-layer label system covering basic classification, enhanced features, and temporal changes;

[0065] Step 4: Use multi-scale cropping to match basic ground objects, terrain, hillsides, cracks, and vegetation features, and introduce temporal data augmentation to combine the robustness of different time image enhancement models to illumination changes;

[0066] Step 5: Construct a CABR-ResUNet model, combining residual connection, CBAM dual attention mechanism, ASPP multi-scale feature extraction, and BRN boundary refinement technology to improve the model's perception ability of complex terrain, adaptability to multi-scale targets, and optimize edge segmentation accuracy;

[0067] Step 6: Use multi-spectral and thermal infrared remote sensing technologies to monitor vegetation cover degree, moisture content, and surface temperature, calculate the vegetation cover impact coefficient ZBX, and compare it with the preset threshold Q1 to determine whether the vegetation cover has an impact on image segmentation. If there is an impact, use CBAM to strengthen the vegetation area features;

[0068] Step 7: Utilize LiDAR point cloud data and digital surface model DSM, combine Histogram of Oriented Gradients (HOG) and Gabor texture analysis techniques to monitor the impact of cracks on image segmentation in real-time, calculate the crack influence coefficient LFX, and compare it with the preset threshold Q2 to determine whether the cracks have an impact on image segmentation. If there is an impact, activate the BoundaryLoss weight, increase the weighted values of HOG and Gabor, strengthen the analysis of slope changes and texture features, and improve the crack detection accuracy.

[0069] Step 8: Optimize the CABR-ResUNet model, dynamically adjust the weights by combining adaptive loss functions, use cross-entropy loss, Dice loss, and Focal loss to improve the segmentation accuracy; set the activation function, optimizer, and multi-task loss function, perform model training, and evaluate the accuracy through evaluation metrics such as the confusion matrix; finally, use the multi-scale fusion of MSF and the edge prediction method of IEP to output the final image segmentation result.

[0070] In this embodiment, the present invention introduces the CABR-ResUNet model, combines residual connection, dual attention mechanism CBAM, ASPP multi-scale feature extraction, and BRN boundary refinement technology, which can effectively improve the segmentation accuracy of remote sensing images in complex hilly mountainous areas, enable the model to have stronger object perception ability in areas with large terrain undulations, optimize the edge segmentation effect at the same time, reduce the phenomena of mis-segmentation and missed segmentation, and improve the overall segmentation accuracy.

[0071] Embodiment 2

[0072] This embodiment is an explanatory description based on Embodiment 1. Specifically, Step 1 includes:

[0073] S11: Utilize GIS (Geographic Information System) technology and relevant surveying and mapping data to obtain the electronic distribution map of the hilly mountainous area; divide the hilly mountainous area into several monitoring areas, and mark the several monitoring areas on the electronic distribution map of the hilly mountainous area as: Jcq1, Jcq2,..., Jcq n ; n represents the number of monitoring areas.

[0074] In this embodiment, the present invention divides the hilly mountainous area through GIS technology to obtain an electronic distribution map and marks multiple monitoring areas, enabling the remote sensing image acquisition and analysis to have a clear spatial reference, and improving the standardization and accuracy of data management.

[0075] Embodiment 3

[0076] This embodiment is an explanatory description based on Embodiment 1. Specifically, Step 1 further includes:

[0077] S12. Use a drone to carry a multi-sensor remote sensing device including an RGB visible light camera, a multi-spectral camera, a hyperspectral camera, a thermal infrared camera, and a LiDAR sensor to collect multi-angle images of the monitoring area; for the first round of image collection, use a standard overhead angle of 90° to obtain vertically high-resolution remote sensing images and identify the basic ground features in the monitoring area; for the second round of image collection, use an oblique angle of 45°, adjust the attitude of the drone, and collect oblique images to perform three-dimensional perception of the terrain, hillsides, and cracks in the monitoring area; for the third round of image collection, use low-altitude flight at a distance of 15m - 30m to collect the mountain slopes and vegetation-covered areas in the monitoring area and identify the mountain coverage; and use multi-temporal image collection to perform image recognition on the shadows in the hilly mountainous areas of the monitoring area at different time periods, including early morning, noon, and dusk.

[0078] In this embodiment, the present invention uses a drone to carry a multi-sensor remote sensing device to perform multi-angle and multi-temporal image collection, which can not only comprehensively obtain the topographic and geomorphic information of hilly mountainous areas, but also effectively overcome the influence of light changes and shadow interference, improve the accuracy and robustness of remote sensing image segmentation, and provide high-quality data support for the precise analysis of complex terrain areas.

[0079] Embodiment 4

[0080] This embodiment is an explanatory description based on Embodiment 1. Specifically, Step 2 includes:

[0081] S21. For the remote sensing images obtained at different time points, use spatio-temporal filtering technology to eliminate temporal noise, perform atmospheric correction and shadow removal, and perform light normalization.

[0082] S22. Crop the remote sensing image data after atmospheric correction and shadow removal, remove irrelevant background areas, and retain the image data within the study area.

[0083] S23. Adopt point cloud image registration technology to perform geometric correction and elevation information matching on optical remote sensing images and LiDAR point cloud images to obtain a high-precision digital surface model DSM. Use multi-view image registration technology to perform geometric correction and radiation equalization on remote sensing images obtained at different angles. Adopt sparse representation fusion technology to perform spectral information fusion on multi-spectral and hyperspectral images, extract high-dimensional spectral features, and remove redundant information to finally obtain preprocessed image data.

[0084] In this embodiment, through spatio-temporal filtering, atmospheric correction, shadow removal, and light normalization processing, the influence of noise and light changes in remote sensing images is effectively eliminated, and combined with point cloud image registration and spectral fusion technology, the geometric accuracy and spectral information integrity of the images are improved, thus providing high-quality preprocessed data for subsequent precise segmentation of ground features.

[0085] Example 5

[0086] This example is an explanatory note based on Example 1. Specifically, Step 3 includes:

[0087] S31. Based on the preprocessed image data, combined with multi-source data including visible light, multispectral, LiDAR, and thermal infrared, perform ground object annotation;

[0088] S32. Adopt semi-automatic label assistance technology to construct a multi-layer label system, including basic classification, enhanced features, and temporal changes.

[0089] In this example, by fusing visible light, multispectral, LiDAR, and thermal infrared data for ground object annotation and introducing semi-automatic label assistance technology to construct a multi-layer label system, the ground object classification is made more accurate, which not only improves the annotation efficiency but also enhances the model's recognition ability for different ground object features and their temporal changes.

[0090] Example 6

[0091] This example is an explanatory note based on Example 1. Specifically, Step 4 includes:

[0092] S41. Adopt multi-scale cropping including optional 256*256, 512*512, and 1024*1024 to match the features of basic ground objects, terrain, hillsides, cracks, and vegetation-covered ground objects;

[0093] S42. Introduce temporal data augmentation, pair with images at different times including early morning, noon, and dusk, and enhance the model's robustness to light changes.

[0094] In this example, through multi-scale cropping technology to match ground object features at different scales and combined with the temporal data augmentation strategy, introducing multi-period image data of early morning, noon, and dusk, the model's adaptability to light changes is effectively improved, and the robustness to the complex terrain and its dynamic changes in hilly and mountainous areas is enhanced.

[0095] Example 7

[0096] This example is an explanatory note based on Example 1. Specifically, Step 5 includes:

[0097] S51. Construct the CABR-ResUNet model: Based on the U-Net network, the residual connection (Residual Connection) is combined to improve the deep feature transfer ability. The CBAM dual attention mechanism is adopted, and the attention to important feature regions is enhanced through channel attention (Channel Attention) and spatial attention (Spatial Attention), so as to improve the model's perception ability of complex terrains. The ASPP multi-scale feature extraction module is introduced, and dilated convolutions with different dilation rates are used to capture ground object information at different scales, enhancing the model's adaptability to multi-scale targets in hilly and mountainous areas. The BRN boundary refinement technology is adopted to optimize the edge information in the decoding stage, and the accuracy of the segmentation result is improved through feature enhancement and edge constraint, reducing edge blurring and misjudgment.

[0098] In this embodiment, by constructing the CABR-ResUNet model, combining the residual connection, the dual attention mechanism CBAM, the multi-scale feature extraction ASPP, and the boundary refinement technology BRN, the model's perception ability of complex terrains in hilly and mountainous areas is significantly improved. This model can effectively capture multi-scale target information and optimize the edge segmentation accuracy, thereby reducing boundary blurring and misjudgment, and improving the segmentation accuracy and reliability of remote sensing images.

[0099] Embodiment 8

[0100] This embodiment is an explanatory description based on Embodiment 1. Specifically, Step 6 includes:

[0101] S61. Through multi-spectral remote sensing technology and thermal infrared remote sensing technology, the vegetation coverage degree, vegetation water content, terrain slope, and surface temperature are monitored in real time. After dimensionless processing, the vegetation coverage influence coefficient ZBX is calculated and obtained by the following formula:

[0102] ZBX = w1 * Fndvi + w2 * Fsavi + w3 * Fθ + w4 * Flst + w5 * Fnir + w6 * Fswir;

[0103] In the formula, Fndvi represents the normalized difference vegetation index, Fsavi represents the soil-adjusted vegetation index, Fθ represents the influence factor of slope terrain on vegetation growth, Flst represents the surface temperature, Fnir represents the near-infrared reflectance, reflecting the vegetation density, Fswir represents the short-wave infrared reflectance, reflecting the vegetation water condition, and w1, w2, w3, w4, w5, and w6 represent weight coefficients, where 0 < w1 < 1, 0 < w2 < 1, 0 < w3 < 1, 0 < w4 < 1, 0 < w5 < 1, 0 < w6 < 1, and w1 + w2 + w3 + w4 + w5 + w6 = 1;

[0104]

[0105] Where, NIR represents the reflection value in the near-infrared band, and Red represents the reflection value in the red band;

[0106]

[0107] Where, L represents the adjustment factor, and 0 < L < 1;

[0108]

[0109] Where, h represents the elevation, x represents the horizontal component of the slope, and y represents the vertical component of the slope;

[0110] S62. By presetting the first threshold Q1 in advance and comparing and analyzing the vegetation coverage influence coefficient ZBX with the first threshold Q1, the first evaluation result is obtained, including:

[0111] When the vegetation coverage influence coefficient ZBX < the first threshold Q1, it indicates that the vegetation coverage has no influence on image segmentation, and continuous monitoring is carried out;

[0112] When the vegetation coverage influence coefficient ZBX ≥ the first threshold Q1, it indicates that the vegetation coverage has an influence on image segmentation, triggering the first warning instruction and generating the first strategy: adjusting the image contrast and further strengthening the characteristics of the vegetation coverage area using CBAM.

[0113] In this embodiment, by real-time monitoring the vegetation coverage degree, vegetation water content, terrain slope and surface temperature, and calculating the vegetation coverage influence coefficient ZBX, the present invention can accurately evaluate the influence of vegetation coverage on image segmentation. When the vegetation coverage influence coefficient exceeds the preset threshold, the system can automatically trigger an alarm and generate corresponding adjustment strategies, effectively improving the image segmentation accuracy by enhancing the image contrast and further strengthening the characteristics of the vegetation area. The specific implementation cases are as follows in the table:

[0114]

[0115] Table 1

[0116]

[0117] Table 2.

[0118] Example 9

[0119] This embodiment is an explanatory description carried out in Example 1. Specifically, step seven includes:

[0120] S71. By combining LiDAR point cloud data and digital surface model DSM, using the histogram of oriented gradients HOG and Gabor texture analysis technology, real-time monitoring the influence of cracks on image segmentation, after dimensionless processing, calculating and obtaining the crack influence coefficient LFX, and the formula is as follows:

[0121]

[0122] In the formula, represents the gradient change of the slope in the x direction, represents the gradient change of the slope in the y direction, HOG represents the feature value extracted by the histogram of oriented gradients, Gabor represents the texture feature extracted by the Gabor filter, Boundary represents the boundary enhancement loss, optimizing the boundary segmentation accuracy of the crack area, and a1, a2, a3, and a4 represent weight coefficients, where 0 < a1 < 1, 0 < a2 < 1, 0 < a3 < 1, and 0 < a4 < 1, and a1 + a2 + a3 + a4 = 1;

[0123] S72. By presetting a second threshold Q2 in advance and comparing and analyzing the crack influence coefficient LFX with the second threshold Q2, the second evaluation result is obtained, including:

[0124] When the crack influence coefficient LFX < the second threshold Q2, it means that the crack has no influence on image segmentation, and continuous monitoring is carried out;

[0125] When the crack influence coefficient LFX ≥ the second threshold Q2, it means that the crack has an influence on image segmentation, triggering a second warning instruction and generating a second strategy: activating the weight of BoundaryLoss, increasing the weighted values of HOG and Gabor, strengthening the analysis of slope change and texture features, further enhancing the accuracy of crack detection, and ensuring the accurate identification of crack features.

[0126] In this embodiment, by continuously monitoring the influence of cracks on image segmentation and calculating the crack influence coefficient LFX, the present invention can effectively evaluate the influence of cracks on the accuracy of image segmentation. When the crack influence coefficient exceeds the preset threshold, the system automatically triggers an alarm and generates an adjustment strategy, by enhancing the boundary segmentation accuracy of the crack area, activating the weight of BoundaryLoss, and optimizing the weighted values of HOG and Gabor texture features, thereby improving the crack detection accuracy. The specific implementation cases are as follows in the table:

[0127]

[0128] Table 3

[0129]

[0130] Table 4.

[0131] Embodiment 10

[0132] This embodiment is an explanatory description carried out in Embodiment 1. Specifically, Step 8 includes:

[0133] S81. Further optimize the performance of the CABR-ResUNet model, combine the adaptive loss function mechanism, dynamically adjust the weights, and adopt the categorical cross-entropy loss, Dice loss, and Focal loss to improve the class balance and overall segmentation accuracy. On this basis, set the training parameters through the activation function, optimizer, and multi-task loss function of the segmentation task, perform model training, and evaluate the model accuracy through the confusion matrix and other evaluation metrics. Finally, use the MSF multi-scale fusion and IEP ignore edge prediction methods to predict the segmentation results of high-resolution remote sensing images, and adjust the model according to the accuracy requirements to output the final segmentation results.

[0134] In this embodiment, by further optimizing the CABR-ResUNet model and combining the adaptive loss function mechanism and multi-task loss function, the weights can be dynamically adjusted, and the class balance and overall segmentation accuracy can be improved. The use of categorical cross-entropy loss, Dice loss, and Focal loss effectively improves the model's recognition ability for different classes and optimizes the activation function and optimizer settings during training. The model accuracy is evaluated through evaluation metrics such as the confusion matrix, and the MSF multi-scale fusion and IEP ignore edge prediction methods are adopted, which can significantly improve the segmentation result accuracy of high-resolution remote sensing images and ensure that the finally output image segmentation results are more accurate and reliable.

[0135] The setting of the threshold value is for the convenience of comparison. Regarding the size of the threshold value, it depends on the amount of sample data and the base quantity set by those skilled in the art for each group of sample data; as long as it does not affect the proportional relationship between the parameters and the quantized values.

[0136] The above formulas are all obtained by collecting a large amount of data for software simulation and selecting a formula close to the true value. The coefficients in the formula are set by those skilled in the art according to the actual situation. The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, according to the technical solution and inventive concept of the present invention, makes equivalent substitutions or changes, and should be covered by the protection scope of the present invention.

Claims

1. A method for segmenting remote sensing images in hilly and mountainous areas based on the CABR-ResUNet model, characterized in that, It includes the following steps: Step 1: Use GIS technology and surveying data to obtain an electronic distribution map of hilly and mountainous areas, and divide it into several monitoring areas; Use drones equipped with RGB visible light cameras, multispectral cameras, hyperspectral cameras, thermal infrared cameras and LiDAR sensors to collect multi-angle images, obtain images from top-down, oblique and low-altitude angles respectively, identify basic ground objects, terrain, hillsides, cracks, vegetation cover and shadow information, and complete multi-temporal image collection; Step 2: Perform spatio-temporal filtering on remotely sensed images obtained at different time points, eliminate noise and perform atmospheric correction, shadow removal and illumination normalization; Then crop the images, remove irrelevant backgrounds, and retain the image data within the study area; Next, use point cloud image registration and multi-view image registration techniques to perform geometric correction and elevation matching to generate a high-precision digital surface model DSM; Finally, use sparse representation fusion technology to perform spectral fusion and feature extraction on multispectral and hyperspectral images to obtain the final preprocessed image data; Step 3: Based on the preprocessed image data and multi-source data including visible light, multispectral, LiDAR and thermal infrared, perform ground object annotation, and use semi-automatic label-assisted technology to construct a multi-layer label system covering basic classification, enhanced features and temporal changes; Step 4: Adopt multi-scale cropping to match basic ground objects, terrain, hillsides, cracks and vegetation features, and introduce temporal data augmentation to combine the robustness of different time image enhancement models to illumination changes; Step 5: Build a CABR-ResUNet model, combine residual connection, CBAM dual attention mechanism, ASPP multi-scale feature extraction and BRN boundary refinement technology to improve the model's perception ability of complex terrain, adaptability to multi-scale targets, and optimize edge segmentation accuracy; Step 6: Use multispectral and thermal infrared remote sensing technology to monitor vegetation coverage, moisture content and surface temperature, calculate the vegetation coverage impact coefficient ZBX, and compare it with the preset threshold Q1 to judge whether the vegetation coverage has an impact on image segmentation. If there is an impact, use CBAM to strengthen the vegetation area features; Step 7: Use LiDAR point cloud data and digital surface model DSM, combine histogram of oriented gradients HOG and Gabor texture analysis technology to monitor the impact of cracks on image segmentation in real time, calculate the crack impact coefficient LFX, and compare it with the preset threshold Q2 to judge whether the cracks have an impact on image segmentation. If there is an impact, activate the BoundaryLoss weight, increase the weighted values of HOG and Gabor, strengthen the analysis of slope changes and texture features, and improve crack detection accuracy; Step 8: Optimize the CABR-ResUNet model, combine an adaptive loss function to dynamically adjust the weights, use cross-entropy loss, Dice loss and Focal loss to improve segmentation accuracy; Set activation functions, optimizers and multi-task loss functions, perform model training, and evaluate the accuracy through evaluation metrics such as confusion matrices; Finally, use MSF multi-scale fusion and IEP ignore edge prediction methods to output the final image segmentation result.

2. The method for segmenting remote sensing images in hilly and mountainous areas based on the CABR-ResUNet model according to claim 1, characterized in that, Step 1 includes: S11. Use GIS geographic information system technology and related surveying and mapping data to obtain an electronic distribution map of hilly and mountainous areas; divide the hilly and mountainous areas into several monitoring regions, and mark the several monitoring regions on the electronic distribution map of the hilly and mountainous areas as: Jcq1, Jcq2, ..., Jcq n ; n represents the number of monitoring regions.

3. A method for segmenting remote sensing images in hilly and mountainous areas based on the CABR-ResUNet model according to claim 1, characterized in that Step 1 also includes: S12. Use a drone equipped with multi-sensor remote sensing devices including an RGB visible light camera, a multi-spectral camera, a hyperspectral camera, a thermal infrared camera, and a LiDAR sensor to collect multi-angle images of the monitoring area; for the first round of image collection, use a standard overhead angle of 90° to obtain vertically high-resolution remote sensing images to identify the basic ground features in the monitoring area; for the second round of image collection, use an oblique angle of 45°, adjust the attitude of the drone, and collect oblique images to perform three-dimensional perception of the terrain, hillsides, and cracks in the monitoring area; for the third round of image collection, use low-altitude flight at a distance of 15 m - 30 m to collect the mountain slopes and vegetation-covered areas in the monitoring area to identify the mountain coverage; and use multi-temporal image collection to perform image recognition on the shadows in the hilly mountainous areas of the monitoring area at different time periods including early morning, noon, and dusk.

4. A method for segmenting remote sensing images in hilly and mountainous areas based on the CABR-ResUNet model according to claim 1, characterized in that, Step 2 includes: S21. For the remote sensing images obtained at different time points, use spatio-temporal filtering technology to eliminate temporal noise, perform atmospheric correction and shadow removal, and perform illumination normalization; S22. Crop the remote sensing image data after atmospheric correction and shadow removal to remove irrelevant background areas and retain the image data within the study area; S23. Use point cloud image registration technology to perform geometric correction and elevation information matching on the optical remote sensing image and the LiDAR point cloud image to obtain a high-precision digital surface model DSM. Use multi-view image registration technology to perform geometric correction and radiometric equalization on the remote sensing images obtained at different angles. Use sparse representation fusion technology to perform spectral information fusion on the multi-spectral and hyperspectral images, extract high-dimensional spectral features, and remove redundant information to finally obtain preprocessed image data.

5. A method for segmenting remote sensing images in hilly and mountainous areas based on the CABR-ResUNet model according to claim 1, characterized in that, Step 3 includes: S31. Based on the preprocessed image data, combine multi-source data including visible light, multi-spectral, LiDAR, and thermal infrared to perform ground object annotation; S32. Use semi-automatic label assistance technology to construct a multi-layer label system, including basic classification, enhanced features, and temporal changes.

6. A method for segmenting remote sensing images in hilly and mountainous areas based on the CABR-ResUNet model according to claim 1, characterized in that Step 4 includes: S41. Use multi-scale cropping including optional 256*256, 512*512, and 1024*1024 to match the features of basic ground objects, terrain, hillsides, cracks, and vegetation-covered ground objects; S42. Introduce temporal data augmentation, pair different time images including early morning, noon, and dusk to enhance the robustness of the model to illumination changes.

7. A method for segmenting remote sensing images in hilly and mountainous areas based on the CABR-ResUNet model according to claim 1, characterized in that, Step 5 includes: S51. Construct the CABR-ResUNet model: Based on the U-Net network, combine the Residual Connection to improve the deep feature transfer ability. Adopt the CBAM dual attention mechanism, enhance the attention to important feature regions through Channel Attention and Spatial Attention, and improve the model's perception ability of complex terrains. Introduce the ASPP multi-scale feature extraction module, use dilated convolutions with different dilation rates to capture ground object information at different scales, and enhance the model's adaptability to multi-scale targets in hilly and mountainous areas. Adopt the BRN boundary refinement technology to optimize the edge information in the decoding stage, and improve the accuracy of the segmentation results through feature enhancement and edge constraints, reducing edge blurring and misjudgment.

8. A method for segmenting remote sensing images in hilly and mountainous areas based on the CABR-ResUNet model according to claim 1, characterized in that, Step six includes: S61. Through multi-spectral remote sensing technology and thermal infrared remote sensing technology, monitor the vegetation coverage degree, vegetation moisture content, terrain slope, and surface temperature in real time. After dimensionless processing, calculate and obtain the vegetation coverage influence coefficient ZBX, and the formula is as follows: ZBX = w1 * Fndvi + w2 * Fsavi + w3 * Fθ + w4 * Flst + w5 * Fnir + w6 * Fswir; In the formula, Fndvi represents the normalized difference vegetation index, Fsavi represents the soil-adjusted vegetation index, Fθ represents the influence factor of slope terrain on vegetation growth, Flst represents the surface temperature, Fnir represents the near-infrared reflectance, reflecting the density of vegetation, Fswir represents the short-wave infrared reflectance, reflecting the moisture condition of vegetation, and w1, w2, w3, w4, w5, and w6 represent weight coefficients; In the formula, NIR represents the reflectance value in the near-infrared band, and Red represents the reflectance value in the red band; In the formula, L represents the adjustment factor, 0 < L < 1; In the formula, h represents the elevation, x represents the horizontal component of the slope, and y represents the vertical component of the slope; S62. By presetting the first threshold Q1 in advance and comparing and analyzing the vegetation coverage influence coefficient ZBX with the first threshold Q1, the first evaluation result is obtained, including: When the vegetation coverage influence coefficient ZBX < the first threshold Q1, it means that the vegetation coverage has no influence on image segmentation, and continuous monitoring is carried out; When the vegetation coverage influence coefficient ZBX ≥ the first threshold Q1, it means that the vegetation coverage has an influence on image segmentation, trigger the first warning instruction, and generate the first strategy: adjust the image contrast and further strengthen the features of the vegetation coverage area using CBAM.

9. A method for segmenting remote sensing images in hilly and mountainous areas based on the CABR-ResUNet model according to claim 1, characterized in that, Step seven includes: S71. By combining LiDAR point cloud data and digital surface model DSM, use the histogram of oriented gradients HOG and Gabor texture analysis technology to monitor the influence of cracks on image segmentation in real time. After dimensionless processing, calculate and obtain the crack influence coefficient LFX, and the formula is as follows: In the formula, represents the gradient change of the slope in the x direction, represents the gradient change of the slope in the y direction, HOG represents the feature value extracted by the histogram of oriented gradients, Gabor represents the texture feature extracted by the Gabor filter, Boundary represents the boundary enhancement loss, optimizing the boundary segmentation accuracy of the crack area, and a1, a2, a3, and a4 represent the weight coefficients; S72. By presetting the second threshold Q2 in advance and comparing and analyzing the crack influence coefficient LFX with the second threshold Q2, the second evaluation result is obtained, including: When the crack influence coefficient LFX < the second threshold Q2, it means that the cracks have no influence on image segmentation, and continuous monitoring is carried out; When the crack influence coefficient LFX ≥ the second threshold Q2, it indicates that the crack has an impact on image segmentation, triggering the second warning instruction and generating the second strategy: activating the weight of BoundaryLoss, increasing the weighted values of HOG and Gabor, strengthening the analysis of slope changes and texture features, further enhancing the accuracy of crack detection, and ensuring the accurate identification of crack features.

10. A method for segmenting remote sensing images in hilly and mountainous areas based on the CABR-ResUNet model according to claim 1, characterized in that, Step eight includes: S81. Further optimize the performance of the CABR-ResUNet model, combine the adaptive loss function mechanism, dynamically adjust the weights, adopt categorical cross-entropy loss, Dice loss, and Focal loss to improve class balance and overall segmentation accuracy; on this basis, set training parameters through the activation function, optimizer, and multi-task loss function of the segmentation task, conduct model training, and evaluate the model accuracy through the confusion matrix and other evaluation metrics; finally, use the MSF multi-scale fusion and IEP ignore-edge prediction methods to predict the segmentation results of high-resolution remote sensing images, and adjust the model according to the accuracy requirements to output the final segmentation results.

Images