A method for road extraction from remote sensing images

Abstract

The present invention discloses a method for extracting roads from remote sensing images with shadow interference. Aiming at the situation where road sections in high-resolution remote sensing images are partially obscured by shadows of objects on both sides, causing interruptions in conventional extraction methods, the shadow areas of the roads in the images are first detected, and the shadow areas are compensated using the grayscale information of the non-obscured road sections. The compensated roads are then extracted. Experiments have shown that this method can greatly reduce the influence of shadows and achieve good extraction results.

Description

Technical Field

[0001] The present invention belongs to the technical field of optical image processing, and in particular relates to a remote sensing image road extraction method, which is mainly used in airborne optoelectronic pods. Background Art

[0002] In high-resolution remote sensing images, there are often trees and buildings on both sides of the road. Due to the angle of the sun, the shadows cast by these objects on the road will cause uneven grayscale on the road surface. The shadow area in the image will appear darker than the surrounding objects due to lack of light. At the same time, the spectral characteristics of the original objects in the shadow-covered area are not completely obscured by the shadow effect, so the original features of the objects are still preserved to a certain extent.

[0003] By combining the grayscale features of the above shadow areas, shadow detection and extraction can be achieved in the image, and the grayscale features of the original objects covered by the shadow areas can be restored to a certain extent to achieve object detection.

[0004] Airborne optoelectronic pods integrate optics, mechanics, automatic control and communication technologies. They are important search and reconnaissance equipment in the aerospace field. They are often equipped with optical sensors such as visible light and near-infrared. Therefore, it is of great significance to study the target extraction technology with shadow interference applied to airborne optoelectronic pods. Summary of the Invention

[0005] In view of the above background, the present invention provides a method for extracting roads from remote sensing images with shadow interference.

[0006] The technical solution adopted by the present invention to solve the technical problem is: a remote sensing image road extraction method, comprising the following steps:

[0007] (1) Color space conversion: Based on the low brightness, high hue, and high saturation characteristics of the road shadow area in the high-resolution remote sensing image, the RGB to HSV color space conversion is first performed;

[0008] (2) Preliminary detection of shadow areas;

[0009] (3) Shadow area filtering: After the shadows are initially detected, a series of indicators are generated based on the road characteristics. The initially extracted shadow areas are further filtered based on the indicators to remove the shadows in the non-road areas and retain the shadows in the road sections.

[0010] (4) Road shadow area compensation: After the road shadow detection is completed, the shadow area is compensated using the Wallis filter, and then the median filter is used to smooth the preliminary compensation results to solve the problem of grayscale value mutation in the edge area caused by light, road material, etc. The region growing method is used to extract the preliminary road, and then morphological processing is performed to improve the extraction effect;

[0011] (5) Road extraction: Finally, the morphological skeleton algorithm is used to extract the mesh road lines, optimize the problems of burrs and interruptions in the extraction results, and obtain a complete road skeleton line.

[0012] The method for extracting roads from remote sensing images comprises the following steps: firstly converting the color space of the original image to HSV, and then splitting the transformed image into three separate bands: H, S, and V.

[0013] The method for extracting roads from remote sensing images comprises the following steps: step (2) is to use the Otsu method to obtain the segmentation threshold δ for each of the three components H, S, and V. h , δ s , δ v , use the segmentation threshold to perform image segmentation respectively, and find the intersection of the results using the following formula:

[0014] f=f h ∩f s ∩f v

[0015]

[0016] The results are then overlaid on the color image, with the extraction results shown in blue.

[0017] The method for extracting roads from remote sensing images, wherein step (3) is to process the adjacent planar shadow areas of the shadow area in units of blocks, calculate the area S of each shadow block, and set the area threshold δ s , remove the block shadows below the threshold; calculate the standard deviation of each shadow area and set the standard deviation threshold δ var , remove the areas that do not meet the conditions;

[0018] The R, G, and B band images are used to generate a differential image. The differences between the blue-purple trees and the real road shadows are analyzed and compared in the differential image. For the shadow areas that are retained after filtering, the overall grayscale value mean μ of all areas in the blue-red band differential image is calculated. ave And the grayscale mean μ corresponding to each shadow area i , in μ aveAs the threshold for judgment, retain all shadow areas S where the average pixel grayscale value of a single block is greater than the average pixel grayscale value of all areas. i , remove the areas that do not meet the conditions, and get the final road shadow result S through the following formula:

[0019] S={S i |μ i ≥μ ave}

[0020] A hole filling process is used to obtain further improved extraction results.

[0021] In the method for extracting roads from remote sensing images, step (4) is to use a Wallis filter to compensate the shadow area finally detected. The Wallis filter is represented as follows:

[0022]

[0023] Where g(x,y) represents the original image, f represents the reference area information, and s f is the standard deviation of the reference area, m f is the grayscale mean of the reference area, s g and m g Then the standard deviation and grayscale mean of the area to be enhanced, g c (x, y) is the grayscale value of the transformed image, c is the variance expansion coefficient, and b is the brightness expansion coefficient. The value range of both is [0, 1].

[0024] Select some ideal road seed points in the image, count the 3×3 or 5×5 neighborhood information of the seed points as the reference area, and assign the neighborhood mean and standard deviation to m f and s f At the same time, the final shadow detection results are counted, the grayscale mean and standard deviation corresponding to each individual area are calculated, and assigned to m g and s g , Wallis filtering is performed on each area as a unit, and the initial compensation result is obtained. The edge of the compensation area is smoothed using the median filter of a 3×3 template.

[0025] The method for extracting roads from remote sensing images, wherein step (5) is to extract roads using a region growing method, select seed points in a road segment, compare the grayscale value with the seed point to see whether a threshold condition is met, and then determine whether the pixel is a road point, thereby realizing road search and discrimination of the entire image.

[0026] The method for extracting roads from remote sensing images uses a morphological skeleton algorithm to process the extraction results, then removes burrs from the extracted skeleton lines, and simultaneously connects the interrupted parts of the skeleton lines to finally obtain a complete road skeleton line, which is then superimposed on the original image to extract a mesh road line.

[0027] The beneficial effects of the present invention are as follows: in order to solve the problem that some trees and shadows are confused and difficult to distinguish, the present invention utilizes the characteristic that blue-red light and blue-green light have little correlation to obtain the mutual differential images of the three bands, and eliminates the mistakenly extracted trees according to the grayscale threshold; the present invention uses Wallis filtering combined with median filtering to process the shadow area, solves the problem of grayscale value mutation in the edge area caused by light, road material and other reasons, and realizes the extraction and compensation of road shadows; the present invention adopts morphological processing to optimize the extraction results, better suppresses noise interference, and the extraction results are more stable. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 The image results of each channel in the HSV space of the present invention;

[0029] Figure 2 It is the shadow area initially extracted by the present invention;

[0030] Figure 3 This is the preliminary filtering result of the shadow area of ​​the present invention;

[0031] Figure 4 It is the result of mutual differentiation of RGB bands in the present invention;

[0032] Figure 5 This is the hole filling optimization result of the present invention;

[0033] Figure 6 is the road compensation result of the present invention;

[0034] Figure 7 This is the road extraction result after compensation of the present invention;

[0035] Figure 8 The road skeleton line result extracted by the present invention;

[0036] Figure 9 This is the second set of road data extraction results of the present invention. DETAILED DESCRIPTION

[0037] The present invention will be described in further detail below with reference to the accompanying drawings.

[0038] The present invention discloses a method for extracting roads from remote sensing images. In combination with the low brightness, high hue, and high saturation characteristics of shadow areas, the method first performs a color space conversion from RGB to HSV to preliminarily detect shadows. Then, a series of indicators are generated in combination with road features. The preliminarily extracted shadow areas are further filtered according to the indicators to remove shadows in non-road areas and retain shadows in road sections. After the road shadow detection is completed, the shadow areas are compensated using a Wallis filter. Subsequently, the preliminary compensation results are smoothed using a median filter to resolve the problem of grayscale value mutations in edge areas caused by light, road material, and other factors. The regional growing method is used to extract the preliminary extracted roads, and then morphological processing is performed to improve the extraction effect. Finally, a morphological skeleton algorithm is used to extract mesh road lines, and problems such as burrs and interruptions in the extraction results are optimized to obtain a complete road skeleton line.

[0039] (1) Color space conversion.

[0040] First, convert the original image's color space to HSV, and then split the transformed image into three separate bands: H, S, and V. The displayed results are shown in Figure 1 , where (a) is the original color image, (b) is the H component image, (c) is the S component image, and (d) is the V component image.

[0041] (2) Preliminary detection of shadow areas.

[0042] Due to the presence of shadow areas, the image histogram Figure 1 Generally, it presents a bimodal feature, and the shadow area is brighter in the H and S components, but darker in the V component. Therefore, threshold segmentation is performed on each of the three components, and the Otsu method is used to obtain the segmentation threshold δ for each of the H, S, and V components. h , δ s , δ v , use the segmentation threshold to segment the image on the three components respectively, and find the intersection of the results, as shown in the following formula, and then superimpose the results on the color image, and the extracted results are displayed in blue, as shown in Figure 2 :

[0043] f=f h ∩f s ∩f v

[0044]

[0045] 3. Filter out shadow areas.

[0046] After the initial extraction, the shadow area mostly exists in pixels. The adjacent planar shadow areas are processed in blocks. The area S of each shadow block is counted. By observing the extraction results, it can be found that there are some small extraction areas in the image, including correct shadows and incorrectly extracted objects. The area threshold δ is manually set. s , remove the block shadows below the threshold. Count the standard deviation of each shadow area and set the standard deviation threshold δ var , remove the areas that do not meet the conditions, and get the shadow result after preliminary filtering as follows Figure 3 shown.

[0047] Comparing the initial elimination results with the original image, we can find that due to the presence of some blue-purple trees, even after filtering using area and variance as indicators, they still cannot be completely removed. According to one of the characteristics of the shadow area mentioned above, the correlation between blue-red light and blue-green light in the shadow area is not large. Due to the influence of plant photosynthesis, there are different degrees of absorption and reflection of the three types of light. Therefore, we try to generate a differential image by subtracting the R, G, and B band images. In the differential image, we analyze and compare the differences between the blue-purple trees and the real road shadows, such as Figure 4 As shown, (a) is the original image, (b) is the green-red band difference image, (c) is the green-blue band difference image, and (d) is the blue-red band difference image.

[0048] Comparing the three differential band images, we can find that Figure 4 In (b) and (c), the real shadows and trees in the corresponding retained shadow areas are still not well distinguished, while Figure 4 In (d), due to the absorption of red and blue light by trees during photosynthesis and the advantage of blue light in scattering, more blue light is retained than red light. Therefore, the spectrum value of the real shadow is brighter than that of the tree area in the shadow, making it distinguishable. For the shadow area that is retained after filtering, calculate the overall gray value mean μ of all areas in the blue-red band difference image ave And the grayscale mean μ corresponding to each shadow area i , in μ ave As the threshold for judgment, retain all shadow areas S where the average pixel grayscale value of a single block is greater than the average pixel grayscale value of all areas. i , remove the areas that do not meet the conditions and get the final road shadow result S: S = {S i |μ i ≥μ ave}.

[0049] Due to the influence of local individual noise, there are still some holes in the final detection results that have not been processed. In order to reduce the interference caused by subsequent processing, hole filling processing is used to obtain further improved extraction results. Figure 5 shown.

[0050] (4) Road shadow area compensation.

[0051] After the road shadow detection is completed, in order to reduce the impact of the shadow and achieve the final complete road segment extraction, shadow compensation is required.

[0052] The final detected shadow area is compensated using the Wallis filter. The commonly used Wallis filter is expressed as:

[0053]

[0054] Where g(x,y) represents the original image, f represents the reference area information, and s f is the standard deviation of the reference area, m f is the grayscale mean of the reference area, s g and m g Then the standard deviation and grayscale mean of the area to be enhanced, g c (x, y) is the grayscale value of the transformed image. c is the variance expansion coefficient, and b is the brightness expansion coefficient. Both values ​​are in the range [0, 1].

[0055] In order to simplify the calculation, it is assumed that the grayscale values ​​of each road segment in the road image are relatively close. Some ideal road seed points are manually selected in the image, and the 3×3 or 5×5 neighborhood information of the seed points is counted as the reference area. The neighborhood mean and standard deviation are assigned to m f and s f At the same time, the final shadow detection results are counted, the grayscale mean and standard deviation corresponding to each individual area are calculated, and assigned to m g and s g , Wallis filter processing is performed on each area, and the initial compensation result is obtained. The edge of the compensation area is smoothed using the median filter of a 3×3 template. The result is as follows Figure 6 shown.

[0056] (5) Road extraction.

[0057] After the shadow compensation is completed, the road is extracted. In this paper, the region growing method is used to extract the road. The seed points are manually selected in the road section. The grayscale value of the seed point is compared to see whether it meets the threshold condition, and then the pixel is judged to be a road point, realizing the road search and discrimination of the entire image. Figure 7(a) is the result obtained by preliminary processing. Due to the noise and other images, the road segments obtained by preliminary region growing may have missing or empty phenomena. Therefore, further morphological processing is required to improve the extraction results, such as Figure 7 (b) shown.

[0058] In order to further extract the network road line, the morphological skeleton algorithm is used to process the extraction results, and then the burrs in the extracted skeleton line are removed. At the same time, the interrupted parts of the skeleton line need to be connected. Finally, the complete road skeleton line is obtained and superimposed on the original image to display the result as shown below. Figure 8 shown.

[0059] The second set of road data extraction results of the present invention is as follows Figure 9 As shown in the figure, (a) is the original road image, (b) is the initially extracted shadow area, (c) is the optimized shadow area, (d) is the Wallis filter result, (e) is the corresponding grayscale image, and (f) is the extracted skeleton line.

[0060] The above embodiments are merely illustrative of the principles and effects of the present invention, as well as some embodiments of its application. A person skilled in the art may make several modifications and improvements without departing from the inventive concept of the present invention, and all of these modifications and improvements fall within the scope of protection of the present invention.

Claims

1. A method for extracting roads from remote sensing images, characterized by: Includes the following steps (1) Color space conversion: Considering the low brightness, high hue, and high saturation characteristics of the road shadow area in the high-resolution remote sensing image, the RGB to HSV color space conversion is first performed, the color space of the original image is converted to HSV, and then the transformed image is split into three separate bands: H, S, and V. (2) Preliminary detection of shadow areas: Use the Otsu method to obtain the segmentation thresholds for the three components H, S, and V respectively. 、 、 , use the segmentation threshold to perform image segmentation respectively, and find the intersection of the results using the following formula: ， The results are then superimposed on the color image, with the extraction results shown in blue; (3) Shadow area filtering: Combine road features to generate a series of indicators, further filter the initially extracted shadow areas, remove the shadows of non-road areas, and retain the shadows of road sections: the adjacent surface shadow areas of the shadow area are processed in blocks, and the area T of each shadow block is counted, and the area threshold is set. , remove the block shadows below the threshold; calculate the standard deviation of each shadow area and set the standard deviation threshold , remove areas that do not meet the conditions; use the R, G, and B bands to make a difference between each other to generate a differential image, analyze and compare the differences between the blue-purple trees and the real road shadows in the differential image; calculate the overall gray value mean of all areas in the blue-red band differential image And the grayscale mean corresponding to each shadow area ,by As the threshold for judgment, retain all shadow areas that meet the requirement that the average pixel grayscale value of a single block is greater than the average pixel grayscale value of all areas. , remove the areas that do not meet the conditions, and use the formula The final road shadow result T is obtained and hole filling is performed to obtain a further improved extraction result; (4) Road shadow area compensation: After the road shadow detection is completed, the shadow area is compensated using the Wallis filter. The Wallis filter is expressed as: ， in Represents the original image, Represents reference area information, is the standard deviation of the reference area, is the grayscale mean of the reference area, and Then the standard deviation and grayscale mean of the area to be enhanced are: is the gray value of the transformed image, c is the expansion coefficient of the variance, b is the brightness expansion coefficient, and the value range of both is [0,1]. Select some ideal road seed points in the image, count the 3×3 or 5×5 neighborhood information of the seed points as the reference area, and assign the neighborhood mean and standard deviation to and At the same time, the final shadow detection results are counted, the grayscale mean and standard deviation corresponding to each individual area are calculated, and assigned to and , Wallis filter processing is performed on each area, and the initial compensation result is obtained. The edge of the compensation area is smoothed using the median filter of a 3×3 template; Then, median filtering is used to smooth the preliminary compensation results to solve the problem of grayscale value mutation in the edge area caused by light and road material. The region growing method is used to extract the preliminary road, and then morphological processing is performed to improve the extraction effect: the region growing method is used to extract the road, and the seed point is selected in the road segment. The grayscale value is compared with the seed point to see whether it meets the threshold condition, and then the pixel is judged as a road point, realizing road search and discrimination of the entire image. (5) Road extraction: Finally, the morphological skeleton algorithm is used to extract the network road line, and the burrs and interruptions in the extraction results are optimized to obtain the complete road skeleton line: the morphological skeleton algorithm is used to process the extraction results, and then the burrs in the extracted skeleton line are removed. At the same time, the interruptions in the skeleton line need to be connected, and finally the complete road skeleton line is obtained. It is superimposed on the original image to extract the network road line.

Images