Green vegetation segmentation method for multispectral remote sensing image

Abstract

The invention discloses a green vegetation segmentation method for a multispectral remote sensing image, and the method comprises the steps: extracting an RGB image and a near-infrared image from the multispectral remote sensing image, determining a vegetation index according to the actual demands, carrying out the calculation to obtain a vegetation index map, and carrying out the segmentation to obtain a pre-segmentation result, and according to the pre-segmentation result, generating a pre-segmentation mask map to process each channel image, calculating a color index map according to the channel image after mask processing, and then performing segmentation to obtain a green vegetation segmentation result. According to the method, multi-stage segmentation is performed by combining the vegetation index and the color index, so that the precision of green vegetation segmentation is improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of remote sensing image processing, and more specifically relates to a green vegetation segmentation method for multispectral remote sensing images. BACKGROUND

[0002] In the existing vegetation segmentation technology, the traditional RGB image processing method often cannot effectively distinguish green vegetation from complex backgrounds (such as bright soil, dry yellow vegetation or mixed scenes), and lacks a simple and effective extraction means. Although the method based on a single vegetation index (such as NDVI) can distinguish vegetation and non-vegetation to some extent, there are still obvious deficiencies in fine segmentation of green vegetation. Especially in agricultural applications, when it is necessary to accurately distinguish green crops from other vegetation or backgrounds, the existing method often has the problems of over-segmentation or under-segmentation. In addition, the insufficient registration accuracy between the channels of the multispectral image, the radiation distortion caused by the difference between the sensors and other problems will further affect the final segmentation effect. SUMMARY

[0003] The purpose of the present application is to overcome the deficiencies of the prior art and provide a green vegetation segmentation method for multispectral remote sensing images, which combines vegetation index and color index for multi-stage segmentation to improve the accuracy of green vegetation segmentation.

[0004] In order to achieve the above-mentioned purpose of the application, the green vegetation segmentation method for multispectral remote sensing images comprises the following steps:

[0005] S1: Extracting channel images from a multispectral remote sensing image , , R represents a red channel, G represents a green channel, B represents a blue channel, NIR represents a near-infrared channel;

[0006] S2: Determining a vegetation index and a segmentation method according to actual needs, calculating a vegetation index map according to the four channel images , and then segmenting the vegetation index map by using the segmentation method to obtain a pre-segmentation result;

[0007] S3: Generating a pre-segmentation mask map according to the pre-segmentation result, taking the value of the vegetation area as a region of interest, and the value is 1, and the value of other areas is 0; using the pre-segmentation mask map to mask each channel image respectively to obtain channel images ;

[0008] S4: Determining a color index of vegetation according to actual needs, and calculating a color index map according to the four channel images The color index map is calculated, and then the color index map is segmented using a segmentation method to obtain the green vegetation segmentation result.

[0009] This invention provides a green vegetation segmentation method for multispectral remote sensing images. It extracts RGB and near-infrared images from multispectral remote sensing images, determines vegetation indices according to actual needs, calculates and segments the vegetation index map to obtain pre-segmentation results, generates a pre-segmentation mask map based on the pre-segmentation results, processes the images of each channel based on the pre-segmentation results, calculates the color index map based on the channel images after mask processing, and then performs segmentation to obtain the green vegetation segmentation results.

[0010] The present invention has the following beneficial effects:

[0011] 1) This invention adopts a two-stage segmentation, with each stage using different types of indices, which can handle vegetation segmentation tasks in complex scenarios (bright vegetation, mixed vegetation, etc.) and improve segmentation accuracy.

[0012] 2) The present invention can also perform channel registration preprocessing on RGB images based on near-infrared images before segmentation, thereby improving the quality of channel images and further improving segmentation accuracy. Attached Figure Description

[0013] Figure 1 This is a flowchart illustrating a specific implementation of the green vegetation segmentation method for multispectral remote sensing images according to the present invention.

[0014] Figure 2 This is a flowchart of the channel registration preprocessing in this embodiment;

[0015] Figure 3 This is an example diagram of key point matching in this embodiment;

[0016] Figure 4 These are image comparisons before and after channel registration in this embodiment;

[0017] Figure 5 This is a result image of pre-segmentation of vegetation and non-vegetation based on the NDVI index and SAVI index in this embodiment.

[0018] Figure 6 This is a comparison diagram of the pre-segmentation results and the final segmentation results of different remote sensing images in this embodiment;

[0019] Figure 7 This is a comparison of the segmentation results of the ground dataset by the NDVI-ExG and SAVI-ExG methods in this embodiment;

[0020] Figure 8 This is a comparison chart of the segmentation results of the UAV dataset by the NDVI-ExG and SAVI-ExG methods in this embodiment. Detailed Implementation

[0021] The specific embodiments of the present invention will now be described with reference to the accompanying drawings to enable those skilled in the art to better understand the invention. It should be particularly noted that in the following description, detailed descriptions of known functions and designs that might obscure the main content of the invention will be omitted here.

[0022] Example

[0023] Figure 1 This is a flowchart illustrating a specific implementation of the green vegetation segmentation method for multispectral remote sensing images according to the present invention. Figure 1 As shown, the green vegetation segmentation method for multispectral remote sensing images of the present invention includes the following steps:

[0024] S101: Acquire channel image:

[0025] Extracting channel images from multispectral remote sensing images , , Indicates the red channel. Indicates a green channel. Indicates the blue channel. This indicates the near-infrared channel.

[0026] In this embodiment, to improve the accuracy of subsequent green vegetation segmentation, the channel image is processed before segmentation. As a benchmark for the other three channel images Perform registration preprocessing. . Figure 2 This is a flowchart of the channel registration preprocessing in this embodiment. For example... Figure 2 As shown, the specific method for channel registration preprocessing in this invention includes:

[0027] S201: Extracting the Gaussian pyramid:

[0028] For each channel image Construct Gaussian pyramids separately to obtain image representations at different scales. , , This indicates the number of layers in the Gaussian pyramid. The specific method for constructing the Gaussian pyramid in this embodiment is as follows:

[0029] For the original image Using smoothing coefficients respectively Gaussian kernel Perform convolution to obtain the original image. Spatial representation at different scales :

[0030]

[0031] in, Representing an image medium pixel pixel values, Gaussian kernel The formula is as follows:

[0032]

[0033] S202: Key Point Detection

[0034] Image representation at different scales Keypoint detection is performed separately to obtain a set of keypoints. .

[0035] In this embodiment, the Difference of Gaussians (DoG) is used to approximate the Laplacian Gaussian (LoG) to achieve efficient keypoint detection:

[0036]

[0037]

[0038] This method precisely locates the detected key points and subtracts two adjacent Gaussian kernel-smoothed images to detect blobs (extreme points) in the image, thereby eliminating low-contrast and edge response points.

[0039] S203: Image Registration

[0040] Based on the key point set obtained from detection Represented by images Image representation of the other three channels as a benchmark Registration is performed to obtain the image representation. In this embodiment, the RANSAC algorithm is used to estimate the optimal projection transformation matrix. This enables precise registration between RGB and NIR image representations. Figure 3 This is an example diagram of key point matching in this embodiment. For example... Figure 3 As shown, in this embodiment, registration from RGB image to NIR image can be effectively achieved through key point matching.

[0041] S204: Restore channel image:

[0042] According to the registration Layer image representation Restored channel image By registering the RGB three-channel image with the NIR image, the registered RGB image can better match the characteristics of the vegetation.

[0043] To further improve the quality of the registered channel images, the restored channel images can also be further processed. Further corrections will be performed, specifically as follows:

[0044] First, the restored channel image is processed. Median filtering is performed to remove noise, and then grayscale correction is performed using the following formula:

[0045]

[0046] in, , NIR images Image to be corrected The standard deviation of gray level , NIR images Image to be corrected The average gray level.

[0047] In this embodiment, grayscale correction is performed using a relative calibration method, which can effectively eliminate the effects of differences in remote sensing image sensors and uneven illumination.

[0048] Figure 4 These are image comparisons before and after channel registration in this embodiment. Figure 4 As shown, after channel registration preprocessing, the vegetation features of the RGB image are more prominent, and the illumination of the three-channel image is more uniform, which can make the subsequent index calculation more accurate.

[0049] S102: Vegetation and Non-Vegetation Pre-Segmentation:

[0050] The vegetation index and segmentation method are determined based on actual needs, using images from four channels. The vegetation index map is calculated, and then the vegetation index map is segmented using a segmentation method to obtain the pre-segmentation result.

[0051] Commonly used vegetation indices include the ratio vegetation index (RVI), difference vegetation index (DVI), normalized difference vegetation index (NDVI), and soil-light adjusted vegetation index (SDVI). Experiments have shown that the normalized difference vegetation index (NDVI) and the soil-light adjusted vegetation index (SDVI) perform well in the pre-segmentation process of this invention. Their calculation formulas are as follows:

[0052]

[0053]

[0054] in, , These are NIR images With R channel image grayscale image, This represents the soil background adjustment parameter, which is usually taken as 0.5.

[0055] In this embodiment, the segmentation method adopted is the Otsu algorithm, also known as the maximum inter-class variance method, which can automatically determine the optimal segmentation threshold. Figure 5 This is a result image of pre-segmentation of vegetation and non-vegetation based on different vegetation indices in this embodiment. Figure 5 Row (a) shows the pre-segmentation results obtained based on the NDVI index map, row (b) shows the pre-segmentation results obtained based on the RVI index map, row (c) shows the pre-segmentation results obtained based on the SAVI index map, row (d) shows the pre-segmentation results obtained based on the NIR image, and row (e) shows the pre-segmentation results obtained based on the RGB image. Figure 5 As shown, both the NDVI index and the SAVI index can achieve good segmentation results during pre-segmentation, effectively removing background pixels.

[0056] S103: Pre-segmentation masking process:

[0057] A pre-segmentation mask is generated based on the pre-segmentation results. The vegetation region is designated as the region of interest (ROI), with a value of 1, while other regions have values ​​of 0. The pre-segmentation mask is then used to segment the channel image. Perform masking to obtain channel images. .

[0058] S104: Color Segmentation:

[0059] The color index of vegetation is determined based on actual needs, using images from four channels. The color index map is calculated, and then the color index map is segmented using a segmentation method to obtain the vegetation segmentation results.

[0060] In this embodiment, the color index is either the Super Green Index ExG or the Super Green minus the Super Red Index ExG-ExR, and the calculation formulas are as follows:

[0061]

[0062]

[0063] In practical applications, to obtain more accurate vegetation regions, the vegetation segmentation results can be further optimized. The specific methods are as follows:

[0064] First, a 3×3 structuring element is used to perform an opening operation on the vegetation segmentation results. This effectively eliminates small noise points while maintaining the integrity of the main vegetation areas. Then, a Gaussian smoothing filter is applied to process the segmentation boundaries, making the edges more natural and continuous.

[0065] The above methods can effectively remove isolated noise points and improve the accuracy of the final green vegetation map. Figure 6 This is a comparison chart of the pre-segmentation results and the final segmentation results of different remote sensing images in this embodiment. Figure 6 The first three rows of column (a) are pre-segmentation results based on NDVI, RVI, and SAVI vegetation indices; the fourth row is a pre-segmentation result based on the NIR image; the fifth row is the original RGB image; column (b) is the final segmentation result based on the ExG index; column (c) is the final segmentation result based on the G index; column (d) is the final segmentation result based on the ExG-ExR index; and column (e) is the final segmentation result based on the GCI index. Figure 6 As shown, the final segmentation result removes non-vegetation pixels better than the pre-segmentation result, improving the accuracy of vegetation segmentation. Furthermore, the final results based on the ExG index and the G index show the best performance for non-green areas. In (b,1), a small portion of soil remains, but these isolated points can be removed through subsequent opening operations. In (b,3), the plant stems are removed, leaving only the leaves. Note the red box in row (3), where the shadow overlaps with the leaves, and there are also stems. The final segmentation method based on the ExG index effectively segments these small leaves, achieving the best results. In the results based on RGB and NIR, because the first stage did not separate the vegetation from the background well, the second stage could only extract some green leaves, retaining scattered non-green vegetation, as shown in the red box in row (4). The two-stage segmentation based on GCI performed the worst, losing almost all leaf information.

[0066] To better illustrate the technical effects of this invention, specific examples are used to experimentally verify it. In this embodiment, a comparative experiment was conducted on a UAV dataset and a ground dataset. The UAV dataset includes 50 multispectral images with a resolution of 2048×2048, while the ground dataset includes 50 images acquired by a handheld device with a resolution of 6000×4000. In this embodiment, different combinations of indices are used for vegetation segmentation, and accuracy, precision, and recall are used as metrics to evaluate the segmentation performance.

[0067] To demonstrate the effectiveness of this invention, drawing inspiration from ablation experiments, each stage was evaluated to observe whether a single index could yield good results. Therefore, NDVI, SAVI, ExG, NDVI-ExG two-stage segmentation, SAVI-ExG two-stage segmentation, and conventional RGB segmentation were all selected, and adaptive thresholding was used for segmentation of the selected images. Then, the three indices were calculated for both the ground dataset and the UAV dataset. Table 1 compares the indices for segmenting green vegetation in the ground dataset using different methods in this embodiment.

[0068] Accuracy↑ Precision↑ Recall↑ NDVI 0.8891 0.7621 0.7845 SAVI 0.8467 0.7851 0.7714 ExG 0.8521 0.7450 0.8585 RGB 0.5123 0.4236 0.8433 NDVI-ExG 0.9376 0.9426 0.8710 SAVI-ExG 0.9421 0.9512 0.8312

[0069] Table 1

[0070] As shown in Table 1, in the ground dataset, the SAVI-ExG method achieves the best accuracy and precision (0.9421 and 0.9521 respectively), while the NDVI-ExG method has the highest recall (0.8710). Therefore, SAVI-ExG can predict most green pixels, and its segmentation accuracy is the best among all compared methods. In terms of segmentation completeness, the NDVI-ExG algorithm is superior to the SAVI-ExG method. The RGB-based method has the lowest accuracy and precision. Although its recall is not the lowest, this is related to its excessive number of misclassified samples, so it is not very meaningful for reference. Its overall performance is the worst.

[0071] Table 2 is a comparison table of the metrics for segmenting green vegetation in the UAV dataset using different methods in this embodiment.

[0072] Accuracy↑ Precision↑ Recall↑ NDVI 0.9213 0.8245 0.8451 SAVI 0.9120 0.8864 0.8021 ExG 0.8921 0.8675 0.8721 RGB 0.7145 0.6578 0.8523 NDVI-ExG 0.9610 0.9257 0.8624 SAVI-ExG 0.9624 0.9812 0.8753

[0073] Table 2

[0074] As shown in Table 2, in the UAV dataset, the SAVI-ExG method achieved the best segmentation results, with Accuracy, Precision and Recall of 0.9624, 0.9812 and 0.8753, respectively.

[0075] To further compare the segmentation detail performance of the NDVI-ExG and SAVI-ExG methods in various scenarios, a more detailed comparison of the segmentation results of the two methods was conducted. Figure 7 This is a comparison chart of the segmentation results of the ground dataset by the NDVI-ExG and SAVI-ExG methods in this embodiment. Figure 8 This is a comparison chart of the segmentation results of the UAV dataset by the NDVI-ExG and SAVI-ExG methods in this embodiment. Figure 7 , Figure 8Column (a) shows sample images from two corresponding datasets, column (b) shows the NDVI-ExG segmentation results, and column (c) shows the SAVI-ExG segmentation results. Figure 7 As shown, for the first ground sample image, both methods can segment the green plants well, and the road and soil can be distinguished well; for the second ground sample image, although there are some green branches and leaves on the tree trunk in the yellow box and half of the tree trunk in the red box is covered with green moss, both methods can still segment the green parts well. Figure 8 As shown, there are many yellow-green interlaced positions in the drone sample image, especially the red box in row (2), and these pixels can be well distinguished by both methods from a visual perspective.

[0076] Overall, RGB image-based segmentation performed the worst, with nearly half of the pixels being misclassified. While ExG is sensitive to green, it struggles to distinguish between bright soil and green vegetation, resulting in a large number of soil pixels being mixed in the final result. Methods based solely on SAVI and NDVI can differentiate between vegetation and soil, but they are not sensitive to vegetation color, making them suitable only for distinguishing between soil and vegetation. The two-stage segmentation method combining vegetation index and color index effectively segmented green vegetation, achieving the best results.

[0077] Although the illustrative specific embodiments of the present invention have been described above to enable those skilled in the art to understand the invention, it should be understood that the invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the invention as defined and determined by the appended claims, and all inventions utilizing the concept of the present invention are protected.

Claims

1. A method for green vegetation segmentation in multispectral remote sensing images, characterized in that, Includes the following steps: S1: Extract channel images M from multispectral remote sensing images c c∈{R,G,B,NIR}, where R represents the red channel, G represents the green channel, B represents the blue channel, and NIR represents the near-infrared channel; S2: Determine the vegetation index and segmentation method according to actual needs, based on the 4-channel image M. c The vegetation index map is calculated, and then the vegetation index map is segmented using a segmentation method to obtain the pre-segmentation result; S3: Generate a pre-segmentation mask based on the pre-segmentation results, using the vegetation region as the region of interest with a value of 1, and other regions with values ​​of 0; apply the pre-segmentation mask to each channel image M. c Masking is performed separately to obtain channel images. S4: Determine the vegetation color index based on actual needs, using the 4-channel images. The color index map is calculated, and then the color index map is segmented using a segmentation method to obtain the green vegetation segmentation result.

2. The green vegetation segmentation method according to claim 1, characterized in that, In step S1, the channel image M is used. NIR As a benchmark, compare the other three channel images M c′ For registration preprocessing, c′∈{R,G,B}, the specific method is as follows: S1.1: For each channel image M c Construct Gaussian pyramids separately to obtain image representations at different scales. c,i , i = 1, 2, ..., L, where L represents the number of layers in the Gaussian pyramid; S1.2: Image representation at different scales I c,i Keypoint detection is performed separately to obtain a keypoint set P. c,i ; S1.3: Based on the detected keypoint set P c,i Representing I in image NIR,i The image representation of the other three channels is used as a benchmark. c′,i Registration is performed to obtain the image representation. S1.4: Represent I based on the registered L-layer image c′,i The channel image M is restored c′ .

3. The green vegetation segmentation method according to claim 2, characterized in that, In step S1.3, the restored channel image M is also processed. c′ The correction process is performed as follows: First, the restored channel image M... c′ Median filtering is performed to remove noise, and then grayscale correction is performed using the following formula: Among them, std NIR std c′ NIR image M NIR Image M to be corrected c′ grayscale standard deviation, Ave NIR Ave c′ NIR image M NIR Image M to be corrected c′ The average gray level.

4. The green vegetation segmentation method according to claim 1, characterized in that, In step S2, the vegetation index used is the Normalized Difference Vegetation Index (NDVI) or the SDVI (Soil Brightness Adjusted Vegetation Index).

5. The method for segmenting green vegetation according to claim 1, characterized in that, In step S4, the color index is either the super green index ExG or the super green minus the super red index ExG-ExR.

6. The green vegetation segmentation method according to claim 1, characterized in that, Step S4 also includes optimizing the vegetation segmentation results, specifically as follows: First, the vegetation segmentation results are opened using a 3×3 structuring element, and then Gaussian smoothing filter is applied to process the segmentation boundaries.

Images