Vegetation coverage detection methods, devices, equipment and storage media

Abstract

This application provides a method, apparatus, device, and storage medium for vegetation cover detection. The method includes: performing orthorectification and stitching fusion on an original image to obtain a remote sensing image to be detected; inputting the remote sensing image to be detected into a pre-trained vegetation segmentation model; having the encoder in the vegetation segmentation model extract overall and detailed features; having the decoder decode the extracted features to obtain a vegetation segmentation image; and finally determining the vegetation cover of the detection area based on the vegetation pixel information in the vegetation segmentation image. This application can effectively improve the segmentation effect of remote sensing images and achieves efficient and convenient intelligent measurement of vegetation cover.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and more specifically, to a method, apparatus, equipment, and storage medium for detecting vegetation cover. Background Technology

[0002] Vegetation, as an important component of terrestrial ecosystems, serves as a natural link between soil, atmosphere, and water, acting as an "indicator" in global energy transfer and material cycling. Furthermore, the distribution of vegetation in a region not only reflects the quality of the local ecological environment but is also closely related to local economic and social development; therefore, monitoring vegetation coverage has significant practical implications.

[0003] In existing technologies, vegetation coverage detection is performed in two ways. The first is to segment and extract vegetation in the image by setting the HSV (Hue Saturation Value, HSV color model) segmentation threshold range to obtain the vegetation coverage. The second is to first judge the pixels in the image and select the vegetation range line, then manually correct the vegetation range line, and finally determine the vegetation coverage based on the corrected vegetation range line.

[0004] However, the first method, based on color segmentation, suffers from poor adaptability; the second method requires manual correction of vegetation pixels, resulting in low automation and high labor costs. Therefore, how to achieve simple, efficient, and highly adaptable vegetation coverage detection has become an urgent problem to be solved. Summary of the Invention

[0005] The purpose of this application is to address the shortcomings of the prior art by providing a vegetation coverage detection method, apparatus, device, and storage medium to solve the problem that it is difficult to achieve simple, efficient, and highly adaptable vegetation coverage detection in the prior art.

[0006] To achieve the above objectives, the technical solution adopted in this application is as follows:

[0007] In a first aspect, this application provides a method for detecting vegetation coverage, the method comprising:

[0008] Acquire multiple raw images of the detection area;

[0009] Orthorectification and stitching fusion are performed on the multiple original images to obtain the remote sensing image of the detection area;

[0010] The remote sensing image to be detected is input into a pre-trained vegetation segmentation model. The encoder in the vegetation segmentation model extracts the overall features and details of the remote sensing image to be detected. The decoder in the vegetation segmentation model decodes the overall features and details extracted by the encoder to obtain the vegetation segmentation image corresponding to the remote sensing image to be detected.

[0011] The vegetation coverage of the detection area is determined based on the vegetation pixel information in the vegetation segmentation image.

[0012] Optionally, the step of performing orthorectification and stitching fusion on the plurality of original images to obtain the remote sensing image to be detected in the detection area includes:

[0013] Orthorectification is performed on each of the original images to obtain the orthorectified image of each original image and the spatial position information of the orthorectified image;

[0014] The orthophoto images are stitched together based on the spatial location information to obtain a stitched image.

[0015] The stitched image is then subjected to gap removal to obtain the remote sensing image to be detected.

[0016] Optionally, the encoder includes a lightweight feature extraction module, a pyramid pooling module, and a convolution module, and the decoder includes an efficient channel attention module and a convolution sampling module.

[0017] The process involves the encoder in the vegetation segmentation model extracting overall and detail features from the remote sensing image to be detected, and the decoder in the vegetation segmentation model decoding the extracted overall and detail features to obtain a vegetation segmentation image corresponding to the remote sensing image to be detected. This includes:

[0018] The lightweight feature extraction module extracts features from the remote sensing image to be detected to obtain multiple first feature vectors, and inputs the multiple first feature vectors into the pyramid pooling module and the efficient channel attention module in the decoder, respectively. The multiple first feature vectors are used to characterize the overall features of the remote sensing image to be detected.

[0019] The pyramid pooling module extracts features from the plurality of first feature vectors to obtain a plurality of second feature vectors, and concatenates the plurality of second feature vectors to obtain a concatenation vector, which is then input into the convolution module. The plurality of second feature vectors are used to characterize the detailed features of the remote sensing image to be detected.

[0020] The convolution module performs convolution processing on the connection vector, and the convolutional connection vector is input into the convolution sampling module of the decoder. The efficient channel attention module and the convolution sampling module obtain the vegetation segmentation image corresponding to the remote sensing image to be detected based on the multiple first feature vectors and the convolutional connection vector.

[0021] Optionally, the convolutional sampling module includes: a first convolutional layer, a first upsampling layer, a connection layer, a second convolutional layer, and a second upsampling layer;

[0022] The process of obtaining the vegetation segmentation image corresponding to the remote sensing image to be detected by the efficient channel attention module and the convolutional sampling module based on the multiple first feature vectors and the convolutional concatenation vector includes:

[0023] The efficient channel attention module enhances the multiple first feature vectors to obtain enhanced feature vectors, and then inputs the enhanced feature vectors into the first convolutional layer.

[0024] The enhanced feature vector is convolved by the first convolutional layer to obtain the convolved feature vector.

[0025] The first upsampling layer upsamples the convolutional connection vector to obtain the upsampled connection vector.

[0026] The convolutional feature vector and the upsampled concatenation vector are concatenated by the connection layer, and the concatenated vector is input into the second convolutional layer. After processing by the second convolutional layer and the second upsampling layer, the vegetation segmentation image corresponding to the remote sensing image to be detected is obtained.

[0027] Optionally, determining the vegetation coverage of the detection area based on the vegetation pixel information in the vegetation segmentation image includes:

[0028] Based on the vegetation pixel information in the vegetation segmentation image, determine the total number of vegetation pixels in the vegetation segmentation image;

[0029] The ratio of the total number of vegetation pixels to the total number of pixels in the vegetation segmentation image is used as the vegetation coverage rate of the detection area.

[0030] Optionally, before inputting the stitched image into the pre-trained vegetation segmentation model, the method further includes:

[0031] Acquire sample images and process them to obtain multiple labeled samples;

[0032] Each labeled sample is input into the initial vegetation segmentation model. The encoder in the initial vegetation segmentation model extracts the overall features and details of each labeled sample. The decoder in the initial vegetation segmentation model decodes the overall features and details extracted by the encoder to obtain the predicted segmentation image of each labeled sample.

[0033] The initial vegetation segmentation model is iteratively corrected based on the predicted segmentation images of each labeled sample and the labeled information to obtain the vegetation segmentation model.

[0034] Optionally, the step of acquiring and processing sample images to obtain multiple labeled samples includes:

[0035] Multiple sample images are acquired, and the sample images are orthorectified and stitched together to obtain multiple stitched sample images;

[0036] Add annotation information to each of the stitched sample images to obtain multiple annotated samples. The annotation information includes: vegetation areas and non-vegetation areas.

[0037] Secondly, this application provides a vegetation coverage detection device, the device comprising:

[0038] The acquisition module is used to acquire multiple raw images of the detection area;

[0039] The preprocessing module is used to perform orthorectification and stitching fusion on the multiple original images to obtain the remote sensing image to be detected in the detection area;

[0040] The segmentation module is used to input the remote sensing image to be detected into a pre-trained vegetation segmentation model. The encoder in the vegetation segmentation model extracts the overall features and details of the remote sensing image to be detected, and the decoder in the vegetation segmentation model decodes the overall features and details extracted by the encoder to obtain the vegetation segmentation image corresponding to the remote sensing image to be detected.

[0041] The determination module is used to determine the vegetation coverage of the detection area based on the vegetation pixel information in the vegetation segmentation image.

[0042] Optionally, the preprocessing module is specifically used for:

[0043] Orthorectification is performed on each of the original images to obtain the orthorectified image of each original image and the spatial position information of the orthorectified image;

[0044] The orthophoto images are stitched together based on the spatial location information to obtain a stitched image.

[0045] The stitched image is then subjected to gap removal to obtain the remote sensing image to be detected.

[0046] Optionally, the encoder includes a lightweight feature extraction module, a pyramid pooling module, and a convolution module, and the decoder includes an efficient channel attention module and a convolution sampling module.

[0047] The segmentation module is specifically used for:

[0048] The lightweight feature extraction module extracts features from the remote sensing image to be detected to obtain multiple first feature vectors, and inputs the multiple first feature vectors into the pyramid pooling module and the efficient channel attention module in the decoder, respectively. The multiple first feature vectors are used to characterize the overall features of the remote sensing image to be detected.

[0049] The pyramid pooling module extracts features from the plurality of first feature vectors to obtain a plurality of second feature vectors, and concatenates the plurality of second feature vectors to obtain a concatenation vector, which is then input into the convolution module. The plurality of second feature vectors are used to characterize the detailed features of the remote sensing image to be detected.

[0050] The convolution module performs convolution processing on the connection vector, and the convolutional connection vector is input into the convolution sampling module of the decoder. The efficient channel attention module and the convolution sampling module obtain the vegetation segmentation image corresponding to the remote sensing image to be detected based on the multiple first feature vectors and the convolutional connection vector.

[0051] Optionally, the convolutional sampling module includes: a first convolutional layer, a first upsampling layer, a connection layer, a second convolutional layer, and a second upsampling layer;

[0052] The segmentation module is also specifically used for:

[0053] The efficient channel attention module enhances the multiple first feature vectors to obtain enhanced feature vectors, and then inputs the enhanced feature vectors into the first convolutional layer.

[0054] The enhanced feature vector is convolved by the first convolutional layer to obtain the convolved feature vector.

[0055] The first upsampling layer upsamples the convolutional connection vector to obtain the upsampled connection vector.

[0056] The convolutional feature vector and the upsampled concatenation vector are concatenated by the connection layer, and the concatenated vector is input into the second convolutional layer. After processing by the second convolutional layer and the second upsampling layer, the vegetation segmentation image corresponding to the remote sensing image to be detected is obtained.

[0057] Optionally, the determining module is specifically used for:

[0058] Based on the vegetation pixel information in the vegetation segmentation image, determine the total number of vegetation pixels in the vegetation segmentation image;

[0059] The ratio of the total number of vegetation pixels to the total number of pixels in the vegetation segmentation image is used as the vegetation coverage rate of the detection area.

[0060] Thirdly, this application provides an electronic device, including: a processor, a storage medium, and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of the vegetation coverage detection method described above.

[0061] Fourthly, this application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the above-described vegetation coverage detection method.

[0062] The beneficial effects of this application are as follows: By orthorectifying and stitching together the original image, a more standardized remote sensing image to be detected can be obtained, and the accuracy of subsequent vegetation segmentation model prediction can be improved. By improving the encoder and decoder in the vegetation segmentation model, the encoder can extract both global and multi-scale local features of the image to be detected, expanding the receptive field of feature extraction and making it more adaptable to the high resolution characteristics of remote sensing images. The decoder performs decoding processing based on both global and local features, and incorporates an attention mechanism, enabling it to more comprehensively focus on each segmentation target in the image to be detected, thereby improving the segmentation effect. Attached Figure Description

[0063] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0064] Figure 1This illustration shows a schematic diagram of the image effect before and after segmentation according to an embodiment of this application;

[0065] Figure 2 A flowchart of a vegetation cover detection method provided in an embodiment of this application is shown;

[0066] Figure 3 A structural diagram of a vegetation segmentation model provided in an embodiment of this application is shown;

[0067] Figure 4 This document illustrates a flowchart of an embodiment of obtaining a remote sensing image to be detected.

[0068] Figure 5 This paper illustrates a detailed architecture diagram of a vegetation segmentation model provided in an embodiment of this application.

[0069] Figure 6 This document illustrates a flowchart of image processing using a vegetation segmentation model provided in an embodiment of this application.

[0070] Figure 7 This document illustrates a flowchart of image processing using another vegetation segmentation model provided in an embodiment of this application.

[0071] Figure 8 This application provides a flowchart for calculating vegetation coverage according to an embodiment of the present application.

[0072] Figure 9 This document illustrates a flowchart of a vegetation segmentation model trained according to an embodiment of this application.

[0073] Figure 10 This application provides a flowchart of an embodiment for obtaining labeled samples;

[0074] Figure 11 This paper shows a schematic diagram of the structure of a vegetation coverage detection device provided in an embodiment of this application;

[0075] Figure 12 A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0076] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0077] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0078] It should be noted that the term "comprising" will be used in the embodiments of this application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0079] In existing technologies, vegetation coverage detection is performed in two ways. The first is to segment and extract vegetation in the image by setting an HSV segmentation threshold range to obtain the vegetation coverage. The second is to first judge the pixels in the image and select the vegetation range line, then manually correct the vegetation range line, and finally determine the vegetation coverage based on the corrected vegetation range line.

[0080] The first method segments the detection image by setting an HSV segmentation threshold range. Since vegetation is mainly green, the segmentation threshold range can be determined based on the component range of green in the HSV color space. Then, noise processing is performed on the segmented image to obtain the vegetation segmentation result.

[0081] However, this method, which only uses color for segmentation and extraction, has poor adaptability. For example, it is difficult to distinguish when there are non-vegetation objects in the detected image that appear in the threshold color range.

[0082] The second method requires manual correction of any errors in the system's initial assessment after the vegetation boundary line has been selected. However, manual correction suffers from low automation and high labor costs.

[0083] Therefore, how to achieve simple, efficient and highly adaptable vegetation coverage monitoring has become an urgent problem to be solved.

[0084] To address the aforementioned issues, this application proposes a vegetation coverage detection method. An improved semantic segmentation network is trained using sample images to obtain a vegetation segmentation model. This model is then used to extract features from orthorectified and stitched remote sensing images of the detection area, resulting in the final vegetation segmentation image. This method can effectively improve the efficiency of vegetation coverage calculation in land resource statistics, agricultural monitoring, and disaster surveys.

[0085] like Figure 1 The diagram shown is an application scenario illustration provided in this application. Figure 1 ,Will Figure 1 The remote sensing image to be detected, after orthorectification and stitching fusion of the original image, is input into the vegetation segmentation model. The vegetation segmentation model outputs the final vegetation segmentation image, in which vegetation-covered areas are black and vegetation-uncovered areas are white. Therefore, the vegetation coverage rate can be determined based on the proportion of black pixels in the image.

[0086] The following is a description of the vegetation coverage detection method in this application. The subject of this method can be electronic equipment, such as... Figure 2 As shown, the method includes:

[0087] S201: Acquire multiple raw images of the detection area.

[0088] Optionally, the detection area can be an area where vegetation coverage needs to be detected. For example, the detection area can be mountainous areas, cities, towns, etc., where vegetation coverage needs to be detected.

[0089] In this application, the location information and related geographical information of the detection area can be obtained from a 3D map. The location information can be the latitude and longitude of the detection area, and the geographical information can be the slope, mountain direction, etc. of the detection area.

[0090] For example, based on the location and geographical information of the detection area, a drone can be controlled to take off to a set position and altitude, and vertically photograph the detection area to obtain multiple raw images of the detection area, and the coordinates and shooting angle of the drone during the shooting can be recorded.

[0091] S202: Perform orthorectification and stitching fusion on multiple original images to obtain the remote sensing image of the detection area.

[0092] Optionally, in this application, multiple original images can be orthorectified based on the location information and geographic information of the detection area, as well as the shooting coordinates and shooting angle information of the UAV, so as to simultaneously perform tilt correction and projection difference correction on the images and resample the images into orthorectified images.

[0093] It is worth noting that when performing orthorectification on the original image, the location and geographic information of the original image can be added to the attribute information of the orthophoto, so that the orthophoto can be stitched and fused based on the location and geographic information.

[0094] After orthorectifying the original images to obtain orthophotos of each original image, this application can stitch together and fuse the orthophotos of each original image in the detection area to obtain the remote sensing image to be detected in the detection area.

[0095] S203: Input the remote sensing image to be detected into the pre-trained vegetation segmentation model. The encoder in the vegetation segmentation model extracts the overall features and details of the remote sensing image to be detected. The decoder in the vegetation segmentation model decodes the overall features and details extracted by the encoder to obtain the vegetation segmentation image corresponding to the remote sensing image to be detected.

[0096] Optionally, the vegetation segmentation model can be a model trained on the improved semantic segmentation network deeplabv3+ using pre-labeled sample images.

[0097] like Figure 3 The diagram shown is a schematic of the architecture of a vegetation segmentation model proposed in this application. The encoder is used to extract overall features and detail features from the input remote sensing image to be detected. Overall feature extraction can extract the overall features in the image to be detected, while detail feature extraction can expand the receptive field based on the extracted overall features and further extract more local features at more scales in the image to be detected.

[0098] The decoder can decode the features extracted by the encoder and assign a semantic category to each pixel in the image based on the previously extracted overall and local features, thereby obtaining the vegetation segmentation image corresponding to the remote sensing image to be detected. The semantic category can include: vegetation and non-vegetation.

[0099] S204: Determine the vegetation coverage of the detection area based on the vegetation pixel information in the vegetation segmentation image.

[0100] Optionally, vegetation pixel information may include: the number of pixels in the vegetation segmentation image, and the semantic category of each pixel in the vegetation segmentation image.

[0101] The vegetation coverage of the detection area can be determined by the proportion of pixels with the semantic category of vegetation in the vegetation segmentation image.

[0102] In this embodiment, the original image is first orthorectified and stitched together to obtain the remote sensing image to be detected. Then, the remote sensing image to be detected is input into a pre-trained vegetation segmentation model. The encoder in the vegetation segmentation model extracts features, and the decoder decodes the extracted features to obtain a vegetation segmentation image. Finally, the vegetation coverage of the detection area is determined based on the vegetation pixel information in the vegetation segmentation image.

[0103] By orthorectifying and stitching together the original images, a more standardized remote sensing image to be detected can be obtained, improving the accuracy of subsequent vegetation segmentation model predictions. Improvements to the encoder and decoder in the vegetation segmentation model enable the encoder to extract both global and multi-scale local features of the image to be detected, expanding the receptive field of feature extraction and making it more adaptable to the high resolution of remote sensing images. The decoder performs decoding based on both global and local features and incorporates an attention mechanism, enabling it to more comprehensively focus on each segmentation target in the image to be detected, thereby improving segmentation performance.

[0104] The following is a further explanation of the orthorectification and stitching fusion of multiple original images in step S202 above to obtain the remote sensing image of the detection area. Figure 4 As shown, the above step S202 includes:

[0105] S401: Perform orthorectification on each original image to obtain the orthophoto of each original image and the spatial location information of the orthophoto.

[0106] In this application, the location information and geographical information of the detection area, as well as the shooting angle information and shooting coordinate information of the UAV, can be used to orthorectify each original image to obtain the orthorectified image of each original image.

[0107] The spatial location information of orthophotos can include the location information and geographic information of the original image corresponding to the orthophoto.

[0108] S402: Stitch together the orthophotos based on their spatial location information to obtain the stitched image.

[0109] Optionally, based on the spatial location information of each orthophoto, the location of the region contained in the orthophoto within the detection area, as well as the positional relationship of the regions contained in each orthophoto, can be determined. Therefore, each orthophoto can be arranged according to the location of its contained region within the detection area based on the spatial location information, and the orthophotos can be stitched together according to the arrangement result to obtain the stitched image.

[0110] S403: Perform gap removal on the stitched image to obtain the remote sensing image to be detected.

[0111] Optionally, the gaps between adjacent orthophotos in the stitched image can be fused to eliminate the stitching seams and obtain the remote sensing image to be detected.

[0112] In this embodiment of the application, by performing orthorectification and stitching fusion on the original image, the original image can be standardized, thereby improving the accuracy of subsequent vegetation segmentation of the image.

[0113] like Figure 5 The diagram shown is a schematic representation of the specific architecture of a vegetation segmentation model provided in this application. Figure 5 The encoder includes a lightweight feature extraction module, a pyramid pooling module, and a convolution module, while the decoder includes an efficient channel attention module and a convolution sampling module.

[0114] In the vegetation segmentation model, the encoder part optimizes the backbone network and pyramid pooling layer. The lightweight feature extraction module can be the lightweight feature extraction network MobileNetV3, and the pyramid pooling module can be the Dense ASPP module obtained by improving the Atrous Spatial Pyramid Pooling (ASPP) module in a dense connection manner.

[0115] Furthermore, an attention mechanism is introduced into the decoder part of the vegetation segmentation model to improve the segmentation effect.

[0116] The efficient channel attention module can be an Efficient Channel Attention (ECA) module. ECA uses a local cross-channel interaction strategy to avoid dimensionality reduction, and the size of the one-dimensional convolutional kernel used to generate channel weights is adaptive, enabling channel feature enhancement of the input feature map. The convolutional sampling module can include multiple convolutional layers and upsampling layers.

[0117] In step S203 above, the encoder in the vegetation segmentation model extracts overall and detailed features from the remote sensing image to be detected, and the decoder in the vegetation segmentation model decodes the overall and detailed features extracted by the encoder to obtain the vegetation segmentation image corresponding to the remote sensing image to be detected, such as... Figure 6 As shown, it includes:

[0118] S601: The lightweight feature extraction module extracts features from the remote sensing image to be detected, obtaining multiple first feature vectors. These multiple first feature vectors are then input into the pyramid pooling module and the efficient channel attention module in the decoder, respectively. The multiple first feature vectors are used to characterize the overall features of the remote sensing image to be detected.

[0119] It should be noted that remote sensing images have high resolution, and using traditional semantic segmentation networks may result in high computational cost and low algorithm efficiency. Therefore, this application uses a lightweight feature extraction module to extract features from the remote sensing images to be detected, which can improve the algorithm's running speed.

[0120] Reference Figure 5 The lightweight feature extraction module can include multiple convolutional blocks, the number of which can be set by the user. Each convolutional block sequentially extracts features from the input remote sensing image to be detected, obtaining multiple first feature vectors, and then inputs these first feature vectors into the pyramid pooling module and the efficient channel attention module, respectively.

[0121] S602: The pyramid pooling module extracts features from multiple first feature vectors to obtain multiple second feature vectors, and then concatenates the multiple second feature vectors to obtain a concatenation vector. The concatenation vector is then input into the convolution module. The multiple second feature vectors are used to characterize the detailed features of the remote sensing image to be detected.

[0122] Reference Figure 5 DenseASPP employs skip-layer connections to share information, and convolutions with varying dilatancy rates are interconnected to form a dense feature pyramid, thereby achieving a larger receptive field. Figure 5 The receptive field is denoted by r. It can be seen that the pyramid pooling module obtains receptive fields of different sizes through dilated convolutions with different expansion rates.

[0123] By extracting features from the first feature vector at different scales using receptive fields of different sizes, multiple second feature vectors can be obtained. Then, these second feature vectors are concatenated to obtain a concatenation vector. These multiple second feature vectors are used to characterize the detailed features of the remote sensing image to be detected. Therefore, the concatenation vector can characterize the detailed features of the remote sensing image to be detected under different receptive fields.

[0124] S603: The convolution module performs convolution processing on the connection vectors and inputs the convolutional connection vectors into the convolutional sampling module of the decoder. The efficient channel attention module and the convolutional sampling module obtain the vegetation segmentation image corresponding to the remote sensing image to be detected based on multiple first feature vectors and the convolutional connection vectors.

[0125] Optionally, the convolution module can perform convolution processing on the connection vector output by the pyramid pooling module to obtain the convolved connection vector. For example, the connection vector can be subjected to 1×1 convolution processing.

[0126] Optionally, the efficient channel attention module of the decoder receives the first feature vector input to it from the lightweight feature extraction module, and the convolution sampling module receives the convolutional connection vector input to it from the convolution module. The decoder can decode multiple first feature vectors and convolutional connection vectors to obtain the vegetation segmentation image corresponding to the remote sensing image to be detected.

[0127] Reference Figure 5 The convolutional sampling module includes: a first convolutional layer, a first upsampling layer, a connection layer, a second convolutional layer, and a second upsampling layer.

[0128] The first convolutional layer can be a 1×1 convolutional block, and the second convolutional layer can be a 3×3 convolutional block.

[0129] Furthermore, the step in S603 above, where the efficient channel attention module and the convolutional sampling module obtain the vegetation segmentation image corresponding to the remote sensing image to be detected based on multiple first feature vectors and the convolutional concatenation vector, is as follows: Figure 7 As shown, it includes:

[0130] S701: The efficient channel attention module enhances multiple first feature vectors to obtain enhanced feature vectors, and then inputs the enhanced feature vectors into the first convolutional layer.

[0131] Optionally, the efficient channel attention module adopts a local cross-channel interaction strategy to avoid dimensionality reduction. The size of the one-dimensional convolutional kernel used to generate channel weights is adaptive, which can enhance the channel features of the input first feature vector to obtain the enhanced feature vector.

[0132] By introducing an efficient channel attention module, a significant performance gain can be achieved by adding a small number of parameters. Furthermore, by incorporating the ECA attention mechanism, the vegetation segmentation model can focus more on the target to be segmented, thereby improving the segmentation effect.

[0133] S702: The enhanced feature vector is convolved by the first convolutional layer to obtain the convolved feature vector.

[0134] Optionally, the first convolutional layer can perform a 1×1 convolution on the enhanced feature vector to obtain the convolutional feature vector.

[0135] It should be noted that the size of the first convolutional layer, the second convolutional layer, and the convolutional module in this application can be set according to the actual situation. This application only provides a possible size example, and the specific size of the convolutional block is not limited here.

[0136] S703: The first upsampling layer upsamples the convolutional connection vector to obtain the upsampled connection vector.

[0137] Optionally, the first upsampling layer can upsample the convolutional connection vector input from the pyramid pooling module to obtain an upsampled connection vector. The upsampling process can be a quadrupled upsampling of the convolutional connection vector.

[0138] S704: The convolutional feature vector and the upsampled concatenation vector are concatenated by the concatenation layer, and the concatenated vector is input into the second convolutional layer. After processing by the second convolutional layer and the second upsampling layer, the vegetation segmentation image corresponding to the remote sensing image to be detected is obtained.

[0139] Optionally, the connection layer can concatenate the convolutional feature vector and the upsampled connection vector, and input the concatenated vector into the second convolutional layer for convolution processing. The second convolutional layer can perform 3×3 convolution processing on the concatenated vector.

[0140] After the second convolutional layer completes the convolution process, the vector after convolution can be input into the second upsampling layer for upsampling. The second upsampling layer can upsample the vector after convolution by a factor of four to obtain the final vegetation segmentation image.

[0141] It is worth noting that the decoder performs the above decoding process, which is the process of segmenting the remote sensing image to be detected based on the extracted features. In this process, semantic classification is performed on each pixel in the image. After completing the semantic classification of all pixels, the vegetation segmentation image is obtained.

[0142] In this embodiment, by employing a lightweight feature extraction module, the algorithm's running speed can be improved, making it more suitable for the detection requirements of remote sensing images; by adopting the DenseASPP module with dense connections, the receptive field can be expanded without sacrificing feature space resolution, and feature reuse efficiency can be improved; by introducing an efficient channel attention module ECA in the decoder, clear target boundaries can be restored, making the vegetation segmentation model pay more attention to the target to be segmented and improving the segmentation effect.

[0143] The following is a further explanation of determining the vegetation coverage of the detection area based on the vegetation pixel information in the vegetation segmentation image, such as... Figure 8 As shown, the above step S204 includes:

[0144] S801: Determine the total number of vegetation pixels in the vegetation segmentation image based on the vegetation pixel information in the vegetation segmentation image.

[0145] Optionally, vegetation pixel information can indicate the number of pixels in the vegetation segmentation image and the semantic classification result of each pixel. Therefore, the total number of pixels in the vegetation segmentation image whose semantic classification result is vegetation can be determined, i.e., the total number of vegetation pixels.

[0146] S802: The ratio of the total number of vegetation pixels to the total number of pixels in the vegetation segmentation image is used as the vegetation coverage of the detection area.

[0147] In this application, the proportion of the total number of vegetation pixels in the total number of pixels in the vegetation segmentation image can be used as the vegetation coverage of the detection area.

[0148] Before inputting the stitched image into the pre-trained vegetation segmentation model, the model needs to be trained first to obtain the vegetation segmentation model, such as... Figure 9 As shown, the process includes:

[0149] S901: Acquire sample images and process them to obtain multiple labeled samples.

[0150] Optionally, the sample images can be multiple original images of multiple detection areas captured by a drone, with multiple original images captured for each detection area. Processing the sample images can involve orthorectifying and stitching together sample images belonging to the same detection area to obtain a remote sensing image of the detection area, and then labeling the remote sensing image.

[0151] Optionally, the labeled samples may include labeling information, which may be vegetation pixel information pre-added by the user to indicate vegetation areas and non-vegetation areas in the image.

[0152] S902: Input each labeled sample into the initial vegetation segmentation model. The encoder in the initial vegetation segmentation model extracts the overall features and details of each labeled sample. The decoder in the vegetation segmentation model decodes the overall features and details extracted by the encoder to obtain the predicted segmentation image of each labeled sample.

[0153] After the labeled samples are input into the initial vegetation segmentation model, the encoder and decoder in the model process them sequentially to obtain the predicted segmentation images corresponding to each labeled sample. The steps for the initial vegetation segmentation model to perform prediction and classification can be the same as steps S601-S603 described above, and will not be repeated here.

[0154] S903: The initial vegetation segmentation model is iteratively corrected based on the predicted segmentation images and annotation information of each labeled sample to obtain the vegetation segmentation model.

[0155] In this application, the loss value can be calculated based on the vegetation pixel information in the predicted segmentation image of each labeled sample and the pre-labeled vegetation pixel information, and the initial vegetation segmentation model can be iteratively corrected based on the loss value to obtain the vegetation segmentation model.

[0156] Furthermore, the steps in S901 above, such as acquiring sample images and processing them to obtain multiple labeled samples, are as follows: Figure 10 As shown, it includes:

[0157] S1001: Acquire multiple sample images, perform orthorectification and stitching fusion on the sample images to obtain multiple stitched sample images.

[0158] In this application, the location and geographic information of the detection area can be obtained through a 3D map, and a drone can be controlled to take vertical pictures of the detection area to obtain multiple sample images of the detection area.

[0159] After orthorectifying multiple sample images of the detection area, the images can be stitched together according to the location and geographic information of each sample image, and the stitching seams can be fused to obtain the stitched sample image of the detection area.

[0160] S1002: Add annotation information to each stitched sample image to obtain multiple annotated samples. The annotation information includes: vegetation area and non-vegetation area.

[0161] Optionally, users can manually divide the stitched sample images into vegetated and non-vegetated areas and add annotation information to the stitched sample images. For example, the Labelme image annotation tool can be used to annotate the stitched images with vegetated and non-vegetated areas.

[0162] It is worth noting that this application can take sample images of multiple detection areas with different geographical conditions, such as taking multiple sample images of forests, towns and cities respectively, to obtain various types of labeled samples for training the model, so as to improve the accuracy of the model for vegetation segmentation in different geographical conditions.

[0163] Based on the same inventive concept, this application also provides a vegetation coverage detection device corresponding to the vegetation coverage detection method. Since the principle of the device in this application is similar to the vegetation coverage detection method described above in this application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0164] Reference Figure 11 The diagram shown is a schematic of a vegetation coverage detection device provided in an embodiment of this application. The device includes: an acquisition module 1101, a preprocessing module 1102, a segmentation module 1103, and a determination module 1104.

[0165] The acquisition module 1101 is used to acquire multiple original images of the detection area;

[0166] The preprocessing module 1102 is used to perform orthorectification and stitching fusion on multiple original images to obtain the remote sensing image of the detection area;

[0167] The segmentation module 1103 is used to input the remote sensing image to be detected into the pre-trained vegetation segmentation model. The encoder in the vegetation segmentation model extracts the overall features and details of the remote sensing image to be detected, and the decoder in the vegetation segmentation model decodes the overall features and details extracted by the encoder to obtain the vegetation segmentation image corresponding to the remote sensing image to be detected.

[0168] The determination module 1104 is used to determine the vegetation coverage of the detection area based on the vegetation pixel information in the vegetation segmentation image.

[0169] Optionally, the preprocessing module 1102 is specifically used for:

[0170] Orthorectification is performed on each original image to obtain the orthorectified image of each original image and the spatial location information of the orthorectified image;

[0171] The orthophotos are stitched together based on their spatial location information to obtain the stitched image.

[0172] The stitched images are then subjected to gap removal to obtain the remote sensing image to be detected.

[0173] Optionally, the encoder includes a lightweight feature extraction module, a pyramid pooling module, and a convolution module, and the decoder includes an efficient channel attention module and a convolution sampling module.

[0174] The segmentation module 1103 is specifically used for:

[0175] The lightweight feature extraction module extracts features from the remote sensing image to be detected, obtaining multiple first feature vectors. These multiple first feature vectors are then input into the pyramid pooling module and the efficient channel attention module in the decoder, respectively. The multiple first feature vectors are used to characterize the overall features of the remote sensing image to be detected.

[0176] The pyramid pooling module extracts features from multiple first feature vectors to obtain multiple second feature vectors. The multiple second feature vectors are then concatenated to obtain a concatenation vector, which is then input into the convolution module. The multiple second feature vectors are used to characterize the detailed features of the remote sensing image to be detected.

[0177] The convolution module performs convolution processing on the connection vectors, and the convolutional connection vectors are input into the convolutional sampling module of the decoder. The efficient channel attention module and the convolutional sampling module obtain the vegetation segmentation image corresponding to the remote sensing image to be detected based on multiple first feature vectors and the convolutional connection vectors.

[0178] Optionally, the convolutional sampling module includes: a first convolutional layer, a first upsampling layer, a connection layer, a second convolutional layer, and a second upsampling layer;

[0179] The segmentation module 1103 is also specifically used for:

[0180] The efficient channel attention module enhances multiple first feature vectors to obtain enhanced feature vectors, and then inputs the enhanced feature vectors into the first convolutional layer.

[0181] The enhanced feature vector is convolved by the first convolutional layer to obtain the convolved feature vector;

[0182] The first upsampling layer upsamples the convolutional connection vector to obtain the upsampled connection vector.

[0183] The convolutional feature vector and the upsampled concatenation vector are concatenated by the concatenation layer, and the concatenated vector is input into the second convolutional layer. After processing by the second convolutional layer and the second upsampling layer, the vegetation segmentation image corresponding to the remote sensing image to be detected is obtained.

[0184] Optionally, module 1104 is specifically used for:

[0185] Based on the vegetation pixel information in the vegetation segmentation image, determine the total number of vegetation pixels in the vegetation segmentation image;

[0186] The ratio of the total number of vegetation pixels to the total number of pixels in the vegetation segmentation image is used as the vegetation coverage rate of the detection area.

[0187] Optionally, the device also includes a training module for:

[0188] Acquire sample images and process them to obtain multiple labeled samples;

[0189] Each labeled sample is input into the initial vegetation segmentation model. The encoder in the initial vegetation segmentation model extracts the overall features and details of each labeled sample. The decoder in the initial vegetation segmentation model decodes the overall features and details extracted by the encoder to obtain the predicted segmentation image of each labeled sample.

[0190] The initial vegetation segmentation model is iteratively corrected based on the predicted segmentation images and annotation information of each labeled sample to obtain the vegetation segmentation model.

[0191] Optionally, the training module is also specifically used for:

[0192] Multiple sample images are acquired, and orthorectified and stitched together to obtain multiple stitched sample images.

[0193] Add annotation information to each stitched sample image to obtain multiple annotated samples. The annotation information includes: vegetation area and non-vegetation area.

[0194] The processing flow of each module in the device and the interaction flow between each module can be referred to the relevant descriptions in the above method embodiments, and will not be detailed here.

[0195] This application's embodiments, through orthorectification and stitching fusion of the original image, can obtain a more standardized remote sensing image to be detected, and improve the accuracy of subsequent vegetation segmentation model predictions. By improving the encoder and decoder in the vegetation segmentation model, the encoder can extract the overall features and multi-scale local features of the image to be detected, expanding the receptive field of feature extraction and making it more adaptable to the high-resolution characteristics of remote sensing images. In the decoder part, decoding processing is performed based on overall and local features, and an attention mechanism is incorporated to comprehensively focus on each segmentation target in the image to be detected, thereby improving the segmentation effect.

[0196] This application also provides an electronic device, such as... Figure 12 The diagram shown is a schematic representation of an electronic device structure provided in an embodiment of this application, including: a processor 1201, a memory 1202, and a bus. The memory 1202 stores machine-readable instructions executable by the processor 1201 (e.g., ...). Figure 11 The device includes the acquisition module 1101, preprocessing module 1102, segmentation module 1103, and determination module 1104 (and the corresponding execution instructions, etc.). When the computer device is running, the processor 1201 and the memory 1202 communicate via a bus. When the machine-readable instructions are executed by the processor 1201, the above-mentioned vegetation coverage detection method is processed.

[0197] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the above-described vegetation coverage detection method.

[0198] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the method embodiments, and will not be repeated here. In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection can be through some communication interfaces; the indirect coupling or communication connection of devices or modules can be electrical, mechanical, or other forms.

[0199] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. If the functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0200] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A method for detecting vegetation cover, characterized in that, include: Acquire multiple raw images of the detection area; Orthorectification and stitching fusion are performed on the multiple original images to obtain the remote sensing image of the detection area; The remote sensing image to be detected is input into a pre-trained vegetation segmentation model. The encoder in the vegetation segmentation model extracts the overall features and details of the remote sensing image to be detected. The decoder in the vegetation segmentation model decodes the overall features and details extracted by the encoder to obtain the vegetation segmentation image corresponding to the remote sensing image to be detected. The vegetation coverage of the detection area is determined based on the vegetation pixel information in the vegetation segmentation image. The encoder includes a lightweight feature extraction module, a pyramid pooling module, and a convolution module; the decoder includes an efficient channel attention module and a convolution sampling module. The process involves the encoder in the vegetation segmentation model extracting overall and detail features from the remote sensing image to be detected, and the decoder in the vegetation segmentation model decoding the extracted overall and detail features to obtain a vegetation segmentation image corresponding to the remote sensing image to be detected. This includes: The lightweight feature extraction module extracts features from the remote sensing image to be detected to obtain multiple first feature vectors, and inputs the multiple first feature vectors into the pyramid pooling module and the efficient channel attention module in the decoder, respectively. The multiple first feature vectors are used to characterize the overall features of the remote sensing image to be detected. The pyramid pooling module extracts features from the plurality of first feature vectors to obtain a plurality of second feature vectors, and concatenates the plurality of second feature vectors to obtain a concatenation vector, which is then input into the convolution module. The plurality of second feature vectors are used to characterize the detailed features of the remote sensing image to be detected. The convolution module performs convolution processing on the connection vector, and the convolutional connection vector is input into the convolution sampling module of the decoder. The efficient channel attention module and the convolution sampling module obtain the vegetation segmentation image corresponding to the remote sensing image to be detected based on the multiple first feature vectors and the convolutional connection vector.

2. The method according to claim 1, characterized in that, The process of orthorectifying and stitching together the multiple original images to obtain the remote sensing image of the detection area includes: Orthorectification is performed on each of the original images to obtain the orthorectified image of each original image and the spatial position information of the orthorectified image; The orthophotos are stitched together based on the spatial location information to obtain a stitched image. The stitched image is then subjected to gap removal to obtain the remote sensing image to be detected.

3. The method according to claim 1, characterized in that, The convolutional sampling module includes: a first convolutional layer, a first upsampling layer, a connection layer, a second convolutional layer, and a second upsampling layer; The process of obtaining the vegetation segmentation image corresponding to the remote sensing image to be detected by the efficient channel attention module and the convolutional sampling module based on the multiple first feature vectors and the convolutional concatenation vector includes: The efficient channel attention module enhances the multiple first feature vectors to obtain enhanced feature vectors, and then inputs the enhanced feature vectors into the first convolutional layer. The enhanced feature vector is convolved by the first convolutional layer to obtain the convolved feature vector. The first upsampling layer upsamples the convolutional connection vector to obtain the upsampled connection vector. The convolutional feature vector and the upsampled concatenation vector are concatenated by the connection layer, and the concatenated vector is input into the second convolutional layer. After processing by the second convolutional layer and the second upsampling layer, the vegetation segmentation image corresponding to the remote sensing image to be detected is obtained.

4. The method according to claim 1, characterized in that, Determining the vegetation coverage of the detection area based on the vegetation pixel information in the vegetation segmentation image includes: Based on the vegetation pixel information in the vegetation segmentation image, determine the total number of vegetation pixels in the vegetation segmentation image; The ratio of the total number of vegetation pixels to the total number of pixels in the vegetation segmentation image is used as the vegetation coverage rate of the detection area.

5. The method according to claim 1, characterized in that, Before inputting the remote sensing image to be detected into the pre-trained vegetation segmentation model, the method further includes: Acquire sample images and process them to obtain multiple labeled samples; Each labeled sample is input into the initial vegetation segmentation model. The encoder in the initial vegetation segmentation model extracts the overall features and details of each labeled sample. The decoder in the initial vegetation segmentation model decodes the overall features and details extracted by the encoder to obtain the predicted segmentation image of each labeled sample. The initial vegetation segmentation model is iteratively corrected based on the predicted segmentation images and annotation information of each labeled sample to obtain the vegetation segmentation model.

6. The method according to claim 5, characterized in that, The process of acquiring and processing sample images yields multiple labeled samples, including: Multiple sample images are acquired, and the sample images are orthorectified and stitched together to obtain multiple stitched sample images; Add annotation information to each of the stitched sample images to obtain multiple annotated samples. The annotation information includes: vegetation areas and non-vegetation areas.

7. A vegetation cover detection device, characterized in that, include: The acquisition module is used to acquire multiple raw images of the detection area; The preprocessing module is used to perform orthorectification and stitching fusion on the multiple original images to obtain the remote sensing image to be detected in the detection area; The segmentation module is used to input the remote sensing image to be detected into a pre-trained vegetation segmentation model. The encoder in the vegetation segmentation model extracts the overall features and details of the remote sensing image to be detected, and the decoder in the vegetation segmentation model decodes the overall features and details extracted by the encoder to obtain the vegetation segmentation image corresponding to the remote sensing image to be detected. The determination module is used to determine the vegetation coverage of the detection area based on the vegetation pixel information in the vegetation segmentation image; The encoder includes a lightweight feature extraction module, a pyramid pooling module, and a convolution module; the decoder includes an efficient channel attention module and a convolution sampling module. The segmentation module is used for: The lightweight feature extraction module extracts features from the remote sensing image to be detected to obtain multiple first feature vectors, and inputs the multiple first feature vectors into the pyramid pooling module and the efficient channel attention module in the decoder, respectively. The multiple first feature vectors are used to characterize the overall features of the remote sensing image to be detected. The pyramid pooling module extracts features from the plurality of first feature vectors to obtain a plurality of second feature vectors, and concatenates the plurality of second feature vectors to obtain a concatenation vector, which is then input into the convolution module. The plurality of second feature vectors are used to characterize the detailed features of the remote sensing image to be detected. The convolution module performs convolution processing on the connection vector, and the convolutional connection vector is input into the convolution sampling module of the decoder. The efficient channel attention module and the convolution sampling module obtain the vegetation segmentation image corresponding to the remote sensing image to be detected based on the multiple first feature vectors and the convolutional connection vector.

8. An electronic device, characterized in that, include: The device includes a processor, a storage medium, and a bus. The storage medium stores program instructions executable by the processor. When the electronic device is running, the processor communicates with the storage medium via the bus. The processor executes the program instructions to perform the steps of the vegetation coverage detection method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the vegetation coverage detection method as described in any one of claims 1 to 6.

Images