Deep learning-based vegetation cover identification method

Through deep learning methods combined with vegetation index, the U-Net network and attention module are used to extract vegetation characteristics, which solves the problem of low recognition accuracy in complex environments in the existing technology, and achieves higher recognition accuracy and freedom Great.

CN117237811BActive Publication Date: 2024-11-12AEROSPACE NEWSKY TECHNOLOGY CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202311405910.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-10-26
Publication Date
2024-11-12
Estimated Expiration
2043-10-26

AI Technical Summary

Technical Problem

The existing vegetation coverage recognition method has low recognition accuracy in complex environments, making it difficult to adapt to the influence of external interference such as weeds and light, and is poorly robust.

Method used

Using the vegetation coverage recognition method based on deep learning, vegetation images are segmented through segmentation networks, combined with vegetation index, deep semantic features and shallow texture features are integrated, and feature extraction and segmentation are extracted and segmented using U-Net network and attention module, and calculated Vegetation coverage.

Benefits of technology

It improves the recognition accuracy of vegetation coverage, reduces the impact of interference in complex environments, and enhances the robustness of recognition.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117237811B_ABST
    Figure CN117237811B_ABST
Patent Text Reader

Abstract

The present application discloses a vegetation coverage recognition method based on deep learning, which relates to the field of computer vision technology. The method includes: obtaining a vegetation image to be identified; inputting the vegetation image into a segmentation network for image segmentation to obtain a vegetation segmentation result, wherein the vegetation segmentation result includes the confidence that each pixel in the vegetation image belongs to vegetation; based on the RGB value of each pixel in the vegetation image, determining the vegetation index of each pixel, the vegetation index is used to indicate whether the pixel belongs to vegetation or not; based on the vegetation segmentation result and the vegetation index, determining the target pixel belonging to vegetation in the vegetation image; determining the proportion of the target pixel in the vegetation image as the coverage of vegetation in the vegetation image. The method provided in the embodiment of the present application can reduce interference from complex environments such as weeds, light, and water surface, thereby further improving the recognition accuracy of vegetation coverage.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a vegetation coverage recognition method based on deep learning. Background Art

[0002] Vegetation coverage is a common parameter reflecting the growth status of crops. It directly determines the amount of light intercepted by crops, which in turn affects the efficiency of photosynthesis. It is one of the most commonly used traits for remote sensing monitoring of crop canopy structure in the early stages of growth. Rapid and accurate acquisition of crop coverage plays an important role in refined farmland management and yield estimation.

[0003] At present, the extraction of vegetation coverage is generally achieved by using various two-dimensional image threshold segmentation methods, such as Lab-Kmeans segmentation method, bimodal threshold segmentation method, and visible light vegetation index segmentation method. Although the above methods are simple to implement, they are easily affected by the external environment, and it is difficult to remove the influence of green weeds and light. They have poor robustness and great limitations in practical engineering applications.

[0004] In recent years, with the development of computer vision technology, image segmentation algorithms represented by deep learning have been applied to vegetation cover detection. Although the segmentation accuracy has been improved to varying degrees, it is still difficult to adapt to recognition in complex environments. Summary of the invention

[0005] In response to the above problems and technical requirements, the applicant proposes a vegetation coverage recognition method based on deep learning. The technical solution of this application is as follows:

[0006] On the one hand, a vegetation coverage recognition method based on deep learning is provided, comprising the following steps:

[0007] Acquire a vegetation image to be identified;

[0008] Inputting the vegetation image into a segmentation network to perform image segmentation to obtain a vegetation segmentation result, wherein the vegetation segmentation result includes a confidence level that each pixel in the vegetation image belongs to vegetation;

[0009] Determine a vegetation index of each pixel point based on the RGB value of each pixel point in the vegetation image, wherein the vegetation index is used to indicate whether the pixel point belongs to vegetation or not;

[0010] Based on the vegetation segmentation result and the vegetation index, determining target pixel points belonging to vegetation in the vegetation image;

[0011] The proportion of the target pixel points in the vegetation image is determined as the coverage of the vegetation in the vegetation image.

[0012] Among them, the further solutions are:

[0013] The segmentation network includes a first extraction module, a second extraction module and a segmentation module;

[0014] The step of inputting the vegetation image into a segmentation network to perform image segmentation and obtain a vegetation segmentation result comprises:

[0015] Inputting the vegetation image into the first extraction module to perform feature extraction to obtain deep vegetation features, wherein the deep vegetation features contain semantic information of the vegetation image;

[0016] Inputting the vegetation image into the second extraction module to perform feature extraction to obtain shallow vegetation features, wherein the shallow vegetation features include texture information of the vegetation image;

[0017] Splicing the deep vegetation features, the shallow vegetation features and the vegetation image to obtain a spliced ​​feature map;

[0018] The spliced ​​feature map is input into the segmentation module to perform image segmentation to obtain the vegetation segmentation result.

[0019] Optionally, the first extraction module includes a first U-Net network, the second extraction module includes a second U-Net network, and the segmentation module includes a third U-Net network;

[0020] The first U-Net network and the second U-Net network are in parallel structure, and the third U-Net network is in series structure with the first U-Net network and the second U-Net network respectively.

[0021] Optionally, the number of network layers of the first U-Net network is higher than the number of network layers of the second U-Net network, and the encoder layers in the first U-Net network and the second U-Net network share network parameters.

[0022] The step of inputting the vegetation image into the first extraction module to extract features and obtain deep vegetation features includes:

[0023] Inputting the vegetation image into at least two encoders of the first U-Net network for encoding to obtain encoding features, wherein the encoder includes a bneck module;

[0024] The encoded features are input into at least two decoders of the first U-Net network for decoding to obtain the deep vegetation features, and the decoders correspond to the encoders one by one.

[0025] The step of inputting the encoded features into at least two decoders of the first U-Net network for decoding to obtain the deep vegetation features includes:

[0026] The attention module is used to fuse the n-1th decoding feature output by the n-1th decoder and the nth encoding feature output by the encoder corresponding to the nth decoder to obtain the nth feature to be decoded;

[0027] Inputting the nth feature to be decoded into the nth decoder for feature decoding to obtain the nth decoded feature;

[0028] The decoded features output by the last decoder are determined as the deep vegetation features.

[0029] Based on the RGB value of each pixel in the vegetation image, the vegetation index of each pixel is determined as follows:

[0030]

[0031] Among them, GGLI is the vegetation index, γ is the gamma enhancement value, and R, G, and B are the three color components of the RGB color space corresponding to the pixel.

[0032] The determining, based on the vegetation segmentation result and the vegetation index, target pixel points belonging to vegetation in the vegetation image includes:

[0033] Adjusting the RGB values ​​of non-vegetation pixels indicated by the vegetation segmentation result in the vegetation image to zero, and performing grayscale processing on the vegetation image after the RGB value adjustment to obtain a first grayscale image;

[0034] Adjusting the RGB values ​​of the non-vegetation pixels indicated by the vegetation index in the vegetation image to zero, and performing grayscale processing on the vegetation image after the RGB value adjustment to obtain a second grayscale image;

[0035] Common non-zero pixels in the first grayscale image and the second grayscale image are determined as target pixels belonging to vegetation in the vegetation image, wherein the non-zero pixels are pixels with non-zero pixel values.

[0036] The step of adjusting the RGB value of the non-vegetation pixel indicated by the vegetation index in the vegetation image to zero includes:

[0037] When the vegetation index of a pixel point is less than an index threshold, determining the pixel point as the non-vegetation pixel point;

[0038] The RGB value of the non-vegetation pixel is adjusted to zero.

[0039] The step of adjusting the RGB value of the non-vegetation pixel indicated by the vegetation segmentation result in the vegetation image to zero includes:

[0040] When the confidence of a pixel point in the vegetation segmentation result is less than a confidence threshold, determining the pixel point to be the non-vegetation pixel point;

[0041] The RGB value of the non-vegetation pixel is adjusted to zero.

[0042] The beneficial technical effects of this application are:

[0043] In an embodiment of the present application, for a vegetation image to be identified, on the one hand, a segmentation network is used to perform image segmentation to obtain pixel points belonging to vegetation in the vegetation image; on the other hand, a vegetation index is calculated based on the RGB values ​​of the pixel points, and the pixel points belonging to vegetation in the vegetation image are determined based on the vegetation index; then, the two are fused to obtain target pixel points belonging to vegetation in the vegetation image, and the vegetation coverage is determined based on the proportion of the target pixel points. In this way, the segmentation results of the segmentation network and the vegetation index are used for comprehensive judgment, which can reduce interference from complex environments such as weeds, light, and water surfaces, thereby further improving the recognition accuracy of vegetation coverage. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 is a schematic diagram of a segmentation network structure provided by an exemplary embodiment of the present application;

[0045] Figure 2 This is a flowchart of a vegetation coverage identification method based on deep learning provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0046] The specific implementation of the present application is further described below in conjunction with the accompanying drawings.

[0047] The embodiment of the present application provides a vegetation coverage recognition method based on deep learning, and the method is described by taking the execution of a computer device as an example. The method comprises the following steps:

[0048] Step S1, obtaining a vegetation image to be identified.

[0049] The vegetation image may be an image from which vegetation coverage needs to be extracted, such as a crop image from which crop coverage needs to be extracted.

[0050] Step S2: Input the vegetation image into the segmentation network for image segmentation to obtain a vegetation segmentation result, wherein the vegetation segmentation result includes the confidence that each pixel in the vegetation image belongs to vegetation.

[0051] The segmentation network is an image segmentation network based on deep learning and is pre-trained for image segmentation of vegetation images. Optionally, the segmentation network is a network improved based on the U-Net network, which is used to achieve refined segmentation of vegetation images. The segmentation network can be trained in advance using training samples, wherein the training samples are vegetation images with real labels. The segmentation network is trained by the difference between the prediction results of the segmentation network and the real labels, and can be used for the segmentation of vegetation images after the training is completed.

[0052] The vegetation segmentation result is obtained by segmenting the vegetation image using the segmentation network. The vegetation segmentation result includes the segmentation result of each pixel point, and the confidence or probability that each pixel point belongs to vegetation can be obtained. Subsequently, the pixel point belonging to vegetation can be determined based on the confidence that the pixel point belongs to vegetation.

[0053] Step S3, determining a vegetation index of each pixel point based on the RGB value of each pixel point in the vegetation image, where the vegetation index is used to indicate whether the pixel point belongs to vegetation or not.

[0054] In an embodiment of the present application, in order to reduce interference such as weeds, light and water surface, in addition to using the segmentation network to extract vegetation pixels, a vegetation index is also introduced, which is a green enhancement index, and the vegetation area can be extracted according to the RGB value of the pixel in the RGB image.

[0055] In a possible implementation, the vegetation index corresponding to each pixel is calculated using the RGB value of each pixel in the vegetation image, and the vegetation attribute of the pixel, that is, whether it is an attribute of vegetation, can be determined according to the vegetation index.

[0056] Step S4: determining target pixels belonging to vegetation in the vegetation image based on the vegetation segmentation result and the vegetation index.

[0057] The computer equipment can analyze the vegetation segmentation results to obtain the pixel points belonging to vegetation in the vegetation image; and can determine the pixel points belonging to vegetation in the vegetation image according to the vegetation index corresponding to each pixel point, obtain two vegetation extraction results, and then fuse the two vegetation extraction results to obtain the target pixel points belonging to vegetation in the vegetation image.

[0058] Step S5: determining the proportion of the target pixel points in the vegetation image as the coverage of the vegetation in the vegetation image.

[0059] After the target pixel is determined, the proportion of the target pixel in the vegetation image can be determined according to the ratio of the number of target pixel points to the total number of pixels. The proportion is the coverage of vegetation in the vegetation image.

[0060] In an embodiment of the present application, for a vegetation image to be identified, on the one hand, a segmentation network is used to perform image segmentation to obtain pixel points belonging to vegetation in the vegetation image; on the other hand, a vegetation index is calculated based on the RGB values ​​of the pixel points, and the pixel points belonging to vegetation in the vegetation image are determined based on the vegetation index; then, the two are fused to obtain target pixel points belonging to vegetation in the vegetation image, and the vegetation coverage is determined based on the proportion of the target pixel points. In this way, the segmentation results of the segmentation network and the vegetation index are used for comprehensive judgment, which can reduce interference from complex environments such as weeds, light, and water surfaces, thereby further improving the recognition accuracy of vegetation coverage.

[0061] In the embodiment of the present application, the segmentation network is a network improved based on the U-Net network, which includes a first extraction module, a second extraction module and a segmentation module. The process of segmenting the vegetation image using the segmentation network includes the following steps:

[0062] Step S21: input the vegetation image into the first extraction module to perform feature extraction to obtain deep vegetation features, which contain semantic information of the vegetation image.

[0063] The first extraction module is used to extract deep semantic features from vegetation images. The vegetation images are input into the first extraction module for feature extraction to obtain deep vegetation features. The first extraction module includes a first U-Net network. The process of extracting deep vegetation features using the first U-Net network includes the following steps:

[0064] Step 1: Input the vegetation image into at least two encoders of the first U-Net network for encoding to obtain encoding features, wherein the encoder includes a bneck module.

[0065] The U-Net network includes an encoder and a decoder. In the embodiment of the present application, a bneck module is introduced into the U-Net network, and the bneck module in MobileNetV3 is used to replace the downsampling process in the U-Net network, that is, the bneck module is introduced into the encoder, and the bneck module is used to extract features.

[0066] The first U-Net network contains several encoders, each of which further extracts features from the features encoded by the previous encoder, and each encoder is composed of bneck modules. The number of bnecks contained in each encoder is different. The deeper the feature depth extracted, the more bnecks there are, and more advanced features can be extracted. That is, the higher the number of layers, the more bneck modules there are.

[0067] Indicatively, Figure 1As shown, it shows a schematic diagram of the structure of the segmentation network provided in an embodiment of the present application. Among them, the first U-Net network 101 includes an encoder, and each layer is feature extracted by a different number of bneck modules. As shown in the figure, 4 layers of bneck modules are used to extract features in sequence.

[0068] Step 2: Input the encoded features into at least two decoders of the first U-Net network for decoding to obtain deep vegetation features, and the decoders correspond to the encoders one by one.

[0069] The first U-Net network also includes several decoders, wherein the decoders correspond to the encoders one by one and are used to perform feature decoding in sequence. The feature decoding process includes a convolution process and an upsampling process.

[0070] Indicatively, Figure 1 As shown, it includes 4 decoders, and the decoder includes a convolutional layer. When the decoder receives the features output from the previous decoder, it decodes the features of the previous output to obtain the final features.

[0071] The U-Net network includes a skip connection module. In the embodiment of the present application, an attention module is introduced into the skip connection module to better integrate features. The process includes the following steps:

[0072] Step 1: Use the attention module to fuse the n-1th decoding feature output by the n-1th decoder and the nth encoding feature output by the encoder corresponding to the nth decoder to obtain the nth feature to be decoded.

[0073] Optionally, the attention module adopts a convolutional block attention module (CBAM), wherein the CBAM module can fuse features from two levels: spatial and channel. In the embodiment of the present application, the CBAM module is introduced in the process of feature jump connection between the U-Net network encoder and the decoder to improve feature fusion.

[0074] The process of skip connection is to introduce the features of the same scale as the encoding process into the decoding process to retain the high-level feature information to a greater extent. The encoding features output by the encoder corresponding to the decoder have the same feature dimensions as the features input to the decoder. That is, the nth encoding feature output by the encoder corresponding to the nth decoder has the same feature dimension as the n-1th decoding feature input to the nth decoder.

[0075] In a possible implementation, the nth encoding feature and the n-1th decoding feature are fused through a CBAM module to obtain the nth feature to be decoded.

[0076] Step 2: Input the nth feature to be decoded into the nth decoder for feature decoding to obtain the nth decoding feature.

[0077] The nth feature to be decoded is input into the nth decoder for feature decoding to obtain the nth decoded feature.

[0078] Indicatively, Figure 1 As shown, the features of each decoder input are obtained by fusing the features decoded by the previous decoder and the features encoded by the corresponding encoder.

[0079] Step 3: Determine the decoded features output by the last decoder as deep vegetation features.

[0080] After being decoded by several layers of decoders, the decoded features output by the last decoder are the deep vegetation features.

[0081] Step S22, inputting the vegetation image into the second extraction module to perform feature extraction to obtain shallow vegetation features, wherein the shallow vegetation features contain texture information of the vegetation image.

[0082] The first U-Net network is used to extract deep features. In the process of extracting deep features, detail information may be lost. In order to compensate for the detail information, a second extraction module is also introduced in the embodiment of the present application. The second extraction module is used to extract shallow texture features of vegetation images. The vegetation image is input into the second extraction module to extract shallow vegetation features.

[0083] The first extraction module and the second extraction module are in parallel structure, and can simultaneously extract features from the vegetation image to obtain deep vegetation features and shallow vegetation features respectively.

[0084] The second extraction module includes a second U-Net network. The first U-Net network and the second U-Net network are in a parallel structure. The second U-Net network is used to extract shallow vegetation features, and the first U-Net network is used to extract deep vegetation features. The two use the same network structure, but the number of network layers of the first U-Net network is higher than that of the second U-Net network, and the same encoder layer in the first U-Net network and the second U-Net network shares network parameters.

[0085] The bneck module is also introduced in the second U-Net network as an encoder to achieve more refined feature extraction. At the same time, the CBAM module is introduced at the jump connection between the encoder and the decoder in the second U-Net network to better fuse features, which helps to improve the accuracy of feature extraction.

[0086] Step S23, stitching the deep vegetation features, the shallow vegetation features and the vegetation image to obtain a stitching feature map.

[0087] In the embodiment of the present application, a third U-Net network is also introduced for more refined feature fusion and segmentation. After extracting the deep vegetation features and the shallow vegetation features, the two are fused with the original vegetation image. In a possible implementation, the three are spliced ​​to obtain a spliced ​​feature map.

[0088] Step S24, inputting the spliced ​​feature map into a segmentation module to perform image segmentation to obtain a vegetation segmentation result.

[0089] The segmentation module includes a third U-Net network, which is a series structure with the first U-Net network and the second U-Net network, and is used to fuse and finely segment the shallow vegetation features and deep vegetation features extracted by the two networks and the original vegetation image. The third U-Net network also introduces the bneck module and the CBAM attention mechanism module to achieve fine segmentation of the image.

[0090] like Figure 1 As shown, the first U-Net network 101 performs feature extraction on the vegetation image to obtain deep vegetation features; the second U-Net network 102 performs feature extraction on the vegetation image to obtain shallow vegetation features; the deep vegetation features are spliced ​​with the shallow vegetation features and the original vegetation image, and then input into the third U-Net network 103 for image segmentation, and the third U-Net network 103 is connected with a softmax layer 104 to convert the segmentation result into confidence, that is, the segmentation module is composed of the third U-Net network 103 and the softmax layer 104, and the confidence that each pixel in the vegetation image belongs to vegetation can be obtained.

[0091] In the embodiment of the present application, the vegetation index uses the GGLI vegetation index for vegetation segmentation. This method constructs a gamma enhanced green leaf index by enhancing the spectral reflectance of vegetation to achieve the purpose of accurately extracting vegetation information. Based on the RGB value of each pixel in the vegetation image, the method for determining the vegetation index of each pixel is as follows:

[0092]

[0093] Among them, GGLI is the vegetation index, γ is the gamma enhancement value, and R, G, and B are the three color components of the RGB color space corresponding to the pixel.

[0094] According to the above method, the vegetation segmentation result and the vegetation index can be obtained, wherein the vegetation segmentation result and the vegetation index respectively indicate different vegetation pixels. In a possible implementation, the vegetation coverage is obtained by comprehensive identification of the two. The process of determining the target pixel points belonging to vegetation in the vegetation image based on the vegetation segmentation result and the vegetation index includes the following steps:

[0095] Step S41, adjusting the RGB values ​​of non-vegetation pixels indicated by the vegetation segmentation result in the vegetation image to zero, and performing grayscale processing on the vegetation image after the RGB value adjustment to obtain a first grayscale image.

[0096] The vegetation segmentation result includes the confidence that each pixel belongs to vegetation, and the pixels belonging to vegetation can be screened according to the confidence. In a possible implementation, the pixels whose confidence is greater than the confidence threshold are determined as pixels belonging to vegetation, wherein the confidence threshold can be set based on experience, and schematically, the confidence threshold can be 0.8. For pixels whose confidence is less than the confidence threshold, it can be determined that they do not belong to vegetation, that is, they are non-vegetation pixels.

[0097] After determining the non-vegetation pixel points based on the vegetation segmentation result, the RGB value of the non-vegetation pixel points can be assigned to 0, while the pixel values ​​of other vegetation pixel points (i.e., the pixel points belonging to vegetation) remain unchanged. After adjusting the pixel values ​​of the non-vegetation pixel points, the vegetation image is grayed to obtain a first gray image.

[0098] Step S42, adjusting the RGB values ​​of the non-vegetation pixels indicated by the vegetation index in the vegetation image to zero, and performing grayscale processing on the vegetation image after the pixel value adjustment to obtain a second grayscale image.

[0099] In the above process, the GGLI vegetation index of each pixel is calculated according to the RGB value of each pixel, and whether it is a vegetation pixel can be determined according to the GGLI vegetation index of each pixel.

[0100] Optionally, an index threshold may be pre-set, and when the vegetation index of a pixel is less than the index threshold, the pixel is determined to be a non-vegetation pixel. When the vegetation index of a pixel is greater than the index threshold, the pixel is determined to be a vegetation pixel (i.e., a pixel belonging to vegetation). For non-vegetation pixels determined according to the vegetation index in the vegetation image, the RGB value is adjusted to 0, while the RGB value of the vegetation pixel remains unchanged. After the RGB value of the pixel is adjusted, grayscale processing may be performed to obtain a second grayscale image.

[0101] Indicatively, the index threshold is 0. When GGLI>0, the RGB value of the pixel is kept unchanged; when GGLI<0, the RGB value of the pixel is assigned to 0.

[0102] It should be noted that step S42 and step S41 can be executed simultaneously or one after the other. This embodiment only describes the implementation method and does not limit the execution sequence. In the process of adjusting the RGB value of the pixel point, the original vegetation image is adjusted.

[0103] Step S43, determining common non-zero pixels in the first grayscale image and the second grayscale image as target pixels belonging to vegetation in the vegetation image, wherein the non-zero pixels are pixels with non-zero pixel values.

[0104] The first grayscale image indicates a first vegetation pixel point (a point with a non-zero pixel value in the first grayscale image), and the second grayscale image indicates a second vegetation pixel point (a point with a non-zero pixel value in the second grayscale image). In this embodiment, the target pixel point that ultimately belongs to vegetation is determined by combining the two results. The method is as follows:

[0105] S={I(x,y)∩J(x,y)}

[0106] S is the result of the set operation, I(x,y) is the pixel point in the first grayscale image, J(x,y) is the pixel point in the second grayscale image, ∩ is the intersection operation, {·} represents the set operation, and the returned result is a set containing elements set to logical value 1 or logical value 0. If I(x,y) and J(x,y) both contain non-zero elements at the same set position, the element at the corresponding position in the output set will be set to logical value 1 (that is, if the pixel values ​​of the pixel points at the same position in the first grayscale image and the second grayscale image are both non-zero, then the point is set to logical value 1); otherwise, the element at the corresponding position of the set is set to logical 0.

[0107] After determining the target pixel point, the vegetation coverage can be calculated as follows:

[0108]

[0109] cov is vegetation coverage, N plant-pixels is the number of elements in set S whose value is 1, N all-pixels is the sum of the elements in the set S.

[0110] like Figure 2 As shown, it shows a flowchart of a vegetation coverage recognition method based on deep learning provided in an embodiment of the present application, which includes the following process:

[0111] Step 201: Obtain a vegetation image to be identified.

[0112] Step 202: Calculate the GGLI vegetation index.

[0113] Step 203, determine whether GGLI is greater than 0, if so, execute step 204; if not, execute step 205.

[0114] Step 204: determine the pixel point as a vegetation pixel point.

[0115] Step 205: determine the pixel point as a background pixel point.

[0116] Step 206: grayscale the image to obtain a second grayscale image.

[0117] Step 207, segmenting the vegetation image using the improved U-Net segmentation network.

[0118] Step 208: Determine vegetation pixels and background pixels according to the confidence level.

[0119] Step 209: grayscale the image to obtain a first grayscale image.

[0120] Step 210, combining the first grayscale image and the second grayscale image to determine vegetation coverage.

[0121] The implementation of steps 201 - 210 may refer to the above embodiment, and will not be described in detail in this embodiment.

[0122] The above is only a preferred embodiment of the present application, and the present application is not limited to the above embodiments. It is understood that other improvements and changes directly derived or associated by those skilled in the art without departing from the spirit and concept of the present application should be considered to be included in the protection scope of the present application.

Claims

1. A vegetation coverage recognition method based on deep learning, characterized in that: The method comprises: Acquire a vegetation image to be identified; Inputting the vegetation image into a segmentation network for image segmentation to obtain a vegetation segmentation result, wherein the vegetation segmentation result includes a confidence level that each pixel in the vegetation image belongs to vegetation; the segmentation network includes a first extraction module, a second extraction module and a segmentation module; Inputting the vegetation image into the first extraction module to perform feature extraction to obtain deep vegetation features, wherein the deep vegetation features contain semantic information of the vegetation image; Inputting the vegetation image into the second extraction module to perform feature extraction to obtain shallow vegetation features, wherein the shallow vegetation features include texture information of the vegetation image; Splicing the deep vegetation features, the shallow vegetation features and the vegetation image to obtain a spliced ​​feature map; Inputting the spliced ​​feature map into the segmentation module to perform image segmentation to obtain the vegetation segmentation result; Determine a vegetation index of each pixel point based on the RGB value of each pixel point in the vegetation image, wherein the vegetation index is used to indicate whether the pixel point belongs to vegetation or not; Based on the vegetation segmentation result and the vegetation index, determining target pixel points belonging to vegetation in the vegetation image; The proportion of the target pixel points in the vegetation image is determined as the coverage of the vegetation in the vegetation image.

2. The method according to claim 1, characterized in that The first extraction module includes a first U-Net network, the second extraction module includes a second U-Net network, and the segmentation module includes a third U-Net network; The first U-Net network and the second U-Net network are in parallel structure, and the third U-Net network is in series structure with the first U-Net network and the second U-Net network respectively.

3. The method according to claim 2, characterized in that The number of network layers of the first U-Net network is higher than the number of network layers of the second U-Net network, and the encoder layers in the first U-Net network and the second U-Net network share network parameters.

4. The method according to claim 2, characterized in that: The step of inputting the vegetation image into the first extraction module to extract features and obtain deep vegetation features includes: Inputting the vegetation image into at least two encoders of the first U-Net network for encoding to obtain encoding features, wherein the encoder includes a bneck module; The encoded features are input into at least two decoders of the first U-Net network for decoding to obtain the deep vegetation features, and the decoders correspond to the encoders one by one.

5. The method according to claim 4, characterized in that The step of inputting the encoded features into at least two decoders of the first U-Net network for decoding to obtain the deep vegetation features includes: The attention module is used to fuse the n-1th decoding feature output by the n-1th decoder and the nth encoding feature output by the encoder corresponding to the nth decoder to obtain the nth feature to be decoded; Inputting the nth feature to be decoded into the nth decoder for feature decoding to obtain the nth decoded feature; The decoded features output by the last decoder are determined as the deep vegetation features.

6. The method according to any one of claims 1 to 5, characterized in that: Based on the RGB value of each pixel in the vegetation image, the vegetation index of each pixel is determined as follows: in, is the vegetation index, is the gamma enhancement value, The pixel corresponding to The three color components of the color space.

7. The method according to any one of claims 1 to 5, characterized in that: The determining, based on the vegetation segmentation result and the vegetation index, target pixel points belonging to vegetation in the vegetation image includes: Adjusting the RGB values ​​of non-vegetation pixels indicated by the vegetation segmentation result in the vegetation image to zero, and performing grayscale processing on the vegetation image after the RGB value adjustment to obtain a first grayscale image; Adjusting the RGB values ​​of the non-vegetation pixels indicated by the vegetation index in the vegetation image to zero, and performing grayscale processing on the vegetation image after the RGB value adjustment to obtain a second grayscale image; Common non-zero pixels in the first grayscale image and the second grayscale image are determined as target pixels belonging to vegetation in the vegetation image, wherein the non-zero pixels are pixels with non-zero pixel values.

8. The method according to claim 7, characterized in that The step of adjusting the RGB value of the non-vegetation pixel indicated by the vegetation index in the vegetation image to zero includes: When the vegetation index of a pixel point is less than an index threshold, determining the pixel point as the non-vegetation pixel point; The RGB value of the non-vegetation pixel is adjusted to zero.

9. The method according to claim 7, characterized in that: The step of adjusting the RGB value of the non-vegetation pixel indicated by the vegetation segmentation result in the vegetation image to zero includes: When the confidence of a pixel point in the vegetation segmentation result is less than a confidence threshold, determining the pixel point to be the non-vegetation pixel point; The RGB value of the non-vegetation pixel is adjusted to zero.