Power transmission line corridor multispectral image greenhouse identification method

Abstract

The invention relates to a power transmission line corridor multispectral image greenhouse recognition method, which comprises the following steps of: constructing RGB image data, degraded greenhouse index data RPGI and plastic covered land index data PMLI, constructing a three-channel semantic segmentation model based on priori knowledge to process three input data in parallel, performing feature extraction by utilizing a Transform architecture, and performing feature extraction by utilizing a three-channel semantic segmentation model based on priori knowledge; fusing the three-channel features, outputting an identification result through a decoding module, and optimizing a training process in combination with a cross entropy loss function and a pixel comparison loss function; and finally, identifying the greenhouse in the multispectral remote sensing image by using the trained three-channel semantic segmentation model. According to the method, the greenhouse can be efficiently and accurately identified from the multispectral remote sensing image, and the universality and the adaptability are high.

Description

Technical Field

[0001] The invention relates to the technical field of remote sensing image recognition, and in particular to a method for recognizing greenhouses in multispectral images of a transmission line corridor. Background Art

[0002] As one of the core components of the power system, transmission lines are responsible for delivering electricity from power plants to end users. However, greenhouse facilities on the transmission line corridors may cause a series of problems, including equipment damage, reduced insulation performance, and increased risk of accidents. In view of the continuous development and expansion of the power system, ensuring the safety of transmission line corridors has become an increasingly important issue. Therefore, timely discovery and accurate identification of greenhouse areas in the corridor is of great significance to maintaining the stability and reliability of the power system.

[0003] Traditionally, the application of remote sensing technology in this field relies on manual analysis or rule-based image processing, which is not only inefficient but also easily affected by subjective judgment. Therefore, it is necessary to develop a more advanced automated greenhouse identification method to solve the problems of existing technologies. Summary of the invention

[0004] The purpose of the present invention is to provide a method for identifying greenhouses in multispectral images of transmission line corridors. The method is conducive to efficiently and accurately identifying greenhouses from multispectral remote sensing images and has strong versatility and adaptability.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is: a method for identifying greenhouses in multispectral images of transmission line corridors, comprising the following steps:

[0006] Step S1: Acquire multispectral remote sensing images and preprocess the remote sensing images; then mark the greenhouse areas in the remote sensing images to create labels, and then construct a remote sensing image greenhouse recognition dataset for subsequent greenhouse target recognition training and verification;

[0007] Step S2: constructing RGB image data with three color channels based on the red, green and blue band data in the remote sensing image;

[0008] Step S3: constructing the degradation greenhouse index data RPGI based on the blue light, green light and near infrared band data in the remote sensing image;

[0009] Step S4: constructing the plastic covered land index data PMLI based on the red light and short-wave infrared band data in the remote sensing image;

[0010] Step S5: constructing a three-channel semantic segmentation model based on prior knowledge; the three-channel semantic segmentation model has three branches, which respectively correspond to three types of input data: RGB image data, greenhouse index data RPGI and plastic covered land index data PMLI; the three-channel semantic segmentation model performs feature extraction and feature fusion on the three types of input data, and then decodes the extracted features to obtain recognition results;

[0011] Step S6: construct a loss function of the three-channel semantic segmentation model, introduce the pixel contrast loss function as an auxiliary function and combine it with the cross entropy loss function to optimize the category imbalance problem;

[0012] Step S7: Use the remote sensing image greenhouse recognition data set obtained in step S1 to train the three-channel semantic segmentation model; for the remote sensing images in the remote sensing image greenhouse recognition data set, use steps S2-S4 to generate RGB image data, greenhouse index data RPGI and plastic covered land index data PMLI as the input of the model, and use the marked labels as the target output of the model;

[0013] Step S8: Use the trained three-channel semantic segmentation model to identify the greenhouse in the multispectral remote sensing image. After the multispectral remote sensing image to be identified is processed through steps S2-S4, it is input into the trained three-channel semantic segmentation model to obtain the greenhouse identification result.

[0014] Furthermore, in step S1, preprocessing the remote sensing image includes performing radiation correction and atmospheric correction on the remote sensing image to reduce interference factors in the image; then, pixel-level marking is performed on the greenhouse area in the remote sensing image to create a label.

[0015] Furthermore, in step S2, an RGB color model that matches human visual perception is used to construct RGB image data. The model selects red, green, and blue band data in the remote sensing image and merges the data of these three bands to form RGB image data with three color channels.

[0016] Furthermore, in step S3, based on the blue light, green light and near infrared band data in the remote sensing image, the degradation greenhouse index data RPGI is constructed, and its calculation formula is as follows:

[0017]

[0018] Among them, Blue, Green, and NIR are blue light, green light, and near-infrared band data respectively, and mean() is the mean function.

[0019] Furthermore, in step S4, based on the red light and short-wave infrared band data in the remote sensing image, the plastic covered land index data PMLI is constructed, and its calculation formula is as follows:

[0020]

[0021] Among them, Red and SWIR1 are red light and shortwave infrared band data respectively.

[0022] Furthermore, in step S5, the three-channel semantic segmentation model includes a feature extraction module and a decoding module. The feature extraction module includes multiple stages of feature extraction and feature fusion. Each stage performs feature extraction through three Transformer modules, and then performs feature fusion, and then sends it to the next stage for feature extraction and feature fusion; the decoding module decodes the features extracted by the feature extraction module to obtain the recognition result.

[0023] Furthermore, the feature extraction module includes an Overlap Patch Merging operation. The three input data are respectively subjected to the Overlap Patch Merging operation. The Overlap Patch Merging operation controls the size and depth of the feature map by setting different convolution kernel sizes, step sizes and padding sizes. The Overlap Patch Merging operation is set before the Transformer Block to generate feature maps of different resolutions. These feature maps have multi-scale characteristics, which help the model capture visual information at different levels. The three input data are subjected to the Overlap Patch Merging operation. After the Merging operation, feature extraction is performed through three first Transformer modules respectively, and then the features of the three branches are fused, and the output feature size is 1 / 4 of the input image; the features after the first fusion are respectively extracted through three second Transformer modules, and then the features of the three branches are fused, and the output feature size is 1 / 8 of the input image; the features after the second fusion are respectively extracted through three third Transformer modules, and then the features of the three branches are fused, and the output feature size is 1 / 16 of the input image; the features after the third fusion are respectively extracted through three fourth Transformer modules, and then the features of the three branches are fused, and the final output feature size is 1 / 32 of the input image.

[0024] Furthermore, the decoding module first extracts features through DWConv convolution, and the DWConv convolution module includes three operations: Conv2D, BatchNorm2D, and ReLU. The feature map is then restored to its original scale through Upsampling, and then DWConv convolution and Upsampling are performed again, and finally Softmax is used for category determination and output.

[0025] Furthermore, in step S6, the cross entropy loss function is as follows:

[0026]

[0027] in, is the cross entropy loss value, C is the number of predicted categories, y c is the real class; y c′ is the probability of predicting the category c′, exp() is the natural exponential function;

[0028] The pixel contrast loss function is as follows:

[0029]

[0030] in, is the pixel contrast loss value, i represents the i-th pixel, I represents the normalized feature vector of pixel i, P i and N i Represent the positive and negative sample sets of pixel i respectively; τ is a hyperparameter; I + is the positive sample feature vector, I - is the negative sample feature vector;

[0031] Therefore, the overall loss function of the model is as follows:

[0032]

[0033] Among them, λ is the set weight coefficient.

[0034] Furthermore, in step S7, during the training process, the resolution of the input image is 512×512 pixels, the learning rate is set to 0.0001, the number of epochs is set to 256, and each batch contains 20 samples.

[0035] Compared with the prior art, the present invention has the following beneficial effects: The present invention provides a method for identifying greenhouses in multispectral images of transmission line corridors based on prior knowledge, aiming to more effectively capture the characteristics of greenhouses in remote sensing images of transmission line corridors, thereby identifying greenhouse areas. The method designs three parallel processing channels to process RGB images, degraded greenhouse indexes, and plastic coverage indexes respectively, uses the Transformer architecture for feature extraction, and then fuses the three-channel features. Finally, the recognition results are output through the decoding module, and the cross entropy loss function and the pixel contrast loss function are combined to optimize the training process, thereby improving the accuracy of greenhouse target recognition. The method has strong versatility and adaptability, and has strong practicality and broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1is an architecture diagram of a three-channel semantic segmentation model based on prior knowledge in an embodiment of the present invention;

[0037] Figure 2 It is a structural diagram of a decoding module in an embodiment of the present invention. DETAILED DESCRIPTION

[0038] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0039] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0040] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0041] This embodiment provides a method for identifying greenhouses in a transmission line corridor using multispectral images, including the following steps:

[0042] Step S1: Acquire multispectral remote sensing images, extract remote sensing images from multispectral remote sensing images, and preprocess the remote sensing images; then mark the greenhouse areas in the remote sensing images to create labels, and then construct a remote sensing image greenhouse recognition dataset for subsequent greenhouse target recognition training and verification.

[0043] Among them, the acquired multispectral remote sensing images (such as Landsat-8, Sentinel-2 and other multispectral satellite data) have multiple spectral bands such as red, green, and blue visible light, as well as near infrared (NIR) and shortwave infrared (SWIR1). Before remote sensing image analysis, a series of preprocessing is required for the remote sensing images, including radiation correction and atmospheric correction of the images, in order to reduce interference factors in the images, such as noise and errors, to improve the quality and reliability of the data. Then, the greenhouse areas in the remote sensing images are carefully labeled at the pixel level to create accurate labels. The remote sensing image greenhouse recognition dataset is constructed through the preprocessed remote sensing images and the created labels, providing a basis for training and verification for subsequent greenhouse target recognition.

[0044] Step S2: constructing RGB image data with three color channels based on the red, green and blue band data in the remote sensing image.

[0045] Specifically, the RGB color model that matches human visual perception is used to construct RGB image data. The model selects the red, green and blue band data in the remote sensing image and merges the data of these three bands to form RGB image data with three color channels. This processing method can intuitively display the color details of the image, which facilitates subsequent image analysis and processing.

[0046] Step S3: Based on the blue, green and near infrared (NIR) band data in the remote sensing image, the degradation greenhouse index data RPGI is constructed. This index is based on the unique spectral characteristics of agricultural greenhouses and is a highly robust greenhouse index.

[0047] Based on the blue light, green light and near infrared band data in the remote sensing image, the degradation greenhouse index data RPGI is constructed, and its calculation formula is as follows:

[0048]

[0049] Among them, Blue, Green, and NIR are blue light, green light, and near-infrared band data respectively, and mean() is the mean function.

[0050] Step S4: Based on the red light and short-wave infrared band data in the remote sensing image, the plastic covered land index data PMLI is constructed. The index combines the rising trend and high reflectivity characteristics of plastic mulch in the visible light band and the falling trend characteristics in the short-wave infrared band to improve the accuracy of plastic plot identification.

[0051] Based on the red light (Red) and shortwave infrared (SWIR1) band data in remote sensing images, the plastic coverage index data PMLI is constructed, and its calculation formula is as follows:

[0052]

[0053] Among them, Red and SWIR1 are red light and shortwave infrared band data respectively.

[0054] Step S5: Construct a three-channel semantic segmentation model based on prior knowledge; the three-channel semantic segmentation model has three branches, which correspond to three types of input data: RGB image data, greenhouse index data RPGI and plastic coverage index data PMLI; the three-channel semantic segmentation model performs feature extraction and feature fusion on the three types of input data, and then decodes the extracted features to obtain recognition results.

[0055] like Figure 1As shown, the three-channel semantic segmentation model includes a feature extraction module and a decoding module. The feature extraction module includes multiple stages of feature extraction and feature fusion. Each stage performs feature extraction through three Transformer modules, and then performs feature fusion, and then sends it to the next stage for feature extraction and feature fusion; the decoding module decodes the features extracted by the feature extraction module to obtain the recognition result.

[0056] The feature extraction module includes an Overlap Patch Merging operation. The three input data are respectively subjected to the Overlap Patch Merging operation. The Overlap Patch Merging operation controls the size and depth of the feature map by setting different convolution kernel sizes, step sizes, and padding sizes. The Overlap Patch Merging operation is set before the Transformer Block to generate feature maps of different resolutions. These feature maps have multi-scale characteristics, which help the model capture visual information at different levels. The three input data are subjected to the Overlap Patch Merging operation. After the Merging operation, feature extraction is performed through three first Transformer modules respectively, and then the features of the three branches are fused, and the output feature size is 1 / 4 of the input image; the features after the first fusion are respectively extracted through three second Transformer modules, and then the features of the three branches are fused, and the output feature size is 1 / 8 of the input image; the features after the second fusion are respectively extracted through three third Transformer modules, and then the features of the three branches are fused, and the output feature size is 1 / 16 of the input image; the features after the third fusion are respectively extracted through three fourth Transformer modules, and then the features of the three branches are fused, and the final output feature size is 1 / 32 of the input image.

[0057] like Figure 2 As shown in the figure, the decoding module first extracts features through DWConv convolution. The DWConv convolution module includes three operations: Conv2D, BatchNorm2D, and ReLU. The feature map is then restored to its original scale through Upsampling, and then DWConv convolution and Upsampling are performed again. Finally, Softmax is used to determine the category output.

[0058] Step S6: Construct the loss function of the three-channel semantic segmentation model, introduce the pixel contrast loss function as an auxiliary function and combine it with the cross entropy loss function to optimize the category imbalance problem.

[0059] In satellite images, greenhouse pixels only account for a small proportion of the entire image, resulting in serious class imbalance. Due to this situation, it is difficult to effectively solve the problem using only the standard cross entropy loss function. Therefore, this method introduces the pixel contrast loss function as an auxiliary function in the training process and combines it with the cross entropy loss function to optimize the class imbalance problem.

[0060] The cross entropy loss function is as follows:

[0061]

[0062] in, is the cross entropy loss value; C is the number of predicted categories; y c For the real class, the default value is 1; y c′ is the probability of predicting category c′; exp() is the natural exponential function.

[0063] The pixel contrast loss function is as follows:

[0064]

[0065] in, is the pixel contrast loss value, i represents the i-th pixel, I represents the normalized feature vector of pixel i, P i and N i denote the positive sample and negative sample set of pixel i respectively; τ is a hyperparameter, and τ=0.1 is set in this embodiment; I + is the positive sample feature vector, I - is the negative sample feature vector.

[0066] Therefore, the overall loss function of the model is as follows:

[0067]

[0068] Among them, λ is the set weight coefficient.

[0069] Step S7: Use the remote sensing image greenhouse recognition dataset obtained in step S1 to train the three-channel semantic segmentation model; for the remote sensing images in the remote sensing image greenhouse recognition dataset, use steps S2-S4 to generate RGB image data, greenhouse index data RPGI and plastic coverage index data PMLI as the input of the model, and use the marked labels as the target output of the model.

[0070] In this embodiment, PaddlePaddle is selected as the deep learning framework to achieve this goal. During the training process, the resolution of the input image is 512×512 pixels, the learning rate is set to 0.0001, the number of epochs is set to 256, and each batch size contains 20 samples.

[0071] Step S8: Use the trained three-channel semantic segmentation model to identify the greenhouse in the multispectral remote sensing image. After the multispectral remote sensing image to be identified is processed through steps S2-S4, it is input into the trained three-channel semantic segmentation model to obtain the greenhouse identification result.

[0072] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0073] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0074] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0075] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0076] The above is only a preferred embodiment of the present invention, and does not limit the present invention in other forms. Any technician familiar with the profession may use the above disclosed technical content to change or modify it into an equivalent embodiment with equivalent changes. However, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the technical solution of the present invention still belongs to the protection scope of the technical solution of the present invention.

Claims

1. A method for identifying greenhouses in a transmission line corridor using multispectral images, characterized in that: The following steps are involved: Step S1: Acquire multispectral remote sensing images and preprocess the remote sensing images; then mark the greenhouse areas in the remote sensing images to create labels, and then construct a remote sensing image greenhouse recognition dataset for subsequent greenhouse target recognition training and verification; Step S2: constructing RGB image data with three color channels based on the red, green and blue band data in the remote sensing image; Step S3: constructing the degradation greenhouse index data RPGI based on the blue light, green light and near infrared band data in the remote sensing image; Step S4: constructing the plastic covered land index data PMLI based on the red light and short-wave infrared band data in the remote sensing image; Step S5: constructing a three-channel semantic segmentation model based on prior knowledge; the three-channel semantic segmentation model has three branches, which respectively correspond to three types of input data: RGB image data, greenhouse index data RPGI and plastic covered land index data PMLI; the three-channel semantic segmentation model performs feature extraction and feature fusion on the three types of input data, and then decodes the extracted features to obtain recognition results; Step S6: construct a loss function of the three-channel semantic segmentation model, introduce the pixel contrast loss function as an auxiliary function and combine it with the cross entropy loss function to optimize the category imbalance problem; Step S7: Use the remote sensing image greenhouse recognition data set obtained in step S1 to train the three-channel semantic segmentation model; for the remote sensing images in the remote sensing image greenhouse recognition data set, use steps S2-S4 to generate RGB image data, greenhouse index data RPGI and plastic covered land index data PMLI as the input of the model, and use the marked labels as the target output of the model; Step S8: Use the trained three-channel semantic segmentation model to identify the greenhouse in the multispectral remote sensing image. After the multispectral remote sensing image to be identified is processed through steps S2-S4, it is input into the trained three-channel semantic segmentation model to obtain the greenhouse identification result.

2. The method for identifying greenhouses in a transmission line corridor multispectral image according to claim 1 is characterized in that: In step S1, the remote sensing image is preprocessed, including performing radiation correction and atmospheric correction on the remote sensing image to reduce interference factors in the image; then, the greenhouse area in the remote sensing image is pixel-level marked to create a label.

3. The method for identifying greenhouses in a transmission line corridor multispectral image according to claim 1 is characterized in that: In step S2, an RGB color model that matches human visual perception is used to construct RGB image data. The model selects red, green, and blue band data in the remote sensing image and merges the data of these three bands to form RGB image data with three color channels.

4. The method for identifying greenhouses in a transmission line corridor multispectral image according to claim 1, characterized in that: In step S3, based on the blue light, green light and near infrared band data in the remote sensing image, the degradation greenhouse index data RPGI is constructed, and its calculation formula is as follows: Among them, Blue, Green, and NIR are blue light, green light, and near-infrared band data respectively, and mean() is the mean function.

5. The method for identifying greenhouses in multispectral images of power transmission line corridors according to claim 1, characterized in that: In step S4, based on the red light and short-wave infrared band data in the remote sensing image, the plastic covered land index data PMLI is constructed, and its calculation formula is as follows: Among them, Red and SWIR1 are red light and shortwave infrared band data respectively.

6. The method for identifying greenhouses in a transmission line corridor multispectral image according to claim 1, characterized in that: In step S5, the three-channel semantic segmentation model includes a feature extraction module and a decoding module. The feature extraction module includes multiple stages of feature extraction and feature fusion. Each stage performs feature extraction through three Transformer modules, and then performs feature fusion, and then sends it to the next stage for feature extraction and feature fusion; the decoding module decodes the features extracted by the feature extraction module to obtain the recognition result.

7. The method for identifying greenhouses in a transmission line corridor multispectral image according to claim 6 is characterized in that: The feature extraction module includes an Overlap Patch Merging operation. The three input data are respectively subjected to the Overlap Patch Merging operation. The Overlap Patch Merging operation controls the size and depth of the feature map by setting different convolution kernel sizes, step sizes, and padding sizes. The Overlap Patch Merging operation is set before the Transformer Block to generate feature maps of different resolutions. These feature maps have multi-scale characteristics, which help the model capture visual information at different levels. The three input data are subjected to the Overlap Patch Merging operation. After the Merging operation, feature extraction is performed through three first Transformer modules respectively, and then the features of the three branches are fused, and the output feature size is 1 / 4 of the input image; the features after the first fusion are respectively extracted through three second Transformer modules, and then the features of the three branches are fused, and the output feature size is 1 / 8 of the input image; the features after the second fusion are respectively extracted through three third Transformer modules, and then the features of the three branches are fused, and the output feature size is 1 / 16 of the input image; the features after the third fusion are respectively extracted through three fourth Transformer modules, and then the features of the three branches are fused, and the final output feature size is 1 / 32 of the input image.

8. The method for identifying greenhouses in multispectral images of power transmission line corridors according to claim 6 is characterized in that: The decoding module first extracts features through DWConv convolution. The DWConv convolution module includes three operations: Conv2D, BatchNorm2D, and ReLU. The feature map is then restored to its original scale through Upsampling. Then, DWConv convolution and Upsampling are performed again, and finally, Softmax is used to determine the category and output.

9. The method for identifying greenhouses in a transmission line corridor multispectral image according to claim 1, characterized in that: In step S6, the cross entropy loss function is as follows: in, is the cross entropy loss value, C is the number of predicted categories, y c is the real class; c′ is the probability of predicting the category c′, exp() is the natural exponential function; The pixel contrast loss function is as follows: in, is the pixel contrast loss value, i represents the i-th pixel, I represents the normalized feature vector of pixel i, P i and N i Represent the positive and negative sample sets of pixel i respectively; τ is a hyperparameter; I + is the positive sample feature vector, I - is the negative sample feature vector; Therefore, the overall loss function of the model is as follows: Among them, λ is the set weight coefficient.

10. The method for identifying greenhouses in multispectral images of power transmission line corridors according to claim 1, characterized in that: In step S7, during the training process, the resolution of the input image is 512×512 pixels, the learning rate is set to 0.0001, the number of epochs is set to 256, and each batch contains 20 samples.

Images