A photovoltaic panel extraction technology based on satellite images

Abstract

The present invention relates to the field of satellite imagery ground feature extraction technology, and more specifically to a method for extracting photovoltaic panels based on satellite images, comprising: performing median filtering and adaptive histogram equalization on an image; the image enters an image encoder to obtain an image feature map; dense prompt points are automatically generated, and converted into prompt maps using a prompt encoder; the image feature map and the prompt map are input to a mask decoder to generate an image segmentation result; a classifier classifies each mask in the segmentation result to extract a photovoltaic panel. The present invention can accurately identify photovoltaic panels under different imaging conditions by combining a SAM model with an SVM classification model.

Description

Technical Field

[0001] The present invention relates to the technical field of satellite image ground object extraction, and more specifically to a photovoltaic panel extraction method based on satellite images. Background Art

[0002] Satellite remote sensing technology is a technical means to quickly and efficiently obtain information about surface objects, and can be used to monitor various features and resources on the earth's surface. In recent years, with the improvement of spatial resolution, spectral resolution, and temporal resolution of satellite sensors, it has become possible to extract photovoltaic panels based on satellite images.

[0003] At present, most methods for extracting photovoltaic panels using satellite remote sensing images are based on deep learning models, such as DeepLab V3+, U-Net, Mask R-CNN, etc. These models can effectively segment the outline of photovoltaic panels from complex backgrounds and improve the recognition accuracy of photovoltaic panels. However, because large-scale photovoltaic power stations are mostly built in deserts, mountains and other areas, satellite images are subject to the complex mixture of the surrounding environment and changes in different time and weather conditions, resulting in inconsistent spectral characteristics of photovoltaic panels in the same area under different imaging conditions. The photovoltaic panel extraction method based on deep learning models is greatly reduced in accuracy when facing photovoltaic panel extraction under different imaging conditions. To improve this situation, most of them use the method of increasing samples, which greatly increases the cost of manual annotation and brings difficulties to the extraction of photovoltaic panels. The Segment Anything Model (SAM) is trained on 11 million images to obtain a segmentation model that can segment everything. However, the segmentation results of SAM do not contain any category information. Therefore, it is necessary to combine SAM with the classification model to accurately detect and extract photovoltaic panels. Summary of the invention

[0004] In view of this, the present invention provides a photovoltaic panel recognition technology based on satellite remote sensing images, which effectively solves the problem of accurate recognition of photovoltaic panels under different imaging conditions by combining the SAM model with the SVM classification model.

[0005] In order to achieve the above object, the present invention adopts the following technical solution:

[0006] The photovoltaic panel recognition model based on satellite remote sensing images includes five parts: image enhancement, image encoder, prompt encoder, mask decoder and classifier. The specific technical process is as follows:

[0007] S1: Perform image enhancement on the image;

[0008] S2: The image enters the image encoder to obtain the image feature map;

[0009] S3: The cue encoder converts the automatically generated dense points into cue maps;

[0010] S4: The image feature map and the hint map are input to the mask decoder to generate the image segmentation result;

[0011] S5: The classifier classifies each mask in the segmentation result and extracts the photovoltaic panel.

[0012] Further, the image enhancement comprises:

[0013] Perform median filtering and adaptive histogram equalization on the image. Median filtering performs convolution operation on the filter kernel and the image to eliminate image noise and prevent the noise from affecting subsequent recognition. Adaptive histogram equalization adjusts the brightness distribution and increases the image contrast by calculating the local histogram of the image.

[0014] Further, the image enters the image encoder to obtain an image feature map, including:

[0015] First, input the image, and map it to a vector sequence of size 4096×1280 through a convolution layer with kernel_size of 16, stride of 16, and number of convolution kernels of 1280. Initialize the vector representing the category, and concatenate it with the mapped image vector sequence to obtain a vector sequence of size 4097×1280. Then, assign position encoding information to each row of vectors and add it to the output vector sequence. After completing the position encoding of the vector sequence, repeat the EncoderBlock operation 32 times, and the output vector sequence reduces the image feature dimension through convolution, and finally obtains a 1×64×64×256 image feature vector.

[0016] The Encoder Block includes:

[0017] First, perform layer normalization (LN), then perform multi-head self-attention operation, then perform residual and layer normalization, then perform MLP, and finally perform residual. The MLP operation includes:

[0018] Perform a fully connected layer to linearly stretch the vector sequence;

[0019] Use GELU activation function for activation;

[0020] Enter the fully connected layer to transform the vector sequence scale;

[0021] Furthermore, the cue encoder converts the automatically generated dense points into a cue map, including:

[0022] First, dense hints are automatically generated, and parameters are set to control the density of sampling points and the threshold for removing low-quality or repeated masks. Then, two convolutions with kernel_size of 2 and stride of 2 are performed with layer normalization, and the GELU activation function is used for activation. Finally, a convolution with kernel_size of 1 and stride of 1 is performed, and the hint map with the output size of 1×256×64×64 is output.

[0023] Furthermore, the image feature map and the hint map are input to the mask decoder to generate the image segmentation result, including:

[0024] First, the prompt map and the target map are self-attentioned, and the output prompt map and target map are used as query, and the image map is used as key and value for cross-attention. MLP is used to update each vector sequence point by point, and the output prompt map and target map are used as key and value, and the image map is used as query for cross-attention.

[0025] Then repeat the above operation twice, output the image features that integrate the prompt mapping, and use the output image features as the key and value, and the prompt mapping as the query for cross attention.

[0026] Finally, the output image features are upsampled twice to restore the image size; the prompt vector sequence undergoes four MLPs and is dot-multiplied with the restored image features to obtain the segmentation result.

[0027] Furthermore, the classifier classifies each mask in the segmentation result and extracts the photovoltaic panels, including:

[0028] Label samples, extract spectral features of near-infrared, red, green, and blue bands of photovoltaic panels, build SVM classifiers, and train classification models;

[0029] The image segmentation results generated by the above mask decoder are input into the classification model, all masks are classified, and the mask of the photovoltaic panel is output.

[0030] Through the above technical solution, it can be known that compared with the prior art, the present invention discloses a photovoltaic panel recognition technology based on satellite remote sensing images. First, the image is enhanced, and then the image enters the image encoder to obtain the image feature map. Secondly, the prompt encoder converts the automatically generated dense points into the prompt map, and inputs the image feature map into the mask decoder to generate the image segmentation result. Finally, the classifier is used to classify each mask in the segmentation result to extract the photovoltaic panel. The present invention can better overcome the photovoltaic panel extraction problem of satellite remote sensing images under different imaging conditions, greatly reduce the dependence on the sample size, and improve the photovoltaic panel extraction efficiency and recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only embodiments. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0032] Figure 1 A flow chart of a photovoltaic panel identification method based on satellite remote sensing technology provided by the present invention;

[0033] Figure 2 The image enhancement flow chart provided by the present invention;

[0034] Figure 3 A structural diagram of an image encoder provided by the present invention;

[0035] Figure 4 A structural diagram of a prompt encoder provided by the present invention;

[0036] Figure 5 A structural diagram of the mask decoder provided by the present invention;

[0037] Figure 6 This is a mask classification flow chart provided by the present invention. DETAILED DESCRIPTION

[0038] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0039] like Figure 1 As shown, the embodiment of the present invention discloses a photovoltaic panel identification method based on satellite remote sensing technology, comprising the following steps:

[0040] The photovoltaic panel recognition model based on satellite remote sensing images includes five parts: image enhancement, image encoder, prompt encoder, mask decoder and classifier. The specific technical process is as follows:

[0041] S1: Perform image enhancement on the image;

[0042] S2: The image enters the image encoder to obtain the image feature map;

[0043] S3: The cue encoder converts the automatically generated dense points into cue maps;

[0044] S4: The image feature map and the hint map are input to the mask decoder to generate the image segmentation result;

[0045] S5: The classifier classifies each mask in the segmentation result and extracts the photovoltaic panel.

[0046] In a specific embodiment, Figure 2 As shown, the image is enhanced, including:

[0047] Perform median filtering and adaptive histogram equalization on the image. Median filtering performs convolution operation on the filter kernel and the image to eliminate image noise and prevent the noise from affecting subsequent recognition. Adaptive histogram equalization adjusts the brightness distribution and increases the image contrast by calculating the local histogram of the image.

[0048] In a specific embodiment, Figure 3 As shown, the image enters the image encoder to obtain the image feature map, including:

[0049] First, input the image, and map it to a vector sequence of size 4096×1280 through a convolution layer with kernel_size of 16, stride of 16, and number of convolution kernels of 1280. Initialize the vector representing the category, and concatenate it with the mapped image vector sequence to obtain a vector sequence of size 4097×1280. Then, assign position encoding information to each row of vectors and add it to the output vector sequence. After completing the position encoding of the vector sequence, repeat the EncoderBlock operation 32 times, and the output vector sequence reduces the image feature dimension through convolution, and finally obtains a 1×64×64×256 image feature vector.

[0050] The Encoder Block includes:

[0051] First, perform layer normalization (LN), then perform multi-head self-attention operation, then perform residual and layer normalization, then perform MLP, and finally perform residual. The MLP operation includes:

[0052] Perform a fully connected layer to linearly stretch the vector sequence;

[0053] Use GELU activation function for activation;

[0054] Enter the fully connected layer to transform the vector sequence scale;

[0055] In a specific embodiment, Figure 4 As shown, the prompt encoder converts the automatically generated dense points into a prompt map, including:

[0056] First, dense hints are automatically generated, and parameters are set to control the density of sampling points and the threshold for removing low-quality or repeated masks. Then, two convolutions with kernel_size of 2 and stride of 2 are performed with layer normalization, and the GELU activation function is used for activation. Finally, a convolution with kernel_size of 1 and stride of 1 is performed, and the hint map with the output size of 1×256×64×64 is output.

[0057] In a specific embodiment, Figure 5 As shown, the image feature map and the prompt map are input to the mask decoder to generate the image segmentation result, including:

[0058] First, the prompt map and the target map are self-attentioned, and the output prompt map and target map are used as query, and the image map is used as key and value for cross-attention. MLP is used to update each vector sequence point by point, and the output prompt map and target map are used as key and value, and the image map is used as query for cross-attention.

[0059] Then repeat the above operation twice, output the image features that incorporate the prompt mapping, and use the output image features as the key and value, and the prompt mapping as the query for cross attention;

[0060] Finally, the output image features are upsampled twice to restore the image size; the prompt vector sequence undergoes four MLPs and is dot-multiplied with the restored image features to obtain the segmentation result.

[0061] In a specific embodiment, Figure 6 As shown, the classifier classifies each mask in the segmentation result and extracts the photovoltaic panels, including:

[0062] Label samples, extract spectral features of near-infrared, red, green, and blue bands of photovoltaic panels, build SVM classifiers, and train classification models;

[0063] The image segmentation results generated by the above mask decoder are input into the classification model, all masks are classified, and the mask of the photovoltaic panel is output.

[0064] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0065] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A photovoltaic panel extraction method based on satellite images, characterized in that: The following steps are involved: S1: Perform image enhancement on the image; S2: The image enters the image encoder to obtain the image feature map; S3: The cue encoder converts the automatically generated dense points into cue maps; S4: The image feature map and the hint map are input to the mask decoder to generate the image segmentation result; S5: The classifier classifies each mask in the segmentation result and extracts the photovoltaic panel; The image feature map and the prompt map are input to the mask decoder to generate an image segmentation result, including: First, the prompt map and the target map are self-attentioned, and the output prompt map and target map are used as query, and the image map is used as key and value for cross attention. Then, MLP is used to update each vector sequence point by point, and the output prompt map and target map are used as key and value, and the image map is used as query for cross attention. Then repeat the self-attention of the prompt mapping and the target mapping twice, output the image features that merge the prompt mapping, and use the output image features as the key and value, and the prompt mapping as the query for cross attention; The final output image features are upsampled twice to restore the image size; the prompt vector sequence undergoes four MLPs and is dot-multiplied with the restored image features to obtain the segmentation result.

2. The photovoltaic panel extraction method based on satellite images according to claim 1 is characterized in that: The step of performing image enhancement on the image comprises: The image is subjected to median filtering and adaptive histogram equalization. The median filter performs convolution operation on the image through the filter kernel to eliminate image noise and avoid the influence of noise on subsequent recognition. The adaptive histogram equalization adjusts the brightness distribution and increases the image contrast by calculating the local histogram of the image.

3. The photovoltaic panel extraction method based on satellite images according to claim 1 is characterized in that: The image enters the image encoder to obtain an image feature map, including: First, input the image, and map the image to a vector sequence of size 4096×1280 through a convolution layer with kernal_size of 16, stride of 16, and number of convolution kernels of 1280. Initialize the vector representing the category and concatenate it with the mapped image vector sequence to obtain a vector sequence of size 4097×1280. Then, assign position encoding information to each row of vectors and add it to the output vector sequence. After completing the position encoding of the vector sequence, repeat the Encoder Block operation 32 times, and the output vector sequence reduces the image feature dimension through convolution, and finally obtains an image feature vector of 1×64×64×256.

4. The photovoltaic panel extraction method based on satellite images according to claim 3 is characterized in that: The Encoder Block includes: First, layer normalization (LN) is performed, and multi-head self-attention operation is performed, then residual and layer normalization are performed, further MLP is performed, and finally residual is performed.

5. The photovoltaic panel extraction method based on satellite images according to claim 4 is characterized in that: The MLP comprises: Perform a fully connected layer to linearly stretch the vector sequence; Use GELU activation function for activation; Enter the fully connected layer to transform the vector sequence scale.

6. The photovoltaic panel extraction method based on satellite images according to claim 1, characterized in that: The cue encoder converts the automatically generated dense points into a cue map, including: First, dense hints are automatically generated, and parameters are set to control the density of sampling points and the threshold for removing low-quality or repeated masks. Then, two convolutions with kernel_size of 2 and stride of 2 are performed with layer normalization, and the GELU activation function is used for activation. Finally, a convolution with kernel_size of 1 and stride of 1 is performed, and the hint map with the output size of 1×256×64×64 is output.

7. The photovoltaic panel extraction method based on satellite images according to claim 1, characterized in that: The classifier classifies each mask in the segmentation result and extracts the photovoltaic panel, including: Label samples, extract spectral features of near-infrared, red, green, and blue bands of photovoltaic panels, build SVM classifiers, and train classification models; The image segmentation results generated by the above mask decoder are input into the classification model, all masks are classified, and the mask of the photovoltaic panel is output.

Images