Visual feature constraint-based photovoltaic identification method and related device

Abstract

The invention discloses a photovoltaic identification method based on visual feature constraint and a related device, and relates to the technical field of photovoltaic cell panel identification, and the method comprises the steps: obtaining a remote sensing image data set; an Upnet segmentation network model is constructed; constructing a feature constraint module based on an adversarial auto-encoder, wherein the feature constraint module comprises a color feature constraint module, a texture feature constraint module and a shape feature constraint module; performing adversarial training on the feature constraint module and the Upnet segmentation network model by using the remote sensing image data set to obtain an optimal Upnet segmentation network model; and obtaining a to-be-detected remote sensing image of the target area, inputting the to-be-detected remote sensing image into the optimal Upnet segmentation network model, and predicting to obtain an image segmentation result corresponding to the to-be-detected remote sensing image of the target area. According to the invention, the accuracy of identifying the photovoltaic cell panel can be effectively improved.

Description

Technical Field

[0001] The present application relates to the technical field of photovoltaic panel identification, and in particular to a photovoltaic identification method and related devices based on visual feature constraints. Background Art

[0002] As the demand for renewable energy continues to grow, photovoltaic power generation, as a clean and sustainable form of energy, has received widespread attention and has developed rapidly. Photovoltaic panels are the core components of photovoltaic power generation systems, and their performance and layout directly affect the efficiency of photovoltaic power generation systems. In order to improve the maintenance efficiency and operational reliability of photovoltaic panels, accurate identification and segmentation of photovoltaic panels is particularly important in various application scenarios.

[0003] Photovoltaic panel segmentation is an important problem in computer vision, which aims to identify and extract the photovoltaic panel area from the complex image background. When dealing with photovoltaic panel segmentation, traditional image processing methods often make the photovoltaic panel recognition results inaccurate and unreliable due to reasons such as changing lighting and complex background.

[0004] With the rapid development of deep learning technology, deep learning-based segmentation models have demonstrated powerful performance in computer vision tasks, and photovoltaic panel segmentation has also benefited greatly from this. However, the visual features of photovoltaic panels in images are unique, such as uniform texture, significant color differences, and regular shapes. Given the above visual features of photovoltaic panels, traditional general deep learning segmentation models often fail to fully utilize these features when applied to photovoltaic panel segmentation tasks, thereby limiting the further improvement of segmentation effects, and the final photovoltaic panel recognition results are not very accurate. Summary of the invention

[0005] The purpose of this application is to provide a photovoltaic identification method and related devices based on visual feature constraints, which can effectively improve the accuracy of identifying photovoltaic panels.

[0006] To achieve the above objectives, this application provides the following solutions:

[0007] In a first aspect, the present application provides a photovoltaic recognition method based on visual feature constraints, comprising the following steps:

[0008] Acquire a remote sensing image dataset; the remote sensing image dataset includes a plurality of remote sensing images and corresponding real labels, each remote sensing image corresponds to a real label, and the real label is used to characterize whether the surface of the remote sensing image contains photovoltaic panels and the area where the photovoltaic panels are located.

[0009] An Upernet segmentation network model is constructed; the Upernet segmentation network model takes the remote sensing image as input and takes the image segmentation result corresponding to the remote sensing image as output, the Upernet segmentation network model includes a feature pyramid network, a feature aggregation module and a semantic segmentation detection head, the feature pyramid network is used to extract and fuse the multi-scale features of the remote sensing image, the feature aggregation module is used to integrate the feature map corresponding to the fused multi-scale features, and the semantic segmentation detection head is used to output the image segmentation result according to the integrated feature map; the image segmentation result is the result of segmenting the area where the photovoltaic panel is located in the predicted remote sensing image.

[0010] Construct a feature constraint module based on an adversarial autoencoder; the feature constraint module includes a color feature constraint module, a texture feature constraint module and a shape feature constraint module, and the color feature constraint module, the texture feature constraint module and the shape feature constraint module all include an autoencoder, a decoder and a discriminator, the autoencoder is used to encode the input feature vector into a low-dimensional manifold, the decoder is used to decode and reconstruct the input vector, and the discriminator is used to distinguish the difference between the image segmentation result corresponding to the remote sensing image predicted by the Upernet segmentation network model and the true label of the remote sensing image.

[0011] Using the remote sensing image data set, adversarial training is performed on the feature constraint module and the Upernet segmentation network model to obtain an optimal Upernet segmentation network model.

[0012] A remote sensing image to be measured of a target area is obtained, and the remote sensing image to be measured is input into the optimal Upernet segmentation network model to predict the image segmentation result corresponding to the remote sensing image to be measured of the target area.

[0013] Optionally, after the step of acquiring the remote sensing image data set, the photovoltaic recognition method based on visual feature constraints further comprises the following steps:

[0014] Preprocess each remote sensing image in the remote sensing image dataset to obtain a preprocessed remote sensing image dataset; the preprocessed remote sensing image dataset is used to perform adversarial training on the feature constraint module and the Upernet segmentation network model to obtain the optimal Upernet segmentation network model.

[0015] Optionally, preprocessing each remote sensing image in the remote sensing image dataset to obtain a preprocessed remote sensing image dataset specifically includes the following steps:

[0016] Each of the remote sensing images in the remote sensing image data set is cropped to obtain a cropped remote sensing image data set; the cropped remote sensing image data set includes a plurality of cropped remote sensing images.

[0017] Data enhancement processing is performed on each of the cropped remote sensing images in the cropped remote sensing image dataset to obtain the preprocessed remote sensing image dataset; the data enhancement processing includes image random rotation processing, and the image random rotation processing includes image random horizontal flipping processing and image random vertical flipping processing.

[0018] Optionally, constructing the color feature constraint module specifically includes the following steps:

[0019] For the color feature constraint module, the predicted mask and the true label of the Upernet segmentation network model are respectively accepted, and the predicted mask and the true label are respectively masked with the corresponding remote sensing image to determine the target area image.

[0020] A color histogram of the target area is calculated according to the target area image.

[0021] Based on the color histogram of the target area, the automatic encoder of the color feature constraint module is used to respectively accept the color histogram feature vector of the positive area predicted by the Upernet segmentation network model and the color histogram feature vector of the actual positive area, and map each color feature in the color histogram feature vector to a low-dimensional vector; the positive area is the area where the photovoltaic panel is located.

[0022] The decoder of the color feature constraint module is used to capture the color-related features in the low-dimensional vector while discarding the color-irrelevant features, and reconstruct the input color features.

[0023] The discriminator of the color feature constraint module is used to distinguish whether the input color feature comes from the image segmentation result predicted by the Upernet segmentation network model or from a real remote sensing image sample.

[0024] Optionally, constructing the texture feature constraint module specifically includes the following steps:

[0025] For the texture feature constraint module, an LBP algorithm is used to calculate the local binary pattern coding of the remote sensing image.

[0026] According to the local binary pattern coding of the remote sensing image, an LBP coding map corresponding to the remote sensing image is determined.

[0027] The LBP encoding image is masked with the corresponding prediction mask of the Upernet segmentation network model and the true label to obtain the LBP value of the target area.

[0028] The LBP values ​​of the target area are used as texture feature vectors and sent to the automatic encoders of the texture feature constraint module respectively, and the discriminator of the texture feature constraint module is used to distinguish the texture feature differences in the adversarial training process.

[0029] Optionally, constructing the shape feature constraint module specifically includes the following steps:

[0030] For the shape feature constraint module, the image segmentation result predicted by the Upernet segmentation network model and the true label are input into the autoencoder of the shape feature constraint module, and the autoencoder is trained using the adversarial training method so that the autoencoder of the shape feature constraint module encodes two types of shapes and captures the difference between the two types of shapes. At the same time, the Upernet segmentation network model is used to antagonize the autoencoder of the shape feature constraint module so that the autoencoder of the shape feature constraint module cannot capture the difference between the two types of shapes; wherein the two types of shapes refer to the shape of the binary image of the image segmentation result predicted by the Upernet segmentation network model and the shape of the binary image of the true label.

[0031] Optionally, the expression of the objective loss function used in the adversarial training process is:

[0032]

[0033] Among them, L all represents the total objective loss function of adversarial training, L seg , L s , L t , L c They represent segmentation loss, shape loss, texture loss, and color loss respectively; λ1, λ2, and λ3 are hyperparameters; G and D represent the Upernet segmentation network model and the feature constraint module respectively.

[0034] In a second aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any of the above-described photovoltaic identification methods based on visual feature constraints.

[0035] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described photovoltaic identification methods based on visual feature constraints.

[0036] In a fourth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-mentioned photovoltaic identification methods based on visual feature constraints.

[0037] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0038] The present application provides a photovoltaic identification method and related device based on visual feature constraints, which combines an Upernet segmentation network model with a feature constraint module based on an adversarial autoencoder, and constructs an Upernet segmentation network model and a feature constraint module respectively, wherein the feature constraint module includes a color feature constraint module, a texture feature constraint module, and a shape feature constraint module, so that when adversarial training is performed between the feature constraint module and the Upernet segmentation network model, the Upernet segmentation network model fully learns the features of the color, texture, and shape of the photovoltaic panels in the remote sensing image. The Upernet segmentation network model is continuously approached to the real photovoltaic panel contour by adversarial training, and the feature constraint module attempts to distinguish the difference between the predicted results and the real results of the Upernet segmentation network model, and guides the Upernet segmentation network model to focus on the color features, texture features, and shape features of the photovoltaic panels. After the adversarial training is completed, the optimal Upernet segmentation network model can be obtained. In practical applications, the remote sensing image to be tested in the target area is obtained and input into the optimal Upernet segmentation network model, so that a more accurate and reliable image segmentation result can be predicted, which effectively improves the accuracy of identifying photovoltaic panels. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0040] Figure 1 A schematic flow chart of a photovoltaic identification method based on visual feature constraints provided in one embodiment of the present application.

[0041] Figure 2 A flowchart of an overall network architecture provided for an embodiment of the present application.

[0042] Figure 3 A schematic diagram of the structure of the UperNet segmentation network model provided in one embodiment of the present application.

[0043] Figure 4 A schematic diagram of the structure of a color feature constraint module provided in one embodiment of the present application.

[0044] Figure 5 A schematic diagram of the structure of a texture feature constraint module provided in one embodiment of the present application.

[0045] Figure 6 A schematic diagram of the structure of a shape feature constraint module provided in one embodiment of the present application. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0047] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0048] like Figure 1 and Figure 2 As shown, this embodiment provides a photovoltaic recognition method based on visual feature constraints, which aims to accurately identify the outline of photovoltaic panels from remote sensing images and segment the area where the photovoltaic panels are located. The method specifically includes the following steps:

[0049] Step S1, obtaining a remote sensing image dataset.

[0050] In this embodiment, the remote sensing image data set includes a plurality of remote sensing images and corresponding real labels, each of the remote sensing images corresponds to one real label, and the real label is used to characterize whether the surface of the remote sensing image contains photovoltaic panels and the area where the photovoltaic panels are located.

[0051] Step S2: construct an Upernet (unified perceptual parsing network) segmentation network model.

[0052] In this embodiment, the Upernet segmentation network model takes the remote sensing image as input and takes the image segmentation result corresponding to the remote sensing image as output. The Upernet segmentation network model includes a feature pyramid network (FPN), a feature aggregation module and a semantic segmentation detection head, the feature pyramid network is used to extract and fuse the multi-scale features of the remote sensing image, the feature aggregation module is used to integrate the feature map corresponding to the fused multi-scale features, and the semantic segmentation detection head is used to output the image segmentation result according to the integrated feature map; the image segmentation result is the result of segmenting the area where the photovoltaic panel is located in the predicted remote sensing image.

[0053] Step S3: construct a feature constraint module based on the adversarial autoencoder.

[0054] In this embodiment, the feature constraint module includes a color feature constraint module, a texture feature constraint module and a shape feature constraint module. The color feature constraint module, the texture feature constraint module and the shape feature constraint module all include an autoencoder, a decoder and a discriminator. The autoencoder is used to encode the input feature vector into a low-dimensional manifold, the decoder is used to decode and reconstruct the input vector, and the discriminator is used to distinguish the difference between the image segmentation result corresponding to the remote sensing image predicted by the Upernet segmentation network model and the true label of the remote sensing image.

[0055] Step S4: using the remote sensing image data set, performing adversarial training on the feature constraint module and the Upernet segmentation network model to obtain an optimal Upernet segmentation network model.

[0056] Step S5, obtaining a remote sensing image to be measured of the target area, and inputting the remote sensing image to be measured into the optimal Upernet segmentation network model to predict the image segmentation result corresponding to the remote sensing image to be measured of the target area.

[0057] In this embodiment, after the remote sensing image data set is acquired in step S1, the following steps are also included:

[0058] Step S1.5, preprocess each remote sensing image in the remote sensing image dataset to obtain a preprocessed remote sensing image dataset. The preprocessed remote sensing image dataset is used to perform adversarial training on the feature constraint module and the Upernet segmentation network model to obtain the optimal Upernet segmentation network model. Specifically, the following steps are included:

[0059] Step S1.51, cropping each remote sensing image in the remote sensing image data set to obtain a cropped remote sensing image data set; the cropped remote sensing image data set includes a plurality of cropped remote sensing images.

[0060] Step S1.52, perform data enhancement processing on each of the cropped remote sensing images in the cropped remote sensing image dataset to obtain the preprocessed remote sensing image dataset; the data enhancement processing includes image random rotation processing, and the image random rotation processing includes image random horizontal flipping processing and image random vertical flipping processing.

[0061] In this embodiment, step S3 constructs a feature constraint module based on the adversarial autoencoder, which specifically includes the following steps:

[0062] Step S31: for the color feature constraint module, respectively accept the predicted mask and the true label of the Upernet segmentation network model, and respectively mask the predicted mask and the true label with the corresponding remote sensing image to determine the target area image.

[0063] Step S32: Calculate the color histogram of the target area according to the target area image.

[0064] Step S33, based on the color histogram of the target area, using the automatic encoder of the color feature constraint module, respectively accepting the color histogram feature vector of the positive area predicted by the Upernet segmentation network model and the color histogram feature vector of the actual positive area, and mapping each color feature in the color histogram feature vector to a low-dimensional vector; the positive area is the area where the photovoltaic panel is located.

[0065] Step S34: using the decoder of the color feature constraint module, capturing the color-related features in the low-dimensional vector while discarding the color-irrelevant features, and reconstructing the input color features.

[0066] Step S35, using the discriminator of the color feature constraint module, distinguish whether the input color feature comes from the image segmentation result predicted by the Upernet segmentation network model or from a real remote sensing image sample.

[0067] Step S36: For the texture feature constraint module, use LBP (Local Binary Patterns) algorithm to calculate the local binary pattern code of the remote sensing image.

[0068] Step S37: Determine the LBP coding image corresponding to the remote sensing image according to the local binary pattern coding of the remote sensing image.

[0069] Step S38, masking the LBP encoding image with the corresponding prediction mask of the Upernet segmentation network model and the true label to obtain the LBP value of the target area.

[0070] Step S39: The LBP values ​​of the target area are used as texture feature vectors and sent to the automatic encoders of the texture feature constraint module respectively, and the discriminator of the texture feature constraint module is used to distinguish the texture feature differences in the adversarial training process.

[0071] Step S40: for the shape feature constraint module, the image segmentation result predicted by the Upernet segmentation network model and the true label are input into the autoencoder of the shape feature constraint module, and the autoencoder is trained by the adversarial training method so that the autoencoder of the shape feature constraint module encodes two types of shapes and captures the difference between the two types of shapes. At the same time, the Upernet segmentation network model is used to antagonize the autoencoder of the shape feature constraint module so that the autoencoder of the shape feature constraint module cannot capture the difference between the two types of shapes; wherein the two types of shapes refer to the shape of the binary image of the image segmentation result predicted by the Upernet segmentation network model and the shape of the binary image of the true label.

[0072] In order to make the technical solution of this embodiment clearer, the specific implementation process of the technical solution of this embodiment is described in detail below in the form of examples. Specifically, the following implementation steps are included:

[0073] Step 1: Data collection and preprocessing.

[0074] S1-1: Collect training data sets, including remote sensing image data sets and corresponding manually annotated sample images. In this embodiment, remote sensing images with a spatial resolution of 0.15 m and three bands of red, green and blue from Google Earth are used.

[0075] S1-2: Perform image cropping processing. In this embodiment, a sliding window is used to crop the remote sensing image into blocks of 256×256 pixels.

[0076] S1-3: Perform data augmentation, including random image rotation, random horizontal flipping, and random vertical flipping, to enhance the generalization ability of the model.

[0077] Step 2, build the Upernet segmentation network model. The structure of the Upernet segmentation network model is as follows Figure 3 As shown in the figure, the Upernet segmentation network model uses a feature pyramid network to extract and fuse multi-scale features, integrating feature maps at different levels to obtain richer feature representations. Additional feature aggregation modules are used to further integrate and process feature maps. Finally, the segmentation result is output through the semantic segmentation detection head.

[0078] Step 3, construct a feature constraint module based on adversarial autoencoder.

[0079] S3-1: The feature constraint module includes a color feature constraint module, a texture feature constraint module, and a shape feature constraint module. Each feature constraint module includes an autoencoder, a decoder, and a discriminator. The autoencoder encodes the input feature vector into a low-dimensional vector, the decoder reconstructs the input vector, and the discriminator is used to distinguish the difference between the real sample and the prediction result of the Upernet segmentation network model, that is, the difference between the image segmentation result corresponding to the remote sensing image predicted by the Upernet segmentation network model and the real label of the remote sensing image.

[0080] In this embodiment, the characteristics of different features are taken into consideration. The autoencoders in the color feature constraint module and the texture feature constraint module are composed of multiple linear layers, and the autoencoder in the shape feature constraint module is composed of multiple convolutional layers. The steps for constructing each feature constraint module are as follows:

[0081] S3-2: For the color feature constraint module, accept the predicted mask and true label of the Upernet segmentation network model, respectively mask the corresponding remote sensing image to obtain the target area image, and calculate the color histogram of the target area. Build an autoencoder, which is a trainable neural network that accepts the color histogram feature vectors of the positive area predicted by the Upernet segmentation network model and the true area, maps each color feature to a low-dimensional vector, captures its significant features while discarding irrelevant features, and reconstructs the input color features as much as possible. Figure 4 As shown in the figure, in the color feature constraint module, multiple linear layers are used to build an autoencoder, where the discriminator is used to distinguish whether the input feature comes from the Upernet segmentation network model prediction or the real sample.

[0082] S3-3: For the texture feature constraint module, such as Figure 5 As shown, the LBP algorithm is first used to calculate the local binary pattern coding of the remote sensing image to construct the texture feature vector of the target area. The formula is as follows:

[0083]

[0084] Among them, LBP(·) is local binary pattern coding, (x c ,y c ) represents the center pixel of the neighborhood window, i p is the value of the center pixel of the neighborhood window, i c is the value of other pixels in the neighborhood, p is the number of sampled points in the neighborhood, and S(x) is the sign function, expressed as:

[0085]

[0086] Then, the LBP encoding map of the remote sensing image is masked with the corresponding prediction mask and true label of the Upernet segmentation network model to obtain the LBP value of the target area as the texture feature vector.

[0087] The LBP values ​​of the target area are respectively sent to an autoencoder composed of multiple linear layers to learn the texture features of the photovoltaic panel image. Through adversarial training, it is difficult for the discriminator to distinguish the differences in texture features between them, thereby optimizing the parameters of the Upernet segmentation network model.

[0088] S3-4: For shape feature constraint modules, such as Figure 6 As shown, the prediction results of the Upernet segmentation network model and the true label are directly fed into the autoencoder to reconstruct the input shape as much as possible. The autoencoder is trained through an adversarial training scheme, forcing the autoencoder to better encode the two types of shapes and capture their subtle differences, while encouraging the Upernet segmentation network model to trick the autoencoder into not being able to capture subtle differences. Since the input is in the form of an image, the shape feature constraint module consists of multiple convolutional layers.

[0089] Step 4: Train the Upernet segmentation network model and feature constraint module. This includes the following steps:

[0090] S4-1: The training method adopts adversarial training, and the Upernet segmentation network model and the feature constraint module are optimized alternately. The Upernet segmentation network model attempts to minimize the difference between the segmentation result and the true label, while the feature constraint module attempts to learn the encoding to maximize their distance. Therefore, the total target loss function is defined as:

[0091]

[0092] Among them, L all represents the total objective loss function of adversarial training, L seg , L s , L t , L c Represent segmentation loss, shape loss, texture loss, and color loss, respectively. λ1, λ2, and λ3 are hyperparameters, which are used to control the weights of shape loss, texture loss, and color loss, respectively. G and D represent the Upernet segmentation network model and the autoencoder of the feature constraint module, respectively. During the training process of G, the results predicted by G will gradually approach the true labels on the ground, and the feature constraint module encourages the autoencoder to encode the subtle differences between them.

[0093] S4-2: The segmentation loss uses cross entropy loss, which is defined as:

[0094]

[0095] Among them, L seg represents the segmentation loss, which uses the cross entropy loss, q i is the probability that the model predicts the positive class, p i is the true label of the i-th sample.

[0096] S4-3: The autoencoder has two loss terms, reconstruction loss and adversarial loss.

[0097] The reconstruction loss is defined as:

[0098]

[0099] Among them, L rec is the reconstruction loss, x represents the input feature vector, y represents its corresponding true label, G(x) represents the predicted mask of the segmentation module, D(y) and D(G(x)) represent the reconstruction results of the true label and the predicted mask, respectively.

[0100] The adversarial loss encourages the discriminator to distinguish different inputs, thereby guiding the Upernet segmentation network model to deceive the autoencoder. For the shape feature constraint module, texture feature constraint module, and color feature constraint module, they are adversarial shape loss, texture loss, and color loss, respectively. It is defined as:

[0101]

[0102] Among them, L d represents the feature constraint loss, which refers to shape loss, texture loss and color loss in the shape feature constraint module, texture feature constraint module and color feature constraint module respectively. E(y) and E(G(x)) are the encodings of the true label and predicted mask.

[0103] During training, we first optimize G by fixing D and using the following loss:

[0104]

[0105] Among them, L G Represents the segmentation loss and feature constraint loss of the Upernet segmentation network model.

[0106] Optimize L G G will be encouraged to output PV panel predictions that are consistent with the ground truth labels. Then, when G is fixed, D is optimized by the following loss:

[0107]

[0108] Among them, λ is a hyperparameter used to control the weights of shape loss, texture loss, and color loss, and ω is a hyperparameter used to make the autoencoder training more stable.

[0109] Throughout the training process, the Upernet segmentation network model will continue to approach the actual PV panel outline. When the feature constraint module cannot distinguish the difference between the predicted results and the actual results, it no longer provides effective supervision. At this point, the Upernet segmentation network model can independently output better segmentation results.

[0110] In this embodiment, the output results of the UperNet segmentation network model are input into the feature constraint module. The feature constraint module (including the color feature constraint module, the texture feature constraint module and the shape feature constraint module) simultaneously receives the output results of the UperNet segmentation network model, the remote sensing images in the training set and the real labels. The feature constraint module is only used in the training stage. During training, the UperNet segmentation network model and the feature constraint module update parameters alternately. The UperNet segmentation network model first outputs the results, and then the results are sent to the feature constraint module. The feature constraint module extracts texture, color and shape features respectively, and distinguishes the differences in texture, color and shape between the output results of the UperNet segmentation network model and the corresponding areas of the real labels. Use texture loss, color loss and shape loss to measure the size of the difference, and add the loss value to the segmentation loss to control the optimization direction of the next iteration of the UperNet segmentation network model.

[0111] In this embodiment, texture loss, color loss, shape loss and segmentation loss of UperNet segmentation network model are weighted and added. That is, the overall objective function. The final UperNet segmentation network model is jointly affected by segmentation loss, texture loss, color loss and shape loss. The segmentation loss is calculated by the output result of the UperNet segmentation network model and the true label, the texture loss is calculated in the texture feature constraint module, the color loss is calculated in the color feature constraint module, and the shape loss is calculated in the shape feature constraint module.

[0112] Step 5, save the parameters of the optimal segmentation model and make predictions.

[0113] Remove the feature constraint module, save the trained UperNet segmentation network model, and apply it to the recognition of large-scale remote sensing images. The purpose is to further improve the segmentation accuracy without increasing the model parameters of the UperNet segmentation network model in the prediction application stage, and to improve the accuracy and robustness of the UperNet segmentation network model in identifying photovoltaic panels on the basis of effectively controlling the computational cost. This has high practicality for large-scale data recognition tasks.

[0114] The present application provides a photovoltaic recognition method based on visual feature constraints, including remote sensing image preprocessing; constructing an UperNet segmentation network model; constructing a color feature constraint module, a texture feature constraint module and a shape feature constraint module based on an autoencoder; using a color histogram to extract the color features of photovoltaic panels in remote sensing images, using local binary pattern encoding to extract the texture features of photovoltaic panels in remote sensing images, and using a binary image to extract the shape features of photovoltaic panels in remote sensing images; through adversarial training, the UperNet segmentation network model is continuously approached to the actual photovoltaic panel contour, and the feature constraint module attempts to distinguish the difference between the predicted result and the actual result, guiding the UperNet segmentation network model to focus on the color, texture and shape features of the photovoltaic panel; finally, after the adversarial training is completed, the feature constraint module is removed, and the optimized optimal UperNet segmentation network model is applied to the recognition of photovoltaic panels, so as to accurately identify and segment the photovoltaic panel area, and finally accurately delineate the area where the photovoltaic panel is located from the remote sensing image. Without increasing the number of parameters of the UperNet segmentation network model, targeted optimization of the task of photovoltaic panel identification can be achieved, effectively improving the accuracy of the UperNet segmentation network model in identifying and segmenting photovoltaic panels.

[0115] In an exemplary embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0116] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0117] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0118] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0119] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0120] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A photovoltaic recognition method based on visual feature constraints, characterized in that: The photovoltaic recognition method based on visual feature constraints includes: Acquire a remote sensing image data set; the remote sensing image data set includes a plurality of remote sensing images and corresponding true labels, each remote sensing image corresponds to one true label, and the true label is used to characterize whether the surface of the remote sensing image contains a photovoltaic panel and the area where the photovoltaic panel is located; Constructing an Upernet segmentation network model; the Upernet segmentation network model takes the remote sensing image as input and takes the image segmentation result corresponding to the remote sensing image as output, the Upernet segmentation network model includes a feature pyramid network, a feature aggregation module and a semantic segmentation detection head, the feature pyramid network is used to extract and fuse the multi-scale features of the remote sensing image, the feature aggregation module is used to integrate the feature map corresponding to the fused multi-scale features, and the semantic segmentation detection head is used to output the image segmentation result according to the integrated feature map; the image segmentation result is the result of segmenting the area where the photovoltaic panel is located in the predicted remote sensing image; Constructing a feature constraint module based on an adversarial autoencoder; the feature constraint module includes a color feature constraint module, a texture feature constraint module and a shape feature constraint module, and the color feature constraint module, the texture feature constraint module and the shape feature constraint module all include an autoencoder, a decoder and a discriminator, the autoencoder is used to encode an input feature vector into a low-dimensional manifold, the decoder is used to decode and reconstruct the input vector, and the discriminator is used to distinguish the difference between the image segmentation result corresponding to the remote sensing image predicted by the Upernet segmentation network model and the true label of the remote sensing image; Using the remote sensing image data set, the feature constraint module and the Upernet segmentation network model are subjected to adversarial training to obtain an optimal Upernet segmentation network model; A remote sensing image to be measured of a target area is obtained, and the remote sensing image to be measured is input into the optimal Upernet segmentation network model to predict the image segmentation result corresponding to the remote sensing image to be measured of the target area.

2. The photovoltaic identification method based on visual feature constraints according to claim 1 is characterized in that: After the step of acquiring the remote sensing image data set, the photovoltaic recognition method based on visual feature constraints further includes: Preprocess each remote sensing image in the remote sensing image dataset to obtain a preprocessed remote sensing image dataset; the preprocessed remote sensing image dataset is used to perform adversarial training on the feature constraint module and the Upernet segmentation network model to obtain the optimal Upernet segmentation network model.

3. The photovoltaic identification method based on visual feature constraints according to claim 2 is characterized in that: Preprocessing each remote sensing image in the remote sensing image dataset to obtain a preprocessed remote sensing image dataset specifically includes: Performing cropping processing on each of the remote sensing images in the remote sensing image data set to obtain a cropped remote sensing image data set; the cropped remote sensing image data set includes a plurality of cropped remote sensing images; Data enhancement processing is performed on each of the cropped remote sensing images in the cropped remote sensing image dataset to obtain the preprocessed remote sensing image dataset; the data enhancement processing includes image random rotation processing, and the image random rotation processing includes image random horizontal flipping processing and image random vertical flipping processing.

4. The photovoltaic identification method based on visual feature constraints according to claim 1 is characterized in that: Constructing the color feature constraint module specifically includes: For the color feature constraint module, the predicted mask and the true label of the Upernet segmentation network model are respectively accepted, and the predicted mask and the true label are respectively masked with the corresponding remote sensing image to determine the target area image; Calculating a color histogram of the target area according to the target area image; Based on the color histogram of the target area, the automatic encoder of the color feature constraint module is used to respectively accept the color histogram feature vector of the positive area predicted by the Upernet segmentation network model and the color histogram feature vector of the real positive area, and map each color feature in the color histogram feature vector to a low-dimensional vector; the positive area is the area where the photovoltaic panel is located; Using a decoder of the color feature constraint module, the color-related features in the low-dimensional vector are captured while the color-irrelevant features are discarded, and the input color features are reconstructed; The discriminator of the color feature constraint module is used to distinguish whether the input color feature comes from the image segmentation result predicted by the Upernet segmentation network model or from a real remote sensing image sample.

5. The photovoltaic identification method based on visual feature constraints according to claim 1, characterized in that: Constructing the texture feature constraint module specifically includes: For the texture feature constraint module, the LBP algorithm is used to calculate the local binary pattern coding of the remote sensing image; Determine the LBP coding map corresponding to the remote sensing image according to the local binary pattern coding of the remote sensing image; Masking the LBP encoding map with the corresponding prediction mask of the Upernet segmentation network model and the true label to obtain the LBP value of the target area; The LBP values ​​of the target area are used as texture feature vectors and sent to the automatic encoders of the texture feature constraint module respectively, and the discriminator of the texture feature constraint module is used to distinguish the texture feature differences in the adversarial training process.

6. The photovoltaic identification method based on visual feature constraints according to claim 1, characterized in that: Constructing the shape feature constraint module specifically includes: For the shape feature constraint module, the image segmentation result predicted by the Upernet segmentation network model and the true label are input into the autoencoder of the shape feature constraint module, and the autoencoder is trained using the adversarial training method so that the autoencoder of the shape feature constraint module encodes two types of shapes and captures the difference between the two types of shapes. At the same time, the Upernet segmentation network model is used to antagonize the autoencoder of the shape feature constraint module so that the autoencoder of the shape feature constraint module cannot capture the difference between the two types of shapes; wherein the two types of shapes refer to the shape of the binary image of the image segmentation result predicted by the Upernet segmentation network model and the shape of the binary image of the true label.

7. The photovoltaic identification method based on visual feature constraints according to claim 1, characterized in that: The expression of the objective loss function used in the adversarial training process is: Among them, L all Represents the total objective loss function of adversarial training, L seg , L s , L t , L c They represent segmentation loss, shape loss, texture loss, and color loss respectively; λ1, λ2, and λ3 are hyperparameters; G and D represent the Upernet segmentation network model and the feature constraint module respectively.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the photovoltaic identification method based on visual feature constraints as described in any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the photovoltaic identification method based on visual feature constraints described in any one of claims 1 to 7 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the photovoltaic identification method based on visual feature constraints described in any one of claims 1 to 7 is implemented.

Images