Method, device and equipment for processing semantic segmentation model of photovoltaic panel
A semantic segmentation model for photovoltaic panels, composed of MobileNetV2 and Deeplab V3+ networks, solves the problem of accurate monitoring of photovoltaic power plants, realizes efficient and high-precision photovoltaic panel detection, and supports intelligent monitoring of photovoltaic power plants.
Patent Information
- Application Number
- CN202411172267.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-26
- Publication Date
- 2024-11-22
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing technologies cannot achieve accurate monitoring of photovoltaic power plants, especially efficient and high-precision detection of photovoltaic panels, resulting in low statistical efficiency, low data accuracy, and susceptibility to human factors.
A semantic segmentation model for photovoltaic panels, consisting of MobileNetV2 and Deeplab V3+ networks, is used to process photovoltaic power plant images through semantic segmentation, generate photovoltaic panel masks, and output high-level feature maps through the ASPP structure for upsampling and convolution operations to achieve high-precision prediction of photovoltaic panels.
It achieves efficient and high-precision photovoltaic panel image detection, supports intelligent monitoring of photovoltaic power plants, and improves detection efficiency and accuracy.
Smart Images

Figure CN119006827A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the photovoltaic field, and specifically to a processing method, device and processing equipment for a semantic segmentation model of a photovoltaic panel. Background Art
[0002] At present, solar energy technology is mainly solar photovoltaic, which directly converts sunlight into electricity through photovoltaic arrays. Since the construction of large-scale photovoltaic power stations requires a large amount of land, understanding the distribution of photovoltaic power stations is crucial for evaluating the ecological and environmental impact and predicting the actual power generation of photovoltaic power stations.
[0003] There are many publicly released statistical data on photovoltaic installed capacity. Although manual statistical monitoring methods have low requirements on technology and equipment, they consume a lot of time, manpower and material resources, and are easily affected by environmental conditions and other factors, resulting in low statistical efficiency, low data accuracy and a large impact of human factors.
[0004] In addition, the spatial distribution data of the published photovoltaic maps are mainly concentrated in the medium spatial resolution of 30m and the high spatial resolution of 10m. In this case, the extraction results of the complete centralized photovoltaic power station using the existing images of different resolutions are relatively rough. The main identification results are the complete individual information of the photovoltaic power station, which contains more non-panel data, including roads between photovoltaic facilities and nearby houses.
[0005] The mainstream methods for extracting images from photovoltaic power plants mainly focus on using random forest models to extract power plants by inputting a large amount of spectral information, or using the comparison of deep learning semantic segmentation models to extract large areas of power plants. The former can extract large-scale photovoltaic power plants in a large range, but there is more complex data pre-processing; the latter reduces parameters and data is quickly input, but most of them only compare models.
[0006] It can be seen that the existing technology for image detection of photovoltaic power stations remains at the rough photovoltaic power station level and cannot penetrate into the smaller granularity photovoltaic panels, and cannot meet the precise monitoring needs of photovoltaic power stations. In addition, image detection of photovoltaic power stations itself also has the problem of difficulty in balancing detection accuracy and detection efficiency. Summary of the invention
[0007] The present application provides a processing method, device and processing equipment for a semantic segmentation model of a photovoltaic panel. For the image detection target of a photovoltaic panel, based on the introduction of semantic segmentation to configure the semantic segmentation model of the photovoltaic panel, a more specific model configuration scheme is also provided, thereby obtaining an efficient and high-precision photovoltaic panel image detection effect, which is helpful to realize the intelligent monitoring of photovoltaic power stations.
[0008] In a first aspect, the present application provides a device for processing a semantic segmentation model of a photovoltaic panel, the device comprising:
[0009] Obtain sample PV power plant images from different data sources;
[0010] Perform semantic segmentation of photovoltaic panels on sample photovoltaic power station images to obtain photovoltaic panel masks, and form a photovoltaic panel dataset consisting of sample photovoltaic power station images and photovoltaic panel masks;
[0011] The photovoltaic panel semantic segmentation model is trained through the photovoltaic panel dataset. The photovoltaic panel semantic segmentation model is used to predict the mask of the photovoltaic panel in the photovoltaic power station image input by the model. The model architecture of the photovoltaic panel semantic segmentation model consists of the MobileNetV2 network and the Deeplab V3+ network. The MobileNetV2 network generates the initial feature map of the photovoltaic power station image input by the model, and then inputs the initial feature map into the ASPP structure used in the encoder part of the Deeplab V3+ network. The high-level feature map is output through the network structure with different expansion rates and superimposed, and then upsampling and convolution operations are performed to obtain the final output feature map, and finally the photovoltaic panel prediction is performed.
[0012] In a second aspect, the present application provides a processing device for a photovoltaic panel semantic segmentation model, the device comprising:
[0013] An acquisition unit, used for acquiring sample photovoltaic power station images from different data sources;
[0014] A segmentation unit is used to perform semantic segmentation processing on photovoltaic panels of sample photovoltaic power station images to obtain photovoltaic panel masks and form a photovoltaic panel data set consisting of sample photovoltaic power station images and photovoltaic panel masks;
[0015] The training unit is used to train a photovoltaic panel semantic segmentation model using a photovoltaic panel data set, wherein the photovoltaic panel semantic segmentation model is used to predict the mask of the photovoltaic panel in the photovoltaic power station image input by the model. The model architecture of the photovoltaic panel semantic segmentation model consists of a MobileNetV2 network and a Deeplab V3+ network. The MobileNetV2 network generates an initial feature map of the photovoltaic power station image input by the model, and then inputs the initial feature map into the ASPP structure used in the encoder part of the Deeplab V3+ network. The high-level feature map is output through the network structure with different expansion rates and superimposed, and then upsampling and convolution operations are performed to obtain the final output feature map, and finally the photovoltaic panel prediction is performed.
[0016] In a third aspect, the present application provides a processing device, including a processor and a memory, wherein a computer program is stored in the memory, and when the processor calls the computer program in the memory, the method provided in the first aspect of the present application or any possible implementation method of the first aspect of the present application is executed.
[0017] In a fourth aspect, the present application provides a computer-readable storage medium, which stores multiple instructions, and the instructions are suitable for a processor to load to execute the method provided by the first aspect of the present application or any possible implementation of the first aspect of the present application.
[0018] From the above content, it can be concluded that the present application has the following beneficial effects:
[0019] For the image detection target of photovoltaic panels, after obtaining sample photovoltaic power station images from different data sources, this application performs photovoltaic panel semantic segmentation processing on the sample photovoltaic power station images to obtain photovoltaic panel masks, and forms a photovoltaic panel data set consisting of sample photovoltaic power station images and photovoltaic panel masks. Then, the photovoltaic panel semantic segmentation model is trained through the photovoltaic panel data set. The model architecture of the photovoltaic panel semantic segmentation model consists of a MobileNetV2 network and a DeeplabV3+ network. The MobileNetV2 network generates the initial feature map of the photovoltaic power station image input to the model, and then inputs the initial feature map into the ASPP structure used in the encoder part of the Deeplab V3+ network. The high-level feature map is output through the network structure with different expansion rates and superimposed, and then upsampling and convolution operations are performed to obtain the final output feature map, and finally photovoltaic panel prediction is performed. In this way, on the basis of introducing semantic segmentation to configure the photovoltaic panel semantic segmentation model, a more specific model configuration scheme is also provided, thereby obtaining efficient and high-precision photovoltaic panel image detection effects, which is helpful to realize intelligent monitoring of photovoltaic power stations. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.
[0021] Figure 1 A flowchart of a method for processing a semantic segmentation model of a photovoltaic panel in this application;
[0022] Figure 2 A schematic diagram of a scenario for annotation processing in this application;
[0023] Figure 3A schematic diagram of a scenario for photovoltaic panel feature enhancement processing in this application;
[0024] Figure 4 A schematic diagram of the architecture of the photovoltaic panel semantic segmentation model for this application;
[0025] Figure 5 A schematic diagram of a scenario for the actual application of the application model;
[0026] Figure 6 A schematic diagram of a scenario showing the results of this application;
[0027] Figure 7 A logical schematic diagram of the overall scheme of this application;
[0028] Figure 8 A structural schematic diagram of a processing device for a semantic segmentation model of a photovoltaic panel in this application;
[0029] Fig. 9 A schematic diagram of the structure of the processing equipment of this application. DETAILED DESCRIPTION
[0030] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.
[0031] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described here can be implemented in an order other than that illustrated or described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or modules is not necessarily limited to those steps or modules clearly listed, but may include other steps or modules that are not clearly listed or inherent to these processes, methods, products or devices. The naming or numbering of steps in this application does not mean that the steps in the method flow must be executed in the time / logical sequence indicated by the naming or numbering. The process steps that have been named or numbered can change the execution order according to the technical purpose to be achieved, as long as the same or similar technical effects can be achieved.
[0032] The division of modules in this application is a logical division. There may be other division methods when it is implemented in actual applications. For example, multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection between modules can be electrical or other similar forms, which are not limited in this application. In addition, the modules or submodules described as separate components may or may not be physically separated, may or may not be physical modules, or may be distributed in multiple circuit modules, and some or all of the modules may be selected according to actual needs to achieve the purpose of the present application.
[0033] Before introducing the processing method of the photovoltaic panel semantic segmentation model provided by this application, the background content involved in this application is first introduced.
[0034] The processing method, device and computer-readable storage medium of the photovoltaic panel semantic segmentation model provided in this application can be applied to processing equipment. For the image detection target of photovoltaic panels, on the basis of introducing semantic segmentation to configure the photovoltaic panel semantic segmentation model, a more specific model configuration scheme is also provided, thereby obtaining efficient and high-precision photovoltaic panel image detection effects, which is helpful to realize intelligent monitoring of photovoltaic power stations.
[0035] The processing method of the photovoltaic panel semantic segmentation model mentioned in this application can be executed by a processing device of the photovoltaic panel semantic segmentation model, or a server, a physical host or a user equipment (UE) and other different types of processing devices integrated with the processing device of the photovoltaic panel semantic segmentation model. Among them, the processing device of the photovoltaic panel semantic segmentation model can be implemented in hardware or software, and the UE can be a terminal device such as a smart phone, a tablet computer, a laptop computer, a desktop computer or a personal digital assistant (PDA), and the processing device can be set in a device cluster.
[0036] It can be understood that the data processing involved in the processing method of the photovoltaic panel semantic segmentation model of the present application is mainly the training processing of the model. Therefore, for the processing equipment that executes the processing method of the photovoltaic panel semantic segmentation model provided by the present application, or the processing equipment equipped with the application service corresponding to the processing method of the photovoltaic panel semantic segmentation model provided by the present application, in actual circumstances, it is only necessary to have the required data processing capabilities.
[0037] If specific acquisition operations involving photovoltaic power station images are required, the processing equipment needs to be equipped with corresponding image acquisition capabilities, which can be achieved by adjusting the composition structure of the processing equipment.
[0038] Of course, in actual situations, the present application solution usually only needs to involve existing photovoltaic power station images, and does not need to involve specific collection operations for photovoltaic power station images. The collection of photovoltaic power station images can be handled by the data source.
[0039] Next, we will introduce the processing method of the photovoltaic panel semantic segmentation model provided by this application.
[0040] First, see Figure 1 , Figure 1 A flow chart of a method for processing a semantic segmentation model of a photovoltaic panel in the present application is shown. The method for processing a semantic segmentation model of a photovoltaic panel provided in the present application may specifically include the following steps S101 to S103:
[0041] Step S101, obtaining sample photovoltaic power station images from different data sources;
[0042] It can be understood that, corresponding to the training requirements of the photovoltaic panel semantic segmentation model, it is necessary to configure sample photovoltaic power station images. Among them, for the photovoltaic power station images involved in this application, the image content involves a large number of photovoltaic panels, which are usually obtained by collecting images from above the photovoltaic power station.
[0043] The sample photovoltaic power station images here come from different data sources and correspond to different types of sample photovoltaic power station images. This helps to improve the diversity of samples, thereby improving the training efficiency and detection accuracy of the model, and ultimately helps to improve the generalization of the trained model.
[0044] As an exemplary embodiment here, the sample photovoltaic power station images involved in the present application may specifically include aerial images, satellite remote sensing images and Google images obtained through the QGIS platform, wherein the (ultra-high spatial) resolution of the Google images is 0.3m.
[0045] Among them, the QGIS platform is a free desktop geographic information system (GIS) software that can pull or import relevant Google images. It has the advantages of convenient operation and visualization, and can provide data display, editing and analysis functions.
[0046] In addition, the aerial images and satellite remote sensing images involved are actually suitable for easily obtaining samples of large-scale photovoltaic power stations, while Google images can help to flexibly and actively capture samples at different locations according to needs, and can also capture samples of smaller-scale photovoltaic power stations.
[0047] It can be understood that the embodiments herein provide an easy-to-implement supporting solution for configuring a variety of sample photovoltaic power station images.
[0048] Of course, in practical applications, other types of data sources or data acquisition methods may also be involved, which can be adjusted according to actual needs.
[0049] In addition, the sample PV power station images also take into account different PV panel styles and different installation environment backgrounds to further ensure a high degree of sample diversity.
[0050] As an example, sample PV power station images of different provinces or different geographical regions can be configured. In actual situations, the PV panel style is usually adjusted according to the regional characteristics of different geographical regions. There are also different installation environment backgrounds. In this way, sample diversity can be easily ensured through different geographical regions.
[0051] Step S102, performing photovoltaic panel semantic segmentation processing on the sample photovoltaic power station image to obtain a photovoltaic panel mask, and forming a photovoltaic panel data set consisting of the sample photovoltaic power station image and the photovoltaic panel mask;
[0052] It can be understood that the training samples used for training the model usually need to be added / assigned annotations, and the annotation content corresponds to the prediction results of the model, that is, the photovoltaic panel detection results. In this regard, based on the introduction of semantic segmentation, the present application can construct the semantic segmentation results of the photovoltaic panel by means of a mask. Compared with the conventionally understood rectangular detection box with a horizontal bottom edge, the mask can specifically reflect the image area of the identified photovoltaic panel, corresponding to the shape contour of the identified photovoltaic panel. In other words, the prediction effect to be achieved by the model of the present application involves the specific shape contour of the photovoltaic panel.
[0053] Specifically, refer to Figure 2 A schematic diagram of a scenario of the annotation processing of this application is shown. Figure 2 The a part on the left is the original image. Figure 2 Part b on the right side of the middle is a photovoltaic panel mask that identifies the image area where the photovoltaic panel is located in the original image, thus vividly reflecting the mask setting involved in the present application.
[0054] In addition, in addition to annotating the photovoltaic panel detection results (masks), the annotation link can also configure the corresponding annotation content corresponding to the prediction logic of the model design. For example, considering the subtle impact of different photovoltaic panel styles and different installation environment backgrounds on the model prediction effect in actual situations, the sample photovoltaic panel style and sample installation environment background corresponding to the sample photovoltaic power station image can also be annotated. Furthermore, the geographical area where the photovoltaic power station is located in the image can also be annotated. In this way, in actual situations, the model can be guided to achieve a more delicate prediction effect for the photovoltaic panels in the image input by the model based on these subtle factors, rather than focusing only on the image characteristics of the photovoltaic panels themselves.
[0055] In addition, to ensure the effectiveness of model training, this application can also standardize the samples to unify the image size.
[0056] In this regard, as another exemplary embodiment, before semantic segmentation of photovoltaic panels is performed on the sample photovoltaic power station image, the processing method of the photovoltaic panel semantic segmentation model of the present application may further include:
[0057] The sample photovoltaic power station images were uniformly cropped to a size of 1024*1024 pixels.
[0058] It is easy to see that the samples involved in this application are unified in size of 1024*1024 pixels. Under the setting of this image size, it is beneficial to take into account both the model training effect and the actual use effect of the model in the actual application of the present application solution.
[0059] Of course, in actual situations, they can also be uniformly cut to other specific sizes, which can be adaptively adjusted according to actual needs.
[0060] In addition, the annotation results can also be saved in specific image formats such as PNG, which can also be adaptively adjusted according to actual needs.
[0061] At the same time, after completing the labeling process, you can continue to make corresponding settings for the training samples to further enhance the performance of the training samples.
[0062] At this time, the training sample is a photovoltaic panel data set (sample photovoltaic power station image + photovoltaic panel mask). Specifically, as an exemplary embodiment, the sample photovoltaic power station image is subjected to photovoltaic panel semantic segmentation processing to obtain a photovoltaic panel mask, and a photovoltaic panel data set consisting of the sample photovoltaic power station image and the photovoltaic panel mask is formed. The processing method of the photovoltaic panel semantic segmentation model of the present application may also include:
[0063] Enhance the photovoltaic panel features of the photovoltaic panel dataset in terms of images and generate pseudo labels to increase the number of samples.
[0064] Among them, it can be understood that enhancing the characteristics of photovoltaic panels at the image level can also be understood by enhancing the characteristics of the images of photovoltaic panels. The specific image enhancement operation means involved can be adjusted under actual circumstances, such as adjusting parameters such as sharpness.
[0065] For the photovoltaic panel feature enhancement processing involved here, please refer to Figure 3 A schematic diagram of a scenario of photovoltaic panel feature enhancement processing in this application is shown, Figure 3 Part a on the left side of the image is the photovoltaic panel. Figure 3 Part b on the middle right is the photovoltaic panel mask with enhanced photovoltaic panel features.
[0066] In addition, at the level of sample quantity, the number of samples can also be further increased by generating pseudo labels. The application of pseudo labels, in layman's terms, is a process of using a model trained on labeled data to make predictions on unlabeled data, screening samples based on the prediction results, and then inputting them into the model for training again.
[0067] It can be seen that although the setting of the embodiment here is aimed at the configuration work of training samples, it extends to the subsequent model training process. Therefore, it can also be understood as the solution content that can be involved in the details of the model training work involved in the subsequent step S103.
[0068] Step S103, training a photovoltaic panel semantic segmentation model through a photovoltaic panel data set, wherein the photovoltaic panel semantic segmentation model is used to predict the mask of the photovoltaic panel in the photovoltaic power station image input by the model, and the model architecture of the photovoltaic panel semantic segmentation model is composed of a MobileNetV2 network and a Deeplab V3+ network. The MobileNetV2 network generates an initial feature map of the photovoltaic power station image input by the model, and then inputs the initial feature map into the ASPP structure adopted in the encoder part of the Deeplab V3+ network. The high-level feature map is output through the network structure with different expansion rates and superimposed, and then upsampling and convolution operations are performed to obtain the final output feature map, and finally the photovoltaic panel prediction is performed.
[0069] Among them, ASPP, namely Atrous Spatial Pyramid Pooling.
[0070] It can be understood that, based on the introduction of semantic segmentation processing for photovoltaic panels, the present application also configures a specific model implementation architecture for the semantic segmentation model of photovoltaic panels.
[0071] Specifically, the photovoltaic panel semantic segmentation model of this application uses a semantic segmentation algorithm composed of a backbone feature extraction network MobileNetV2 network and an improved Deeplab V3+ network to design the model, including:
[0072] The MobileNetV2 network is mainly composed of an inverted residual structure and an Inverted Residuals structure;
[0073] The Deeplab V3+ network uses the ASPP structure in the encoder part, which can effectively capture feature maps of different sizes and enhance the robustness of the model. Secondly, the Deeplab V3+ network uses dilated convolution to increase the depth and receptive field of the network, so that it can better extract contextual information. Finally, the Deeplab V3+ network uses multi-scale prediction technology in the decoder part, which can improve the prediction accuracy of the model at different scales and realize multi-scale semantic segmentation.
[0074] For the MobileNetV2 network, the following parameters can also be included:
[0075] Table 1-MobileNetV2 network parameter examples
[0076] Input Operator t c n s 2242×3 conv2d - 32 1 2 1122×32 bottleneck 1 16 1 1 1122×16 bottleneck 6 24 2 2 42×24 bottleneck 6 32 3 2 282×32 bottleneck 6 64 4 2 142×64 bottleneck 6 96 3 1 142×96 bottleneck 6 160 3 2 72×160 bottleneck 6 320 1 1 72×320 conv2d 1×1 - 1280 1 1 72×1280 avgpool 7×7 - - 1 - 1×1×1280 conv2d 1×1 - k -
[0077] Among them, t represents the dimensionality increase ratio of the 1×1 convolution in the inverted residual structure block, c is the channel information of the output feature matrix, n represents the number of times the bottleneck structure is repeated, and s represents the step size, but it only represents the step size of the hollow convolution in the first bottleneck structure, and the subsequent repeated step sizes are all equal to 1.
[0078] Based on the above model structure, the general working logic of the photovoltaic panel semantic segmentation model in this application is as follows:
[0079] The MobileNetV2 network generates / extracts the initial feature map of the photovoltaic power station image input to the model, and then inputs the initial feature map into the ASPP structure used in the encoder part of the Deeplab V3+ network (at this time, the model's work is transferred from the MobileNetV2 network to the Deeplab V3+ network). The high-level feature map is output and superimposed through the network structure with different expansion rates, and then upsampling and convolution operations are performed to obtain the final output feature map. Finally, the photovoltaic panel is predicted, and the mask corresponding to the predicted photovoltaic panel image area is output.
[0080] Furthermore, the ASPP structure and the upsampling structure may also be added with an attention mechanism implemented by a convolutional block attention module CBAM. The attention involved in the attention mechanism configured in the present application may specifically include a channel attention module M c (F) and spatial attention M s (F) and can be expressed as:
[0081] M c (F)=σ(MLP(AvgPool(F))+MLP(MaxPool(F)),
[0082] M s (F) = σ(f 7×7 ([AvgPool(F);MaxPool(F)])),
[0083] Among them, F represents the input feature map, σ represents the Sigmoid activation function, MLP represents the multi-layer perceptron, AvgPool represents the global average pooling operation, MaxPool represents the maximum pooling operation, and f 7×7 Represents a 7×7 convolution operation, and [AvgPool(F); MaxPool(F)] means concatenating the global average pooling operation result and the maximum pooling operation result along the channel axis.
[0084] It can be understood that a specific implementation scheme is provided here for the attention mechanism involved in this application.
[0085] On the other hand, the present application also specifically designs the photovoltaic panel semantic segmentation model as a three-layer structure, which can be specifically a feature extraction and reorganization layer, a nonlinear mapping layer, and a semantic segmentation and reorganization layer, thus forming a three-layer model structure, specifically, there are:
[0086] The feature extraction and reorganization layer uses a single-layer convolution structure with a size of f 1 ×f 1 The convolution kernel W 1 , set the bias vector B 1 , used to extract features from the input image and reorganize the features in the input image into a feature matrix for nonlinear mapping;
[0087] The nonlinear mapping layer uses a size of f 2 ×f 2 The convolution kernel W 2 , set the bias vector B 2 , used to remap the feature matrix output by the feature extraction and reconstruction layer into the required features of the semantic segmentation and reconstruction layer;
[0088] The semantic segmentation reorganization layer uses a size of f 3 ×f3 The convolution kernel W 3 , set the bias vector B 3 , used to convert the mapped required features into a semantic segmentation prediction matrix, and use the softmax activation function to convert the semantic segmentation prediction matrix into a semantic segmentation prediction mask;
[0089] The corresponding mathematical definitions are:
[0090] F 1 (Y)=max(0,W 1 ×Y+B 1 ),
[0091] F 2 (Y)=max(0,W 2 ×F 1 (Y)+B 2 ),
[0092] F 3 (Y)=softmax(0,W 3 ×F 2 (Y)+B 3 ),
[0093] Among them, F 1 (Y) represents the feature extraction and reorganization layer, F 2 (Y) represents the nonlinear mapping layer, F 3 (Y) represents the semantic segmentation reorganization layer.
[0094] Correspondingly, the three-layer structure setting here can also refer to Figure 4 A schematic diagram of the architecture of the photovoltaic panel semantic segmentation model of the present application is shown for a more vivid understanding.
[0095] In addition, the model training process includes:
[0096] In each round of model training, the training samples (sample PV power station images) are input into the model so that the model predicts the corresponding PV panel mask to achieve forward propagation. Then, based on the predicted PV panel mask, the loss function is calculated in combination with the annotation results of the training sample configuration, i.e., the PV panel mask. The model parameters are optimized in reverse based on the loss function calculation results to achieve back propagation. When the model training conditions such as training time, training times, and prediction accuracy are met, the model training is completed and the model can be put into practical use.
[0097] Among them, the loss function adopted can adopt any type of loss function, which can be flexibly adjusted according to actual conditions.
[0098] As an example, the loss function used in this application can specifically adopt a pixel-by-pixel cross entropy loss function L, and the corresponding mathematical definition is as follows:
[0099]
[0100] Where i = 0, 1, 2, ... N, i represents the pixel number, c = 0, 1, 2, ... M, c represents the semantic segmentation category number, y (i,c) Represents the unique hot encoding value of the labeled data of the pixel numbered i. If the true category of the pixel i is equal to c, then y (i,c) =1, otherwise, y (i,c) =0, p (i,c) Represents the predicted probability of the pixel numbered i.
[0101] During the training process, the expected threshold T of the loss function can be set. If L < T, it means that the model can correctly divide the area where the photovoltaic panels in the image input to the model are located, that is, the semantic segmentation model meets the requirements; if L > T, it means that the area where the photovoltaic panels in the image input to the model cannot be correctly divided, that is, the semantic segmentation model does not meet the requirements, and training continues until L < T.
[0102] After completing the model training and obtaining a photovoltaic panel semantic segmentation model that can be put into practical use, it is obvious that photovoltaic panel prediction processing can be carried out under actual conditions. To this end, this application also provides a specific application solution for the photovoltaic panel semantic segmentation model.
[0103] Specifically, the processing method of the photovoltaic panel semantic segmentation model of the present application may also include:
[0104] The photovoltaic panel semantic segmentation model is used to predict the photovoltaic panel mask corresponding to the current photovoltaic power station image;
[0105] The photovoltaic panel mask corresponding to the current photovoltaic power station image is gray-processed, and then the obtained gray-scale image is binarized. Then, based on the obtained grid image, the grid information of the photovoltaic panel that does not conform to the actual area is removed through the area algorithm to form the overall grid information of the photovoltaic panel;
[0106] The overall raster information of the photovoltaic panel is converted into vector information, and the vector area is calculated. Both are imported into the attribute table and output as the photovoltaic panel attribute features of the current photovoltaic power station image.
[0107] The area algorithm involved may be pre-set or set in real time. In addition, the area algorithm itself may be an existing algorithm, such as extracted from an opencv library. Of course, the use of improved or even self-developed algorithms is not excluded.
[0108] It is easy to see that the area algorithm is used to calculate the area of the grid graphics (the graphics of the detected, unit / single photovoltaic panel), so as to eliminate the grid information of some photovoltaic panels that do not conform to the actual situation (part of the photovoltaic panels detected), so as to effectively optimize the prediction results of the model. Figure 5 A schematic diagram of a scenario of actual application of the model of the present application is shown to help more vividly understand the optimization processing of the model prediction results here.
[0109] After the optimization of the model prediction results is completed, a photovoltaic panel attribute that can be used for output can be processed through vector conversion, and the corresponding vector area can also be calculated as another photovoltaic panel attribute that can be used for output, completing a process that meets the current photovoltaic panel identification needs. Figure 6 A schematic diagram of a scenario showing the results of the present application is shown to help more vividly understand the display processing of photovoltaic panel properties here.
[0110] In addition, in addition to directly displaying one / two photovoltaic panel properties mentioned above, the corresponding analysis results can also be displayed. The so-called analysis can be understood as starting from an overall perspective, corresponding to multiple or even all photovoltaic panels detected. The analysis strategies involved in this process can be adjusted according to actual needs. For example, the distribution of photovoltaic panels in the current photovoltaic power station can be analyzed, the impact of the current photovoltaic power station on the ecological environment can be evaluated, or the actual power generation of the current photovoltaic power station can be predicted.
[0111] It can be understood that a specific implementation scheme is provided here for the practical application of the photovoltaic panel semantic segmentation model of the present application, and a better photovoltaic panel image detection effect can be obtained, further improving the practical value of the present application scheme.
[0112] For the above solutions, you can also combine Figure 7 A logical schematic diagram of the overall solution of the present application is shown for a more complete understanding.
[0113] In general, for the image detection target of photovoltaic panels, after obtaining the photovoltaic power station images from different data sources, this application performs semantic segmentation processing on the photovoltaic panels of the photovoltaic power station images to obtain photovoltaic panel masks, and forms a photovoltaic panel data set consisting of photovoltaic power station images and photovoltaic panel masks. Then, the photovoltaic panel semantic segmentation model is trained through the photovoltaic panel data set. The model architecture of the photovoltaic panel semantic segmentation model consists of a MobileNetV2 network and a DeeplabV3+ network. The MobileNetV2 network generates the initial feature map of the photovoltaic power station image input by the model, and then inputs the initial feature map into the ASPP structure used in the encoder part of the Deeplab V3+ network. The high-level feature map is output through the network structure with different expansion rates and superimposed, and then upsampling and convolution operations are performed to obtain the final output feature map, and finally photovoltaic panel prediction is performed. In this way, on the basis of introducing semantic segmentation to configure the photovoltaic panel semantic segmentation model, a more specific model configuration scheme is also provided, thereby obtaining efficient and high-precision photovoltaic panel image detection effects, which is helpful to realize intelligent monitoring of photovoltaic power stations.
[0114] The above is an introduction to the processing method of the photovoltaic panel semantic segmentation model provided by this application. In order to better implement the processing method of the photovoltaic panel semantic segmentation model provided by this application, this application also provides a processing device for the photovoltaic panel semantic segmentation model from the perspective of functional modules.
[0115] See also Figure 8 , Figure 8 Schematic diagram of a structure of a processing device for a semantic segmentation model of a photovoltaic panel in the present application. In the present application, the processing device 800 for a semantic segmentation model of a photovoltaic panel may specifically include the following structure:
[0116] An acquisition unit 801 is used to acquire sample photovoltaic power station images from different data sources;
[0117] The segmentation unit 802 is used to perform semantic segmentation processing on the photovoltaic panels of the sample photovoltaic power station image to obtain a photovoltaic panel mask and form a photovoltaic panel data set consisting of the sample photovoltaic power station image and the photovoltaic panel mask;
[0118] The training unit 803 is used to train a photovoltaic panel semantic segmentation model using a photovoltaic panel data set, wherein the photovoltaic panel semantic segmentation model is used to predict the mask of the photovoltaic panel in the photovoltaic power station image input by the model, and the model architecture of the photovoltaic panel semantic segmentation model consists of a MobileNetV2 network and a Deeplab V3+ network. The MobileNetV2 network generates an initial feature map of the photovoltaic power station image input by the model, and then inputs the initial feature map into the ASPP structure used in the encoder part of the Deeplab V3+ network. The high-level feature map is output through the network structure with different expansion rates and superimposed, and then upsampling and convolution operations are performed to obtain the final output feature map, and finally the photovoltaic panel prediction is performed.
[0119] In an exemplary embodiment, the sample photovoltaic power station images specifically include aerial images, satellite remote sensing images, and Google images obtained through the QGIS platform, wherein the resolution of the Google images is 0.3 m;
[0120] The sample PV power station images take into account different PV panel styles and different installation environment backgrounds.
[0121] In yet another exemplary embodiment, the apparatus further includes a cutting unit 804, configured to:
[0122] The sample photovoltaic power station images were uniformly cropped to a size of 1024*1024 pixels.
[0123] In yet another exemplary embodiment, the apparatus further includes an enhancement unit 805, configured to:
[0124] Enhance the photovoltaic panel features of the photovoltaic panel dataset in terms of images and generate pseudo labels to increase the number of samples.
[0125] In another exemplary embodiment, the ASPP structure and the upsampling structure are added with an attention mechanism implemented by a convolutional block attention module CBAM, and the attention involved in the attention mechanism specifically includes a channel attention M c (F) and spatial attention M s (F), and expressed as:
[0126] M c (F)=σ(MLP(AvgPool(F))+MLP(MaxPool(F)),
[0127] M s (F) = σ(f 7×7 ([AvgPool(F);MaxPool(F)])),
[0128] Among them, F represents the input feature map, σ represents the Sigmoid activation function, MLP represents the multi-layer perceptron, AvgPool represents the global average pooling operation, MaxPool represents the maximum pooling operation, and f 7×7 Represents a 7×7 convolution operation, and [AvgPool(F); MaxPool(F)] means concatenating the global average pooling operation result and the maximum pooling operation result along the channel axis.
[0129] In another exemplary embodiment, the photovoltaic panel semantic segmentation model is designed as a three-layer structure, which includes a feature extraction and reorganization layer, a nonlinear mapping layer, and a semantic segmentation and reorganization layer;
[0130] The feature extraction and reorganization layer uses a single-layer convolution structure with a size of f 1 ×f 1 The convolution kernel W 1 , set the bias vector B 1 , used to extract features from the input image and reorganize the features in the input image into a feature matrix for nonlinear mapping;
[0131] The nonlinear mapping layer uses a size of f 2 ×f 2 The convolution kernel W 2 , set the bias vector B 2 , used to remap the feature matrix output by the feature extraction and reconstruction layer into the required features of the semantic segmentation and reconstruction layer;
[0132] The semantic segmentation reorganization layer uses a size of f 3 ×f 3 The convolution kernel W 3 , set the bias vector B 3 , used to convert the mapped required features into a semantic segmentation prediction matrix, and use the softmax activation function to convert the semantic segmentation prediction matrix into a semantic segmentation prediction mask;
[0133] The corresponding mathematical definitions are:
[0134] F 1 (Y)=max(0,W 1 ×Y+B 1 ),
[0135] F 2 (Y)=max(0,W 2 ×F 1 (Y)+B 2 ),
[0136] F 3 (Y)=softmax(0,W 3 ×F 2(Y)+B 3 ),
[0137] Among them, F 1 (Y) represents the feature extraction and reorganization layer, F 2 (Y) represents the nonlinear mapping layer, F 3 (Y) represents the semantic segmentation reorganization layer.
[0138] In yet another exemplary embodiment, the apparatus further includes an application unit 806, configured to:
[0139] The photovoltaic panel semantic segmentation model is used to predict the photovoltaic panel mask corresponding to the current photovoltaic power station image;
[0140] The photovoltaic panel mask corresponding to the current photovoltaic power station image is gray-processed, and then the obtained gray-scale image is binarized. Then, based on the obtained grid image, the grid information of the photovoltaic panel that does not conform to the actual area is removed through the area algorithm to form the overall grid information of the photovoltaic panel;
[0141] The overall raster information of the photovoltaic panel is converted into vector information, and the vector area is calculated. Both are imported into the attribute table and output as the photovoltaic panel attribute features of the current photovoltaic power station image.
[0142] This application also provides a processing device from the perspective of hardware structure, see Fig. 9 , Fig. 9 901, a computer program stored in the memory 902, and an input / output device 903. The processor 901 is used to execute the computer program stored in the memory 902. Figure 1 The steps of the processing method of the photovoltaic panel semantic segmentation model in the corresponding embodiment; or, the processor 901 is used to execute the computer program stored in the memory 902 to implement the following Figure 8 Corresponding to the functions of each unit in the embodiment, the memory 902 is used to store the processor 901 executing the above Figure 1 The computer program required for the processing method of the photovoltaic panel semantic segmentation model in the corresponding embodiment.
[0143] Exemplarily, the computer program may be divided into one or more modules / units, one or more modules / units are stored in the memory 902, and executed by the processor 901 to complete the present application. One or more modules / units may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program in the computer device.
[0144] The processing device may include, but is not limited to, a processor 901, a memory 902, and an input / output device 903. Those skilled in the art will appreciate that the illustration is merely an example of a processing device and does not constitute a limitation on the processing device, and may include more or fewer components than shown in the illustration, or a combination of certain components, or different components. For example, the processing device may also include a network access device, a bus, etc., and the processor 901, the memory 902, the input / output device 903, etc. are connected via a bus.
[0145] The processor 901 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the processing device, and uses various interfaces and lines to connect various parts of the entire device.
[0146] The memory 902 can be used to store computer programs and / or modules. The processor 901 implements various functions of the computer device by running or executing the computer programs and / or modules stored in the memory 902 and calling the data stored in the memory 902. The memory 902 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function, etc.; the data storage area can store data created according to the use of the processing device, etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.
[0147] When the processor 901 is used to execute the computer program stored in the memory 902, the following functions can be implemented:
[0148] Obtain sample PV power plant images from different data sources;
[0149] Perform semantic segmentation of photovoltaic panels on sample photovoltaic power station images to obtain photovoltaic panel masks, and form a photovoltaic panel dataset consisting of sample photovoltaic power station images and photovoltaic panel masks;
[0150] The photovoltaic panel semantic segmentation model is trained through the photovoltaic panel dataset. The photovoltaic panel semantic segmentation model is used to predict the mask of the photovoltaic panel in the photovoltaic power station image input by the model. The model architecture of the photovoltaic panel semantic segmentation model consists of the MobileNetV2 network and the Deeplab V3+ network. The MobileNetV2 network generates the initial feature map of the photovoltaic power station image input by the model, and then inputs the initial feature map into the ASPP structure used in the encoder part of the Deeplab V3+ network. The high-level feature map is output through the network structure with different expansion rates and superimposed, and then upsampling and convolution operations are performed to obtain the final output feature map, and finally the photovoltaic panel prediction is performed.
[0151] Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the processing device, processing equipment and corresponding units of the photovoltaic panel semantic segmentation model described above can refer to the following. Figure 1 The description of the processing method of the photovoltaic panel semantic segmentation model in the corresponding embodiment will not be repeated here.
[0152] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.
[0153] To this end, the present application provides a computer-readable storage medium, in which a plurality of instructions are stored, and the instructions can be loaded by a processor to execute the present application as follows: Figure 1 The steps of the processing method of the photovoltaic panel semantic segmentation model in the corresponding embodiment, the specific operation can refer to the following Figure 1 The description of the processing method of the photovoltaic panel semantic segmentation model in the corresponding embodiment will not be repeated here.
[0154] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.
[0155] Due to the instructions stored in the computer-readable storage medium, the present application can be executed. Figure 1 The steps of the processing method of the photovoltaic panel semantic segmentation model in the corresponding embodiment, therefore, the present application can be implemented as follows Figure 1The beneficial effects that can be achieved by the processing method of the photovoltaic panel semantic segmentation model in the corresponding embodiment are detailed in the previous description and will not be repeated here.
[0156] The above is a detailed introduction to the processing method, device, processing equipment and computer-readable storage medium of the photovoltaic panel semantic segmentation model provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the idea of the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.
Claims
1. A processing method for the semantic segmentation model of photovoltaic panels, It is characterized in that The method comprises: Obtain sample PV power plant images from different data sources; Performing photovoltaic panel semantic segmentation processing on the sample photovoltaic power station image to obtain a photovoltaic panel mask, and forming a photovoltaic panel data set consisting of the sample photovoltaic power station image and the photovoltaic panel mask; The photovoltaic panel semantic segmentation model is trained by the photovoltaic panel data set, wherein the photovoltaic panel semantic segmentation model is used to predict the mask of the photovoltaic panel in the photovoltaic power station image input by the model, and the model architecture of the photovoltaic panel semantic segmentation model consists of a MobileNetV2 network and a Deeplab V3+ network. The MobileNetV2 network generates an initial feature map of the photovoltaic power station image input by the model, and then inputs the initial feature map into the ASPP structure adopted in the encoder part of the Deeplab V3+ network. The high-level feature map is output and superimposed through the network structure with different expansion rates, and then upsampling and convolution operations are performed to obtain the final output feature map, and finally the photovoltaic panel prediction is performed.
2. The method according to claim 1, It is characterized in that The sample photovoltaic power station images specifically include aerial images, satellite remote sensing images and Google images obtained through the QGIS platform, wherein the resolution of the Google images is 0.3m; The sample photovoltaic power station images take into account different photovoltaic panel styles and different installation environment backgrounds.
3. The method according to claim 1, It is characterized in that Before performing semantic segmentation processing on photovoltaic panels of the sample photovoltaic power station image, the method further includes: The sample photovoltaic power station images are uniformly cropped to a size of 1024*1024 pixels.
4. The method according to claim 1, It is characterized in that After performing semantic segmentation processing on photovoltaic panels of the sample photovoltaic power station image, the method further includes: The photovoltaic panel features of the photovoltaic panel dataset are enhanced in terms of images, and pseudo labels are generated to increase the number of samples.
5. The method according to claim 1, It is characterized in that The ASPP structure and the upsampling structure are added with an attention mechanism implemented by a convolutional block attention module CBAM. The attention involved in the attention mechanism specifically includes channel attention M c (F) and spatial attention M s (F), and expressed as: M c (F)=σ(MLP(AvgPool(F))+MLP(MaxPool(F)), M s (F)=σ(f 7×7 ([AvgPool(F);MaxPool(F)])), Among them, F represents the input feature map, σ represents the Sigmoid activation function, MLP represents the multi-layer perceptron, AvgPool represents the global average pooling operation, MaxPool represents the maximum pooling operation, and f 7×7 Represents a 7×7 convolution operation, and [AvgPool(F); MaxPool(F)] means concatenating the global average pooling operation result and the maximum pooling operation result along the channel axis.
6. The method according to claim 1, It is characterized in that The photovoltaic panel semantic segmentation model is designed as a three-layer structure, which includes a feature extraction and reorganization layer, a nonlinear mapping layer, and a semantic segmentation and reorganization layer. The feature extraction and reorganization layer uses a single-layer convolution structure with a size of f 1 ×f 1 The convolution kernel W 1 , set the bias vector B 1 , used to extract features in an input image and reorganize the features in the input image into a feature matrix for nonlinear mapping; The nonlinear mapping layer uses a size of f 2 ×f 2 The convolution kernel W 2 , set the bias vector B 2 , used to remap the feature matrix output by the feature extraction and reorganization layer into the required features of the semantic segmentation and reorganization layer; The semantic segmentation reorganization layer uses a size of f 3 ×f 3 The convolution kernel W 3 , set the bias vector B 3 , used to convert the mapped required features into a semantic segmentation prediction matrix, and use a softmax activation function to convert the semantic segmentation prediction matrix into a semantic segmentation prediction mask; The corresponding mathematical definitions are: F 1 (Y)=max(0,W 1 ×Y+B 1 ), F 2 (Y)=max(0,W 2 ×F 1 (Y)+B 2 ), F 3 (Y)=softmax(0,W 3 ×F 2 (Y)+B 3 ), Among them, F 1 (Y) represents the feature extraction and reorganization layer, F 2 (Y) represents the nonlinear mapping layer, F 3 (Y) represents the semantic segmentation reorganization layer.
7. The method according to claim 1, It is characterized in that The method further comprises: Predicting the photovoltaic panel mask corresponding to the current photovoltaic power station image by using the photovoltaic panel semantic segmentation model; Grayscale processing is performed on the photovoltaic panel mask corresponding to the current photovoltaic power station image, and then the obtained grayscale image is binarized, and then, based on the obtained grid image, the grid information of the photovoltaic panel that does not conform to the actual area is eliminated through an area algorithm to form the overall grid information of the photovoltaic panel; The overall raster information of the photovoltaic panel is converted into vector information, and the vector area is calculated. The two are imported into an attribute table and output as attribute features of the photovoltaic panel of the current photovoltaic power station image.
8. A processing device for a semantic segmentation model of photovoltaic panels, It is characterized in that The device comprises: An acquisition unit, used for acquiring sample photovoltaic power station images from different data sources; A segmentation unit, configured to perform semantic segmentation processing on photovoltaic panels of the sample photovoltaic power station image to obtain a photovoltaic panel mask, and form a photovoltaic panel data set consisting of the sample photovoltaic power station image and the photovoltaic panel mask; A training unit is used to train a photovoltaic panel semantic segmentation model through the photovoltaic panel data set, wherein the photovoltaic panel semantic segmentation model is used to predict the mask of the photovoltaic panel in the photovoltaic power station image input by the model, and the model architecture of the photovoltaic panel semantic segmentation model is composed of a MobileNetV2 network and a Deeplab V3+ network. The MobileNetV2 network generates an initial feature map of the photovoltaic power station image input by the model, and then inputs the initial feature map into the ASPP structure adopted in the encoder part of the Deeplab V3+ network. The high-level feature map is output through the network structure with different expansion rates and superimposed, and then upsampling and convolution operations are performed to obtain the final output feature map, and finally photovoltaic panel prediction is performed.
9. A processing device, It is characterized in that The method comprises a processor and a memory, wherein the memory stores a computer program, and the processor executes the method according to any one of claims 1 to 7 when calling the computer program in the memory.
10. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute the method according to any one of claims 1 to 7.
Citation Information
Patent Citations
Solar photovoltaic panel identification method and device, electronic equipment and storage medium
CN112348030A
Unmanned aerial vehicle visual angle photovoltaic module visual detection method and device and processing equipment
CN114463308A
Photovoltaic panel hot spot detection and area proportion calculation method
CN116433591A
Semantic segmentation method and apparatus based on multispectral image, and storage medium
WO2023221328A1
