Photovoltaic region segmentation method and device, storage medium and computer equipment
The photovoltaic region segmentation method based on multi-layer feature extraction and feature fusion solves the problem of low accuracy in photovoltaic region segmentation caused by environmental factors in remote sensing technology, and achieves higher accuracy and generalization ability in photovoltaic region segmentation.
Patent Information
- Application Number
- CN202411243575.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-05
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-09-05
AI Technical Summary
In existing technologies, images acquired by remote sensing technology are greatly affected by factors such as season, lighting conditions, and geographical location, resulting in poor generalization ability of photovoltaic region segmentation models, which in turn affects the accuracy of photovoltaic region segmentation.
A multi-layer feature extraction method is adopted, including a first feature extraction layer, a second feature extraction layer, and a point rendering layer. A separation attention module, an average pooling module, a multi-scale feature extraction module, and a dual attention module are used for deep feature extraction and photovoltaic region segmentation. Feature fusion is performed by combining multi-scale feature extraction and dual attention modules, and the channel and spatial dimensions are dynamically adjusted to improve the accuracy of photovoltaic region segmentation.
By employing multi-layer feature extraction and feature fusion, the generalization ability and prediction performance of the photovoltaic region segmentation model are improved, the feature capture capability of the photovoltaic region is enhanced, the impact of complex backgrounds and object shapes on segmentation accuracy is reduced, and the accuracy of photovoltaic region edge detection is improved.
Smart Images

Figure CN119027433A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of photovoltaic region identification, and in particular to a photovoltaic region segmentation method, device, storage medium and computer equipment. Background Art
[0002] Photovoltaic areas refer to areas on the ground where photovoltaic power generation equipment is installed, which can convert solar energy into electrical energy. With the increasing global demand for renewable energy, photovoltaic power generation has been widely promoted as a major way to utilize solar energy. Therefore, the precise extraction of photovoltaic areas can help relevant institutions understand the distribution of photovoltaic resources, so as to make more scientific decisions when planning photovoltaic power generation projects and further promote the development of renewable energy.
[0003] In recent years, remote sensing technology has developed rapidly and has become one of the means to obtain basic image data for photovoltaic area segmentation. At present, photovoltaic area segmentation can be completed by training the corresponding task model through deep learning, but because the images obtained by remote sensing technology are greatly affected by factors such as season, lighting conditions and geographical location, the generalization ability of the model is poor, which leads to low accuracy of photovoltaic area segmentation. Summary of the invention
[0004] The purpose of this application is to solve at least one of the above-mentioned technical defects, especially the technical defect in the prior art that the images obtained by remote sensing technology are greatly affected by factors such as season, lighting conditions and geographical location, which makes the generalization ability of the model poor, thereby leading to low accuracy of photovoltaic area segmentation.
[0005] In a first aspect, the present application provides a photovoltaic region segmentation method, the method comprising:
[0006] Identify satellite images;
[0007] Obtaining a pre-trained region segmentation model; the region segmentation model includes a first feature extraction layer, a second feature extraction layer and a point rendering layer;
[0008] Inputting the satellite image into the first feature extraction layer to perform preliminary feature extraction;
[0009] Inputting the feature map output by the first feature extraction layer into the second feature extraction layer, and performing deep feature extraction by the multi-scale feature extraction module and the dual attention module of the second feature extraction layer to obtain a regional feature map corresponding to the satellite image;
[0010] The region feature map is input into the point rendering layer to obtain a photovoltaic region division image.
[0011] In one of the embodiments, the first feature extraction layer includes a separation attention module and an average pooling module;
[0012] The step of inputting the satellite image into the first feature extraction layer comprises:
[0013] Inputting the satellite image into the separation attention module to obtain a first feature map, and adjusting the spatial dimension of the first feature map using the separation attention module;
[0014] The first feature map after spatial dimension adjustment is input into the average pooling module to reduce the spatial dimension of the first feature map after spatial dimension adjustment to obtain a second feature map, and the second feature map is used as the feature map output by the first feature extraction layer.
[0015] In one embodiment, the feature map output by the first feature extraction layer is input into the second feature extraction layer, and the multi-scale feature extraction module and the dual attention module of the second feature extraction layer perform deep feature extraction, including:
[0016] Inputting the feature map output by the first feature extraction layer into the multi-scale feature extraction module to extract a multi-scale feature map based on the feature map;
[0017] Inputting the multi-scale features into the dual attention module to determine the channel weight and spatial weight of each position in the multi-scale feature map;
[0018] Based on the channel weight and the spatial weight of each position in the multi-scale feature map, weighted processing is performed on the multi-scale feature map to obtain a multi-scale weighted feature map;
[0019] Feature fusion is performed on the multi-scale weighted feature map to obtain a regional feature map.
[0020] In one embodiment, the feature map output by the first feature extraction layer is input into the second feature extraction layer, and the multi-scale feature extraction module and the dual attention module of the second feature extraction layer perform deep feature extraction, including:
[0021] Inputting the feature map output by the first feature extraction layer into the multi-scale feature extraction module and the dual attention module respectively, so as to respectively determine a third feature map output by the multi-scale extraction module and a fourth feature map output by the dual attention module;
[0022] Feature fusion is performed on the third feature map and the fourth feature map to obtain a regional feature map.
[0023] In one embodiment, the step of inputting the region feature map into the point rendering layer to obtain a photovoltaic region division image includes:
[0024] Determining a plurality of key points in the region feature map;
[0025] For each key point, locally refine the key point to obtain the refined features of the key point;
[0026] When the refined features of each key point are determined, the refined features of each key point are rendered into the regional feature map to obtain a photovoltaic regional feature map;
[0027] The satellite image is annotated according to the photovoltaic region characteristic map to obtain a photovoltaic region division image.
[0028] In one embodiment, the training process of the region segmentation model includes:
[0029] Obtain a training sample set and its corresponding labeled sample set;
[0030] Determine an initial model; the initial model includes a first feature extraction layer, a second feature extraction layer and a point rendering layer to be trained;
[0031] Iteratively training the initial model according to the training sample set until a preset training stop condition is met, and determining the trained initial model as the region segmentation model;
[0032] Among them, in each iteration of iterative training, the predicted photovoltaic area of each training sample is determined according to the output results of the training sample set in this iteration, and the loss function of this iteration is determined according to the predicted photovoltaic area of each training sample and the labeled photovoltaic area of each labeled sample in the labeled sample set to adjust the parameters of the initial model.
[0033] In one embodiment, the loss function is determined according to the following expression:
[0034]
[0035] In the formula, represents the loss function, Indicates the height of the image. represents the weight of the image, Represents pixels The predicted classification of Represents pixels The annotation classification of Indicates the smoothing factor to prevent letters from being zero.
[0036] In a second aspect, the present application provides a photovoltaic region segmentation device, the device comprising:
[0037] An image acquisition module, used to determine satellite images;
[0038] A model determination module, used to obtain a pre-trained region segmentation model; the region segmentation model includes a first feature extraction layer, a second feature extraction layer and a point rendering layer;
[0039] A preliminary feature extraction module, used for inputting the satellite image into the first feature extraction layer to perform preliminary feature extraction;
[0040] A deep feature extraction module, used for inputting the feature map output by the first feature extraction layer into the second feature extraction layer, and performing deep feature extraction by the multi-scale feature extraction module and the dual attention module of the second feature extraction layer to obtain a regional feature map corresponding to the satellite image;
[0041] The region division module is used to input the region feature map into the point rendering layer to obtain a photovoltaic region division image.
[0042] In a third aspect, the present application provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the photovoltaic area segmentation method described in any of the above embodiments.
[0043] In a fourth aspect, the present application provides a computer device, comprising: one or more processors, and a memory;
[0044] The memory stores computer-readable instructions, and when the one or more processors execute the computer-readable instructions, the steps of the photovoltaic area segmentation method described in any one of the above embodiments are performed.
[0045] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:
[0046] The photovoltaic region segmentation method, device, storage medium and computer equipment provided by the present application include: determining a satellite image, obtaining a pre-trained region segmentation model, and then inputting the satellite image into the first feature extraction layer of the region segmentation model to perform preliminary feature extraction, and then inputting the feature map output by the first feature extraction layer into the second feature extraction layer of the region segmentation model, and performing deep feature extraction by the multi-scale feature extraction module and the dual attention module of the second feature extraction layer to obtain a regional feature map, and finally inputting the regional feature map into the point rendering layer of the regional segmentation model to obtain a photovoltaic region segmentation image. Through multi-layer feature extraction, a richer data representation of the photovoltaic region can be obtained, and more feature information of different dimensions can be captured to improve the generalization ability and prediction performance of the photovoltaic region segmentation model, and the multi-scale feature extraction module and the dual attention module are combined for further feature extraction, which can adaptively fuse local features with global context information and improve the accuracy of photovoltaic region segmentation. The dual attention module can also dynamically adjust the channel dimension and spatial dimension to adapt to images of different scales and complexities, thereby reducing the influence of easily confused backgrounds and complex object morphologies on the accuracy of photovoltaic region segmentation. BRIEF DESCRIPTION OF THE DRAWINGS
[0047] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0048] Figure 1 A schematic diagram of a flow chart of a photovoltaic region segmentation method provided in an embodiment of the present application;
[0049] Figure 2 A schematic flow chart of the steps of inputting a satellite image into a first feature extraction layer for preliminary feature extraction provided in an embodiment of the present application;
[0050] Figure 3 A schematic flow chart of the step of inputting a feature map output by a first feature extraction layer into a second feature extraction layer for deep feature extraction provided in an embodiment of the present application;
[0051] Figure 4 A schematic flow chart of the steps of inputting a regional feature map into a point rendering layer to obtain a photovoltaic region division image provided in an embodiment of the present application;
[0052] Figure 5 A schematic diagram of constructing a region segmentation model provided in an embodiment of the present application;
[0053] Figure 6 A schematic diagram of the structure of a photovoltaic region segmentation device provided in an embodiment of the present application;
[0054] Figure 7 An internal structure diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0055] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0056] In one embodiment, the present application provides a photovoltaic area segmentation method, and the following embodiment is described by applying the method to a server. It can be understood that the photovoltaic area segmentation method can be executed by a single server or a server cluster composed of multiple servers, and the present application does not make specific restrictions on this.
[0057] like Figure 1 As shown, the present application provides a photovoltaic region segmentation method, the method comprising:
[0058] Step S101: Determine a satellite image.
[0059] Among them, satellite images refer to images of the earth's surface taken by artificial satellites, and the types of satellite images include but are not limited to optical satellite images, radar satellite images, infrared satellite images, synthetic aperture radar (SAR) images, etc. It can be understood that the type of satellite image determined in this embodiment can be any one of the aforementioned multiple types of satellite images, and the satellite image determined in this embodiment can also be an image obtained by fusing two or more of the aforementioned multiple types of satellite images. Therefore, the former can be called a single-modal image, and the latter can be called a multi-modal image, that is, the satellite image can be a single-modal image or a multi-modal image.
[0060] In this step, when it is necessary to segment and identify the photovoltaic area of a certain area, a type of satellite image of the area is obtained, or multiple types of satellite images of the area are obtained, and the multiple types of satellite images are merged to obtain a final satellite image. When the satellite image to be used is determined, the photovoltaic area segmentation is performed based on the satellite image.
[0061] Step S102: Obtain a pre-trained region segmentation model.
[0062] The region segmentation model includes a first feature extraction layer, a second feature extraction layer and a point rendering layer.
[0063] Specifically, the first feature extraction layer includes a separation attention module and an average pooling module for preliminary feature extraction. The separation attention module is used to better capture the key features in the image through the separation attention mechanism and improve the classification accuracy. The average pooling module is used to reduce the spatial dimension of the data. The second feature extraction layer includes a multi-scale feature extraction module and a dual attention module. The multi-scale feature extraction module is used to extract features of different scales from the image. The dual attention module is used to combine channel attention and spatial attention to capture the semantic information of the image. The point rendering layer is used to refine the edges of the photovoltaic area.
[0064] It can be understood that based on the region segmentation model of the structure described above, multiple layers of feature extraction are set to capture more feature information of different dimensions, and the specific structural composition in each feature extraction layer can obtain feature information to the greatest extent, thereby improving the accuracy of photovoltaic region segmentation. In addition, using a point rendering layer to refine the edge of the photovoltaic region in the feature map can improve the accuracy of photovoltaic region edge detection.
[0065] Step S103: input the satellite image into the first feature extraction layer to perform preliminary feature extraction.
[0066] In this step, the satellite image is feature extracted using the separate attention module and the average pooling module in the first feature extraction layer to obtain a feature map corresponding to the satellite image output by the first feature extraction layer.
[0067] Step S104: The feature map output by the first feature extraction layer is input into the second feature extraction layer, and the multi-scale feature extraction module and the dual attention module of the second feature extraction layer perform deep feature extraction to obtain a regional feature map corresponding to the satellite image.
[0068] Among them, the regional feature map refers to the feature representation of the photovoltaic area in the satellite image, which is used to identify and segment the photovoltaic area.
[0069] In this step, since the photovoltaic area in the satellite image is small and the photovoltaic area segmentation is easily affected by the complex environment, the extraction of photovoltaic area features is strengthened. When the feature map corresponding to the satellite image output by the first feature extraction layer is obtained, the feature map is input into the second feature extraction layer, and further feature extraction is performed by the multi-scale feature extraction module and the dual attention module of the second feature extraction layer. Since the dual attention module includes two types of attention modules, which respectively describe the semantic interdependence in the spatial dimension and the channel dimension, so as to enhance the semantic consistency, the accuracy of segmentation can be maintained when segmenting the photovoltaic area of a complex image.
[0070] Step S105: inputting the region feature map into the point rendering layer to obtain a photovoltaic region division image.
[0071] The photovoltaic region division image refers to an image corresponding to a satellite image on which photovoltaic regions are marked.
[0072] In this step, the regional feature map is passed through the point rendering layer to refine the edge of the photovoltaic region to improve the clarity of the photovoltaic region boundary, and the satellite image is annotated using the point rendering layer to generate a photovoltaic region division image.
[0073] The photovoltaic region segmentation method, device, storage medium and computer equipment provided by the present application include: determining a satellite image, obtaining a pre-trained region segmentation model, and then inputting the satellite image into the first feature extraction layer of the region segmentation model to perform preliminary feature extraction, and then inputting the feature map output by the first feature extraction layer into the second feature extraction layer of the region segmentation model, and performing deep feature extraction by the multi-scale feature extraction module and the dual attention module of the second feature extraction layer to obtain a regional feature map, and finally inputting the regional feature map into the point rendering layer of the regional segmentation model to obtain a photovoltaic region segmentation image. Through multi-layer feature extraction, a richer data representation of the photovoltaic region can be obtained, and more feature information of different dimensions can be captured to improve the generalization ability and prediction performance of the photovoltaic region segmentation model, and the multi-scale feature extraction module and the dual attention module are combined for further feature extraction, which can adaptively fuse local features with global context information and improve the accuracy of photovoltaic region segmentation. The dual attention module can also dynamically adjust the channel dimension and spatial dimension to adapt to images of different scales and complexities, thereby reducing the influence of easily confused backgrounds and complex object morphologies on the accuracy of photovoltaic region segmentation.
[0074] like Figure 2 As shown, in one embodiment, the first feature extraction layer includes a separation attention module and an average pooling module; inputting the satellite image into the first feature extraction layer includes:
[0075] Step S201: Input the satellite image into the separation attention module to obtain a first feature map, and use the separation attention module to adjust the spatial dimension of the first feature map.
[0076] In this step, the separation attention module is used to adjust the spatial dimension of the first feature map. This adjustment can enhance the resolution of the feature map and improve the distinguishability of the features, thereby making the image analysis more accurate. By adjusting the spatial dimension, the redundant information of the first feature map can be effectively reduced and the most valuable features can be retained. In addition, adjusting the spatial dimension can also reduce the consumption of computing resources and improve the processing speed.
[0077] Step S202: Input the first feature map after spatial dimension adjustment into the average pooling module to reduce the spatial dimension of the first feature map after spatial dimension adjustment to obtain a second feature map, and use the second feature map as the feature map output by the first feature extraction layer.
[0078] In this step, by calculating the average value of each local area on the first feature map, the spatial dimension of the first feature map is effectively reduced while retaining important feature information. Specifically, the average pooling module sets a preset window, slides the preset window on the first feature map, calculates the average value of all pixel values in the window, and then uses this average value as a new pixel value of the output feature map. In this way, the spatial resolution of the first feature map is reduced, but the key visual information is retained.
[0079] It is understandable that by adjusting and reducing the spatial dimension, the regional segmentation model can focus more on global features, which helps capture the overall structure and pattern in satellite images, thereby improving the generalization ability of the regional segmentation model for images and reducing the model's sensitivity to noise and details in satellite images. This improves the accuracy of photovoltaic region segmentation.
[0080] like Figure 3 As shown, in one embodiment, the feature map output by the first feature extraction layer is input into the second feature extraction layer, and the multi-scale feature extraction module and the dual attention module of the second feature extraction layer perform deep feature extraction, including:
[0081] Step S301: inputting the feature map output by the first feature extraction layer into the multi-scale feature extraction module to extract a multi-scale feature map based on the feature map.
[0082] In this step, the multi-scale feature extraction module is used to analyze the input feature map to extract feature information of different scales. For example, scale space theory or multi-scale filters are used to capture the details and structural information of the image at different resolution levels. In this way, the multi-scale feature extraction module can understand the image content more comprehensively, enhance the model's sensitivity to features of different scales, and improve the richness and accuracy of feature expression.
[0083] Step S302: Input the multi-scale features into the dual attention module to determine the channel weight and spatial weight of each position in the multi-scale feature map.
[0084] Among them, spatial weight refers to the weight applied to each pixel of the image, which is used to emphasize or weaken the characteristics of certain areas. Channel weight refers to the weight applied to each color channel (such as RGB) of the image, which is used to adjust the contribution of different color channels to the model.
[0085] Step S303: Based on the channel weight and spatial weight of each position in the multi-scale feature map, the multi-scale feature map is weighted to obtain a multi-scale weighted feature map.
[0086] In this step, spatial weights and channel weights are applied to the feature maps of each scale. This can be achieved by element-wise multiplication (Hadamard product). Specifically, for the feature map of each scale, the pixel value of each position is multiplied by the corresponding spatial weight, and then the pixel value of each channel is multiplied by the corresponding channel weight. Finally, a multi-scale weighted feature map is obtained.
[0087] Step S304: performing feature fusion on the multi-scale weighted feature map to obtain a regional feature map.
[0088] In this step, the multi-scale weighted feature maps are fused to obtain the final regional feature map. The feature fusion method can be simple stacking, weighted averaging, or other more complex fusion strategies, such as using an attention mechanism to dynamically adjust the contribution of features at different scales.
[0089] It can be understood that from the above steps, the connection structure between the multi-scale feature extraction module and the dual attention module can be known. Through the series connection between the multi-scale feature extraction module and the dual attention module, the multi-scale feature map is first extracted, and then the channel weight and spatial weight are determined to weight the multi-scale feature map, thereby effectively improving the expressiveness of the features and enhancing the model's understanding of image details and contextual information. Further, the feature map is processed in series by the multi-scale feature extraction module and the dual attention module, which helps to avoid possible information loss during feature fusion. The multi-scale feature extraction module is responsible for multi-scale feature extraction, and the dual attention module is responsible for dynamically adjusting the focus area of the feature. The series connection of the two can better optimize the fusion and expression of features.
[0090] In one embodiment, the feature map output by the first feature extraction layer is input into the second feature extraction layer, and the multi-scale feature extraction module and the dual attention module of the second feature extraction layer perform deep feature extraction, including:
[0091] The feature map output by the first feature extraction layer is respectively input into the multi-scale feature extraction module and the dual attention module to respectively determine a third feature map output by the multi-scale extraction module and a fourth feature map output by the dual attention module.
[0092] The third feature map and the fourth feature map are subjected to feature fusion to obtain a regional feature map.
[0093] The solution described in this embodiment is realized based on the parallel connection of the multi-scale feature extraction module and the dual attention module in structure. In addition, the feature fusion process in this embodiment can refer to the description of step S304.
[0094] It can be understood that the parallel connection means that the multi-scale feature extraction module and the dual attention module can run simultaneously to improve the computational efficiency.
[0095] In addition, it can be achieved by structurally connecting the dual attention module and the multi-scale feature extraction module in parallel, or by structurally connecting the dual attention module and the multi-scale feature extraction module in series. The module processing order of these two schemes is different from the above two schemes, and this application does not make any specific restrictions on the selection of the connection structure.
[0096] like Figure 4 As shown, in one embodiment, the regional feature map is input into the point rendering layer to obtain a photovoltaic region division image, including:
[0097] Step S401: Determine multiple key points in the region feature map.
[0098] In this step, the boundary of each photovoltaic region is determined by the regional feature map, and key points are selected on the boundary of each photovoltaic region according to a preset step size.
[0099] Step S402: for each key point, locally refine the key point to obtain a refined feature of the key point.
[0100] In this step, for each key point, the local context information around the key point is learned to locally refine the key point and obtain the refined features of the key point.
[0101] Step S403: when the refined features of each key point are determined, the refined features of each key point are rendered into the regional feature map to obtain a photovoltaic regional feature map.
[0102] It can be understood that when the refined features of each key point are determined, the refined features of each key point are rendered into the regional feature map to obtain a photovoltaic region feature map with clear edges of each photovoltaic region.
[0103] Step S404: labeling the satellite image according to the photovoltaic region characteristic map to obtain a photovoltaic region division image.
[0104] It can be understood that by determining key points to obtain local context information, the surrounding area of each key point is refined to refine the boundary of the photovoltaic area, thereby improving the accuracy of photovoltaic area edge detection.
[0105] In one embodiment, the training process of the region segmentation model includes:
[0106] Obtain a training sample set and its corresponding labeled sample set;
[0107] Determine an initial model; the initial model includes a first feature extraction layer, a second feature extraction layer and a point rendering layer to be trained;
[0108] Iteratively train the initial model according to the training sample set until a preset training stop condition is met, and determine the trained initial model as the region segmentation model;
[0109] Among them, in each iteration of iterative training, the predicted photovoltaic area of each training sample is determined according to the output results of the training sample set in this iteration, and the loss function of this iteration is determined according to the predicted photovoltaic area of each training sample and the labeled photovoltaic area of each labeled sample in the labeled sample set to adjust the parameters of the initial model.
[0110] Specifically, the loss function is used to describe the gap between the predicted photovoltaic area of each sample and the labeled photovoltaic area of each labeled sample. The parameters of the initial model include the training parameters of the first feature extraction layer, the second feature extraction layer and the point rendering layer.
[0111] In another embodiment, when determining the initial model, transfer learning is used to fine-tune the parameters of the initial model to obtain a new initial model, and then iterative training is performed based on the new initial model. In this embodiment, fine-tuning the parameters of the initial model in advance can improve the training efficiency of the initial model.
[0112] It can be understood that by iteratively training the initial model including the first feature extraction layer, the second feature extraction layer and the point rendering layer to be trained, and taking the gap between the predicted photovoltaic area and the labeled photovoltaic area as the loss function, the initial model can be trained in the direction of narrowing the gap between the predicted photovoltaic area and the labeled photovoltaic area, and by updating the training parameters of the first feature extraction layer, the second feature extraction layer and the point rendering layer, the feature extraction ability of the initial model is improved, thereby enhancing the generalization ability of the finally trained model.
[0113] In one example, if Figure 5 As shown, Figure 5 A schematic diagram of the construction of the regional segmentation model provided in the embodiment of the present application. Backbone refers to the backbone network, upgrade refers to pushing the separation attention module into the backbone network, DAM refers to the dual attention module, and integration refers to combining the dual attention module with the multi-scale feature extraction module in the encoding layer (Encoder). Optimization learning can be used to fine-tune the parameters of the initial model and optimize the loss function during the model training process.
[0114] In one embodiment, the loss function is determined according to the following expression:
[0115]
[0116] In the formula, represents the loss function, Indicates the height of the image. represents the weight of the image, Represents pixels The predicted classification of Represents pixels The annotation classification of Indicates the smoothing factor to prevent letters from being zero.
[0117] In this embodiment, by calculating the similarity between the predicted classification and the labeled classification, the overlap degree of the two sets is effectively measured. Compared with the traditional cross entropy loss function, the way of determining the loss function by this expression can solve the problem of class imbalance and reduce the model's neglect of minority classes, thereby improving the performance of the model in the photovoltaic area segmentation task.
[0118] In one embodiment, it is also possible to consider using a focal loss function, an IoU loss function, and a mixed loss function, wherein a mixed loss function refers to a function obtained by mixing two or more types of loss functions. In the process of selecting a loss function, a suitable loss function can be selected based on actual needs, and this application does not impose specific restrictions on this.
[0119] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.
[0120] The photovoltaic region segmentation device provided in the embodiment of the present application is described below. The photovoltaic region segmentation device described below and the photovoltaic region segmentation method described above can be referred to each other.
[0121] like Figure 6As shown, the present application provides a photovoltaic region segmentation device 500, the device comprising:
[0122] An image acquisition module 501 is used to determine a satellite image;
[0123] The model determination module 502 is used to obtain a pre-trained region segmentation model; the region segmentation model includes a first feature extraction layer, a second feature extraction layer and a point rendering layer;
[0124] A preliminary feature extraction module 503 is used to input the satellite image into the first feature extraction layer to perform preliminary feature extraction;
[0125] A deep feature extraction module 504 is used to input the feature map output by the first feature extraction layer into the second feature extraction layer, and the multi-scale feature extraction module and the dual attention module of the second feature extraction layer perform deep feature extraction to obtain a regional feature map corresponding to the satellite image;
[0126] The region division module 505 is used to input the region feature map into the point rendering layer to obtain a photovoltaic region division image.
[0127] In one embodiment, the first feature extraction layer includes a separation attention module and an average pooling module; the preliminary feature extraction module includes:
[0128] A dimension adjustment submodule, used for inputting the satellite image into the separation attention module to obtain a first feature map, and adjusting the spatial dimension of the first feature map using the separation attention module;
[0129] The pooling submodule is used to input the first feature map after spatial dimension adjustment into the average pooling module to reduce the spatial dimension of the first feature map after spatial dimension adjustment, obtain the second feature map, and use the second feature map as the feature map output by the first feature extraction layer.
[0130] In one embodiment, the deep feature extraction module includes:
[0131] A feature extraction submodule, configured to input the feature map output by the first feature extraction layer into the multi-scale feature extraction module, so as to extract a multi-scale feature map based on the feature map;
[0132] A weight determination submodule, used to input the multi-scale features into the dual attention module to determine the channel weight and spatial weight of each position in the multi-scale feature map;
[0133] A feature weighting submodule is used to perform weighted processing on the multi-scale feature map based on the channel weight and spatial weight of each position in the multi-scale feature map to obtain a multi-scale weighted feature map;
[0134] The first feature fusion submodule is used to perform feature fusion on the multi-scale weighted feature map to obtain a regional feature map.
[0135] In one embodiment, the deep feature extraction module includes:
[0136] A feature determination submodule, used to input the feature map output by the first feature extraction layer into the multi-scale feature extraction module and the dual attention module, respectively, to determine a third feature map output by the multi-scale extraction module and a fourth feature map output by the dual attention module;
[0137] The second feature fusion submodule is used to perform feature fusion on the third feature map and the fourth feature map to obtain a regional feature map.
[0138] In one embodiment, the region division module includes:
[0139] A point determination submodule is used to determine multiple key points in the regional feature map;
[0140] A local refinement submodule is used to locally refine each key point to obtain a refined feature of the key point;
[0141] A rendering submodule, for rendering the refined features of each key point into a regional feature map when the refined features of each key point are determined, so as to obtain a photovoltaic regional feature map;
[0142] The annotation submodule is used to annotate the satellite image according to the photovoltaic region characteristic map to obtain the photovoltaic region division image.
[0143] In one embodiment, the photovoltaic region segmentation device further comprises:
[0144] A sample acquisition module is used to obtain a training sample set and its corresponding labeled sample set;
[0145] An initial model determination module is used to determine an initial model; the initial model includes a first feature extraction layer, a second feature extraction layer and a point rendering layer to be trained;
[0146] An iterative training module is used to iteratively train the initial model according to the training sample set until a preset training stop condition is met, and then determine the trained initial model as the region segmentation model;
[0147] Among them, in each iteration of iterative training, the predicted photovoltaic area of each training sample is determined according to the output results of the training sample set in this iteration, and the loss function of this iteration is determined according to the predicted photovoltaic area of each training sample and the labeled photovoltaic area of each labeled sample in the labeled sample set to adjust the parameters of the initial model.
[0148] The division of each module in the above photovoltaic area segmentation device is only for illustration. In other embodiments, the photovoltaic area segmentation device can be divided into different modules as needed to complete all or part of the functions of the above photovoltaic area segmentation device. Each module in the above photovoltaic area segmentation device can be implemented in whole or in part by software, hardware and a combination thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.
[0149] In one embodiment, the present application further provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the photovoltaic area segmentation method as described in any of the above embodiments.
[0150] In one embodiment, the present application further provides a computer device having computer-readable instructions stored therein. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the photovoltaic area segmentation method as described in any one of the above embodiments.
[0151] Indicatively, Figure 7 As shown, Figure 7 This is a schematic diagram of the internal structure of a computer device provided in an embodiment of the present application. The computer device 600 may be provided as a server. Figure 7 The computer device 600 includes a processing component 602, which further includes one or more processors, and a memory resource represented by a memory 601 for storing instructions executable by the processing component 602, such as an application. The application stored in the memory 601 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 602 is configured to execute instructions to perform the photovoltaic region segmentation method of any of the above embodiments.
[0152] The computer device 600 may further include a power supply component 603 configured to perform power management of the computer device 600, a wired or wireless network interface 604 configured to connect the computer device 600 to a network, and an input / output (I / O) interface 605. The computer device 600 may operate based on an operating system stored in the memory 601, such as Windows Server TM, Mac OS X TM, Unix TM, Linux TM, Free BSD TM, or the like.
[0153] Those skilled in the art will understand that Figure 7The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0154] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish an entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the term "include", "comprise" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only include those elements, but also include other elements that are not clearly listed, or also include elements inherent to such process, method, article or equipment. In the absence of more restrictions, the elements limited by the sentence "comprise one..." do not exclude the existence of other identical elements in the process, method, article or equipment including the elements. Herein, the singular "one", "one" and "described / the" may also include plural forms, unless the context clearly indicates another way. It should also be understood that the terms "include / comprise" or "have" etc. specify the existence of stated features, wholes, steps, operations, components, parts or combinations thereof, but do not exclude the possibility of the existence or addition of one or more other features, wholes, steps, operations, components, parts or combinations thereof. At the same time, the term "and / or" used in this specification includes any and all combinations of the relevant listed items.
[0155] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can refer to each other.
[0156] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A photovoltaic region segmentation method, characterized in that: The method comprises: Identify satellite images; Obtaining a pre-trained region segmentation model; the region segmentation model includes a first feature extraction layer, a second feature extraction layer and a point rendering layer; Inputting the satellite image into the first feature extraction layer to perform preliminary feature extraction; Inputting the feature map output by the first feature extraction layer into the second feature extraction layer, and performing deep feature extraction by the multi-scale feature extraction module and the dual attention module of the second feature extraction layer to obtain a regional feature map corresponding to the satellite image; The region feature map is input into the point rendering layer to obtain a photovoltaic region division image.
2. The photovoltaic region segmentation method according to claim 1, characterized in that: The first feature extraction layer includes a separation attention module and an average pooling module; The step of inputting the satellite image into the first feature extraction layer comprises: Inputting the satellite image into the separation attention module to obtain a first feature map, and adjusting the spatial dimension of the first feature map using the separation attention module; The first feature map after spatial dimension adjustment is input into the average pooling module to reduce the spatial dimension of the first feature map after spatial dimension adjustment to obtain a second feature map, and the second feature map is used as the feature map output by the first feature extraction layer.
3. The photovoltaic region segmentation method according to claim 1, characterized in that: The step of inputting the feature map output by the first feature extraction layer into the second feature extraction layer, and performing deep feature extraction by the multi-scale feature extraction module and the dual attention module of the second feature extraction layer, comprises: Inputting the feature map output by the first feature extraction layer into the multi-scale feature extraction module to extract a multi-scale feature map based on the feature map; Inputting the multi-scale features into the dual attention module to determine the channel weight and spatial weight of each position in the multi-scale feature map; Based on the channel weight and the spatial weight of each position in the multi-scale feature map, weighted processing is performed on the multi-scale feature map to obtain a multi-scale weighted feature map; Feature fusion is performed on the multi-scale weighted feature map to obtain a regional feature map.
4. The photovoltaic region segmentation method according to claim 1, characterized in that: The step of inputting the feature map output by the first feature extraction layer into the second feature extraction layer, and performing deep feature extraction by the multi-scale feature extraction module and the dual attention module of the second feature extraction layer, comprises: Inputting the feature map output by the first feature extraction layer into the multi-scale feature extraction module and the dual attention module respectively, so as to respectively determine a third feature map output by the multi-scale extraction module and a fourth feature map output by the dual attention module; Feature fusion is performed on the third feature map and the fourth feature map to obtain a regional feature map.
5. The photovoltaic region segmentation method according to claim 1, characterized in that: The step of inputting the region feature map into the point rendering layer to obtain a photovoltaic region division image comprises: Determining a plurality of key points in the region feature map; For each key point, locally refine the key point to obtain the refined features of the key point; When the refined features of each key point are determined, the refined features of each key point are rendered into the regional feature map to obtain a photovoltaic regional feature map; The satellite image is annotated according to the photovoltaic region characteristic map to obtain a photovoltaic region division image.
6. The photovoltaic region segmentation method according to any one of claims 1 to 5, characterized in that: The training process of the region segmentation model includes: Obtain a training sample set and its corresponding labeled sample set; Determine an initial model; the initial model includes a first feature extraction layer, a second feature extraction layer and a point rendering layer to be trained; Iteratively training the initial model according to the training sample set until a preset training stop condition is met, and determining the trained initial model as the region segmentation model; Among them, in each iteration of iterative training, the predicted photovoltaic area of each training sample is determined according to the output results of the training sample set in this iteration, and the loss function of this iteration is determined according to the predicted photovoltaic area of each training sample and the labeled photovoltaic area of each labeled sample in the labeled sample set to adjust the parameters of the initial model.
7. The photovoltaic region segmentation method according to claim 6, characterized in that: The loss function is determined according to the following expression: In the formula, represents the loss function, Indicates the height of the image. represents the weight of the image, Represents pixels The predicted classification of Represents pixels The annotation classification of Indicates the smoothing factor to prevent letters from being zero.
8. A photovoltaic area segmentation device, characterized in that: The device comprises: An image acquisition module, used to determine satellite images; A model determination module, used to obtain a pre-trained region segmentation model; the region segmentation model includes a first feature extraction layer, a second feature extraction layer and a point rendering layer; A preliminary feature extraction module, used for inputting the satellite image into the first feature extraction layer to perform preliminary feature extraction; A deep feature extraction module, used for inputting the feature map output by the first feature extraction layer into the second feature extraction layer, and performing deep feature extraction by the multi-scale feature extraction module and the dual attention module of the second feature extraction layer to obtain a regional feature map corresponding to the satellite image; The region division module is used to input the region feature map into the point rendering layer to obtain a photovoltaic region division image.
9. A storage medium, characterized in that: The storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the photovoltaic area segmentation method according to any one of claims 1 to 7.
10. A computer device, characterized in that: include: one or more processors, and memory; The memory stores computer-readable instructions, and when the computer-readable instructions are executed by the one or more processors, the steps of the photovoltaic area segmentation method according to any one of claims 1 to 7 are performed.
Citation Information
Patent Citations
Semantic segmentation method and device for transparent object in image and electronic equipment
CN116993973A
Image segmentation method and system based on cascade attention and multi-scale feature fusion
CN118096785A
Pyramid architecture for multi-scale processing in point cloud segmentation
US20240013399A1
