Roof available area identification method based on multi-task deep learning

Abstract

The invention discloses a roof available area identification method based on multi-task deep learning, and belongs to the technical field of image identification. Comprising the steps of obtaining multiple satellite pictures; and cutting and fragmenting each satellite picture, and inputting each satellite picture after fragmenting into the trained improved U-Net semantic segmentation network model to obtain a roof available region prediction result map. According to the improved U-Net semantic segmentation network model, a multi-scale feature extraction module and a detail header module are added on the basis of the U-Net semantic segmentation network model, and the improved U-Net semantic segmentation network model comprises a roof region recognition branch, a roof edge region recognition branch and a roof invalid region recognition branch. And performing post-processing on the roof available region prediction result graph based on the roof connected domain area and the compactness index to obtain a roof available region corresponding to each satellite picture. The method improves the recognition accuracy of the available area of the roof.

Description

Technical Field

[0001] This application belongs to the field of image recognition technology, and in particular relates to a method for identifying rooftop usable areas based on multi-task deep learning. Background Technology

[0002] Solar energy is a clean and renewable energy source that can be converted into electricity through photovoltaic (PV) technology. In urban areas, many building rooftops are unused, making them ideal locations for installing solar PV panels. Utilizing urban rooftops for PV systems not only effectively utilizes space resources but also provides electricity to buildings, reducing reliance on the traditional power grid and lowering carbon emissions. Promoting rooftop PV development and accurately estimating the potential for urban building rooftop PV installations requires precisely identifying suitable rooftop PV installation areas.

[0003] Traditional roof identification methods lack generalization ability, and machine learning-based methods yield roof data with low spatial resolution. Deep learning-based methods, however, are highly effective for accurate identification of urban building roofs. However, urban building roofs contain unusable areas, including rooftop equipment (such as air conditioning units and ventilation ducts), structural obstructions (such as skylights), existing photovoltaic systems, and unsuitable perforated areas. These areas significantly reduce the actual usable photovoltaic installation area. However, some deep learning-based roof identification studies have not considered these unusable areas, failing to accurately identify the existing usable roof areas. This affects the accuracy of subsequent rooftop photovoltaic panel installation and solar energy potential assessment.

[0004] Existing research using deep learning models to identify rooftops does not take into account the problem of ineffective rooftop areas, and cannot accurately identify usable areas on urban rooftops. This results in inaccurate estimation of the potential for installing photovoltaic panels on urban rooftops and low rooftop utilization rates. Summary of the Invention

[0005] This application aims to address at least one of the technical problems existing in the prior art. To this end, this application proposes a roof usable area identification method based on multi-task deep learning, which improves the accuracy of roof usable area identification.

[0006] In a first aspect, this application provides a method for identifying rooftop usable areas based on multi-task deep learning, the method comprising: Acquire multiple satellite images; Each satellite image is cropped and segmented. The segmented satellite image is then input into a trained improved U-Net semantic segmentation network model to obtain a predicted map of the roof usable area. The improved U-Net semantic segmentation network model adds a multi-scale feature extraction module and a detail head module to the U-Net semantic segmentation network model. The improved U-Net semantic segmentation network model includes a roof area recognition branch, a roof edge area recognition branch, and a roof invalid area recognition branch. Post-processing is performed on the predicted roof usable area based on the roof connectivity area and compactness index to obtain the roof usable area corresponding to each satellite image.

[0007] According to one embodiment of this application, the step of inputting each segmented satellite image into a trained improved U-Net semantic segmentation network model to obtain a predicted map of rooftop usable areas includes: Each satellite image after being segmented is input into the roof edge region recognition branch to obtain the roof edge result; Each satellite image after segmentation is input into the roof region recognition branch, and the roof edge results are used as constraints to obtain the roof region segmentation results; Each satellite image after being segmented is input into the roof invalid area identification branch to obtain the roof invalid area segmentation result; Subtracting the invalid roof area segmentation results from the roof area segmentation results yields the predicted map of the usable roof area.

[0008] According to one embodiment of this application, the step of inputting each segmented satellite image into the roof edge region recognition branch to obtain the roof edge result includes: Each satellite image after being segmented is sequentially processed through three downsampling modules for feature extraction to obtain feature extraction information; The feature extraction information is input into the multi-scale feature extraction module, and multi-scale feature extraction is performed using 3D convolution to obtain multi-scale information. Multi-scale information is input into the detail head module to extract edge information and obtain the roof edge result.

[0009] According to one embodiment of this application, the step of inputting each segmented satellite image into the roof region recognition branch, and combining the roof edge results as constraints to obtain the roof region segmentation result includes: Each satellite image after being sliced ​​is preprocessed to obtain a data-enhanced satellite image. The preprocessing includes random rotation, random flipping, random scaling, and normalization. The augmented satellite images are sequentially processed through five downsampling modules for feature extraction to obtain deep feature information; The deep feature information is sequentially passed through five upsampling modules for detailed reconstruction to obtain the first reconstructed feature map; The first reconstructed feature map is concatenated with multi-scale information and then input into the first convolutional layer to obtain the roof area segmentation result.

[0010] According to one embodiment of this application, the step of inputting each segmented satellite image into the roof invalid area identification branch to obtain the roof invalid area segmentation result includes: Each satellite image after being segmented is sequentially processed through five downsampling modules for feature extraction to obtain deep feature information; After the deep feature information is degraded from gradient tracking, it is sequentially passed through five upsampling modules for detailed reconstruction to obtain the second reconstructed feature map. The results output by the last three upsampling modules are ANDed with the roof area segmentation results to obtain invalid area identification constraints. Based on the invalid region identification constraint, the second reconstructed feature map is input into the second convolutional layer to obtain the invalid region segmentation result of the roof.

[0011] According to one embodiment of this application, the post-processing of the roof usable area prediction result map based on the roof connectivity region area and compactness index to obtain the roof usable area corresponding to each satellite image includes: The number of pixels and compactness index of each connected region in the predicted map of the roof's usable area are calculated. The formula for calculating the compactness index is as follows:

[0012] Where A is the area of ​​the connected region, and P is the perimeter. Compactness index; Determine whether the number of pixels in each connected component is less than a first preset threshold. If the number of pixels is less than the first preset threshold, determine whether the compactness index is less than a second preset threshold. If the compactness index is less than the second preset threshold, remove the connected components with both the number of pixels less than the first preset threshold and the compactness index less than the second preset threshold to obtain the rooftop usable area corresponding to each satellite image.

[0013] According to one embodiment of this application, the training process of the improved U-Net semantic segmentation network model includes: Construct an improved U-Net semantic segmentation network model; We acquire an image dataset from the architectural field, perform secondary annotation, mark the ineffective roof areas on each image, and divide the image dataset into training and test sets. The preset improved U-Net semantic segmentation network model is trained based on the training set to obtain a pre-trained improved U-Net semantic segmentation network model; The test set is input into the pre-trained improved U-Net semantic segmentation network model for validation, resulting in the trained improved U-Net semantic segmentation network model.

[0014] Secondly, this application provides a rooftop usable area identification device based on multi-task deep learning, the device comprising: The acquisition module is used to acquire multiple satellite images; The first processing module is used to crop and segment each satellite image, and input each segmented satellite image into the trained improved U-Net semantic segmentation network model to obtain the predicted result map of the roof usable area. The improved U-Net semantic segmentation network model adds a multi-scale feature extraction module and a detail head module to the U-Net semantic segmentation network model. The improved U-Net semantic segmentation network model includes a roof area recognition branch, a roof edge area recognition branch, and a roof invalid area recognition branch. The second processing module is used to post-process the predicted roof usable area map based on the roof connectivity area and compactness index to obtain the roof usable area corresponding to each satellite image.

[0015] Thirdly, this application provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the roof usable area identification method based on multi-task deep learning as described in the first aspect above.

[0016] Fourthly, this application provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the roof usable area identification method based on multi-task deep learning as described in the first aspect above.

[0017] Fifthly, this application provides a chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the roof availability area identification method based on multi-task deep learning as described in the first aspect.

[0018] In a sixth aspect, this application provides a computer program product, including a computer program that, when executed by a processor, implements the roof availability region identification method based on multi-task deep learning as described in the first aspect above.

[0019] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application.

[0020] The present invention provides a rooftop usability area identification method based on multi-task deep learning, which has the following advantages over existing technologies: This invention acquires multiple satellite images, crops and segments them, and inputs each segmented satellite image into a trained improved U-Net semantic segmentation network model to obtain a predicted image of the usable roof area. By adding multi-scale feature extraction and a detail head module, the model's ability to perceive different scales and details is enhanced. By combining roof area recognition branches, roof edge area recognition branches, and roof invalid area recognition branches for multi-task processing, more accurate identification and segmentation of the usable roof area is achieved. In the post-processing stage, the predicted image is optimized based on the roof connected region area and compactness index, effectively improving the identification accuracy and robustness of the usable roof area. Especially in satellite images of complex roof structures and different geographical locations, it can obtain relatively accurate area division results, effectively improving the identification accuracy and generalization ability of the usable roof area, and enabling rapid identification of the usable roof area of ​​buildings in a large urban area. Attached Figure Description

[0021] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which: Figure 1 This is one of the flowcharts illustrating the roof usable area identification method based on multi-task deep learning provided in this application embodiment; Figure 2 This is one of the structural schematic diagrams of the improved U-Net semantic segmentation network model provided in the embodiments of this application; Figure 3 This is the second schematic diagram of the structure of the improved U-Net semantic segmentation network model provided in the embodiments of this application; Figure 4 This is the second flowchart of the roof usable area identification method based on multi-task deep learning provided in the embodiments of this application; Figure 5 RGB satellite images with a resolution of 0.3m downloaded from Google Earth in 2023, showing a portion of the core urban area of ​​a certain city. Figure 6 To improve the U-Net semantic segmentation network model, the identification result map of the rooftop usable area in a part of the core urban area of ​​a city in 2023 was obtained. Figure 7 This is a schematic diagram of the structure of the roof usable area identification device based on multi-task deep learning provided in the embodiments of this application; Figure 8 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0022] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0023] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0024] The following description, in conjunction with the accompanying drawings, details the roof usable area identification method based on multi-task deep learning, the roof usable area identification device based on multi-task deep learning, the electronic device, and the readable storage medium provided in this application, through specific embodiments and application scenarios.

[0025] Among them, the rooftop region identification method based on multi-task deep learning can be applied to the terminal, specifically by the hardware or software in the terminal.

[0026] The terminal includes, but is not limited to, portable communication devices such as mobile phones or tablets with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads). It should also be understood that, in some embodiments, the terminal may not be a portable communication device, but rather a desktop computer with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads).

[0027] The following embodiments describe a terminal including a display and a touch-sensitive surface. However, it should be understood that the terminal may include one or more other physical user interface devices such as a physical keyboard, mouse, and joystick.

[0028] The roof usable area identification method based on multi-task deep learning provided in this application embodiment can be executed by an electronic device or a functional module or entity in an electronic device that can implement the roof usable area identification method based on multi-task deep learning. The electronic devices mentioned in this application embodiment include, but are not limited to, mobile phones, tablets, computers, cameras and wearable devices. The following uses an electronic device as the execution subject to illustrate the roof usable area identification method based on multi-task deep learning provided in this application embodiment.

[0029] Currently, methods for identifying and extracting urban building rooftops mainly fall into three categories: traditional identification methods, machine learning identification methods, and deep learning identification methods. Traditional rooftop identification techniques primarily rely on manual image annotation and traditional image processing methods, which have significant limitations. Manual image annotation is inefficient and costly, making it difficult to apply to large-scale urban assessments. Rule-based image processing methods (such as edge detection and threshold segmentation) lack generalization ability when faced with complex urban rooftop scenes, making it difficult to adapt to rooftop feature variations under different building types and lighting conditions. Machine learning-based rooftop identification techniques mainly rely on multi-source remote sensing information, exhibiting strong generalization and the ability to identify rooftops over a wide area. However, the rooftop spatial distribution data obtained using this method has low resolution, making it unsuitable for detailed photovoltaic panel layout planning for specific cities, thus failing to provide strong support for the accurate installation of rooftop photovoltaic panels. With the rapid development of deep learning in recent years, more and more research is using deep learning methods to identify urban rooftops. Deep learning methods possess multi-scale feature fusion capabilities, simultaneously capturing both local details and global structure of rooftops. Furthermore, deep learning methods have strong generalization capabilities; well-trained models can be applied to rooftop identification in various scenarios. Therefore, using deep learning methods to identify roofs can achieve high accuracy, but it requires a large amount of data to support model training.

[0030] Figure 1 This is one of the flowcharts illustrating the rooftop usable area identification method based on multi-task deep learning provided in this application embodiment, such as... Figure 1 As shown, the rooftop usable area identification method based on multi-task deep learning includes steps 110, 120 and 130.

[0031] Step 110: Acquire multiple satellite images; For example, RGB satellite images of a part of the core urban area of ​​a city in 2023 with a resolution of 0.3m were downloaded from Google Earth, resulting in multiple satellite images.

[0032] Step 120: Crop and segment each satellite image, and input each segmented satellite image into the trained improved U-Net semantic segmentation network model to obtain the predicted roof usable area map. The improved U-Net semantic segmentation network model adds a multi-scale feature extraction module and a detail head module to the U-Net semantic segmentation network model. The improved U-Net semantic segmentation network model includes a roof area recognition branch, a roof edge area recognition branch, and a roof invalid area recognition branch. Furthermore, each satellite image is cropped and segmented. The original satellite image has dimensions X×Y, and the cropped local image has dimensions H×H. The overlap rate between adjacent cropped local images is S. The number of horizontal segments m, the number of vertical segments n, and the row and column coordinates x and y of the top-left corner of the cropped image are calculated using the following formulas:

[0033]

[0034]

[0035]

[0036]

[0037] Where i and j are the row and column indices of the cropped image in the original satellite image, respectively, and Q is the original satellite image. This represents the horizontal dimension of the cropped image. This represents the vertical dimension of the cropped image. Let y be the row coordinate of the top-left corner of the cropped image, y be the column coordinate of the top-left corner of the cropped image, S be the overlap rate between adjacent local images after cropping, X and Y be the dimensions of the original satellite image, and H be the dimensions of the cropped image.

[0038] For example, the total number of rows and columns of pixels X and Y of a core area of ​​a city obtained from Google Earth are 31267 and 41824, respectively. The size H of the cropped local image is 512, and the overlap rate S between adjacent local images is 0.3. The original satellite image is read using GDAL (Geospatial Data Abstraction Library) and the block processing is performed according to the above parameters.

[0039] An improved U-Net semantic segmentation network model was built based on PyTorch, and the optimized model parameters were fed into the improved U-Net semantic segmentation network model to complete the model parameter initialization. The sliced ​​satellite images were fed into the improved U-Net semantic segmentation network model in batches to predict the roof areas and invalid roof areas in the images. The difference between the two was used to obtain the predicted roof usable area map, which was then stitched together to recreate the size of the original satellite image. Since an overlap rate was set for adjacent local images in the cropped slices, a voting method was used to determine the pixel category for overlapping areas.

[0040] In some embodiments, the step of inputting each segmented satellite image into a trained improved U-Net semantic segmentation network model to obtain a predicted map of rooftop usable areas includes: Each satellite image after being segmented is input into the roof edge region recognition branch to obtain the roof edge result; Each satellite image after segmentation is input into the roof region recognition branch, and the roof edge results are used as constraints to obtain the roof region segmentation results; Each satellite image after being segmented is input into the roof invalid area identification branch to obtain the roof invalid area segmentation result; Subtracting the invalid roof area segmentation results from the roof area segmentation results yields the predicted map of the usable roof area.

[0041] In some embodiments, inputting each segmented satellite image into the roof edge region recognition branch to obtain the roof edge result includes: Each satellite image after being segmented is sequentially processed through three downsampling modules for feature extraction to obtain feature extraction information; The feature extraction information is input into the multi-scale feature extraction module, and multi-scale feature extraction is performed using 3D convolution to obtain multi-scale information. Multi-scale information is input into the detail head module to extract edge information and obtain the roof edge result.

[0042] Figure 2 This is one of the structural schematic diagrams of the improved U-Net semantic segmentation network model provided in the embodiments of this application, such as... Figure 2 As shown, the improved U-Net semantic segmentation network model is a multi-task model, including roof recognition, invalid roof region recognition, and roof edge recognition tasks. The improved U-Net semantic segmentation network model optimizes the roof segmentation edges by adding a roof detail edge constraint module based on the original U-Net semantic segmentation network model.

[0043] Figure 3This is the second structural diagram of the improved U-Net semantic segmentation network model provided in the embodiments of this application. The output feature information of the first three downsampling modules of the backbone U-Net is extracted by the multi-scale feature extraction module using 3D convolution to extract the multi-scale information of the roof. This information is then concatenated in the last layer of the backbone U-Net decoding. After passing through a detail head module, the edge information of the roof is extracted to obtain the roof edge result. The loss is calculated with the real roof edge as the segmentation boundary constraint.

[0044] It should be noted that the real roof edge is generated from the real roof label through detailed edges, and does not participate in model training; it is simply the result of generating real boundary information from the real label.

[0045] See Figure 2 In some embodiments, the step of inputting each segmented satellite image into the roof region recognition branch, and combining the roof edge results as constraints to obtain the roof region segmentation result, includes: Each satellite image after being sliced ​​is preprocessed to obtain a data-enhanced satellite image. The preprocessing includes random rotation, random flipping, random scaling, and normalization. The augmented satellite images are sequentially processed through five downsampling modules for feature extraction to obtain deep feature information; The deep feature information is sequentially passed through five upsampling modules for detailed reconstruction to obtain the first reconstructed feature map; The first reconstructed feature map is concatenated with multi-scale information and then input into the first convolutional layer to obtain the roof area segmentation result.

[0046] For example, each satellite image after being sliced ​​is preprocessed and then augmented with data (including random rotation, random flipping, random scaling, and normalization) to obtain an augmented satellite image. This image is then passed through five downsampling modules (including convolutional layers, normalization layers, activation layers, and pooling layers) to extract the deepest features and obtain deep feature information. Finally, the image is passed through five upsampling modules (including 3×3 convolutional layers, normalization layers, activation layers, and 2x interpolation) to obtain the first reconstructed feature map.

[0047] It is worth noting that the multi-scale information, after being constrained by the segmentation boundary, also contains boundary feature information. After the first reconstructed feature map is concatenated with the multi-scale information, it is passed through the first convolutional layer, which includes a 3×3 convolutional layer and an activation layer, to obtain the roof region segmentation result. This allows the multi-scale feature extraction module and the backbone U-Net to work together on the segmentation result, thereby improving the accuracy of roof segmentation.

[0048] See Figure 2In some embodiments, the step of inputting each segmented satellite image into the roof invalid area identification branch to obtain the roof invalid area segmentation result includes: Each satellite image after being segmented is sequentially processed through five downsampling modules for feature extraction to obtain deep feature information; After the deep feature information is degraded from gradient tracking, it is sequentially passed through five upsampling modules for detailed reconstruction to obtain the second reconstructed feature map. The results output by the last three upsampling modules are ANDed with the roof area segmentation results to obtain invalid area identification constraints. Based on the invalid region identification constraint, the second reconstructed feature map is input into the second convolutional layer to obtain the invalid region segmentation result of the roof.

[0049] It is easy to understand that each satellite image after being sliced ​​is sequentially passed through five downsampling modules for feature extraction to obtain deep feature information. Since the gradient backpropagation of the roof invalid area identification will destroy the downsampling structure of the backbone U-net, thus affecting the identification of the roof area, the deep feature information is de-gradient tracked and then sequentially passed through five upsampling modules for detail reconstruction to obtain the second reconstructed feature map.

[0050] Furthermore, the roof region segmentation result is downsampled and then ANDed with the last three upsampling modules to obtain the roof invalid region identification constraint. Based on the invalid region identification constraint, the second reconstructed feature map is input into the second convolutional layer, which includes a 3×3 convolutional layer and an activation layer, to obtain the roof invalid region segmentation result.

[0051] Finally, subtract the invalid roof area segmentation results from the roof area segmentation results to obtain the predicted map of the usable roof area.

[0052] Step 130: Post-process the predicted roof usable area map based on the roof connectivity area and compactness index to obtain the roof usable area corresponding to each satellite image.

[0053] In some embodiments, the post-processing of the roof usable area prediction map based on the roof connectivity area and compactness index to obtain the roof usable area corresponding to each satellite image includes: The number of pixels and compactness index of each connected region in the predicted map of the roof's usable area are calculated. The formula for calculating the compactness index is as follows:

[0054] Where A is the area of ​​the connected region, and P is the perimeter. Compactness index; Determine whether the number of pixels in each connected component is less than a first preset threshold. If the number of pixels is less than the first preset threshold, determine whether the compactness index is less than a second preset threshold. If the compactness index is less than the second preset threshold, remove the connected components with both the number of pixels less than the first preset threshold and the compactness index less than the second preset threshold to obtain the rooftop usable area corresponding to each satellite image.

[0055] It is easy to understand that, since some roofs have a large number of unusable areas, after subtracting the invalid areas from the roof area, the remaining part still needs to be further eliminated.

[0056] The compactness index, also known as the shape factor or shape index, measures how closely a region's shape resembles a circle. The calculation formula is shown below:

[0057]

[0058] Where A is the area of ​​the connected region, and P is the perimeter. For the outer perimeter, For the internal perimeter, The compactness index indicates that the shape is closer to a circle, while the smaller the value, the more irregular the shape.

[0059] The number of pixels and compactness index of each connected region in the predicted roof usable area map are calculated. The irregularity of the remaining connected regions is determined by evaluating the area and compactness index of the roof connected regions. Connected regions with too low compactness are removed. For example, roof areas with less than 300 connected region pixels and a compactness index of less than 0.3 are removed to obtain the roof usable area corresponding to each satellite image.

[0060] In some embodiments, the training process of the improved U-Net semantic segmentation network model includes: Construct an improved U-Net semantic segmentation network model; We acquire an image dataset from the architectural field, perform secondary annotation, mark the ineffective roof areas on each image, and divide the image dataset into training and test sets. The preset improved U-Net semantic segmentation network model is trained based on the training set to obtain a pre-trained improved U-Net semantic segmentation network model; The test set is input into the pre-trained improved U-Net semantic segmentation network model for validation, resulting in the trained improved U-Net semantic segmentation network model.

[0061] Figure 4This is the second flowchart illustrating the roof usable area identification method based on multi-task deep learning provided in this application embodiment, as shown below. Figure 4 As shown, the method includes the following steps: (1) Obtain RGB satellite images of buildings in the core urban area of ​​a city with a resolution of 0.3m as a dataset of typical Chinese buildings; (2) Labelme software was used to perform secondary annotation on the typical Chinese building dataset to label invalid regions, and the dataset was divided into training set and test set; (3) Based on the PyTorch framework, a pre-defined improved U-Net semantic segmentation network model was constructed; (4) Train the pre-set improved U-Net semantic segmentation network model based on the training set to obtain the pre-trained improved U-Net semantic segmentation network model; (5) Input the test set into the pre-trained improved U-Net semantic segmentation network model for verification, and obtain the trained improved U-Net semantic segmentation network model; (6) The trained improved U-Net semantic segmentation network model is used to predict the test area and extract the effective utilization area of ​​urban building roofs; (7) Eliminate roof areas with too small an area and compactness index of roof connectivity domain.

[0062] It should be noted that the improved U-Net semantic segmentation network model is a multi-task model, including roof area recognition, invalid roof area recognition, and roof edge area recognition tasks. The loss function for these three tasks is the Dice loss. The calculation formula is as follows:

[0063] Where N is the total number of pixels in the image. Let be the true label of the i-th pixel, with target class 1 and background class 0. Let be the predicted probability that the i-th pixel belongs to the target category. This is the loss function.

[0064] The formula for calculating the total loss function is shown below:

[0065] in, The loss function for the roof area identification task. The loss function for the task of identifying invalid areas on rooftops. The loss function for the roof edge region identification task. This is the total loss function.

[0066] Table 1 is a configuration table of training parameters provided in the embodiments of this application. As shown in Table 1, the improved U-Net semantic segmentation network model is trained using the parameter configuration in Table 1. After training, the trained improved U-Net semantic segmentation network model is obtained, and the parameters of the model are saved.

[0067] Table 1. Configuration of training parameters

[0068] It should be noted that the evaluation metrics for the improved U-Net semantic segmentation network model include precision, recall, and IoU score. The calculation formulas are shown below:

[0069]

[0070]

[0071] Where N is the total number of pixels in the image. Let be the true label of the i-th pixel, with target class 1 and background class 0. Let be the probability predicted by the model that the i-th pixel belongs to the target category. For accuracy, For recall rate, Score for IoU.

[0072] Ultimately, the trained improved U-Net semantic segmentation network model achieved a precision of 0.8637, a recall of 0.8479, and an IoU score of 0.7425 on the test set.

[0073] For example, the sampling accuracy was verified for multiple local scenes in a core area of ​​a city. Three areas with a range of 500m×500m were randomly sampled in each scene.

[0074] Table 2. Sampling accuracy verification for multiple local scenes

[0075] Table 2 is a verification table of the sampling accuracy of multiple local scenes provided in the embodiments of this application. As shown in Table 2, sampling verification was carried out in three different scenes: urban blocks, factories, and campuses, to verify the accuracy of the improved U-Net semantic segmentation network model in predicting the effective area of ​​urban rooftops in some core areas of a city. Table 2 shows that the improved U-Net semantic segmentation network model performs best in factory recognition, with an IoU score of 0.77. In urban blocks and campuses, due to the complexity of the scenes, the recognition accuracy is slightly lower, but the IoU scores still exceed 0.7, and the precision and recall also exceed 0.8. This indicates that the improved U-Net semantic segmentation network model has high recognition accuracy, recall, and IoU scores in these scenes, proving that the improved U-Net semantic segmentation network model proposed in this application has good practicality.

[0076] Taking a portion of a city's core urban area as an example, the results were mapped using Python's PyTorch and GDAL libraries, combined with the ArcGIS platform. Figure 5 This is a 0.3m resolution RGB satellite image downloaded from Google Earth in 2023 of a part of the core urban area of ​​a certain city. Figure 6 To improve the U-Net semantic segmentation network model, this paper presents the prediction results of the available rooftop areas in a core urban area of ​​a city in 2023.

[0077] The roof usable area identification method based on multi-task deep learning provided in this application involves acquiring multiple satellite images, cropping and segmenting them, and then inputting each segmented satellite image into a trained improved U-Net semantic segmentation network model to obtain a predicted map of the roof usable area. By adding multi-scale feature extraction and detail head modules, the model's ability to perceive different scales and details is enhanced. By combining roof area identification branches, roof edge area identification branches, and roof invalid area identification branches for multi-task processing, more accurate identification and segmentation of the roof usable area is achieved. In the post-processing stage, the predicted map is optimized based on the roof connected region area and compactness index, effectively improving the identification accuracy and robustness of the roof usable area. Especially in satellite images with complex roof structures and different geographical locations, relatively accurate area segmentation results can be obtained, which can effectively improve the identification accuracy and generalization ability of the roof usable area and enable rapid identification of the roof usable area of ​​buildings in a large urban area.

[0078] The roof usable area identification method based on multi-task deep learning provided in this application can be executed by a roof usable area identification device based on multi-task deep learning. This application uses the roof usable area identification device based on multi-task deep learning executing the roof usable area identification method based on multi-task deep learning as an example to illustrate the roof usable area identification device based on multi-task deep learning provided in this application.

[0079] This application also provides a rooftop usable area identification device based on multi-task deep learning, such as... Figure 7 As shown, the rooftop usable area identification device based on multi-task deep learning includes: an acquisition module 710, a first processing module 720, and a second processing module 730.

[0080] Module 710 is used to acquire multiple satellite images; The first processing module 720 is used to crop and segment each satellite image, and input each segmented satellite image into the trained improved U-Net semantic segmentation network model to obtain the roof usable area prediction result map. The improved U-Net semantic segmentation network model adds a multi-scale feature extraction module and a detail head module on the basis of the U-Net semantic segmentation network model. The improved U-Net semantic segmentation network model includes a roof area recognition branch, a roof edge area recognition branch, and a roof invalid area recognition branch. The second processing module 730 is used to post-process the predicted roof usable area map based on the roof connected domain area and compactness index to obtain the roof usable area corresponding to each satellite image.

[0081] The roof usable area identification device based on multi-task deep learning provided in this application embodiment can achieve... Figures 1 to 6 The various processes implemented using the multi-task deep learning-based roof region recognition method are not described in detail here to avoid repetition.

[0082] In some embodiments, such as Figure 8 As shown, this application embodiment also provides an electronic device 800, including a processor 801, a memory 802, and a computer program stored on the memory 802 and executable on the processor 801. When the program is executed by the processor 801, it implements the various processes of the above-described embodiment of the roof usable area identification method based on multi-task deep learning and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0083] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0084] This application also provides a non-transitory computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the above-described embodiments of the roof usable area identification method based on multi-task deep learning, and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0085] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0086] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method for identifying rooftop usable areas based on multi-task deep learning.

[0087] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0088] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface and the processor are coupled. The processor is used to run programs or instructions to implement the various processes of the above-described embodiments of the roof usable area identification method based on multi-task deep learning, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0089] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a device-level chip, device chip, chip device, or on-chip device chip, etc.

[0090] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0091] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the roof usable area identification method based on multi-task deep learning of the various embodiments of this application.

[0092] In the description of this application, "first feature" and "second feature" may include one or more of the features.

[0093] In the description of this application, "multiple" means two or more.

[0094] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

[0095] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0096] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

Claims

1. A method for identifying rooftop usable areas based on multi-task deep learning, characterized in that, The method includes: Acquire multiple satellite images; Each satellite image is cropped and segmented. The segmented satellite image is then input into a trained improved U-Net semantic segmentation network model to obtain a predicted map of the roof usable area. The improved U-Net semantic segmentation network model adds a multi-scale feature extraction module and a detail head module to the U-Net semantic segmentation network model. The improved U-Net semantic segmentation network model includes a roof area recognition branch, a roof edge area recognition branch, and a roof invalid area recognition branch. Post-processing is performed on the predicted roof usable area based on the roof connectivity area and compactness index to obtain the roof usable area corresponding to each satellite image.

2. The rooftop usable area identification method based on multi-task deep learning according to claim 1, characterized in that, The process involves inputting each segmented satellite image into a trained, improved U-Net semantic segmentation network model to obtain a predicted map of rooftop usable areas, including: Each satellite image after being segmented is input into the roof edge region recognition branch to obtain the roof edge result; Each satellite image after segmentation is input into the roof region recognition branch, and the roof edge results are used as constraints to obtain the roof region segmentation results; Each satellite image after being segmented is input into the roof invalid area identification branch to obtain the roof invalid area segmentation result; Subtracting the invalid roof area segmentation results from the roof area segmentation results yields the predicted map of the usable roof area.

3. The rooftop usable area identification method based on multi-task deep learning according to claim 2, characterized in that, The process of inputting each segmented satellite image into the roof edge region recognition branch to obtain roof edge results includes: Each satellite image after being segmented is sequentially processed through three downsampling modules for feature extraction to obtain feature extraction information; The feature extraction information is input into the multi-scale feature extraction module, and multi-scale feature extraction is performed using 3D convolution to obtain multi-scale information. Multi-scale information is input into the detail head module to extract edge information and obtain the roof edge result.

4. The rooftop usable area identification method based on multi-task deep learning according to claim 3, characterized in that, The process of inputting each segmented satellite image into the roof region recognition branch, and combining the roof edge results as constraints, yields the roof region segmentation result, including: Each satellite image after being sliced ​​is preprocessed to obtain a data-enhanced satellite image. The preprocessing includes random rotation, random flipping, random scaling, and normalization. The augmented satellite images are sequentially processed through five downsampling modules for feature extraction to obtain deep feature information; The deep feature information is sequentially passed through five upsampling modules for detailed reconstruction to obtain the first reconstructed feature map; The first reconstructed feature map is concatenated with multi-scale information and then input into the first convolutional layer to obtain the roof area segmentation result.

5. The rooftop usable area identification method based on multi-task deep learning according to claim 3, characterized in that, The step of inputting each segmented satellite image into the roof invalid area identification branch to obtain the roof invalid area segmentation result includes: Each satellite image after being segmented is sequentially processed through five downsampling modules for feature extraction to obtain deep feature information; After the deep feature information is degraded from gradient tracking, it is sequentially passed through five upsampling modules for detailed reconstruction to obtain the second reconstructed feature map. The results output by the last three upsampling modules are ANDed with the roof area segmentation results to obtain invalid area identification constraints. Based on the invalid region identification constraint, the second reconstructed feature map is input into the second convolutional layer to obtain the invalid region segmentation result of the roof.

6. The rooftop usable area identification method based on multi-task deep learning according to claim 1, characterized in that, The prediction results of the roof usable area based on the roof connected domain area and compactness index are post-processed to obtain the roof usable area corresponding to each satellite image, including: The number of pixels and compactness index of each connected region in the predicted map of the roof's usable area are calculated. The formula for calculating the compactness index is as follows: Where A is the area of ​​the connected region, and P is the perimeter. Compactness index; Determine whether the number of pixels in each connected component is less than a first preset threshold. If the number of pixels is less than the first preset threshold, determine whether the compactness index is less than a second preset threshold. If the compactness index is less than the second preset threshold, remove the connected components with both the number of pixels less than the first preset threshold and the compactness index less than the second preset threshold to obtain the rooftop usable area corresponding to each satellite image.

7. The rooftop usable area identification method based on multi-task deep learning according to claim 1, characterized in that, The training process of the improved U-Net semantic segmentation network model includes: Construct an improved U-Net semantic segmentation network model; We acquire an image dataset from the architectural field, perform secondary annotation, mark the ineffective roof areas on each image, and divide the image dataset into training and test sets. The preset improved U-Net semantic segmentation network model is trained based on the training set to obtain a pre-trained improved U-Net semantic segmentation network model; The test set is input into the pre-trained improved U-Net semantic segmentation network model for validation, resulting in the trained improved U-Net semantic segmentation network model.

8. A roof usable area identification device based on multi-task deep learning, implemented using the roof usable area identification method based on multi-task deep learning as described in any one of claims 1 to 7, characterized in that, The device includes: The acquisition module is used to acquire multiple satellite images; The first processing module is used to crop and segment each satellite image, and input each segmented satellite image into the trained improved U-Net semantic segmentation network model to obtain the predicted result map of the roof usable area. The improved U-Net semantic segmentation network model adds a multi-scale feature extraction module and a detail head module to the U-Net semantic segmentation network model. The improved U-Net semantic segmentation network model includes a roof area recognition branch, a roof edge area recognition branch, and a roof invalid area recognition branch. The second processing module is used to post-process the predicted roof usable area map based on the roof connectivity area and compactness index to obtain the roof usable area corresponding to each satellite image.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the roof usability area identification method based on multi-task deep learning as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the roof usability area identification method based on multi-task deep learning as described in any one of claims 1 to 7.

Images