A roof style recognition method based on depth map and deep learning

Abstract

The application provides a roof style recognition method based on a depth map and deep learning, and the recognition method comprises the following steps: training a building roof style type recognition deep learning model; and performing building roof style type recognition according to the building roof style type recognition deep learning model. In the deep learning building roof style recognition based on the depth map, a random rotation technology is used to enhance the stability and accuracy of the model.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of depth map instead of profile map of buildings, and particularly relates to a roof style recognition method based on depth map and deep learning. BACKGROUND

[0002] Roof style plays an important role in the protection of historical buildings and the construction of real three-dimensional, and is the key information for rapid three-dimensional modeling of buildings. According to the roof style, the location and height information of the building can be combined to better and faster construct the building white model of the city. With the promotion of real three-dimensional construction, there are a large amount of three-dimensional data, such as city mesh model, DSM (Digital Surface Model), single model or point cloud. However, how to effectively use these three-dimensional data to mine the roof style information of the building is still in its infancy.

[0003] There are three main methods for obtaining the type of building roof style. The first method is manual discrimination. The manual observation of the building on site or through three-dimensional data visualization software can visually determine the type of roof style. Although this method has high accuracy, it requires a large amount of manual labor, is low in efficiency and slow in speed. The second method is key point rule matching. A series of three-dimensional sample libraries of building roof styles are constructed. The three-dimensional data of the building to be determined is matched with each sample of the sample library. Compared with manual discrimination, this method has greatly improved efficiency, but the accuracy is low and cannot meet the production demand. The third method is a roof style recognition method based on building profile map deep learning (publication number: CN113902720A). This method first obtains the profile map of the building from the three-dimensional data using a profile line, and then uses a deep learning model to classify the profile map to determine the roof style of the building. This scheme has poor stability. The profile map changes with the profile line. The same roof style may have different profile maps due to different profile lines, which increases the difficulty of roof style recognition. The data is not fully utilized. The use of profile map form causes a large amount of loss of three-dimensional data information, which is not conducive to the extraction of features by the deep learning model for roof style recognition. SUMMARY

[0004] In view of the above problems, the present application is proposed to provide a roof style recognition method based on depth map and deep learning to overcome the above problems or at least partially solve the above problems.

[0005] According to one aspect of the present application, a roof style recognition method based on depth map and deep learning is provided, which comprises:

[0006] training a building roof style type recognition deep learning model;

[0007] According to the building roof style type identification deep learning model, building roof style type identification is performed.

[0008] Optionally, the training building roof style type identification deep learning model specifically comprises:

[0009] Prepare each building vector range in the training area and model data in the training area as basic input data;

[0010] For each building: according to the vector range of each building, calculate the circumscribed rectangle, and according to the circumscribed rectangle, obtain the depth map and orthographic map containing the building roof structure based on the model data, repeat the steps through traversal operation, and make the training data of the training area;

[0011] Formulate roof style type classification and visual judgment criteria;

[0012] Based on the training data, construct a building roof style type identification training data set;

[0013] Use a deep learning library to construct a building roof style type identification deep learning model;

[0014] Use the building roof style type identification training data set to train and optimize the building roof style type identification deep learning model.

[0015] Optionally, the model data is any one of a Mesh model, a DSM (Digital Surface Model), a monomerized model, or point cloud three-dimensional data.

[0016] Optionally, the building roof style type identification deep learning model adopts a ResNet or VGG deep learning model.

[0017] Optionally, the building roof style type identification deep learning model specifically comprises:

[0018] In a three-dimensional scene, load three-dimensional data, set a virtual camera in the three-dimensional scene according to the circumscribed rectangle, the main optical axis direction of the virtual camera is the -Z direction of the three-dimensional scene, the viewport of the virtual camera is consistent with the circumscribed rectangle, render the three-dimensional scene, obtain the depth buffer of the virtual camera, and obtain a depth map after normalization;

[0019] Obtain color information of the virtual camera to obtain an orthographic map.

[0020] Optionally, the building roof style type recognition training data set is constructed based on the training data, and specifically includes:

[0021] According to the orthographic map, artificial visual judgment is performed, the roof style type of each building is marked as a label for training of the building roof style type recognition deep learning model according to the classification and the visual judgment standard.

[0022] Each depth map is resampled to 2 n *2 n pixel size as input data of the building roof style type recognition deep learning model.

[0023] Optionally, the building roof style type recognition deep learning model is trained by using the building roof style type recognition training data set, and specifically includes:

[0024] The rotation operation is used to perform training data enhancement, expand the training sample data amount, and ensure the stability and accuracy of the model in recognizing the roof style type at different angles.

[0025] The cross-entropy function is used to measure the loss value between the predicted roof style type and the label roof style type, the building roof style type recognition deep learning model parameters are iteratively optimized until the loss value converges, and the model is saved.

[0026] Optionally, the building roof style type recognition is performed according to the building roof style type recognition deep learning model, and specifically includes:

[0027] Prepare the outer rectangular range of each building in the to-be-recognized area and the model data of the to-be-recognized area as basic input data.

[0028] For each building, calculate the circumscribed rectangle according to the vector range of each building, and obtain the depth map containing the building roof structure based on the model data according to the circumscribed rectangle.

[0029] Each depth map of the to-be-recognized area is normalized and resampled to 512*512 size as input of the building roof style type recognition deep learning model.

[0030] The processed building roof depth map of the to-be-recognized area is input into the trained building roof style type recognition deep learning model to recognize the building roof style type, and the recognition result is output and saved.

[0031] The building roof style type recognition deep learning model is repeatedly used to recognize each building to complete the building roof style recognition of the to-be-recognized area.

[0032] The application provides a roof style recognition method based on a depth map and deep learning, and the recognition method comprises the following steps: training a building roof style type recognition deep learning model; and performing building roof style type recognition according to the building roof style type recognition deep learning model.

[0033] The above description is only a summary of the technical scheme of the application, in order to more clearly understand the technical means of the application, and to be implemented according to the content of the specification, and in order to make the above and other purposes, characteristics and advantages of the application more obvious and easy to understand, the following specific embodiments of the application are described. BRIEF DESCRIPTION OF DRAWINGS

[0034] In order to more clearly illustrate the technical scheme of the embodiments of the application, the following will briefly introduce the drawings needed to be used in the embodiment description, and obviously, the drawings in the following description are only some embodiments of the application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0035] Figure 1 The flow chart of training the building roof style type recognition deep learning model provided by the embodiment of the application;

[0036] Figure 2 The method flow chart of performing building roof style type recognition according to the building roof style type recognition deep learning model provided by the embodiment of the application. DETAILED DESCRIPTION

[0037] The exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided so that the present disclosure can be more thoroughly understood and the scope of the present disclosure can be accurately conveyed to those skilled in the art.

[0038] The terms "include" and "have" and any variations thereof in the specification embodiments of the application and claims and drawings are intended to cover non-exclusive inclusion, for example, including a series of steps or units.

[0039] The technical scheme of the application will be further described in detail below in combination with the drawings and embodiments.

[0040] As shown in Figure 1 and Figure 2 A roof style recognition method based on a depth map and deep learning, comprising:

[0041] 1. Training a deep learning model for building roof style type recognition.

[0042] 1.1. Preparing the training area, each building vector range and one of the three-dimensional data such as mesh model, DSM, single model or point cloud in the training area as the basic input data.

[0043] The training area refers to the area that provides training data for the deep learning model for building roof style type recognition.

[0044] 1.2. Through programming or using existing software, for each building: calculate its bounding rectangle through its vector range, and based on one of the three-dimensional data such as mesh model, DSM, single model or point cloud, obtain the depth map (see 1.2.1) and orthographic map (see 1.2.2) containing the building roof structure according to the bounding rectangle, repeat the above steps through traversal operation to complete the production of training data for the entire training area.

[0045] 1.2.1 Depth map acquisition: in the three-dimensional scene, load the three-dimensional data in 1.2, and set the virtual camera in the three-dimensional scene according to the bounding rectangle calculated in 1.2, the principal axis direction of the virtual camera is the -Z direction of the three-dimensional scene, the viewport of the virtual camera is consistent with the bounding rectangle, at this time the three-dimensional scene is rendered, and the depth buffer of the virtual camera is obtained, after normalization, the orthographic map is obtained.

[0046] 1.2.2 Orthographic map acquisition: in the three-dimensional scene, load the three-dimensional data in 1.2, and set the virtual camera in the three-dimensional scene according to the bounding rectangle calculated in 1.2, the principal axis direction of the virtual camera is the -Z direction of the three-dimensional scene, the viewport of the virtual camera is consistent with the bounding rectangle, at this time the three-dimensional scene is rendered, at this time the color information of the virtual camera is obtained, and the orthographic map is obtained.

[0047] 1.3. Formulate the classification and visual judgment standard of roof style type.

[0048] 1.4. Based on the training data obtained in 1.2, construct the building roof style type recognition training data set.

[0049] 1.4.1. Artificial visual discrimination according to the orthographic map, and mark the roof style type of each building according to the classification and visual judgment standard in 1.3, as the label for training the deep learning model for building roof style type recognition.

[0050] 1.4.2. Resample each depth map to 2nX2n pixel size, for example: 512*512, as the input data of the deep learning model for building roof style type recognition.

[0051] 1.5 Use pytorch or other deep learning libraries to build a building roof style type recognition deep learning model. The model can use deep learning models such as ResNet or VGG.

[0052] 1.6 Use the building roof style type recognition training dataset to train the optimized building roof style type recognition deep learning model.

[0053] 1.6.1 Use rotation operation to do data augmentation, expand the training sample data volume, and ensure the stability and accuracy of the model in identifying roof style types at different angles.

[0054] 1.6.2 Use cross-entropy function to measure the loss value between predicted roof style type and labeled roof style type, iteratively optimize the parameters of building roof style type recognition deep learning model until the loss value converges, and save the model.

[0055] 2 Use the building roof style type recognition deep learning model to identify the building roof style type.

[0056] 2.1 Prepare one of the following three-dimensional data in each building's outer rectangular range and the Mesh model, DSM, single model or point cloud of the area to be identified as the basic input data.

[0057] The area to be identified refers to the area that needs to apply the building roof style type recognition deep learning model to identify the building roof style type.

[0058] 2.2 By writing programs or using existing software, for each building: calculate its circumscribed rectangle through its vector range, and based on one of the three-dimensional data such as Mesh, DSM or point cloud, obtain the depth map containing the building roof structure (see 1.2.1) based on the circumscribed rectangle.

[0059] 2.3 Normalize and resample each depth map of the area to be identified into 512*512 size as the input of the building roof style type recognition deep learning model.

[0060] 2.4 Input the processed building roof depth map of the area to be identified into the building roof style type recognition deep learning model trained in 1.6.2 to identify the building roof style type, and save the identification result.

[0061] 2.5 Repeat steps 2.3 and 2.4 for each building to complete the building roof style identification of the area to be identified.

[0062] Beneficial effects: In the present application, the depth map of the building is used instead of the profile to serve as the input of the deep learning model, the depth map contains more three-dimensional information, and the three-dimensional data (Mesh model, DSM, individualized model or point cloud, etc.) is more fully utilized; the depth map is randomly rotated during the training process, ensuring the stability and accuracy of the roof style recognition deep learning model in identifying the roof style type at different angles, and the type of the building roof style can be more quickly, efficiently, stably and accurately identified.

[0063] The above detailed description of the embodiments of the present application further explains the purposes, technical solutions and beneficial effects of the present application. It should be understood that the above is only a specific embodiment of the present application and is not used to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A roof style recognition method based on depth maps and deep learning, characterized in that, The identification method includes: Training a deep learning model for recognizing building roof style types includes: Prepare the vector range of each building in the training area and the model data within the training area as the basic input data; For each building: Calculate the bounding rectangle based on the vector range of each building; based on the model data and the bounding rectangle, obtain the depth map and orthophoto map containing the building's roof structure; repeat the steps through the traversal operation to create training data for the training area, including: In a 3D scene, 3D data is loaded, and a virtual camera in the 3D scene is set according to the bounding rectangle. The main optical axis of the virtual camera is the -Z direction of the 3D scene, and the viewport of the virtual camera is consistent with the bounding rectangle. The 3D scene is rendered, and the depth cache of the virtual camera is obtained. After normalization, a depth map is obtained. Obtain the color information of the virtual camera to obtain an orthophoto; Establish classification and visual judgment criteria for roof style types; Based on the training data, a training dataset for recognizing building roof style types was constructed. A deep learning model for identifying building roof style types was constructed using a deep learning library. The deep learning model for recognizing building roof style types is trained and optimized using the aforementioned building roof style type recognition training dataset. The building roof style type is identified using a deep learning model for building roof style type recognition.

2. The roof style recognition method based on depth maps and deep learning according to claim 1, characterized in that, The model data can be any one of a Mesh model, a DSM, a monolithic model, or point cloud 3D data.

3. The roof style recognition method based on depth maps and deep learning according to claim 1, characterized in that, The deep learning model for identifying building roof style types uses either ResNet or VGG deep learning models.

4. The roof style recognition method based on depth maps and deep learning according to claim 1, characterized in that, The construction of the building roof style type recognition training dataset based on training data specifically includes: Based on the orthophoto, manual visual judgment is performed, and the roof style type of each building is marked according to the classification and visual judgment criteria. This mark serves as a label for training the deep learning model for building roof style type recognition. Each depth map is resampled into 2 n *2 n The pixel size is used as input data for the deep learning model for recognizing building roof style types.

5. The roof style recognition method based on depth maps and deep learning according to claim 1, characterized in that, The specific steps of using the building roof style type recognition training dataset to train and optimize the building roof style type recognition deep learning model include: Rotation operations are used to augment the training data, expand the amount of training sample data, and ensure the stability and accuracy of the model in recognizing roof style types from different angles. The cross-entropy function is used to measure the loss between the predicted roof style type and the labeled roof style type. The parameters of the deep learning model for building roof style type recognition are iteratively optimized until the loss value converges, and then the model is saved.

6. The roof style recognition method based on depth maps and deep learning according to claim 1, characterized in that, The specific steps of identifying the building roof style type using the deep learning model for building roof style type recognition include: Prepare the rectangular range of the outer perimeter of each building in the area to be identified and the model data of the area to be identified as the basic input data; For each building: Calculate the bounding rectangle based on the vector range of each building, and obtain a depth map containing the building's roof structure based on the model data and this bounding rectangle; Each depth map of the area to be identified is normalized and resampled to a size of 512*512 as input to the deep learning model for identifying building roof style types; The processed building roof depth map of the area to be identified is input into the trained deep learning model for building roof style type recognition, and the building roof style type is identified. The recognition results are then output and saved. For each building, the deep learning model for roof style type recognition is repeatedly applied to complete the roof style recognition of the area to be identified.

Images