Methods, devices, electronic equipment and storage media for obtaining floor information

Abstract

This application provides a method, apparatus, electronic device, and computer-readable storage medium for acquiring floor information. The method includes: acquiring a state image of a building to be detected; performing deep convolutional feature extraction on the state image to obtain floor association features of the building; and performing regression prediction processing based on the floor association features to obtain the floor information of the building. This application can accurately predict the floor information of a building, avoiding the high data acquisition cost caused by the need for on-site entry into the building to obtain accurate floor information.

Description

Technical Field

[0001] This application relates to the field of floor information recognition technology, specifically to a method, apparatus, electronic device, and computer-readable storage medium for obtaining floor information. Background Technology

[0002] Building floor information is one of the important attributes of a building. For map platforms, building floor information can assist in the rendering of 3D maps; urban planning can use building floor information to analyze the rationality of the city skyline; and express delivery and logistics companies also need building floor information to judge the difficulty of delivery.

[0003] However, the building floor information on existing maps only exists in the central areas of some major cities because accurate floor information can only be obtained by physically entering the buildings, which is costly. Summary of the Invention

[0004] This application provides a method, apparatus, electronic device, and computer-readable storage medium for obtaining floor information, aiming to solve the problem of high cost in obtaining building floor information.

[0005] Firstly, this application provides a method for obtaining floor information, the method comprising:

[0006] Acquire the state image of the building to be detected;

[0007] Deep convolution feature extraction is performed on the state image to obtain the floor association features of the building to be detected;

[0008] Regression prediction is performed based on the floor association features to obtain the floor information of the building to be detected.

[0009] In one possible implementation of this application, the floor association features include at least one of the following: the plot building layout information of the building to be detected, the building spacing, the building shadow area, and the building slope length. The step of performing deep convolutional feature extraction on the state image to obtain the floor association features of the building to be detected includes:

[0010] Pixel feature extraction is performed on the state image to obtain the pixel information of the state image;

[0011] A pre-trained floor prediction network is invoked to perform deep convolutional feature extraction based on the pixel information, thereby obtaining at least one of the following information: the plot building layout information, building spacing, building shadow area, and building slope length of the building to be detected. The floor prediction network contains multiple fully connected layers.

[0012] In one possible implementation of this application, the step of performing regression prediction processing based on the floor association features to obtain the floor information of the building to be detected includes:

[0013] The floor prediction network is invoked to perform regression prediction processing based on at least one of the following information: the plot building layout information, building spacing, building shadow area, and building slope length of the building to be detected, so as to obtain the floor information of the building to be detected.

[0014] In one possible implementation of this application, the step of performing deep convolutional feature extraction on the state image to obtain the floor association features of the building to be detected further includes:

[0015] Obtain a training dataset, wherein the training dataset includes sample satellite images of sample buildings, and the sample satellite images are labeled with the actual floor information of the sample buildings;

[0016] The floor prediction network is obtained by training a preset deep regression network based on the training dataset.

[0017] In one possible implementation of this application, training a preset deep regression network based on the training dataset to obtain the floor prediction network includes:

[0018] The deep regression network is invoked to perform deep convolutional feature extraction based on the sample satellite images to obtain sample features of the sample satellite images; and regression is performed based on the sample features to obtain the floor prediction information of the sample buildings;

[0019] Based on the actual floor information and the predicted floor information, the training loss value of the deep regression network is determined;

[0020] The model parameters of the deep regression network are adjusted according to the training loss value until the deep regression network converges, thus obtaining the floor prediction network.

[0021] In one possible implementation of this application, the step of acquiring the state image of the building to be detected further includes:

[0022] Obtain a building database, wherein the building database includes satellite images of each building among multiple buildings, and the center point coordinates of each building, wherein the satellite images of each building and the center point coordinates of each building are associated and recorded.

[0023] The process of acquiring the state image of the building to be detected includes:

[0024] Obtain the coordinates of the target center point of the building to be inspected;

[0025] From the building database, obtain satellite images corresponding to the coordinates of the target center point to serve as the status images.

[0026] In one possible implementation of this application, the step of performing regression prediction processing based on the floor association features to obtain the floor information of the building to be detected further includes:

[0027] Obtain the building block outlines and identification marks of the building to be detected;

[0028] Based on the identification mark, the building block outline and the floor information are associated to obtain the target building block outline of the building to be detected, wherein the target building block outline is used to construct a map.

[0029] Secondly, this application provides a floor information acquisition device, the floor information acquisition device comprising:

[0030] The acquisition unit is used to acquire a state image of the building to be detected.

[0031] The extraction unit is used to perform deep convolutional feature extraction on the state image to obtain the floor association features of the building to be detected;

[0032] The prediction unit is used to perform regression prediction processing based on the floor association features to obtain the floor information of the building to be detected.

[0033] In one possible implementation of this application, the floor association features include at least one of the following: plot building layout information of the building to be detected, building spacing, building shadow area, and building slope length; the extraction unit is specifically used for:

[0034] Pixel feature extraction is performed on the state image to obtain the pixel information of the state image;

[0035] A pre-trained floor prediction network is invoked to perform deep convolutional feature extraction based on the pixel information, thereby obtaining at least one of the following information: the plot building layout information, building spacing, building shadow area, and building slope length of the building to be detected. The floor prediction network contains multiple fully connected layers.

[0036] In one possible implementation of this application, the prediction unit is specifically used for:

[0037] The floor prediction network is invoked to perform regression prediction processing based on at least one of the following information: the plot building layout information, building spacing, building shadow area, and building slope length of the building to be detected, so as to obtain the floor information of the building to be detected.

[0038] In one possible implementation of this application, the floor information acquisition device further includes a training unit. Before the step of performing deep convolutional feature extraction on the state image to obtain the floor association features of the building to be detected, the training unit is specifically used for:

[0039] Obtain a training dataset, wherein the training dataset includes sample satellite images of sample buildings, and the sample satellite images are labeled with the actual floor information of the sample buildings;

[0040] The floor prediction network is obtained by training a preset deep regression network based on the training dataset.

[0041] In one possible implementation of this application, the training unit is specifically used for:

[0042] The deep regression network is invoked to perform deep convolutional feature extraction based on the sample satellite images to obtain sample features of the sample satellite images; and regression is performed based on the sample features to obtain the floor prediction information of the sample buildings;

[0043] Based on the actual floor information and the predicted floor information, the training loss value of the deep regression network is determined;

[0044] The model parameters of the deep regression network are adjusted according to the training loss value until the deep regression network converges, thus obtaining the floor prediction network.

[0045] In one possible implementation of this application, the acquisition unit is specifically used for:

[0046] Obtain a building database, wherein the building database includes satellite images of each building among multiple buildings, and the center point coordinates of each building, wherein the satellite images of each building and the center point coordinates of each building are associated and recorded.

[0047] Obtain the coordinates of the target center point of the building to be inspected;

[0048] From the building database, obtain satellite images corresponding to the coordinates of the target center point to serve as the status images.

[0049] In one possible implementation of this application, the floor information acquisition device further includes an association unit. After the step of performing regression prediction processing based on the floor association features to obtain the floor information of the building to be detected, the association unit is specifically used for:

[0050] Obtain the building block outlines and identification marks of the building to be detected;

[0051] Based on the identification mark, the building block outline and the floor information are associated to obtain the target building block outline of the building to be detected, wherein the target building block outline is used to construct a map.

[0052] Thirdly, this application also provides an electronic device, which includes a processor and a memory, wherein the memory stores a computer program, and when the processor calls the computer program in the memory, it executes the steps in any of the floor information acquisition methods provided in this application.

[0053] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, the computer program being loaded by a processor to execute the steps in the floor information acquisition method.

[0054] This application extracts floor association features from the state image of the building to be detected through deep convolutional features, thereby obtaining floor association features that reflect the floor information of the building. Based on the floor association features, regression prediction is performed to obtain the floor information of the building. Since floor association features that reflect the floor height of the building can be extracted from the image, such as the building's plot layout, building spacing, building shadow area, and building slope length, the floor information of the building can be accurately predicted. This avoids the problem of high data acquisition costs caused by the need to enter the building on-site to obtain accurate floor information. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0056] Figure 1 This is a schematic diagram of a scenario for the floor information acquisition system provided in an embodiment of this application;

[0057] Figure 2 This is a flowchart illustrating a method for obtaining floor information provided in an embodiment of this application;

[0058] Figure 3 This is a schematic flowchart of an embodiment of step 202 provided in this application;

[0059] Figure 4 This is a schematic diagram of an embodiment of the functional structure of the deep regression network provided in this application;

[0060] Figure 5This is a schematic diagram of the model training and floor information prediction process provided in the embodiments of this application;

[0061] Figure 6 This is a schematic diagram of an embodiment of the floor information acquisition device provided in this application.

[0062] Figure 7 This is a schematic diagram of an embodiment of the electronic device provided in this application. Detailed Implementation

[0063] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0064] In the description of the embodiments of the present application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0065] To enable any person skilled in the art to implement and use this application, the following description is provided. In this description, details are set forth for purposes of explanation. It should be understood that those skilled in the art will recognize that this application can be implemented without using these specific details. In other instances, well-known processes will not be described in detail to avoid obscuring the description of the embodiments of this application with unnecessary detail. Therefore, this application is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in the embodiments of this application.

[0066] The execution subject of the floor information acquisition method in this application embodiment can be the floor information acquisition device provided in this application embodiment, or different types of electronic devices such as server equipment, physical host, or user equipment (UE) that integrate the floor information acquisition device. The floor information acquisition device can be implemented in hardware or software. The UE can be a terminal device such as a smartphone, tablet computer, laptop computer, handheld computer, desktop computer, or personal digital assistant (PDA).

[0067] The electronic device can operate independently or in a cluster. By applying the floor information acquisition method provided in this application, the floor information of a building can be accurately predicted, avoiding the problem of high data acquisition costs caused by the need to enter the building in person to obtain accurate floor information.

[0068] See Figure 1 , Figure 1 This is a schematic diagram of a floor information acquisition system provided in an embodiment of this application. The floor information acquisition system may include an electronic device 100, which integrates a floor information acquisition device. For example, the electronic device can acquire a state image of a building to be detected; perform deep convolutional feature extraction on the state image to obtain floor association features of the building to be detected; and perform regression prediction processing based on the floor association features to obtain the floor information of the building to be detected.

[0069] In addition, such as Figure 1 As shown, the floor information acquisition system may also include a memory 200 for storing data, such as image data and video data.

[0070] It should be noted that, Figure 1 The schematic diagram of the floor information acquisition system shown is merely an example. The floor information acquisition system and scenario described in this application are for the purpose of more clearly illustrating the technical solutions of this application, and do not constitute a limitation on the technical solutions provided in this application. As those skilled in the art will know, with the evolution of floor information acquisition systems and the emergence of new business scenarios, the technical solutions provided in this invention are also applicable to similar technical problems.

[0071] The following describes the method for obtaining floor information provided in the embodiments of this application. In this embodiment, an electronic device is used as the execution subject. For the sake of simplicity and ease of description, the execution subject will be omitted in the subsequent method embodiments. The method for obtaining floor information includes: obtaining a state image of a building to be detected; performing deep convolution feature extraction on the state image to obtain the floor association features of the building to be detected; and performing regression prediction processing based on the floor association features to obtain the floor information of the building to be detected.

[0072] Reference Figure 2 , Figure 2 This is a flowchart illustrating a method for obtaining floor information provided in an embodiment of this application. It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown here. The method for obtaining floor information includes steps 201 to 203, wherein:

[0073] 201. Obtain the status image of the building to be detected.

[0074] In this context, the state image refers to the image of the building to be detected. Specifically, the state image can be a panoramic view, an overhead view, an upward view, etc., of the building to be detected.

[0075] In some embodiments, images of the building to be inspected can be captured by aerial photography using a camera, serving as a status image of the building. Specifically, in practical applications, the electronic device provided in this application embodiment can directly include a camera (primarily used for aerial photography of the building to be inspected) in its hardware, and store the images captured by the camera locally, allowing direct retrieval within the electronic device; alternatively, the electronic device can establish a network connection with the camera and acquire the images obtained by the camera online based on the network connection; or, the electronic device can read the images captured by the camera from a related storage medium storing the images captured by the camera. The specific acquisition method is not limited here.

[0076] The camera can capture images according to preset shooting methods, such as setting the shooting height, shooting direction, or shooting distance. The specific shooting method can be adjusted according to the camera itself and is not limited here. Multiple frames captured by the camera can be combined into a video through a timeline.

[0077] In some embodiments, satellite images of the building to be detected can be acquired as status images. Specifically, before the step of acquiring the status image of the building to be detected, the method further includes: acquiring a building database, wherein the building database includes satellite images of each of a plurality of buildings, and the center point coordinates of each of the buildings, wherein the satellite images of each building are associated with the center point coordinates of each building.

[0078] For example, satellite images provided by third-party platforms (such as publicly available sub-meter resolution satellite imagery data from Google Earth) can be segmented according to each building to obtain satellite images for each building. The coordinates of the center point of the spatial projection (i.e., the building's vertical projection onto the ground) of each building can then be obtained, and the satellite images and the center point coordinates of the spatial projection of each building can be linked and recorded to form a building database.

[0079] At this point, the step of obtaining the state image of the building to be detected may specifically include: obtaining the coordinates of the target center point of the building to be detected; and obtaining the satellite image corresponding to the coordinates of the target center point from the building database as the state image.

[0080] The target center point coordinates refer to the center point coordinates of the spatial projection of the building to be inspected.

[0081] Specifically, the coordinates of a center point that is the same as the target center point coordinates are found in the building database, and satellite images associated with the center point coordinates are obtained as the status images of the building to be detected.

[0082] For example, the building database records satellite images of buildings as shown in Table 1. If the target center point coordinates are (10, 10), then satellite slices can be obtained. Figure 1 As a state image of the building to be detected.

[0083]

[0084]

[0085] By linking the center point coordinates with satellite imagery, the satellite imagery corresponding to each building can be quickly located. Furthermore, when building maps based on these buildings, the center point coordinates can indicate the building's location on the map.

[0086] 202. Perform deep convolution feature extraction on the state image to obtain the floor association features of the building to be detected.

[0087] Among them, floor-related features refer to information that can reflect the floor height, such as plot building layout information, building spacing, building shadow area, building slope length, etc.

[0088] Here, the information on the layout of buildings on the plot, the spacing between buildings, the area of ​​building shadows, and the length of building slopes are just examples of floor association features. Specifically, the floor association features obtained after deep convolution feature extraction of the state image in step 202 are not limited to this example. The floor association features may include more or fewer features than those given here.

[0089] Building layout information refers to the spatial distribution of buildings within the same area of ​​interest (AOI). Typically, buildings within an AOI have similar heights or meet the requirement of lower buildings on the south side and higher buildings on the north side for better lighting.

[0090] Building spacing refers to the spatial distance between the building being tested and its surrounding buildings. When the building does not cross a road, its height is directly proportional to its distance from surrounding buildings.

[0091] The area of ​​a building's shadow refers to the size of the shadow cast by the building in a spatial coordinate system when the sun shines obliquely during satellite imagery. The solar altitude angle can be deduced from the building's spatial coordinates and the image acquisition time, thus allowing the calculation of the building's height.

[0092] The building slope length refers to the length by which ground features not located on the optical axis of the satellite image tilt towards the optical axis as their height increases due to non-orthophoto projection. Because ground features not located on the optical axis tilt towards the optical axis as their height increases due to non-orthophoto projection, the height of a building can be calculated by tracing its interior orientation elements in the image.

[0093] To accurately predict the floor information of a building based on its image, it is necessary to extract features that reflect this information. These features can include various elements such as the building layout on the site, building spacing, building shadow area, and building slope length. Therefore, this embodiment employs a deep convolutional neural network (CNN) for feature extraction.

[0094] In some embodiments, the floor association features are hidden features that can be automatically extracted and represented by a deep convolutional neural network. The specific extraction process will be described in detail in steps 301-302 below, and will not be repeated here for the sake of simplicity.

[0095] In some embodiments, the floor association features are specific features that can be obtained by classifying or regressing the state image using a CNN-based network model.

[0096] For example, in the first aspect, the building layout information of the land parcel can be divided into several categories, such as category A, category B, and category C. By collecting sample images, a first classification network used to classify the building layout information of the land parcel is trained, so that the first classification network can extract the convolutional features of the image based on the image of the building; and classify the land parcel building layout information of the building based on the convolutional features of the image to determine whether the building is a category A, category B, or category C.

[0097] Secondly, building spacing can be divided into several categories, such as maximum spacing, large spacing, normal spacing, and small spacing. By collecting sample images, a second classification network for classifying building spacing is trained, so that the second classification network can extract the convolutional features of the image based on the image of the building; and classify the building spacing based on the convolutional features of the image to determine whether the building is a maximum spacing, large spacing, normal spacing, or small spacing.

[0098] Thirdly, sample images can be collected to train a first semantic segmentation network for predicting the building shadow area, so that the first semantic segmentation network can extract the convolutional features of the image based on the building image; and predict the building shadow area based on the convolutional features of the image to determine the building shadow area.

[0099] Fourthly, sample images can be collected to train a second semantic segmentation network for predicting the length of a building's slope. This allows the second semantic segmentation network to extract convolutional features from the building's image and distinguish between the building's top surface and slope based on these convolutional features, thereby determining the length of the building's slope.

[0100] Then, a first classification network is used to extract convolutional features from the state image to obtain the convolutional features of the state image; and classification is performed based on the convolutional features of the state image to obtain the plot building layout information of the building to be detected; a second classification network is used to extract convolutional features from the state image to obtain the convolutional features of the state image; and classification is performed based on the convolutional features of the state image to obtain the building spacing of the building to be detected; a first semantic segmentation network is used to extract convolutional features from the state image to obtain the convolutional features of the state image; and regression is performed based on the convolutional features of the state image to obtain the building shadow area of ​​the building to be detected; a second semantic segmentation network is used to extract convolutional features from the state image to obtain the convolutional features of the state image; and regression is performed based on the convolutional features of the state image to obtain the building slope length of the building to be detected.

[0101] 203. Perform regression prediction processing based on the floor association features to obtain the floor information of the building to be detected.

[0102] In some embodiments, the floor association features are specific features. In this case, step 203 may specifically include: representing the information such as the plot building layout, building spacing, building shadow area, and building slope length of the building to be detected through specific feature vectors; and regressing the floor information of the building to be detected based on the specific feature vectors representing the plot building layout, building spacing, building shadow area, and building slope length through a trained regression model.

[0103] In some embodiments, steps 202 and 203 can be implemented by a trained floor prediction network, the specific implementation of which is described below.

[0104] In this embodiment, deep convolutional feature extraction is performed on the state image of the building to be detected to obtain floor association features that reflect the floor information of the building to be detected; regression prediction processing is performed based on the floor association features to obtain the floor information of the building to be detected; since floor association features that reflect the floor height of the building can be extracted from the image, such as the building's plot layout information, building spacing, building shadow area, and building slope length, the floor information of the building can be accurately predicted, avoiding the problem of high data acquisition costs caused by the need to enter the building on-site to obtain accurate floor information.

[0105] like Figure 3 As shown, step 202 may specifically include steps 301 to 302:

[0106] 301. Extract pixel features from the state image to obtain pixel information of the state image.

[0107] like Figure 4 As shown, in some embodiments, each pixel in the state image can be characterized sequentially to obtain the pixel information of the state image.

[0108] In some embodiments, feature extraction of pixels in the state image can also be performed using two-dimensional convolution to obtain pixel information representing the spatial features of pixels in the state image.

[0109] 302. Call the pre-trained floor prediction network and perform deep convolutional feature extraction based on the pixel information to obtain at least one of the following information: the plot building layout information, building spacing, building shadow area, and building slope length of the building to be detected. The floor prediction network contains multiple fully connected layers.

[0110] Among them, the floor prediction network is a regression model with a deep convolutional neural network.

[0111] For example, the floor prediction network trained using steps a-b and b1-b3 below can be used to perform deep convolutional feature extraction based on pixel information to obtain at least one of the following information about the building plot layout, building spacing, building shadow area, and building slope length of the building to be detected. The building plot layout, building spacing, building shadow area, and building slope length are extracted through the hidden layers of the floor prediction network and are hidden features reflecting the building height.

[0112] Step 203 may specifically include: calling the floor prediction network and performing regression prediction processing based on at least one of the following information of the building plot layout, building spacing, building shadow area, and building slope length of the building to be detected, to obtain the floor information of the building to be detected.

[0113] For example, such as Figure 4 As shown, in order to avoid excessive loss due to a sharp decrease in feature dimension, the floor prediction network in this embodiment uses multiple consecutive fully connected layers (such as three fully connected layers) to successively reduce the feature dimension of the image. The m neurons of the last layer of the multiple fully connected layers each output a scalar, resulting in m scalars. All m scalars are connected to a neuron as output, and finally the building floor information is output.

[0114] Each scalar is used to represent different hidden features. For example, the last layer outputs four scalars from four neurons, which are used to represent the plot building layout information, building spacing, shadow area, and slope length of the building to be detected.

[0115] Since building height can be reflected by a series of hidden features such as plot building layout information, building spacing, building shadow area, and building slope length, and convolutional neural networks have excellent automatic feature extraction capabilities, the prediction accuracy of floor information can be improved by automatically extracting image features reflecting building height through deep convolutional neural networks in deep regression networks and then regressing floor information.

[0116] In some embodiments, satellite images of buildings can be acquired as sample data to train a deep regression network to obtain a floor prediction network. The floor prediction network can be trained through the following steps a to b:

[0117] a. Obtain a training dataset, wherein the training dataset includes sample satellite images of sample buildings, and the sample satellite images are labeled with the actual floor information of the sample buildings.

[0118] For example, third-party map asset providers offer satellite images and floor information for each building. These images and floor information can be obtained from existing map development interfaces (i.e., the third-party map asset providers). Each building with known floor information is used as a sample building, its satellite image is used as a sample satellite image, and the actual floor information of the sample building is used as the label data for the sample satellite image.

[0119] For example, third-party map asset providers offer satellite imagery of various buildings. This satellite imagery can be obtained from existing map development interfaces (i.e., third-party map asset providers). Each building is used as a sample building, and its satellite imagery is used as the sample satellite imagery. Furthermore, the actual floor information of each sample building is recorded through on-site inspections to serve as the label data for the sample satellite imagery.

[0120] b. Train the preset deep regression network based on the training dataset to obtain the floor prediction network.

[0121] Deep regression networks refer to regression models with deep convolutional neural networks. For example, a specific deep regression network can be the open-source ResNet network.

[0122] Because deep neural networks suffer from vanishing and exploding gradient problems, making it difficult to train models with optimal parameters, ResNet networks can be used as deep regression networks. ResNet is a deep CNN architecture whose advantages over other model architectures are: ordinary deep networks have high training errors, while ResNet is easier to optimize and can achieve accuracy with significantly increased depth.

[0123] To avoid excessive loss of feature information during the drastic reduction of feature dimensionality, which could lead to low accuracy in floor information prediction, deep regression networks can incorporate multiple fully connected layers. For example, in this embodiment, three consecutive fully connected layers are used to successively reduce the feature dimensionality, ultimately outputting building floor information.

[0124] Specifically, step b may include b1 to b3:

[0125] b1. Call the deep regression network to perform deep convolution feature extraction based on the sample satellite image to obtain the sample features of the sample satellite image; and perform regression based on the sample features to obtain the floor prediction information of the sample building.

[0126] Among them, sample features refer to information that can reflect the building height extracted from sample satellite images, such as the layout of buildings on the plot, the distance between buildings, the area of ​​building shadows, and the length of building slopes.

[0127] Floor prediction information refers to the floor information of sample buildings obtained by regression from sample satellite images.

[0128] For example, such as Figure 4 As shown, Figure 4 This is a schematic diagram of an embodiment of the functional structure of the deep regression network provided in this application. The deep regression network may include an input layer, a hidden layer, and an output layer.

[0129] The input layer is used to extract pixel features from the building image, and inputs information about each pixel in the image.

[0130] Hidden layers are used to extract deep convolutional features by progressively reducing the feature dimensionality of each pixel in the input layer. This process extracts hidden features that reflect building height, such as plot building layout, building spacing, building shadow area, and building slope length, thus obtaining image features.

[0131] The output layer is used to interface with the hidden layer and regress the building's floor information based on the image features extracted from the hidden layer.

[0132] Specifically, firstly, the input layer of the deep regression network represents the sample satellite image as pixels, obtaining the pixel information of the sample satellite image. Then, the hidden layers of the deep regression network perform deep convolutional feature extraction based on the pixel information of the sample satellite image, progressively reducing the feature dimensionality. This extracts hidden features reflecting building height, such as plot and building layout information, building spacing, building shadow area, and building slope length, thus obtaining the sample features of the sample satellite image. Finally, the output layer of the deep regression network regresses the floor information of the sample building based on the sample features of the sample satellite image, obtaining the predicted floor information of the sample building.

[0133] b2. Based on the actual floor information and the predicted floor information, determine the training loss value of the deep regression network.

[0134] The deep regression network is equipped with a loss function, which is set according to the floor prediction information output by the deep regression network.

[0135] For example, the loss function can be the mean squared error between the predicted floor information and the actual floor information. During training, the training loss value of the deep regression network is calculated using the loss function. By substituting the predicted floor information and the actual floor information into the loss function, the training loss value of the corresponding deep regression network can be obtained.

[0136] b3. Adjust the model parameters of the deep regression network according to the training loss value until the deep regression network converges to obtain the floor prediction network.

[0137] Specifically, the model parameters of the deep regression network are continuously adjusted based on the training loss value of each training iteration until a preset stopping condition is met (i.e., when the deep regression network converges). The parameter-updated deep regression network is then used as the trained floor prediction network. At this point, the trained floor prediction network can be applied to extract deep convolutional features from building images to obtain the building's floor association features; and the building's floor information can be regressed based on these features. Similar to the deep regression network's network structure and function, the floor prediction network also includes an input layer, hidden layers, and an output layer to regress the building's floor information from its image.

[0138] The preset training stop conditions can be set according to actual needs. For example, it could be when the training loss value is less than a preset value, or when the training loss value basically stops changing, that is, when the difference between the training loss values ​​corresponding to multiple adjacent training iterations is less than the preset value; or when the number of iterations of the deep regression network training reaches the maximum number of iterations.

[0139] In another embodiment, aerial images of buildings can be acquired as sample data to train a deep regression network to obtain a floor prediction network. The specific training process can be referred to the descriptions of steps a-b and b1-b3 above.

[0140] As can be seen from the above, by training the deep regression network with the training dataset, the trained floor prediction network is obtained. This allows the floor prediction network to learn hidden features that reflect the height of buildings, such as the layout of buildings on the plot, the spacing between buildings, the area of ​​building shadows, and the length of building slopes, and thus accurately predict the floor information of buildings.

[0141] Furthermore, to facilitate the subsequent construction of a map containing floor information, after obtaining the floor information of the building to be detected, the following steps may also be included: obtaining the building block outline and identification mark of the building to be detected; associating the building block outline and the floor information according to the identification mark to obtain the target building block outline of the building to be detected, wherein the target building block outline is used to construct the map.

[0142] As an example, this method for obtaining floor information can be used to obtain floor information of various buildings in a specific business district, such as... Figure 5 As shown, regression analysis of satellite images of buildings is performed based on a deep convolutional neural network model to obtain the building's floor information. The following describes four parts: data acquisition and preprocessing, model architecture and parameter training, floor information inference, and data post-processing.

[0143] (I) Data Acquisition and Preprocessing

[0144] Deep CNN regression models require a large number of images and their corresponding label images as training datasets. Therefore, satellite images and floor information of buildings are preprocessed into a training dataset usable by the model. The following steps convert the satellite images and floor information of buildings into the training dataset required by the model:

[0145] 1. The model uses publicly available sub-meter resolution satellite imagery from Google Earth as training data. Building block outlines are downloaded from existing map development interfaces. These outlines include the coordinates of the building's center point in its spatial projection, indicating its location on the map. Building block outlines belonging to the same building are linked to the satellite imagery using these center point coordinates.

[0146] 2. Traverse the building outlines with floor information, obtain the center point coordinates of each building outline, save the satellite image centered at those coordinates, and use the building's floor information as the label data for that satellite image. The dataset consisting of images and labels is the training dataset required for the deep CNN regression model.

[0147] (II) Model Architecture and Parameter Training

[0148] CNNs possess excellent automatic feature extraction capabilities, resulting in CNN-based regression models achieving 20%-30% higher accuracy than traditional image feature extraction methods. Since building height can be reflected through a series of hidden features such as building layout information, building spacing, building shadow area, and building slope length, deep CNNs are needed to automatically extract image features reflecting building height. However, deep CNNs suffer from vanishing and exploding gradient problems, making it difficult to train a model with optimal parameters. Therefore, ResNet is used as the deep regression model to be trained. ResNet is a deep CNN network architecture with advantages over other model architectures: ordinary deep networks have high training errors, while ResNet is easier to optimize and can achieve accuracy from significantly increased depth.

[0149] To avoid excessive loss of feature information during the drastic reduction in feature dimension, the ResNet residual network uses three consecutive fully connected layers to successively reduce the feature dimension, ultimately outputting the building's floor information. The loss function is the mean squared error between the predicted floor information and the actual floor information. By optimizing the loss function through stochastic gradient descent, the optimal deep regression model is obtained, which is the floor prediction network.

[0150] (III) Floor Information Inference

[0151] Using the center coordinates of each building as the image center, the satellite image of a certain business district is divided into multiple satellite images of uniform size; each segmented satellite image is used to indicate a building. The floor association features of the buildings are extracted using the floor prediction network trained in (II) to obtain the floor information of the buildings corresponding to each satellite image in the business district.

[0152] (iv) Data Post-processing

[0153] The building database stores the center point coordinates of each building and the outline of each building block.

[0154] Finally, by using the identification identifier (such as ID or center point coordinates) of each building, the floor information of each building in the business district is associated with the building block outline and linked to the corresponding building database, so that the map attribute elements such as the center point coordinates, building block outline, and floor information of each building can be obtained from the building database to construct the map.

[0155] To better implement the floor information acquisition method in the embodiments of this application, based on the floor information acquisition method, the embodiments of this application also provide a floor information acquisition device, such as... Figure 6 The diagram shown is a structural schematic of one embodiment of the floor information acquisition device in this application. The floor information acquisition device 600 includes:

[0156] Acquisition unit 601 is used to acquire a state image of the building to be detected;

[0157] Extraction unit 602 is used to perform deep convolution feature extraction on the state image to obtain the floor association features of the building to be detected;

[0158] The prediction unit 603 is used to perform regression prediction processing based on the floor association features to obtain the floor information of the building to be detected.

[0159] In some embodiments of this application, the floor association features include at least one of the following: plot building layout information of the building to be detected, building spacing, building shadow area, and building slope length. The extraction unit 602 is specifically used for:

[0160] Pixel feature extraction is performed on the state image to obtain the pixel information of the state image;

[0161] A pre-trained floor prediction network is invoked to perform deep convolutional feature extraction based on the pixel information, thereby obtaining at least one of the following information: the plot building layout information, building spacing, building shadow area, and building slope length of the building to be detected. The floor prediction network contains multiple fully connected layers.

[0162] In some embodiments of this application, the prediction unit 603 is specifically used for:

[0163] The floor prediction network is invoked to perform regression prediction processing based on at least one of the following information: the plot building layout information, building spacing, building shadow area, and building slope length of the building to be detected, so as to obtain the floor information of the building to be detected.

[0164] In some embodiments of this application, the floor information acquisition device 600 further includes a training unit (not shown in the figure). Before the step of performing deep convolutional feature extraction on the state image to obtain the floor association features of the building to be detected, the training unit is specifically used for:

[0165] Obtain a training dataset, wherein the training dataset includes sample satellite images of sample buildings, and the sample satellite images are labeled with the actual floor information of the sample buildings;

[0166] The floor prediction network is obtained by training a preset deep regression network based on the training dataset.

[0167] In some embodiments of this application, the training unit is specifically used for:

[0168] The deep regression network is invoked to perform deep convolutional feature extraction based on the sample satellite images to obtain sample features of the sample satellite images; and regression is performed based on the sample features to obtain the floor prediction information of the sample buildings;

[0169] Based on the actual floor information and the predicted floor information, the training loss value of the deep regression network is determined;

[0170] The model parameters of the deep regression network are adjusted according to the training loss value until the deep regression network converges, thus obtaining the floor prediction network.

[0171] In some embodiments of this application, the acquisition unit 601 is specifically used for:

[0172] Obtain a building database, wherein the building database includes satellite images of each building among multiple buildings, and the center point coordinates of each building, wherein the satellite images of each building and the center point coordinates of each building are associated and recorded.

[0173] Obtain the coordinates of the target center point of the building to be inspected;

[0174] From the building database, obtain satellite images corresponding to the coordinates of the target center point to serve as the status images.

[0175] In some embodiments of this application, the floor information acquisition device 600 further includes an association unit (not shown in the figure). After the step of performing regression prediction processing based on the floor association features to obtain the floor information of the building to be detected, the association unit is specifically used for:

[0176] Obtain the building block outlines and identification marks of the building to be detected;

[0177] Based on the identification mark, the building block outline and the floor information are associated to obtain the target building block outline of the building to be detected, wherein the target building block outline is used to construct a map.

[0178] In specific implementation, the above units can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of the above units can be found in the previous method embodiments and will not be repeated here.

[0179] Because this floor information acquisition device can perform the functions described in this application, Figures 1 to 5 Corresponding to the steps in the floor information acquisition method in any embodiment, this application can achieve the following: Figures 1 to 5 For details on the beneficial effects that the floor information acquisition method can achieve in any embodiment, please refer to the preceding description, which will not be repeated here.

[0180] Furthermore, to better implement the floor information acquisition method in this application embodiment, based on the floor information acquisition method, this application embodiment also provides an electronic device, see below. Figure 7 , Figure 7 This illustration shows a structural diagram of an electronic device according to an embodiment of this application. Specifically, the electronic device provided in this embodiment includes a processor 701, which executes a computer program stored in a memory 702 to implement, for example... Figures 1 to 5 Corresponding to each step of the floor information acquisition method in any embodiment; or, when the processor 701 executes the computer program stored in the memory 702, it implements as follows: Figure 6 The functions of each unit in the corresponding embodiment.

[0181] For example, a computer program can be divided into one or more modules / units, one or more of which are stored in memory 702 and executed by processor 701 to complete the embodiments of this application. One or more modules / units can be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in a computer device.

[0182] The electronic device may include, but is not limited to, processor 701 and memory 702. Those skilled in the art will understand that the illustrations are merely examples of an electronic device and do not constitute a limitation on the electronic device. It may include more or fewer components than illustrated, or combine certain components, or different components. For example, the electronic device may also include input / output devices, network access devices, buses, etc., and processor 701, memory 702, input / output devices, and network access devices are connected via a bus.

[0183] The processor 701 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the electronic device, connecting various parts of the electronic device through various interfaces and lines.

[0184] The memory 702 can be used to store computer programs and / or modules. The processor 701 implements various functions of the computer device by running or executing the computer programs and / or modules stored in the memory 702 and by calling data stored in the memory 702. The memory 702 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device (such as audio data, video data, etc.). In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, RAM, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0185] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the floor information acquisition device, electronic equipment, and their corresponding units described above can be found in, for example... Figures 1 to 5 The description of the floor information acquisition method corresponding to any embodiment will not be repeated here.

[0186] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0187] Therefore, embodiments of this application provide a computer-readable storage medium storing a plurality of instructions that can be loaded by a processor to execute the present application. Figures 1 to 5 For the steps in the floor information acquisition method corresponding to any embodiment, please refer to the following for specific operations: Figures 1 to 5 The description of the floor information acquisition method corresponding to any embodiment will not be repeated here.

[0188] The computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0189] Because of the instructions stored in the computer-readable storage medium, the present application can be executed as described above. Figures 1 to 5 Corresponding to the steps in the floor information acquisition method in any embodiment, this application can achieve the following: Figures 1 to 5 For details on the beneficial effects that the floor information acquisition method can achieve in any embodiment, please refer to the preceding description, which will not be repeated here.

[0190] The foregoing has provided a detailed description of a method, apparatus, electronic device, and computer-readable storage medium for obtaining floor information according to embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for obtaining floor information, characterized in that, The method includes: Acquire the state image of the building to be detected; Deep convolution feature extraction is performed on the state image to obtain the floor association features of the building to be detected; Regression prediction is performed based on the floor association features to obtain the floor information of the building to be detected; The floor association features include at least one of the following: the plot building layout information of the building to be detected, the building spacing, and at least one of the following: the building shadow area and the building slope length. The step of performing deep convolutional feature extraction on the state image to obtain the floor association features of the building to be detected includes: A classification network is used to extract convolutional features from the state image to obtain the convolutional features of the state image; and classification is performed based on the convolutional features of the state image to obtain the plot building layout information and / or building spacing of the building to be detected. A semantic segmentation network is used to extract convolutional features from the state image to obtain the convolutional features of the state image, thereby obtaining the building shadow area and / or building slope length of the building to be detected.

2. The method for obtaining floor information according to claim 1, characterized in that, The step of performing deep convolutional feature extraction on the state image to obtain the floor association features of the building to be detected includes: Pixel feature extraction is performed on the state image to obtain the pixel information of the state image; A pre-trained floor prediction network is invoked to perform deep convolutional feature extraction based on the pixel information to obtain the floor association features of the building to be detected. The floor prediction network contains multiple fully connected layers.

3. The method for obtaining floor information according to claim 2, characterized in that, The step of performing regression prediction processing based on the floor association features to obtain the floor information of the building to be detected includes: The floor prediction network is invoked, and regression prediction processing is performed based on the floor association features to obtain the floor information of the building to be detected.

4. The method for obtaining floor information according to claim 2, characterized in that, The step of performing deep convolutional feature extraction on the state image to obtain the floor association features of the building to be detected also includes: Obtain a training dataset, wherein the training dataset includes sample satellite images of sample buildings, and the sample satellite images are labeled with the actual floor information of the sample buildings; The floor prediction network is obtained by training a preset deep regression network based on the training dataset.

5. The method for obtaining floor information according to claim 4, characterized in that, The step of training a preset deep regression network based on the training dataset to obtain the floor prediction network includes: The deep regression network is invoked to perform deep convolutional feature extraction based on the sample satellite images to obtain sample features of the sample satellite images; and regression is performed based on the sample features to obtain the floor prediction information of the sample buildings; Based on the actual floor information and the predicted floor information, the training loss value of the deep regression network is determined; The model parameters of the deep regression network are adjusted according to the training loss value until the deep regression network converges, thus obtaining the floor prediction network.

6. The method for obtaining floor information according to claim 1, characterized in that, The process of acquiring the state image of the building to be detected also includes, prior to: Obtain a building database, wherein the building database includes satellite images of each building among multiple buildings, and the center point coordinates of each building, wherein the satellite images of each building and the center point coordinates of each building are associated and recorded. The process of acquiring the state image of the building to be detected includes: Obtain the coordinates of the target center point of the building to be inspected; From the building database, obtain satellite images corresponding to the coordinates of the target center point to serve as the status images.

7. The method for obtaining floor information according to any one of claims 1-6, characterized in that, The step of performing regression prediction processing based on the floor association features to obtain the floor information of the building to be detected further includes: Obtain the building block outlines and identification marks of the building to be detected; Based on the identification mark, the building block outline and the floor information are associated to obtain the target building block outline of the building to be detected, wherein the target building block outline is used to construct a map.

8. A floor information acquisition device, characterized in that, The floor information acquisition device includes: The acquisition unit is used to acquire a state image of the building to be detected. The extraction unit is used to perform deep convolutional feature extraction on the state image to obtain the floor association features of the building to be detected; The prediction unit is used to perform regression prediction processing based on the floor association features to obtain the floor information of the building to be detected. The floor association features include at least one of the following: the plot building layout information of the building to be detected, the building spacing, and at least one of the following: the building shadow area and the building slope length. The step of performing deep convolutional feature extraction on the state image to obtain the floor association features of the building to be detected includes: A classification network is used to extract convolutional features from the state image to obtain the convolutional features of the state image; and classification is performed based on the convolutional features of the state image to obtain the plot building layout information and / or building spacing of the building to be detected. A semantic segmentation network is used to extract convolutional features from the state image to obtain the convolutional features of the state image, thereby obtaining the building shadow area and / or building slope length of the building to be detected.

9. An electronic device, characterized in that, It includes a processor and a memory, wherein the memory stores a computer program, and the processor executes the floor information acquisition method as described in any one of claims 1 to 7 when it invokes the computer program in the memory.

10. A computer-readable storage medium, characterized in that, It stores a computer program, which is loaded by a processor to execute the steps of the floor information acquisition method according to any one of claims 1 to 7.

Images