Building height and floor area calculation method, device, equipment and storage medium

Abstract

The present application provides a method, device, equipment and storage medium for calculating the height and floor area of ​​a building, belonging to the field of artificial intelligence technology. The method comprises: obtaining a target image and target satellite information of a target building; inputting the target image and target satellite information into a trained recognition model, and using the trained recognition model to output target location-related information, a target roof segmentation image and a target offset angle vector of the target building; the trained recognition model is obtained by training a preset recognition model using a training data set; calculating the height of the target building based on the target satellite information, target location-related information, the target roof segmentation image and the target offset angle vector; and calculating the floor area of ​​the target building based on the target satellite information and the target roof segmentation image. The method of the present application can calculate a more accurate building height and floor area.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, device, equipment and storage medium for calculating the height and floor area of ​​a building. Background Art

[0002] When it is necessary to regularly monitor the height and floor area of ​​a building, the height and floor area of ​​the building can be calculated through satellite remote sensing images, so that data can be obtained without on-site measurement.

[0003] Currently, staff can select the target building's outline and height in satellite remote sensing imagery, or use image segmentation technology to segment the target building's outline and then calculate the target building's height and floor area based on the corresponding satellite information in the satellite remote sensing imagery. The accuracy of the data obtained through the staff's selection method is affected by the staff's operation method and level of care. Using image segmentation technology, due to the large number of buildings in satellite remote sensing images, the presence of shadows can also affect the completeness of building outline segmentation, resulting in less accurate data.

[0004] In summary, in the prior art, the calculation results of building height and floor area are not accurate enough. Summary of the Invention

[0005] The present application provides a method, device, equipment and storage medium for calculating the height and floor area of ​​a building, which are used to solve the problem that the calculation results of the height and floor area of ​​a building in the prior art are not accurate enough.

[0006] According to a first aspect of the present application, a method for calculating building height and floor area is provided, comprising:

[0007] Obtain target images and target satellite information of target buildings;

[0008] Inputting a target image and target satellite information into a trained recognition model, and using the trained recognition model to output target location-related information of a target building, a target roof segmentation image, and a target offset angle vector; the trained recognition model is obtained by training a preset recognition model using a training data set, the training data set including at least one training sample, the training sample including a sample image of a sample building, sample satellite information, a location-related information label, a roof segmentation image label, and an offset angle vector label;

[0009] Calculate the target building height based on target satellite information, target location related information, target roof segmentation image and target offset angle vector;

[0010] The target building area is calculated based on the target satellite information and the target roof segmentation image.

[0011] According to a second aspect of the present application, a device for calculating building height and floor area is provided, comprising:

[0012] An acquisition module is used to obtain target images and target satellite information of a target building;

[0013] a recognition model, configured to input a target image and target satellite information into a trained recognition model, and output target location-related information, a target roof segmentation image, and a target offset angle vector of a target building using the trained recognition model; the trained recognition model is obtained by training the recognition model using a training data set, the training data set including at least one training sample, the training sample including a sample image of a sample building, sample satellite information, a location-related information label, a roof segmentation image label, and an offset angle vector label;

[0014] The first calculation module is used to calculate the height of the target building based on the target satellite information, target position related information, the target roof segmentation image and the target offset angle vector;

[0015] The second calculation module is used to calculate the area of ​​the target building according to the target satellite information and the target roof segmentation image.

[0016] According to a third aspect of the present application, there is provided an electronic device, comprising: a processor and a memory communicatively connected to the processor;

[0017] Memory stores computer-executable instructions;

[0018] The processor executes the computer-executable instructions stored in the memory to implement the method according to the first aspect.

[0019] According to a fourth aspect of the present application, a computer-readable storage medium is provided, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the method described in the first aspect.

[0020] According to a fifth aspect of the present application, a computer program product is provided, comprising a computer program, which implements the method described in the first aspect when executed by a processor.

[0021] The method, apparatus, device, and storage medium for calculating the building height and floor area provided in the present application obtain a target image and target satellite information of a target building; input the target image and target satellite information into a trained recognition model, and use the trained recognition model to output target location-related information, a target roof segmentation image, and a target offset angle vector for the target building; the trained recognition model is obtained by training a preset recognition model using a training data set, the training data set including at least one training sample, the training sample including a sample image of a sample building, sample satellite information, location-related information labels, a roof segmentation image label, and an offset angle vector label; the target building height is calculated based on the target satellite information, target location-related information, the target roof segmentation image, and the target offset angle vector; and the target building floor area is calculated based on the target satellite information and the target roof segmentation image. Since the trained model is obtained by training a preset model using a training data set, and each training sample in the training data set includes a sample image of a sample building, sample satellite information, location-related information labels, a roof segmentation image label, and an offset angle vector label, the trained recognition model can output target location-related information, a target roof segmentation image, and a target offset angle vector for the target building. Among them, the target location-related information is the location-related information of the target building in the target image, which can be determined from the target image without image segmentation. At the same time, the target offset angle can also be determined from the target image without image segmentation. Therefore, as long as the target offset angle and the target location-related information are accurate, the accurate target building height can be calculated based on the principle of trigonometric functions. At the same time, the target roof segmentation image is less affected by the building shadow on the target image. The roof segmentation image separated from the target image is more accurate than the target building outline. The target building area calculated based on the target roof segmentation image can be more accurate. In summary, the solution of the present application can calculate a more accurate building height and area. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0023] Figure 1 This is a network architecture diagram corresponding to the application scenario provided in the embodiment of the present application;

[0024] Figure 2 1 is a flow chart of a method for calculating building height and floor area according to the first embodiment of the present application;

[0025] Figure 3 This is a schematic diagram of the structure of the preset recognition model provided in Example 2 of the present application;

[0026] Figure 4 This is a flow chart of a method for calculating building height and floor area according to the second embodiment of the present application;

[0027] Figure 5 This is a flow chart of a method for calculating building height and floor area according to the third embodiment of the present application;

[0028] Figure 6 This is a schematic diagram of the structure of a positioning network provided according to the third embodiment of the present application;

[0029] Figure 7 1 is a flow chart of a method for calculating building height and floor area according to the fourth embodiment of the present application;

[0030] Figure 8 1 is a flow chart of a method for calculating building height and floor area according to the fifth embodiment of the present application;

[0031] Figure 9 This is a schematic diagram of a structure of an angle analysis network provided according to the fifth embodiment of the present application;

[0032] Figure 10 1 is a schematic diagram of the structure of a device for calculating building height and floor area provided in accordance with Example 6 of the present application;

[0033] Figure 11 This is a block diagram of an electronic device provided according to the seventh embodiment of the present application.

[0034] The above drawings illustrate specific embodiments of the present application, which will be described in more detail below. These drawings and the textual description are not intended to limit the scope of the present application in any way, but rather to illustrate the concepts of the present application to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0035] The following is a detailed description and analysis of the prior art involved in this application.

[0036] Currently, there are two main ways to calculate the height and area of ​​target buildings through satellite remote sensing images.

[0037] The first method uses interactive technology, whereby workers select the outline and height of the target building in the remote sensing image. The height and floor area of ​​the target building are then calculated using satellite information corresponding to the remote sensing image. This method is not only time-consuming and labor-intensive, but its accuracy also depends on the operator's manual operation and attentiveness, resulting in inaccurate data.

[0038] The second approach uses artificial intelligence (AI) to segment remote sensing images, isolating the target building's outline from the entire image for calculation. However, due to the large number of buildings in remote sensing images and the potential for building shadows, the target building outlines isolated from the entire image may not be complete, resulting in inaccurate calculations of the target building's height and floor area.

[0039] Therefore, when faced with the problem that the calculation results of building height and floor area in the existing technology are not accurate enough, the inventors conducted creative research and found that in order to calculate more accurate building height and floor area, it is first necessary to obtain a complete and more accurate target building outline. However, the accuracy of the target building outline cannot be controlled by the staff simply checking the target building outline in the remote sensing image. Therefore, it is still necessary to use artificial intelligence technology to segment the target building outline from the remote sensing image.

[0040] Although the possible building shadows in remote sensing images will affect the completeness and accuracy of the target building outline, the impact of building shadows on the target building roof outline is far less than the impact on the target building outline. Most target buildings are regular rectangular structures with the same roof shape as the floor area. Therefore, after locating the target building using artificial intelligence technology, we can separate the target building outline instead of the target building outline, and then obtain a more accurate building floor area by calculating the target building roof area.

[0041] At the same time, as long as the target building height is not calculated through the target building outline, the completeness and accuracy of the target building outline will not affect the accuracy of the target building height. Therefore, after locating the target building using artificial intelligence technology, it is only necessary to obtain the actual offset angle of the target building and the position-related information of the target building on the remote sensing image, and the accurate target building height can be calculated according to the principle of trigonometric function.

[0042] In summary, the inventors have proposed the technical solution of the present application, which involves obtaining a target image and target satellite information of a target building; inputting the target image and target satellite information into a trained recognition model, and using the trained recognition model to output target location-related information, a target roof segmentation image, and a target offset angle vector for the target building; the trained recognition model is obtained by training a preset recognition model using a training dataset, wherein the training dataset includes at least one training sample, and the training sample includes a sample image of a sample building, sample satellite information, location-related information labels, a roof segmentation image label, and an offset angle vector label; the height of the target building is calculated based on the target satellite information, target location-related information, the target roof segmentation image, and the target offset angle vector; and the area of ​​the target building is calculated based on the target satellite information and the target roof segmentation image. Since the trained model is obtained by training a preset model using a training dataset, and each training sample in the training dataset includes a sample image of a sample building, sample satellite information, location-related information labels, a roof segmentation image label, and an offset angle vector label, the trained recognition model is capable of outputting target location-related information, a target roof segmentation image, and a target offset angle vector for the target building. Among them, the target position related information is the position related information of the target building in the target image, which can be determined from the target image without image segmentation. At the same time, the target offset angle can be determined from the target image without image segmentation. Therefore, as long as the target offset angle and the target position related information are accurate, the accurate target building height can be calculated based on the principle of trigonometric functions. At the same time, the target roof segmentation image is less affected by the building shadow on the target image. The roof segmentation image separated from the target image is more accurate than the target building outline. The target building area calculated based on the target roof segmentation image can be more accurate. Therefore, the solution of the present application can calculate more accurate building height and area.

[0043] The building height and floor area calculation method, device, equipment, and storage medium provided in this application are intended to solve the above-mentioned technical problems in the prior art. The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below using specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0044] The following will introduce the network architecture and application scenarios of the building height and floor area calculation method provided in the embodiment of the present application. When the following description refers to the drawings, unless otherwise indicated, the same data in different drawings represent the same or similar elements.

[0045] Figure 1 This is a network architecture diagram corresponding to the application scenario provided by the embodiment of this application. Figure 1 As shown, a network architecture corresponding to an application scenario provided in an embodiment of the present application includes: a terminal 11, an electronic device 12, and a database 13. Terminal 11 is equipped with a client 14 of software for a building height and floor area calculation method, or terminal 11 accesses a webpage corresponding to a website address for a building height and floor area calculation method. Terminal 11, electronic device 12, and database 13 are in communication. Electronic device 12 is configured with a preset recognition model or a trained recognition model.

[0046] Database 13 stores a training dataset. The training dataset includes at least one training sample, which includes a sample image of a sample building, sample satellite information, a location-related information label, a roof segmentation image label, and an offset angle vector label. Database 13 may also store a target image of a target building and target satellite information. The target image is a remote sensing image of the target building taken by a target satellite.

[0047] The user can input the target image and target satellite information of the target building in the operation interface of the client 14 or in the webpage corresponding to the building height and floor area calculation method website. The terminal 11 sends the target image and target satellite information of the target building to the electronic device 12.

[0048] The electronic device 12 can obtain the target image and target satellite information of the target building sent by the terminal 11 through a communication connection with the terminal 11; it can also obtain the target image and target satellite information of the target building stored in the database 13 through a communication connection with the database 13.

[0049] After obtaining the target image and target satellite information of the target building, the electronic device 12 inputs the target image and target satellite information into the trained recognition model, and uses the trained recognition model to output the target position related information, the target roof segmentation image and the target offset angle vector of the target building; calculates the height of the target building based on the target satellite information, the target position related information, the target roof segmentation image and the target offset angle vector; and calculates the area of ​​the target building based on the target satellite information and the target roof segmentation image.

[0050] The electronic device 12 may send the height and floor area of ​​the target building to the terminal 11 so that the terminal 11 displays the height and floor area of ​​the target building to the user.

[0051] The electronic device 12 may also obtain a training data set from the database 12 and use the training data set to train a preset recognition model to obtain a trained recognition model.

[0052] The following embodiments of the present application will be described in conjunction with the accompanying drawings. The embodiments described in the following embodiments do not represent all embodiments consistent with the present application. Instead, they are merely examples of devices and methods consistent with certain aspects of the present application as detailed in the appended claims.

[0053] Example 1

[0054] Figure 2 This is a flow chart of the method for calculating building height and floor area according to the first embodiment of the present application. Figure 2 As shown, the execution subject of this application is a building height and floor area calculation device, which is located in an electronic device. The building height and floor area calculation method provided in this embodiment includes steps 201 to 204.

[0055] Step 201: Acquire a target image and target satellite information of a target building.

[0056] In this embodiment, the target image is a remote sensing image of the target building taken by the target satellite. The target satellite information is information such as the location, time, and shooting parameters of the target satellite when taking the target image. For example, the target satellite information may include the satellite altitude angle and the satellite resolution.

[0057] Step 202: Input the target image and target satellite information into a trained recognition model, and use the trained recognition model to output target location-related information, a target roof segmentation image, and a target offset angle vector of the target building. The trained recognition model is obtained by training a preset recognition model using a training data set, and the training data set includes at least one training sample, and the training sample includes a sample image of a sample building, sample satellite information, a location-related information label, a roof segmentation image label, and an offset angle vector label.

[0058] In this embodiment, the target location related information may include coordinate information of the target building on the target image. For example, the target location related information may be coordinate information and length and width information of a rectangular frame that frames the target building in the target image.

[0059] The target roof segmentation image may be a black and white image, wherein the black may be the background and the white may be the segmented roof shape of the target building.

[0060] The angle corresponding to the target offset angle vector can be the offset angle of the projection of the target building on the target image. Here, since the target image is a remote sensing image taken by the target satellite, except for the overhead view, the target building will definitely be biased in one direction in the target image. Therefore, the target building presented on the target image has a projection offset angle, which makes it impossible to directly calculate the actual height of the target building by the presentation height of the target building on the target image. For example, this projection offset angle can be understood as the angle between the actual height and the presentation height of the target building.

[0061] Step 203 , calculating the target building height based on the target satellite information, target location related information, the target roof segmentation image, and the target offset angle vector.

[0062] In this embodiment, since the target building appears tilted in the target image, the height of the target building can be calculated using the trigonometric function of the target offset angle. For example, the target building can be located by a rectangular frame, and the target position-related information can be the horizontal coordinate, vertical coordinate, length L, and width W of the rectangular frame that locates the target building from the target image in the target image. The length L1 of the roof of the target building in the target image can be calculated based on the target roof segmentation image and the target satellite-related information. Therefore, the height L2 of the target building in the target image is L-L1. The target offset angle θ is calculated based on the target offset angle vector. Therefore, the height L3 of the target building can be L3=L2sinθ.

[0063] Step 204 : Calculate the target building's floor area based on the target satellite information and the target roof segmentation image.

[0064] In this embodiment, the target satellite information, such as the satellite resolution, and the target roof segmentation image can be input into the OpenCV software, and the algorithm included in the OpenCV software can be used to calculate the roof area of ​​the target building, and the roof area of ​​the target building can be determined as the target building floor area.

[0065] The method for calculating the building height and floor area provided in this embodiment obtains a target image and target satellite information of a target building; inputs the target image and target satellite information into a trained recognition model, and uses the trained recognition model to output target location-related information, a target roof segmentation image, and a target offset angle vector for the target building; the trained recognition model is obtained by training a preset recognition model using a training dataset, wherein the training dataset includes at least one training sample, and the training sample includes a sample image of a sample building, sample satellite information, location-related information labels, a roof segmentation image label, and an offset angle vector label; the target building height is calculated based on the target satellite information, target location-related information, the target roof segmentation image, and the target offset angle vector; and the target building floor area is calculated based on the target satellite information and the target roof segmentation image. Since the trained model is obtained by training a preset model using a training dataset, and each training sample in the training dataset includes a sample image of a sample building, sample satellite information, location-related information labels, a roof segmentation image label, and an offset angle vector label, the trained recognition model can output target location-related information, a target roof segmentation image, and a target offset angle vector for the target building. Among them, the target position related information is the position related information of the target building in the target image, which can be determined from the target image without image segmentation. At the same time, the target offset angle can be determined from the target image without image segmentation. Therefore, as long as the target offset angle and the target position related information are accurate, the accurate target building height can be calculated based on the principle of trigonometric functions. At the same time, the target roof segmentation image is less affected by the building shadow on the target image. The roof segmentation image separated from the target image is more accurate than the target building outline. The target building area calculated based on the target roof segmentation image can be more accurate. Therefore, the solution of the present application can calculate more accurate building height and area.

[0066] Example 2

[0067] Figure 3 This is a schematic diagram of the structure of the preset recognition model provided in Example 2 of the present application; Figure 4 This is a flow chart of the method for calculating building height and floor area provided in Example 2 of the present application.

[0068] The method for calculating building height and floor area provided in this embodiment is based on the first embodiment. Figure 3 As shown, the preset recognition model 30 includes an image segmentation model 31, a positioning network 32, a roof segmentation network 33 and an angle analysis network 34. The image segmentation model 31 is connected to the positioning network 32, the roof segmentation network 33 and the angle analysis network 34 respectively. Figure 4As shown, in step 202, the target image and target satellite information are input into the trained recognition model, and the trained recognition model is used to output the location-related information, roof segmentation image and offset angle vector of the target building. Before that, steps 401 to 402 are also included.

[0069] Step 401: Input the sample image of each training sample into an image segmentation model, and use the image segmentation model to extract and output a sample image feature map and at least one region of interest (ROI) matrix of each training sample.

[0070] In this embodiment, for any training sample, after the sample image of the training sample is input into the image segmentation model, the image segmentation model can output the sample image feature map and at least one ROI matrix. The ROI matrix is ​​a matrix that describes the candidate box on the sample image feature map.

[0071] The image segmentation model may be a convolutional neural network portion of a mask-R-CNN image segmentation model, wherein the convolutional neural network portion of the mask-R-CNN image segmentation model may include a ResNet50 network and an RPN network.

[0072] The ResNet50 network can extract features from sample images of training samples to obtain feature maps of sample images.

[0073] The RPN network can generate at least one candidate box on the sample image feature map and generate a region of interest (ROI) matrix corresponding to each candidate box.

[0074] The ROI matrix uses matrix information to describe the position of the candidate box on the sample image feature map. For example, the ROI matrix can use the elements in the ROI matrix to identify whether the elements in the sample image feature map are selected by the candidate box. For example, the ROI matrix uses element 1 to identify that the elements in the sample image feature map are selected by the candidate box, and uses element 0 to identify that the elements in the sample image feature map are not selected by the candidate box. For example, when the target image feature map is a matrix with 300 rows and 300 columns, the ROI matrix is ​​also 300 rows and 300 columns.

[0075] In step 402, the location-related information labels, sample image feature maps, and ROI matrices of each training sample are input into the positioning network to train the positioning network. The roof segmentation image labels, sample image feature maps, and ROI matrices of each training sample are input into the roof segmentation network to train the roof segmentation network. The offset angle vector labels, sample satellite information, sample image feature maps, and ROI matrices of each training sample are input into the angle analysis network to train the angle analysis network to obtain a trained recognition model.

[0076] In this embodiment, the location-related information tag includes the horizontal coordinates, vertical coordinates, frame length, and frame width of the positioning rectangle that selects the sample building in the sample image. The positioning rectangle can be annotated in the sample image by the staff. Here, the horizontal and vertical coordinates in the location-related information tag can be the lower left corner of the positioning rectangle, the geometric center, or the distance between any specified point and the coordinate origin.

[0077] The roof segmentation image label can be a sample roof segmentation image of a sample building in the sample image. The roof portion can be white, the background portion can be black, and the image label can be aligned with the location and size of the positioning rectangle that encloses the sample building. The roof segmentation image label can be obtained by a worker annotating the roof outline in the positioning rectangle.

[0078] The offset angle vector label may be a vector corresponding to the offset angle of the sample building in the sample image. The angle between the offset angle vector label and the horizontal coordinate in the sample image may be equal to the angle between the actual height and the displayed height of the sample building.

[0079] The localization network is used to determine the location information of the sample rectangle from the ROI matrices corresponding to each candidate rectangle. The sample rectangle is the rectangle that most accurately selects the sample building among the candidate rectangles. Using a supervised learning approach, the location information labels of each training sample, the sample image feature map, and each ROI matrix are input into the localization network. The localization network parameters are trained until the localization network achieves a preset localization accuracy. This yields the parameters of the localization network in the trained recognition model.

[0080] The roof segmentation network is used to isolate the roof segmentation image of the sample building from the area selected by each candidate box in the sample image feature map. Using a supervised learning method, the roof segmentation image labels of each training sample, the sample image feature map, and each ROI matrix are input into the roof segmentation network. The parameters of the roof segmentation network are trained to achieve a preset segmentation accuracy. This obtains the parameters of the roof segmentation network in the trained recognition model.

[0081] The angle analysis network determines the offset angle of the sample building from the area selected by each candidate box in the sample image feature map. Using a supervised learning approach, the angle analysis network is fed with the offset angle vector label of each training sample, the sample satellite information, the sample image feature map, and each ROI matrix. The network's parameters are trained until the network achieves a preset analysis accuracy. This yields the parameters for the trained recognition model.

[0082] The method for calculating the building height and floor area provided in this embodiment inputs a sample image of each training sample into an image segmentation model, and uses the image segmentation model to extract and output a sample image feature map and at least one region of interest (ROI) matrix of each training sample; inputs the position-related information label, sample image feature map, and each ROI matrix of each training sample into a positioning network to train the positioning network; inputs the roof segmentation image label, sample image feature map, and each ROI matrix of each training sample into a roof segmentation network to train the roof segmentation network; inputs the offset angle vector label, sample satellite information, sample image feature map, and each ROI matrix of each training sample into an angle analysis network to train the angle analysis network to obtain a trained recognition model; because the preset recognition model is trained using each training sample in the training data set to obtain the trained recognition model, it can be ensured that after the target image and target satellite information are input into the trained recognition model, the trained recognition model can output accurate target position-related information, target roof segmentation image, and target angle offset vector of the target building, thereby more accurately calculating the height and floor area of ​​the target building.

[0083] Example 3

[0084] Figure 5 This is a flow chart of the method for calculating building height and floor area according to the third embodiment of this application. Figure 5 As shown, the method for calculating the building height and floor area provided in this embodiment is based on the second embodiment, and the step 402 of "inputting the position-related information labels of each training sample, the sample image feature map and each ROI matrix into the positioning network to train the positioning network" is refined, and the refinement includes steps 501 to 503.

[0085] Step 501: Use a positioning network to determine and output sample location information of each training sample, and determine the positioning accuracy of the positioning network based on the location information label of each training sample.

[0086] In this embodiment, the positioning network may include a convolution layer. For any training sample, the convolution layer may perform a convolution calculation on the feature matrix of each ROI matrix corresponding to the feature map of the sample image, thereby obtaining sample position-related information of the training sample. Sample position-related information is information related to the position of the sample building in the sample image. Exemplarily, it may include the horizontal coordinate, vertical coordinate, frame length, and frame width of a sample rectangular frame that selects the sample building in the sample image. In this embodiment, the sample position-related information of the training sample may also be obtained by other methods, which are not limited in this embodiment.

[0087] In this embodiment, after determining the sample location information of each training sample, the proportion of training samples whose sample location information conforms to or matches the location information label in the training data set can be determined as the positioning accuracy of the positioning network.

[0088] In step 502, if it is determined that the positioning accuracy is greater than or equal to the preset positioning accuracy, the parameters of the positioning network are determined to be the parameters of the positioning network in the trained recognition model.

[0089] In this embodiment, the preset positioning accuracy can be the accuracy that the user expects the positioning network to achieve, and for example, can be 95%. If the positioning accuracy is greater than or equal to the preset positioning accuracy, the parameter training of the positioning network is complete, and the parameters of the positioning network can be determined as the parameters of the positioning network in the trained recognition model.

[0090] In step 503, if it is determined that the positioning accuracy is less than the preset positioning accuracy, the parameters of the positioning network are adjusted, and the steps of using the positioning network to determine and output the sample location information of each training sample, and determining the positioning accuracy of the positioning network based on the location-related information label of each training sample are repeated until the positioning accuracy is greater than or equal to the preset positioning accuracy.

[0091] In this embodiment, if the positioning accuracy is lower than the preset positioning accuracy, the parameters of the positioning network can be adjusted by using a gradient descent algorithm, a back propagation algorithm, etc. until the positioning accuracy of the positioning network is greater than or equal to the preset positioning accuracy, thereby ensuring the accuracy of the trained recognition model in extracting the target location-related information of the target building.

[0092] The method for calculating building height and floor area provided in this embodiment uses a positioning network to determine and output sample location-related information for each training sample, and determines the positioning accuracy of the positioning network based on the location-related information labels of each training sample. If the positioning accuracy is determined to be greater than or equal to a preset positioning accuracy, the parameters of the positioning network are determined as the parameters of the positioning network in the trained recognition model. If the positioning accuracy is determined to be less than the preset positioning accuracy, the parameters of the positioning network are adjusted, and the steps of using the positioning network to determine and output sample location-related information for each training sample, and determining the positioning accuracy of the positioning network based on the location-related information labels of each training sample are repeated until the positioning accuracy is greater than or equal to the preset positioning accuracy. Because the positioning network is trained using a training data set, the parameters of the positioning network are not determined as the parameters of the positioning network in the trained recognition model until the positioning accuracy of the positioning network is greater than or equal to the preset positioning accuracy. Therefore, the trained recognition model can accurately obtain target location-related information of the target building, and can further calculate the accurate target building height and floor area based on the accurate target location-related information.

[0093] As an optional implementation, based on Example 3, the step 501 of “using a positioning network to determine and output sample position related information of each training sample” is refined, and the refinement includes step 5011.

[0094] In step 5011, a convolution layer is used to perform a convolution operation on the feature matrix of each ROI matrix of each training sample corresponding to the feature matrix in the sample image feature map to obtain sample position related information of each training sample.

[0095] In this embodiment, for any training sample, a convolution layer is used to perform a convolution operation on the feature matrix corresponding to each ROI matrix in the sample image feature map to obtain sample position information related to the training sample. The feature matrix corresponding to the ROI matrix in the sample image feature map refers to the matrix selected by the candidate box marked by the ROI matrix in the sample image feature map.

[0096] Figure 6 This is a schematic diagram of the structure of a positioning network provided according to Example 3 of this application. As an optional implementation method, Figure 6 As shown, the positioning network 60 includes a convolutional layer 61 and a fully connected layer 62, and in step 501, the positioning network is used to determine and output sample position related information of each training sample, and the refinement also includes step 5012.

[0097] In step 5012, a fully connected layer is used to perform a classification operation on the feature matrix corresponding to each ROI matrix in the feature map of the sample image, and obtain the category information of the area selected by each candidate box on the feature map of the sample image corresponding to the sample image.

[0098] In this embodiment, the category information includes buildings and non-buildings. The classification operation can be a global convolution operation, which is used to map the feature matrix corresponding to the ROI matrix in the feature map of the sample image to the category information space, and classify the area selected by the candidate box in the sample image. The parameters of the fully connected layer can be pre-trained and can be used to distinguish whether the area selected by the candidate box corresponds to a building in the sample image. After classification, it is possible to obtain whether the category information of the area selected by the sample rectangular box identified by the sample position related information of the training sample output by the convolution layer is a building in the sample image. If the category information is a building, it can be further determined whether the sample position related information matches the position related information label, thereby determining the positioning accuracy of the positioning network. If the category information is a non-building, the parameters of the convolution layer can be adjusted to locate the sample rectangular box whose type information is a building. Alternatively, the training sample can be eliminated or marked as a negative sample.

[0099] It is understood that the localization network of the trained recognition model can also include a fully connected layer, which is used to output the category information of the target image corresponding to the area selected by the localization rectangle. This can also avoid the next step of calculation when the category information corresponding to the target image selected by the localization rectangle is non-building, thus saving computing resources.

[0100] Example 4

[0101] Figure 7 This is a flow chart of the method for calculating building height and floor area according to the fourth embodiment of this application. Figure 7 As shown, the method for calculating the building height and floor area provided in this embodiment is based on any of the above embodiments, and the step 402 of "inputting the roof segmentation image label of each training sample, the sample image feature map and each ROI matrix into the roof segmentation network to train the roof segmentation network" is refined, and the refinement includes steps 701 to 703.

[0102] Step 701: Use a roof segmentation network to determine and output a sample roof segmentation image of each training sample, and determine the segmentation accuracy of the roof segmentation network based on the roof segmentation image label of each training sample.

[0103] In this embodiment, the roof segmentation network may include a deconvolution layer. For any training sample, the deconvolution layer may perform a deconvolution operation on the feature matrix corresponding to each ROI matrix in the sample image feature map, thereby obtaining a sample roof segmentation image for the training sample. The size and dimensions of the sample roof segmentation image are consistent with the sample rectangular frame.

[0104] In this embodiment, after determining the sample roof segmentation image of each training sample, the proportion of training samples whose sample roof segmentation images conform to or match the roof segmentation image label in the training data set can be determined as the segmentation accuracy of the roof segmentation network.

[0105] Step 702: If it is determined that the segmentation accuracy is greater than or equal to the segmentation accuracy, the parameters of the roof segmentation network are determined as the parameters of the roof segmentation network in the trained recognition model.

[0106] In this embodiment, the preset segmentation accuracy can be the accuracy that the user expects the roof segmentation network to achieve, and can be the same as the preset positioning accuracy. If the segmentation accuracy is greater than or equal to the preset segmentation accuracy, parameter training of the roof segmentation network is complete, and the parameters of the roof segmentation network can be determined as the parameters of the roof segmentation network in the trained recognition model.

[0107] In step 703, if it is determined that the segmentation accuracy is less than the preset segmentation accuracy, the parameters of the roof segmentation network are adjusted, and the steps of using the roof segmentation network to determine and output the sample roof segmentation image of each training sample, and determining the segmentation accuracy of the roof segmentation network according to the roof segmentation image label of each training sample are repeated until the segmentation accuracy is greater than or equal to the preset segmentation accuracy.

[0108] In this embodiment, if the segmentation accuracy is less than the preset segmentation accuracy, the parameters of the roof segmentation network can be adjusted through the gradient descent algorithm, the back propagation algorithm, etc. until the segmentation accuracy of the roof segmentation network is greater than or equal to the preset segmentation accuracy, thereby ensuring the accuracy of the roof segmentation image of the target building segmented by the trained recognition model.

[0109] The method for calculating building height and floor area provided in this embodiment uses a roof segmentation network to determine and output a sample roof segmentation image for each training sample, and determines the segmentation accuracy of the roof segmentation network based on the roof segmentation image label of each training sample. If the segmentation accuracy is determined to be greater than or equal to the segmentation accuracy, the parameters of the roof segmentation network are determined as the parameters of the roof segmentation network in the trained recognition model. If the segmentation accuracy is determined to be less than a preset segmentation accuracy, the parameters of the roof segmentation network are adjusted, and the steps of determining and outputting a sample roof segmentation image for each training sample using the roof segmentation network and determining the segmentation accuracy of the roof segmentation network based on the roof segmentation image label of each training sample are repeated until the segmentation accuracy is greater than or equal to the preset segmentation accuracy. Because the roof segmentation network is trained using a training data set, the parameters of the roof segmentation network are determined as the parameters of the roof segmentation network in the trained recognition model only after the segmentation accuracy of the roof segmentation network is greater than or equal to the preset segmentation accuracy. Therefore, the trained recognition model can accurately obtain a target roof segmentation image of the target building, and can further calculate the accurate floor area of ​​the target building based on the accurate target roof segmentation image.

[0110] As an optional implementation, based on Example 4, the roof segmentation network includes a deconvolution layer. At the same time, the "using the roof segmentation network to determine and output the sample roof segmentation image of each training sample" in step 701 is refined, and the refinement includes step 7011.

[0111] Step 7011: Use a deconvolution layer to perform a deconvolution operation on the feature matrix of at least one ROI matrix of each training sample corresponding to the feature matrix in the sample image feature map, and obtain and output the sample roof segmentation image of each training sample.

[0112] In this embodiment, it can be understood that if the number of ROI matrices output by the image segmentation model is one, then the feature matrix corresponding to this ROI matrix and also in the sample image feature map can be directly deconvolved to obtain a sample roof segmentation image. If the number of ROI matrices output by the image segmentation model is multiple, the same method as in step 501 can be used to obtain the sample position information of the training sample, locate the ROI matrix corresponding to the sample rectangular box, and then perform a deconvolution operation on the feature matrix of the sample rectangular box in the sample image feature map. The deconvolution operation restores the feature matrix to a picture form, such as a white picture with a black background or a black picture with a white background, to represent the roof outline of the sample building.

[0113] Example 5

[0114] Figure 8 This is a flow chart of the method for calculating building height and floor area according to the fifth embodiment of the present application. Figure 8 As shown, the building height and floor area calculation method provided in this embodiment is based on any of the above embodiments, in which the sample satellite information includes the satellite elevation angle. Simultaneously, the step 402 of "inputting the offset angle vector label of each training sample, the sample satellite information, the sample image feature map, and each ROI matrix into the angle analysis network to train the angle analysis network" is refined, and the refinement includes steps 801 to 805.

[0115] Step 801, determining the first angle and second angle of each training sample according to each sample satellite elevation angle; the first angle corresponds to the first offset direction of the sample building in the sample image, and the second angle corresponds to the second offset direction of the sample building in the sample image.

[0116] In this embodiment, since most of the sample images are non-overhead images, the sample buildings have an offset direction in the sample images, or can be understood as an inclined direction. This makes it impossible to proportionally enlarge the height of the sample buildings in the images to obtain the sample building heights. Therefore, it is necessary to calculate the offset angle of the sample buildings and obtain the sample building heights through trigonometric functions. Here, the offset direction of the sample buildings in the sample images is related to the satellite altitude angle. Depending on the satellite altitude angle, the sample buildings can be northeast, southeast, northwest, or southwest. The deflection direction of the sample buildings is actually the relative position relationship between the sample buildings and the satellite, and can therefore be determined based on the satellite altitude angle.

[0117] In summary, if the angle between the sample building and the horizontal coordinate in the sample image is directly determined as the sample building's offset angle, it is impossible to determine whether to use the sin or cos function to calculate the sample building's height. Therefore, it is necessary to determine whether the sample building is more east or north, and then adjust the sample building's tilt angle so that the same trigonometric function is used to calculate the height of each sample building. For example, when the sample building's tilt direction is northeast, the first angle is 0 degrees and the second angle is 90 degrees.

[0118] Step 802 : Rotate the feature matrix of each ROI matrix corresponding to the feature map of the sample image by a first angle and a second angle respectively to obtain first direction matrices and second direction matrices of each sample image.

[0119] In this embodiment, the feature matrix corresponding to the ROI matrix in the sample image feature map may be rotated by the first angle and the second angle through an orthogonal decomposition method to obtain a first direction matrix and a second direction matrix.

[0120] Step 803: Determine and output the sample offset angle vector of each training sample based on each first direction matrix, each second direction matrix and each sample satellite elevation angle, and determine the analysis accuracy of the angle analysis network based on the offset angle vector label of each training sample.

[0121] In this embodiment, for any sample, a multi-layer convolutional neural network can be used to extract features from at least one first direction matrix and the sample satellite elevation angle to obtain a first offset angle vector corresponding to the first direction matrix. A multi-layer convolutional neural network can also be used to extract features from at least one second direction matrix and the sample satellite elevation angle to obtain a second offset angle vector corresponding to the second direction matrix. The one with the higher confidence level between the first and second offset angles can be determined as the sample offset angle vector.

[0122] In this embodiment, the proportion of training samples whose sample offset angle vectors are consistent with or match the offset angle vector labels in the training data set can be determined as the analysis accuracy of the angle analysis network.

[0123] Step 804: If it is determined that the analysis accuracy is greater than or equal to the preset analysis accuracy, the parameters of the angle analysis network are determined as the parameters of the angle analysis network in the trained recognition model.

[0124] In this embodiment, the preset analysis accuracy can be the accuracy that the user expects the angle analysis network to achieve, and can be the same as the preset positioning accuracy. If the analysis accuracy is greater than or equal to the preset analysis accuracy, the parameter training of the angle analysis network is complete, and the parameters of the angle analysis network can be determined as the parameters of the angle analysis network in the trained recognition model.

[0125] Step 805: If it is determined that the analysis accuracy is less than the preset analysis accuracy, the parameters of the angle analysis network are adjusted, and the steps of determining and outputting the sample offset angle vector of each training sample based on each first direction matrix, each second direction matrix and each sample satellite elevation angle, and determining the analysis accuracy of the angle analysis network based on the offset angle vector label of each training sample are repeated until the analysis accuracy is greater than or equal to the preset analysis accuracy.

[0126] In this embodiment, if the analysis accuracy is less than the preset analysis accuracy, the parameters of the angle analysis network can be adjusted through a gradient descent algorithm, a back propagation algorithm, etc. until the analysis accuracy of the angle analysis network is greater than or equal to the preset analysis accuracy, thereby ensuring the accuracy of the target offset angle vector of the target building determined by the trained recognition model.

[0127] The method for calculating the building height and floor area provided in this embodiment determines the first angle and the second angle of each training sample according to the altitude angle of each sample satellite; the first angle corresponds to the first offset direction of the sample building in the sample image, and the second angle corresponds to the second offset direction of the sample building in the sample image; each ROI matrix is ​​rotated by the first angle and the second angle respectively to obtain the first direction matrix and the second direction matrix of each sample image; the sample offset angle vector of each training sample is determined and output according to each first direction matrix, each second direction matrix and each sample satellite altitude angle, and the analysis accuracy of the angle analysis network is determined according to the offset angle vector label of each training sample; if it is determined that the analysis accuracy is greater than or equal to the preset analysis accuracy, the parameters of the angle analysis network are determined as the parameters of the angle analysis network in the trained recognition model; if it is determined that If the analysis accuracy is less than the preset analysis accuracy, the parameters of the angle analysis network are adjusted, and the steps of determining and outputting the sample offset angle vector of each training sample according to each first direction matrix, each second direction matrix and each sample satellite elevation angle, and determining the analysis accuracy of the angle analysis network according to the offset angle vector label of each training sample are repeated until the analysis accuracy is greater than or equal to the preset analysis accuracy; since the angle analysis network is trained using a training data set, the parameters of the angle analysis network are determined as the parameters of the angle analysis network in the trained recognition model until the analysis accuracy of the angle analysis network is greater than or equal to the preset analysis accuracy. Therefore, the trained recognition model can accurately obtain the target offset angle vector of the target building, and then can calculate the accurate target building height based on the accurate target offset angle vector.

[0128] Figure 9 This is a schematic diagram of a structure of an angle analysis network provided by Example 5 of this application. As an optional implementation method, based on Example 5, Figure 9As shown, the angle analysis network 90 includes a plurality of convolutional layers 91. Meanwhile, the step 803 of "determining and outputting the sample offset angle vector of each training sample based on each first direction matrix, each second direction matrix, and each sample satellite elevation angle" is refined, and the refinement includes steps 8031 ​​to 8032.

[0129] In step 8031, a plurality of convolution layers are used to perform convolution operations on the first direction matrix and the second direction matrix respectively to obtain a first offset angle vector and its probability value and a second offset angle vector and its probability value; the first offset angle vector corresponds to the first direction matrix, and the second offset angle vector corresponds to the second direction matrix.

[0130] In this embodiment, if the image segmentation model outputs one ROI matrix, then the number of the first direction matrix and the second direction matrix is ​​also one. The first direction matrix and the second direction matrix can be directly superimposed with the satellite altitude and then convolved to obtain the first offset angle vector and its probability value and the second offset angle vector and its probability value. If the image segmentation model outputs multiple ROI matrices, the same method as in step 501 can be used to first obtain the sample position related information of the training sample, locate the ROI matrix corresponding to the sample rectangular frame, and then rotate the feature matrix framed by the sample rectangular frame in the sample image feature map to obtain the first direction matrix and the second direction matrix.

[0131] Step 8032: Determine the one with a higher probability value between the first offset angle vector and the second offset angle vector as the sample offset angle vector.

[0132] In this embodiment, the first offset angle vector probability value and the second offset angle vector probability value respectively represent the probabilities of the sample building deviating toward the first offset direction and the second offset direction. Therefore, the offset angle vector with the largest probability needs to be selected here.

[0133] The method for calculating the building height and floor area provided in this embodiment uses multiple convolution layers to perform convolution operations on the first direction matrix and the second direction matrix respectively to obtain a first offset angle vector and its probability value and a second offset angle vector and its probability value; the first offset angle vector corresponds to the first direction matrix, and the second offset angle vector corresponds to the second direction matrix; the item with a higher probability value in the first offset angle vector and the second offset angle vector is determined as the sample offset angle vector; since the item with a larger probability value in the offset angle vector is determined as the sample offset angle vector, an accurate sample offset angle vector can be obtained, thereby improving the analysis accuracy of the angle analysis network.

[0134] As an optional implementation, based on any of the above embodiments, the target location-related information includes the length of the target rectangular frame of the target building in the target image, the target offset angle vector includes the horizontal and vertical offset values ​​of the target building from the coordinate origin, and the target satellite information includes the target satellite resolution. Simultaneously, in step 203, the target building height is calculated based on the target satellite information, the target location-related information, the target roof segmentation image, and the target offset angle vector, and then refined. The refinement of step 203 includes steps 2031 through 2034.

[0135] Step 2031 : Calculate the roof image length of the target building roof in the target image based on the target roof segmentation image.

[0136] In this embodiment, the target roof segmentation image may be input into OpenCV software, and the roof image length of the target roof segmentation image in the target image may be calculated using the open source algorithm of the OpenCV software.

[0137] Step 2032: Calculate the height corresponding to the target building in the target image based on the target rectangular frame length and the roof image length.

[0138] In this embodiment, the difference between the length of the target rectangular frame and the length of the roof image may be determined as the height corresponding to the height of the target building in the target image.

[0139] Step 2033: Calculate the target offset angle according to the target offset angle vector.

[0140] In this embodiment, the target offset angle vector includes the horizontal offset value Op and the vertical offset value Oq of the target building from the coordinate origin. The actual offset angle of the target building can be calculated based on the target offset angle vector (Op, Oq) and the origin (0, 0). The actual offset angle of the target building can be an angle α with the x-axis or an angle β with the y-axis.

[0141] Step 2034 , calculating the target building height based on the target satellite resolution, the trigonometric function of the target offset angle, and the height-corresponding length in the target image.

[0142] In this embodiment, since the displayed height of the target building in the target image is proportional to the actual height of the target building, the height H of the target building in the target image can be calculated based on the trigonometric function sinα or cosβ of the target offset angle and the length L4 corresponding to the height in the target image. For example, H = L4sinα or H = L4cosβ. The height of the target building in the target image is then magnified proportionally according to the target satellite resolution to obtain the height of the target building.

[0143] The method for calculating the building height and floor area provided in this embodiment includes target position related information including the target rectangular frame length of the target building in the target image; the target offset angle vector including the horizontal coordinate offset value and the vertical coordinate offset value of the target building and the coordinate origin; the roof image length of the target building roof in the target image is calculated according to the target roof segmentation image; the height corresponding length of the target building in the target image is calculated according to the target rectangular frame length and the roof image length; the target offset angle is calculated according to the target offset angle vector; the target building height is calculated according to the trigonometric function of the target offset angle and the height corresponding length in the target image; since the height corresponding length of the target building in the target image is calculated according to the target rectangular frame length and the roof image length, under the premise that the target rectangular frame length and the target building roof segmentation image are accurate, the height corresponding length of the target building in the target image can be quickly obtained, and then the target building height can be quickly calculated.

[0144] As an optional implementation, based on any of the above embodiments, the target satellite information includes satellite resolution. Meanwhile, in step 204, the target building area is calculated based on the target satellite information and the roof segmentation image and refined. The refinement step 204 includes steps 2041 and 2042.

[0145] Step 2041: Calculate the roof area of ​​the target building in the roof segmentation image using an image processing algorithm.

[0146] In this embodiment, the image provenance algorithm may be any open source algorithm in the OpenCV software, which can calculate the roof area of ​​the target building in the roof segmentation image.

[0147] Step 2042: Calculate the target building's floor area based on the satellite resolution and the target building's roof area.

[0148] In this embodiment, since the target image is a remote sensing image, it is proportional to the actual height and area of ​​the target building. Therefore, the target building's floor area can be obtained by scaling the target building's roof area proportionally to the target satellite resolution. For example, if the target satellite resolution indicates that one pixel in the target image corresponds to an actual area of ​​10 square meters, the target building's floor area can be calculated based on the number of pixels occupied by the target building's roof area in the target image.

[0149] The method for calculating the building height and floor area provided in this embodiment uses target satellite information including satellite resolution to calculate the roof area of ​​the target building in the roof segmentation image using an image processing algorithm; and calculates the floor area of ​​the target building based on the satellite resolution and the roof area of ​​the target building. Since the image processing algorithm calculates the roof area of ​​the target building in the roof segmentation image and calculates the floor area of ​​the target building based on the target satellite resolution, and since the image processing method is an open source algorithm, the floor area of ​​the target building only needs to be proportionally magnified according to the target satellite resolution. Therefore, the floor area of ​​the target building can be quickly and accurately obtained.

[0150] Example 6

[0151] Figure 10 : is a structural diagram of a building height and floor area calculation device provided in accordance with Example 6 of the present application, such as Figure 10 As shown, the building height and floor area calculation device 100 includes an acquisition module 101 , an identification module 102 , a first calculation module 103 and a second calculation module 104 .

[0152] An acquisition module 101 is used to acquire a target image of a target building and target satellite information;

[0153] Recognition model 102 is configured to input a target image and target satellite information into a trained recognition model, and use the trained recognition model to output target location-related information, a target roof segmentation image, and a target offset angle vector of a target building; the trained recognition model is obtained by training a preset recognition model using a training data set, the training data set including at least one training sample, and the training sample including a sample image of a sample building, sample satellite information, a location-related information label, a roof segmentation image label, and an offset angle vector label;

[0154] A first calculation module 103 is configured to calculate the target building height based on target satellite information, target location related information, target roof segmentation image, and target offset angle vector;

[0155] The second calculation module 104 is configured to calculate the target building's floor area based on the target satellite information and the target roof segmentation image.

[0156] As an optional embodiment, the preset recognition model includes an image segmentation model, a positioning network, a roof segmentation network, and an angle analysis network, and the image segmentation model is connected to the positioning network, the roof segmentation network, and the angle analysis network, respectively. The building height and floor area calculation device 100 also includes a training module, which is used to input the sample image of each training sample into the image segmentation model, use the image segmentation model to extract and output the sample image feature map and at least one region of interest (ROI) matrix of each training sample; input the location-related information label, sample image feature map, and each ROI matrix of each training sample into the positioning network to train the positioning network; input the roof segmentation image label, sample image feature map, and each ROI matrix of each training sample into the roof segmentation network to train the roof segmentation network; input the offset angle vector label, sample satellite information, sample image feature map, and each ROI matrix of each training sample into the angle analysis network to train the angle analysis network to obtain a trained recognition model.

[0157] As an optional implementation, when the training module is used to input the position-related information labels of each training sample, the sample image feature map and each ROI matrix into the positioning network and train the positioning network, it is specifically used to use the positioning network to determine and output the sample position-related information of each training sample, and determine the positioning accuracy of the positioning network based on the position-related information labels of each training sample; if the positioning accuracy is determined to be greater than or equal to the preset positioning accuracy, the parameters of the positioning network are determined as the parameters of the positioning network in the trained recognition model; if the positioning accuracy is determined to be less than the preset positioning accuracy, the parameters of the positioning network are adjusted, and the steps of using the positioning network to determine and output the sample position-related information of each training sample, and determining the positioning accuracy of the positioning network based on the position-related information labels of each training sample are repeated until the positioning accuracy is greater than or equal to the preset positioning accuracy.

[0158] As an optional embodiment, the positioning network includes a convolutional layer and a fully connected layer. The training module is further configured to use the convolutional layer to perform a convolution operation on the feature matrix corresponding to each ROI matrix of each training sample in the sample image feature map to obtain sample position related information of each training sample.

[0159] As an optional implementation, when the training module is used to input the roof segmentation image label, sample image feature map and each ROI matrix of each training sample into the roof segmentation network and train the roof segmentation network, it is specifically used to use the roof segmentation network to determine and output the sample roof segmentation image of each training sample, and determine the segmentation accuracy of the roof segmentation network based on the roof segmentation image label of each training sample; if it is determined that the segmentation accuracy is greater than or equal to the segmentation accuracy, the parameters of the roof segmentation network are determined as the parameters of the roof segmentation network in the trained recognition model; if it is determined that the segmentation accuracy is less than the preset segmentation accuracy, the parameters of the roof segmentation network are adjusted, and the steps of using the roof segmentation network to determine and output the sample roof segmentation image of each training sample, and determining the segmentation accuracy of the roof segmentation network based on the roof segmentation image label of each training sample are repeated until the segmentation accuracy is greater than or equal to the preset segmentation accuracy.

[0160] As an optional embodiment, the roof segmentation network includes a deconvolution layer. The training module is further configured to use the deconvolution layer to perform a deconvolution operation on a feature matrix corresponding to at least one ROI matrix of each training sample in the sample image feature map, thereby obtaining and outputting a sample roof segmentation image for each training sample.

[0161] As an optional implementation manner, the sample satellite information includes satellite altitude angle. The training module is specifically further used to determine the first angle and the second angle of each training sample based on the satellite elevation angle of each sample; the first angle corresponds to the first offset direction of the sample building in the sample image, and the second angle corresponds to the second offset direction of the sample building in the sample image; the feature matrix corresponding to each ROI matrix in the feature map of the sample image is rotated by the first angle and the second angle respectively to obtain each first direction matrix and each second direction matrix of each sample image; the sample offset angle vector of each training sample is determined and output according to each first direction matrix, each second direction matrix and each sample satellite elevation angle, and the analysis accuracy of the angle analysis network is determined according to the offset angle vector label of each training sample; if the analysis accuracy is determined to be greater than or equal to the preset analysis accuracy, the parameters of the angle analysis network are determined as the parameters of the angle analysis network in the trained recognition model; if the analysis accuracy is determined to be less than the preset analysis accuracy, the parameters of the angle analysis network are adjusted, and the steps of determining and outputting the sample offset angle vector of each training sample according to each first direction matrix, each second direction matrix and each sample satellite elevation angle, and determining the analysis accuracy of the angle analysis network according to the offset angle vector label of each training sample are repeated until the analysis accuracy is greater than or equal to the preset analysis accuracy.

[0162] As an optional embodiment, the angle analysis network includes multiple convolutional layers. The training module is further specifically configured to use the multiple convolutional layers to perform convolution operations on the first direction matrix and the second direction matrix, respectively, to obtain a first offset angle vector and its probability value and a second offset angle vector and its probability value; the first offset angle vector corresponds to the first direction matrix, and the second offset angle vector corresponds to the second direction matrix; and the item with a higher probability value between the first offset angle vector and the second offset angle vector is determined as the sample offset angle vector.

[0163] As an optional implementation, the target location-related information includes the target rectangular frame length of the target building in the target image, the target offset angle vector includes the horizontal and vertical offset values ​​of the target building from the coordinate origin, and the target satellite information includes the target satellite resolution. When used to calculate the target building height based on the target satellite information, target location-related information, the target roof segmentation image, and the target offset angle vector, the training module is specifically configured to: calculate the roof image length of the target building roof in the target image based on the target roof segmentation image; calculate the height-corresponding length of the target building in the target image based on the target rectangular frame length and the roof image length; calculate the target offset angle based on the target satellite resolution and the target offset angle vector; and calculate the target building height based on a trigonometric function of the target offset angle and the height-corresponding length in the target image.

[0164] As an optional embodiment, the target satellite information includes satellite resolution. When used to calculate the target building's footprint based on the target satellite information and the roof segmentation image, the training module is specifically configured to use an image processing algorithm to calculate the target building's roof area in the roof segmentation image; and calculate the target building's footprint based on the satellite resolution and the target building's roof area.

[0165] The building height and floor area calculation device provided in this embodiment can execute the building height and floor area calculation method provided in any of the above embodiments. The specific implementation method and principle are similar and will not be repeated here.

[0166] Example 7

[0167] Figure 11 : is a block diagram of an electronic device provided according to the seventh embodiment of the present application. Figure 11 As shown, the electronic device 110 provided in this embodiment includes a processor 1101 and a memory 1102 communicatively connected to the processor 1101 .

[0168] The memory 1101 stores computer-executable instructions;

[0169] The processor 1101 executes the computer-executable instructions stored in the memory 1102 to implement the building height and floor area calculation method provided in any one of the above embodiments.

[0170] The communication connection between the processor and the memory may be a bus connection.

[0171] Memory 1102 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk, etc.

[0172] Optionally, the electronic device 110 further includes a transceiver. The processor, memory, and transceiver may be connected via a bus. The transceiver is used to transmit and receive data.

[0173] In an exemplary embodiment, the electronic device 110 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0174] An embodiment of the present invention further provides a computer-readable storage medium storing computer-executable instructions. When executed by a processor, the computer-executable instructions are used to implement the method for calculating building height and floor area provided in any of the above embodiments. For example, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic tape, a floppy disk, or an optical data storage device.

[0175] An embodiment of the present invention further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the method for calculating the building height and floor area provided in any one of the above embodiments.

[0176] It should be understood that the above device embodiments are merely illustrative, and the device of the present application may be implemented in other ways. For example, the module division in the above embodiments is merely a logical functional division, and actual implementation may employ other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be omitted or not implemented.

[0177] In addition, unless otherwise specified, the functional units in the various embodiments of the present application may be integrated into a single module, each module may exist physically separately, or two or more modules may be integrated together. The aforementioned integrated modules may be implemented in the form of hardware or software program modules.

[0178] It should be noted that for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all optional embodiments, and the actions and modules involved are not necessarily required by this application.

[0179] It should be further noted that, although the various steps in the flowchart are shown in sequence as indicated by the arrows, these steps are not necessarily performed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps may be performed in other orders. Moreover, at least a portion of the steps in the flowchart may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily performed at the same time, but may be performed at different times. The execution order of these sub-steps or stages is not necessarily to be performed in sequence, but may be performed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0180] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present application that follow the general principles of the present application and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0181] It should be understood that the present application is not limited to the exact structure described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for calculating building height and floor area, characterized in that: include: Obtain target images and target satellite information of target buildings; Inputting a target image and target satellite information into a trained recognition model, and using the trained recognition model to output target location-related information of a target building, a target roof segmentation image, and a target offset angle vector; the trained recognition model is obtained by training a preset recognition model using a training data set, the training data set including at least one training sample, the training sample including a sample image of a sample building, sample satellite information, a location-related information label, a roof segmentation image label, and an offset angle vector label; Calculate the target building height according to the target satellite information, the target position related information, the target roof segmentation image and the target offset angle vector; Calculate the target building area based on the target satellite information and the target roof segmentation image; The target location related information includes the length of the target rectangular frame of the target building in the target image; The target offset angle vector includes a horizontal coordinate offset value and a vertical coordinate offset value between the target building and the coordinate origin; The target satellite information includes target satellite resolution; The calculating the target building height according to the target satellite information, the target position related information, the target roof segmentation image and the target offset angle vector includes: Calculating the roof image length of the target building roof in the target image according to the target roof segmentation image; Calculate the height corresponding to the length of the target building in the target image according to the length of the target rectangular frame and the length of the roof image; Calculate the target offset angle according to the target offset angle vector; Calculate the target building height based on the target satellite resolution, the trigonometric function of the target offset angle, and the height corresponding length in the target image; The target satellite information includes satellite resolution, and calculating the target building area based on the target satellite information and the roof segmentation image includes: Calculating the roof area of ​​the target building in the roof segmentation image using an image processing algorithm; The target building area is calculated based on the satellite resolution and the roof area of ​​the target building.

2. The method according to claim 1, characterized in that The preset recognition model includes an image segmentation model, a positioning network, a roof segmentation network and an angle analysis network, and the image segmentation model is connected to the positioning network, the roof segmentation network and the angle analysis network respectively; Before inputting the target image and target satellite information into the trained recognition model and using the trained recognition model to output the location-related information, roof segmentation image and offset angle vector of the target building, the following steps are further included: Inputting a sample image of each training sample into an image segmentation model, extracting and outputting a sample image feature map of each training sample and at least one region of interest (ROI) matrix using the image segmentation model; The location-related information labels, sample image feature maps and ROI matrices of each training sample are input into the positioning network to train the positioning network. The roof segmentation image labels, sample image feature maps and ROI matrices of each training sample are input into the roof segmentation network to train the roof segmentation network. The offset angle vector labels, sample satellite information, sample image feature maps and ROI matrices of each training sample are input into the angle analysis network to train the angle analysis network to obtain a trained recognition model.

3. The method according to claim 2, characterized in that The position-related information labels of each training sample, the sample image feature map and each ROI matrix are input into the positioning network to train the positioning network, including: A positioning network is used to determine and output the sample location information of each training sample, and the positioning accuracy of the positioning network is determined based on the location information labels of each training sample; If it is determined that the positioning accuracy is greater than or equal to the preset positioning accuracy, the parameters of the positioning network are determined to be the parameters of the positioning network in the trained recognition model; If it is determined that the positioning accuracy is less than the preset positioning accuracy, the parameters of the positioning network are adjusted, and the steps of using the positioning network to determine and output sample position-related information of each training sample, and determining the positioning accuracy of the positioning network based on the position-related information label of each training sample are repeatedly performed until the positioning accuracy is greater than or equal to the preset positioning accuracy.

4. The method according to claim 3, characterized in that The positioning network includes a convolutional layer and a fully connected layer; The method of using a positioning network to determine and output information related to the sample location of each training sample includes: The convolution layer is used to perform convolution operation on the feature matrix of each ROI matrix of each training sample corresponding to the feature matrix in the sample image feature map to obtain the sample position related information of each training sample.

5. The method according to claim 2, characterized in that The roof segmentation image label of each training sample, the sample image feature map and each ROI matrix are input into the roof segmentation network to train the roof segmentation network, including: A roof segmentation network is used to determine and output a sample roof segmentation image of each training sample, and the segmentation accuracy of the roof segmentation network is determined according to the roof segmentation image label of each training sample; If it is determined that the segmentation accuracy is greater than or equal to the segmentation accuracy, the parameters of the roof segmentation network are determined as the parameters of the roof segmentation network in the trained recognition model; If it is determined that the segmentation accuracy is less than the preset segmentation accuracy, the parameters of the roof segmentation network are adjusted, and the steps of using the roof segmentation network to determine and output the sample roof segmentation image of each training sample, and determining the segmentation accuracy of the roof segmentation network according to the roof segmentation image label of each training sample are repeated until the segmentation accuracy is greater than or equal to the preset segmentation accuracy.

6. The method according to claim 5, characterized in that The roof segmentation network includes a deconvolution layer; The method of using a roof segmentation network to determine and output a sample roof segmentation image of each training sample includes: A deconvolution layer is used to perform a deconvolution operation on the feature matrix corresponding to at least one ROI matrix of each training sample in the sample picture feature map, and a sample roof segmentation image of each training sample is obtained and output.

7. The method according to claim 2, characterized in that The sample satellite information includes satellite altitude angle; The offset angle vector label of each training sample, sample satellite information, sample image feature map and each ROI matrix are input into the angle analysis network to train the angle analysis network, including: Determine a first angle and a second angle of each training sample according to each sample satellite elevation angle; the first angle corresponds to a first offset direction of the sample building in the sample image, and the second angle corresponds to a second offset direction of the sample building in the sample image; Rotate the feature matrix of each ROI matrix corresponding to the feature map of the sample image by a first angle and a second angle respectively to obtain first direction matrices and second direction matrices of each sample image; Determine and output the sample offset angle vector of each training sample according to each first direction matrix, each second direction matrix and each sample satellite elevation angle, and determine the analysis accuracy of the angle analysis network according to the offset angle vector label of each training sample; If it is determined that the analysis accuracy is greater than or equal to the preset analysis accuracy, the parameters of the angle analysis network are determined as the parameters of the angle analysis network in the trained recognition model; If it is determined that the analysis accuracy is less than the preset analysis accuracy, the parameters of the angle analysis network are adjusted, and the steps of determining and outputting the sample offset angle vector of each training sample based on each first direction matrix, each second direction matrix and each sample satellite elevation angle, and determining the analysis accuracy of the angle analysis network based on the offset angle vector label of each training sample are repeated until the analysis accuracy is greater than or equal to the preset analysis accuracy.

8. The method according to claim 7, characterized in that The angle analysis network includes multiple convolutional layers; The method of determining and outputting a sample offset angle vector of each training sample according to each first direction matrix, each second direction matrix, and each sample satellite elevation angle includes: Using multiple convolution layers to perform convolution operations on the first direction matrix and the second direction matrix respectively to obtain a first offset angle vector and its probability value and a second offset angle vector and its probability value; the first offset angle vector corresponds to the first direction matrix, and the second offset angle vector corresponds to the second direction matrix; The one with a higher probability value between the first offset angle vector and the second offset angle vector is determined as the sample offset angle vector.

9. A device for calculating building height and floor area, characterized in that: include: An acquisition module is used to obtain target images and target satellite information of a target building; a recognition model, configured to input a target image and target satellite information into a trained recognition model, and output target location-related information, a target roof segmentation image, and a target offset angle vector of a target building using the trained recognition model; the trained recognition model is obtained by training a preset recognition model using a training data set, the training data set including at least one training sample, the training sample including a sample image of a sample building, sample satellite information, a location-related information label, a roof segmentation image label, and an offset angle vector label; A first calculation module is used to calculate the target building height based on the target satellite information, the target position related information, the target roof segmentation image and the target offset angle vector; A second calculation module is used to calculate the target building area based on the target satellite information and the target roof segmentation image; The target location related information includes the length of the target rectangular frame of the target building in the target image; The target offset angle vector includes a horizontal coordinate offset value and a vertical coordinate offset value between the target building and the coordinate origin; the target satellite information includes the target satellite resolution; The first calculation module is specifically configured to: Calculating the roof image length of the target building roof in the target image according to the target roof segmentation image; Calculate the height corresponding to the length of the target building in the target image according to the length of the target rectangular frame and the length of the roof image; Calculate the target offset angle according to the target offset angle vector; Calculate the target building height based on the target satellite resolution, the trigonometric function of the target offset angle, and the height corresponding length in the target image; The target satellite information includes satellite resolution, and the second calculation module is specifically used to calculate the roof area of ​​the target building in the roof segmentation image using an image processing algorithm; The target building area is calculated based on the satellite resolution and the roof area of ​​the target building.

10. An electronic device, characterized in that: include: a processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 8 when executed by a processor.

12. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 8 when the computer program is executed by a processor.

Images