A method for automatically estimating building area from remote sensing images

Abstract

This invention provides a method for automatically estimating building area in remote sensing images. By employing prompt word engineering, the visual language model (SAM) prompts the segment anything model (SAM) to perform instance segmentation of buildings in remote sensing images. The area of ​​each building in pixels is calculated based on the coordinates of its polygon vertices. Using a car as a medium, the method automatically estimates the scale and converts the pixel-based area into the actual area. Furthermore, the method automatically fine-tunes the model based on user corrections and user reviews, and learns to adaptively adjust hyperparameter configurations based on remote sensing image quality. This allows the method to automatically accumulate experience during use and achieve improved performance.

Description

Technical Field

[0001] The invention relates to a method for automatically estimating the building floor area from a remote sensing image, and belongs to the field of remote sensing image processing. Background Art

[0002] With the acceleration of urbanization, the rational planning and utilization of urban land resources has become increasingly important. As a crucial component of urban infrastructure, accurate estimation of building footprints is crucial for a variety of fields, including urban planning, land management, real estate valuation, and tax collection. Traditional methods for measuring floor space often rely on ground surveying and manual mapping. These methods are not only time-consuming and labor-intensive, but also inefficient and difficult to adapt to the needs of rapid urbanization.

[0003] With the development of remote sensing technology and the internet, obtaining remote sensing images is now easy. Many mapping software programs offer free satellite imagery. Furthermore, the development of drone technology has significantly reduced the cost of obtaining clear remote sensing images. However, obtaining professional, high-definition remote sensing images remains expensive.

[0004] At present, there are mainly the following methods for estimating the building area in remote sensing images:

[0005] Stereo Pair Analysis: This method uses remote sensing image pairs with a certain degree of stereo disparity, combined with the principles of stereo vision, to extract three-dimensional information about buildings and estimate their floor space. This method requires high-quality stereo geometry and is computationally complex.

[0006] Image classification methods use the spectral information of remote sensing images to separate buildings from other features through supervised or unsupervised classification algorithms. Morphological operations are then used to extract building outlines and calculate their areas. This method works best when the image resolution is high, but is affected by factors such as lighting and shadows.

[0007] Shadow analysis technology: Building shadows in remote sensing images provide height information. Combined with sun position and terrain data, the three-dimensional outline of the building can be calculated, and the area occupied can be estimated. Shadow analysis has specific requirements for the time and season of image capture.

[0008] The above methods have high requirements for remote sensing images, and usually require not only images but also a large amount of metadata.

[0009] Furthermore, the use of machine learning to analyze and process remote sensing imagery has also seen rapid development in recent years. These methods utilize machine learning algorithms, such as support vector machines (SVMs) and neural networks, to train models to identify and estimate building footprints. After training on large, labeled datasets, these methods can automatically process new remote sensing imagery, but they require a large number of training samples. For buildings in different cities and with different construction styles, specialized datasets are often required to ensure accurate building identification.

[0010] The Chinese invention patent application with application number CN202311710875.6 provides a method and system for automatic detection and area estimation of grinding burn areas. By technically combining the K-means algorithm and morphological operations, accurate and efficient automatic detection and area estimation of grinding burn areas are achieved. However, it still has the following technical problems: poor generalization ability, the number K of cluster centers in the K-means algorithm needs to be given in advance, but in practice, the selection of this K value is very difficult to estimate. In addition, since it is not known in advance how many categories a given data set should be divided into to be the most appropriate, the result falls into a local optimal solution rather than a global optimal solution. Summary of the Invention

[0011] In order to solve the technical problems of tedious manual search for drawings and difficulty in calculating the manual floor area of ​​non-rectangular buildings, the present invention aims to provide a method for automatically estimating the floor area of ​​a building based on remote sensing images, comprising the following steps:

[0012] S1 obtains remote sensing images uploaded by users

[0013] S2 performs instance segmentation on buildings in remote sensing images:

[0014] S21 uses the prompt word engineering to instruct the visual language model to determine whether the image submitted by the user is a drone aerial image or a satellite image. If not, the user is prompted to re-upload the correct image. If so, the visual language model indicates the number and location of buildings in the image.

[0015] S22. Convert the position given by the visual language model to the bounding box in the image submitted by the user through denormalization;

[0016] S23. The bounding box obtained in S22 is expanded by 20 pixels in length and width with its center as the origin, and the image is cropped to obtain a frame diagram. 20 pixels is an empirical value that has performed well in multiple experiments, avoiding the addition of extra information while ensuring that the complete information is preserved. The frame diagram is then input into an image classification network pre-trained on building images.

[0017] The image classification network is a variational autoencoder structure, the encoder in the variational autoencoder structure is a five-layer convolutional neural network, and the decoder in the variational autoencoder structure is also a five-layer convolutional neural network;

[0018] The cropped frame image is fed into the encoder to obtain an embedding, and the embedding is decoded back into the original image by the decoder;

[0019] A threshold is set to determine whether the difference between the original image and the frame image exceeds the threshold. If so, the image in the bounding box does not belong to the building and the bounding box is removed. Otherwise, the bounding box is retained.

[0020] S24. The bounding box retained in step S23 is used as a prompt word for the segment anything model, and a mask polygon representing the building instance segmentation result is obtained using the segment anything model, and the vertex coordinates of the mask polygon are extracted;

[0021] S3. Build a human-computer interaction interface to obtain user input, wherein the user input includes vertex coordinates after fine-tuning the vertex coordinates of the mask polygon obtained in S24 and user error marking information indicating that the area covered by the mask polygon does not belong to a building;

[0022] S4. Obtaining an area in pixels based on the fine-tuned vertex coordinates;

[0023] S5. Obtain the scale of pixels and actual length, and convert the area in pixels into actual area for output.

[0024] Preferably, in S4, the area in pixels is obtained based on the fine-tuned vertex coordinates using the following formula:

[0025]

[0026] Among them, x i 、x i+1 、y i 、y i+1 They represent respectively: the x-coordinates of the i-th and i+1-th points of the polygon, and the y-coordinates of the i-th and i+1-th points of the polygon; n represents the total number of points in the polygon.

[0027] Preferably, in S5, obtaining the scale of the pixel and the actual length includes the following steps:

[0028] S51. Obtain an aerial car dataset, and use the aerial car dataset to train a target detection network; use the target detection network to detect the car in the picture and obtain a bounding box of the car in the picture;

[0029] S52. After cropping the bounding box obtained in step S51, since car images captured in aerial or satellite images are usually small and blurry, Linear drawing processing is required to obtain a line drawing image of the car in order to enhance the effect of judging the car's orientation. The line drawing image of the car is input into a regression network. The regression network outputs the angle θ between the car's orientation and the horizontal plane based on the input line drawing image of the car. The length of the car in pixels is obtained by the formula Length = cosθ·width, where width is the x-direction width of the bounding box obtained in step S51.

[0030] Preferably, in S51, when it is detected that the number of cars in the picture exceeds one, in S52, several scales are obtained based on the known actual length of each car, and then the average of the several scales is taken as the scale of pixels and actual length.

[0031] Preferably, when the car cannot be detected in the picture in step S51, the scale of pixels and actual length is obtained by obtaining user input.

[0032] Preferably, a reinforcement learning network is constructed, and based on the user input obtained from the human-computer interaction interface in step S3, the temperature value of the visual language large model and the enabling confidence v of the variational autoencoder are conf , the reconstruction loss threshold v of the variational autoencoder thre Make adaptive adjustments;

[0033] The user input includes: the vertex coordinates after fine-tuning obtained in step S3 and the user's annotation information that the area covered by the mask polygon does not belong to a building.

[0034] Preferably, after step S2, the method further includes obtaining a user's score for the building instance segmentation effect from the human-computer interaction interface; the score range is 0-10 points;

[0035] The structure of the reinforcement learning network is a feedforward neural network. The input of the reinforcement learning network is a 6-dimensional vector, whose elements are the resolution length, resolution width, and Brenner gradient of the image submitted by the user in step S2: D(f) = ∑ y ∑ x |f(x+2,y)-f(x,y)| 2 , the average hue, saturation and brightness of the image;

[0036] The reward value of the reinforcement learning network comes from the user input obtained from the human-computer interaction interface in step S3, and specifically includes two parts:

[0037] First, user fine-tuning:

[0038] If there is no user's incorrect marking information that the area covered by the mask polygon does not belong to a building, and the vertex coordinates after fine-tuning do not have any new vertex coordinates compared to the vertex coordinates of the mask polygon obtained in S24, the reward value is +20.

[0039] If the user incorrectly labels the area covered by the mask polygon as not belonging to a building, or if there are newly added vertex coordinates, then the reward value is -20n if the sum of the number of incorrectly labeled information and the number of newly added vertex coordinates is n.

[0040] Second, the score given by users;

[0041] The final reward value is obtained by adding the reward value obtained by the user's fine-tuning and the score given by the user;

[0042] Using deep Q learning algorithm, exploration & exploitation strategy and eposilon-greedy algorithm according to Bellman optimal formula As a label, the network parameters of the reinforcement learning network are updated according to the gradient descent method after calculating the loss; the output of the reinforcement learning network is a three-dimensional vector, whose elements are the temperature offset value t of the visual language model offset , the enabling confidence v of the variational autoencoder conf , the reconstruction loss threshold v of the variational autoencoder thre ;

[0043] The output of the reinforcement learning network is passed through the sigmoid function Scale to between 0-1;

[0044] For the temperature value of the visual language model, the formula temperature=0.25+0.05(1-t offset ) calculates the final temperature value, that is, the temperature value can float between 0.20-0.30 under the control of the reinforcement learning network, and the variational autoencoder enables the confidence v conf If it is lower than 0.5, the variational autoencoder is not enabled for secondary check. If it is greater than or equal to 0.5, the secondary check is enabled and v thre is the reconstruction error threshold.

[0045] Beneficial effects of the present invention:

[0046] The present invention can replace the tediousness of manually searching for drawings and the difficulty of manually calculating the floor area of ​​non-rectangular buildings.

[0047] The present invention utilizes the understanding ability of the large model. After a certain amount of processing, the large model generates SAM algorithm prompt words, which instruct the SAM algorithm to perform instance segmentation on the building. This method has stronger robustness than the traditional computer vision method based on supervised learning. With the development of society and the advancement of architectural technology, people's aesthetic taste for buildings has been changing. The design and style of buildings no longer follow the fixed patterns of the past, and various eye-catching architectural designs emerge in an endless stream. The shape of a building does not have a unified shape and color feature. Although the traditional computer vision method based on supervised learning also has a certain generalization ability, it is generally dependent on the quality of the data set and the exhaustive enumeration of various types of buildings. For some buildings with novel shapes and colors, the traditional computer vision method based on supervised learning is prone to missed detection problems. The method of the present invention has a better recognition effect for buildings of different styles, which reflects the robustness of the method of the present invention.

[0048] The area calculation of the present invention is performed based on polygon vertex coordinates, and has higher accuracy than the traditional rectangular approximation (grid occupancy) calculation method.

[0049] The present invention incorporates a user feedback mechanism, and through a self-evolutionary mechanism and a reinforcement learning adaptive parameter adjustment mechanism, it can automatically and continuously improve performance based on user feedback. Thanks to the use of reinforcement learning, the present invention can learn parameter adjustment strategies for images of varying quality based on user ratings. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 This is a drone remote sensing image of a certain place.

[0051] Figure 2 This is a satellite remote sensing image of the second teaching building of Hangzhou Dianzi University.

[0052] Figure 3 This is a satellite remote sensing image of Hangzhou city balconies.

[0053] Figure 4 This is a drone remote sensing image.

[0054] Figure 5 Satellite remote sensing image.

[0055] Figure 6 Satellite remote sensing image.

[0056] Figure 7 This is a schematic diagram of the area calculation results in pixels of a drone remote sensing image of a certain place.

[0057] Figure 8 This is a schematic diagram of the area calculation results in pixels of the satellite remote sensing image of the second teaching building of Hangzhou Dianzi University.

[0058] Figure 9 This is a schematic diagram of the area calculation results of the satellite remote sensing image of Hangzhou city balconies in pixels.

[0059] Figure 10 This is a schematic diagram of the area calculation results of a drone remote sensing image of a certain place.

[0060] Figure 11 Schematic diagram of the area calculation results of the satellite remote sensing image of the second teaching building of Hangzhou Dianzi University.

[0061] Figure 12 Schematic diagram of the area calculation results of the satellite remote sensing image of Hangzhou urban balconies.

[0062] Figure 13 Flowchart of the present invention. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solutions and advantages of this application more clearly understood, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0064] A method for automatically estimating building footprints from remote sensing images comprises the following steps:

[0065] s1 obtains remote sensing images uploaded by users, for example: Figure 1 (UAV remote sensing image of a certain place), Figure 2 (Satellite remote sensing image of the second teaching building of Hangzhou Dianzi University), Figure 3 (Satellite remote sensing image of Hangzhou city balconies).

[0066] s2 performs instance segmentation on buildings in remote sensing images. The specific steps are: 1. Use prompt word engineering to instruct the visual language model (VLM) to determine whether the picture submitted by the user is a drone aerial picture or a satellite picture. If not, the user is prompted to re-upload the correct picture. If so, the visual language model indicates the number of buildings in the picture and their approximate location. 3. Through steps such as denormalization, the approximate location given by the visual language model is converted into the approximate coordinates of the actual bounding box in the picture uploaded by the user. 4. The bounding box obtained in step 3 is taken as the origin at the center point of the bounding box, and the length and width are expanded by 20 pixels respectively. The picture is cropped and the cropped picture is input into the picture classification network trained by a large number of building pictures (the specific structure is a variational autoencoder, the encoder is a five-layer convolutional neural network, and the decoder is also a five-layer convolutional neural network. The cropped box image is sent to the encoder to obtain the embedding, and the decoder re-decodes the embedding into the original image. Since the variational autoencoder has been trained on about 1,000 aerial pictures of buildings, the variational autoencoder will It is good at encoding building images into embeddings and then decoding the embeddings, so the decoded images will be similar to the original images, that is, the reconstruction error is small. For non-building images, since the variational autoencoder is not good at encoding and decoding them, the decoded images will be quite different from the original images, that is, the reconstruction error is large. We remove the bounding boxes that are significantly different from the cropped images. This step serves as an automatic inspection step to reduce the false detection rate of buildings in the image. 5. The bounding box coordinates obtained in step three and checked again in step four are used as the prompt words for the segment anything (SAM) model. After the segment anything model, the instance segmentation results of the building will be obtained, and the vertex coordinates of the mask polygon will be extracted. Figure 4 (UAV remote sensing image), Figure 5 (satellite remote sensing image), Figure 6 As shown in the satellite remote sensing image (the square box in the image is the approximate bounding box obtained in steps 1 and 2, and the dark mask is the segmented building area), it can be seen from the image that the present invention can effectively segment buildings of different types, shapes, and styles from the image in a zero-shot manner.

[0067] S3 displays the resulting building instance segmentation results to the user. Users can fine-tune the vertex coordinates of the mask polygon by dragging it, or they can delete an entire building mask. If a user deletes an entire building mask, a pop-up window will appear asking whether they simply do not want to calculate the building's floor area or if it was a false positive. If the user indicates a false positive, the data will be recorded in the backend. Users can also manually add an entire mask polygon; if they do, the data will also be recorded in the backend.

[0068] S4 For each polygon fine-tuned by the user after step S3, use the formula according to its vertex coordinates

[0069]

[0070] Calculate its area. The area obtained in this step is in pixels. Figure 7 , Figure 8 and Figure 9 shown

[0071] s5 Automatically determine the scale 1. Use the object detection network trained on the aerial car dataset to detect the cars in the image and obtain the bounding boxes for the cars in the image. 2. Determine the orientation of these cars. Crop the bounding boxes obtained in step 1. Since the car images captured in aerial or satellite images are usually small and blurry, in order to enhance the effect of judging the orientation of the car, we first perform Linear drawing processing on the image. The processed image is input into

[0072] A regression network is introduced. The network can predict the angle between the direction of the car and the horizontal plane for the input line drawing picture of the car. After obtaining the predicted angle θ, the formula Length = cosθ·width is used.

[0073] Among them, width is the width (x direction) of the bounding box obtained in step 1. The length of the car in pixels can be obtained. If there are multiple cars in the picture, the average value is calculated separately. The ratio of the actual length of the car (preset to 4 meters) and the pixel length can be obtained to obtain the scale, and then the area obtained by s4 can be converted into a specific value. For extreme cases, such as when the picture is too blurry to detect the car normally or there is no car in the picture, the user can manually specify the scale. After obtaining the scale, the actual area can be converted, such as Figure 10 , Figure 11 , Figure 12 shown.

[0074] s6 is a user feedback mechanism where users can rate the model's performance on the automatic segmentation in step s2. The score range is 0-10.

[0075] s7 Self-Evolution Mechanism: The method of the present invention has a self-evolution mechanism. In s4, we record the missed detections and false detections reported by users in the background. For false detections, we trace back to the prompts provided by the visual language model and remove the prompts specifying the location in the image. For missed detections, we use the minimum enclosing rectangle of the polygon drawn by the user to reverse-infer the response from the large visual language model. When the number of error correction samples reaches 200, the large visual language model is automatically fine-tuned using the LORA (low rank adaptation) method at night. Afterwards, the visual language model's ability to provide the segmentanything model prompt word is verified on the test set to determine whether the model should be replaced.

[0076] s8 hyperparameter adaptive mechanism. The method of the present invention uses reinforcement learning to give two hyperparameters, namely the temperature parameter of the visual language model, the enabling parameter of the variational autoencoder, and the abnormality threshold parameter, to its adaptive ability. The temperature parameter affects the behavior of the model by changing the probability distribution of the softmax function output. When the temperature t is introduced, the expression of the softmax function is:

[0077] Therefore, when the temperature parameter is low, the output probability distribution of the softmax function becomes more spiky, meaning the variance increases. This means that the predictions with the highest probability become more prominent, while other low-probability predictions are further suppressed. In this case, the model tends to select the output with the highest probability, making the output more deterministic and reducing randomness. Conversely, when the temperature parameter is high, the output probability distribution of the softmax function becomes flatter. This is because the temperature parameter reduces the relative variance between logits, causing high-probability predictions to become less prominent and low-probability predictions to increase in relative abundance. Therefore, at high temperatures, the model output becomes more random. To ensure industrial application and strong reproducibility of the model, the temperature of the visual language model needs to be locked to a low value. This initial value is set to 0.25, but for images of different resolutions and clarity tones, allowing the temperature to fluctuate slightly may achieve better results. In some cases, the reconstruction error threshold of the variational autoencoder can be set higher, even eliminating the need for a secondary check of the variational autoencoder. In other cases, a more rigorous secondary check is required. This paper designs a simple reinforcement learning network that automatically adjusts the inference hyperparameters of the large model and other structural hyperparameters based on user feedback. The reinforcement learning network structure is a feedforward neural network. The input of the network is a 6-dimensional vector, whose elements are the resolution length, resolution width, and Brenner gradient of the input image: D(f) = ∑ y ∑ x|f(x+2,y)-f(x,y)| 2 (reflects the clarity of the image), the average hue, saturation and brightness of the image. The reward consists of two parts: 1. Whether the user in s3 has deleted an entire segmentation due to false detection, and whether the segmentation has been manually added. If this does not happen, the reward is +20. If this happens, let the total number of masks added by the user and deleted due to false detection be n, then the reward is -20n. 2. The user's score in s6 (0-+10). The final reward is obtained by adding the two items. During the user's use, the deep Q learning algorithm is used, and the exploration and exploitation strategy and the eposilon-greedy algorithm are used according to the Bellman optimal formula.

[0078] As the label, the network parameters are updated according to the gradient descent method after calculating the loss. The output of the network is a three-dimensional vector, whose elements are the temperature offset value t of the visual language model offset , the enabling confidence v of the variational autoencoder conf , the reconstruction loss threshold v of the variational autoencoder thre . The output of the network is passed through the sigmoid function Scaled to between 0 and 1. For the temperature value of the visual language large model, temperature = 0.25 + 0.05 (0.5-t offset ) determines the final temperature value (i.e., the temperature value can float between 0.20-0.30 under the control of the reinforcement learning network), and the variational autoencoder enables the confidence v conf If it is lower than 0.5, the variational autoencoder is not enabled to perform secondary checks on the 1, 2, 3, and 4 steps of s2. If it is greater than or equal to 0.5, the secondary check is enabled and v thre is the reconstruction error threshold. Setting the variational autoencoder's enabling confidence level to 0.5 is a compromise value found after multiple experiments. The temperature parameter is allowed to fluctuate between 0.2 and 0.3, providing a certain adjustment range while maintaining a relatively low temperature value. This ensures that the reinforcement learning neural network can adjust the large visual language model to a certain extent and that the present invention has good reproducibility.

[0079] The above description is only part of the specific implementation methods of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by any person familiar with the art within the technical scope disclosed in the present invention should be covered by the protection scope of the present invention.

Claims

1. A method for automatically estimating building area from remote sensing images, characterized in that: The following steps are involved: S1 obtains remote sensing images uploaded by users S2 performs instance segmentation on buildings in remote sensing images: S21 uses the prompt word engineering to instruct the visual language model to determine whether the image submitted by the user is a drone aerial image or a satellite image. If not, the user is prompted to re-upload the correct image. If so, the visual language model indicates the number and location of buildings in the image. S22. Convert the position given by the visual language model to the bounding box in the image submitted by the user through denormalization; S23. The bounding box obtained in S22 is expanded by 20 pixels in length and width with the center point of the bounding box as the origin, and the image is cropped to obtain a frame image; the frame image is input into the image classification network pre-trained by the building image; The image classification network is a variational autoencoder structure, the encoder in the variational autoencoder structure is a five-layer convolutional neural network, and the decoder in the variational autoencoder structure is also a five-layer convolutional neural network; The cropped frame image is fed into the encoder to obtain an embedding, and the embedding is decoded back into the original image by the decoder; A threshold is set to determine whether the difference between the original image and the frame image exceeds the threshold. If so, the image in the bounding box does not belong to the building and the bounding box is removed. Otherwise, the bounding box is retained. S24. The bounding box retained in step S23 is used as a prompt word for the segment anything model, and the mask polygon representing the building instance segmentation result is obtained using the segment anything model, and the vertex coordinates of the mask polygon are extracted; S3. Build a human-computer interaction interface to obtain user input, wherein the user input includes vertex coordinates after fine-tuning the vertex coordinates of the mask polygon obtained in S24 and user error marking information indicating that the area covered by the mask polygon does not belong to a building; S4. Obtaining an area in pixels based on the fine-tuned vertex coordinates; S5. Obtain the scale of pixels and actual length, and convert the area in pixels into actual area for output.

2. The method for automatically estimating building footprint from remote sensing images according to claim 1, wherein: In S4, the area in pixels is obtained based on the fine-tuned vertex coordinates using the following formula: Among them, x i 、x i+1 、y i 、y i+1 They represent respectively: the x-coordinates of the i-th and i+1-th points of the polygon, and the y-coordinates of the i-th and i+1-th points of the polygon; n represents the total number of points in the polygon.

3. The method for automatically estimating building footprint from remote sensing images according to claim 1, wherein: In S5, obtaining the scale of the pixel and the actual length includes the following steps: S51. Obtain an aerial car dataset, and use the aerial car dataset to train a target detection network; use the target detection network to detect the car in the picture and obtain a bounding box of the car in the picture; S52. After cropping the bounding box obtained in step S51, Linear processing is performed to obtain a line drawing image of the car. The line drawing image of the car is input into a regression network. The regression network outputs the angle θ between the car's orientation and the horizontal plane based on the input line drawing image of the car. The length of the car in pixels is calculated using the formula Length = cosθ·width, where width is the x-direction width of the bounding box obtained in step S51.

4. The method for automatically estimating building footprint from remote sensing images according to claim 3, wherein: In S51, when it is detected that the number of cars in the picture exceeds one, in S52, several scales are obtained based on the known actual length of each car, and then the average of the several scales is taken as the scale of pixels and actual length.

5. The method for automatically estimating building footprint from remote sensing images according to claim 3, wherein: When the car cannot be detected in the image in step S51 , the scale of pixels and actual length is obtained by obtaining user input.

6. The method for automatically estimating building footprint from remote sensing images according to claim 1, wherein: The following steps are also included: Construct a reinforcement learning network, based on the user input obtained from the human-computer interaction interface in step S3, to calculate the temperature value of the visual language model and the enabling confidence v of the variational autoencoder. conf , the reconstruction loss threshold v of the variational autoencoder thre Make adaptive adjustments; The user input includes: the vertex coordinates after fine-tuning obtained in step S3 and the user's annotation information that the area covered by the mask polygon does not belong to a building.

7. The method for automatically estimating building footprint from remote sensing images according to claim 6, wherein: After step S2, the method further includes obtaining a user's score for the building instance segmentation effect from the human-computer interaction interface; the score range is 0-10 points; The structure of the reinforcement learning network is a feedforward neural network. The input of the reinforcement learning network is a 6-dimensional vector, whose elements are the resolution length, resolution width, and Brenner gradient of the image submitted by the user in step S2: D(f) = ∑ y ∑ x |f(x+2,y)-f(x,y)| 2 , the average hue, saturation and brightness of the image; The reward value of the reinforcement learning network comes from the user input obtained from the human-computer interaction interface in step S3, and specifically includes two parts: First, user fine-tuning: If there is no user's incorrect marking information that the area covered by the mask polygon does not belong to a building, and the vertex coordinates after fine-tuning do not have any new vertex coordinates compared to the vertex coordinates of the mask polygon obtained in S24, the reward value is +20. If the user incorrectly labels the area covered by the mask polygon as not belonging to a building, or if there are newly added vertex coordinates, then the reward value is -20n if the sum of the number of incorrectly labeled information and the number of newly added vertex coordinates is n. Second, the user-rated scores; The final reward value is obtained by adding the reward value obtained by the user's fine-tuning and the score given by the user; Using deep Q learning algorithm, exploration & exploitation strategy and eposilon-greedy algorithm according to Bellman optimal formula As a label, the network parameters of the reinforcement learning network are updated according to the gradient descent method after calculating the loss; the output of the reinforcement learning network is a three-dimensional vector, whose elements are the temperature offset value t of the visual language model offset , the enabling confidence v of the variational autoencoder conf , the reconstruction loss threshold v of the variational autoencoder thre ; The output of the reinforcement learning network is passed through the sigmoid function Scale to between 0-1; For the temperature value of the visual language model, the formula temperature=0.25+0.05(1-t offset ) calculates the final temperature value, and the variational autoencoder enables confidence v conf If it is lower than 0.5, the variational autoencoder is not enabled for secondary check. If it is greater than or equal to 0.5, the secondary check is enabled and v thre is the reconstruction error threshold.

Images