Building label alignment method, device, equipment, storage medium and program product

Abstract

The application provides a building label alignment method, device, equipment, storage medium and program product, and relates to the technical field of remote sensing image processing. The method comprises the following steps: obtaining a remote sensing image and a label to be aligned; inputting the remote sensing image and the label to be aligned into a pre-trained label alignment model, performing multi-step iterative reasoning, and obtaining target base position labeling and target roof position labeling output by the label alignment model; the label alignment model is obtained by performing noise training on an initial model based on a sample remote sensing image, a sample building label corresponding to the sample remote sensing image, and a sample correction vector corresponding to the sample building label. Through the above method, the model has good label correction capability and noise removal capability, effectively improves the accuracy of label alignment, and is more suitable for actual remote sensing image application scenarios.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image processing technology, and in particular to a method, apparatus, device, storage medium, and program product for aligning building labels. Background Technology

[0002] Building extraction from remote sensing images is a fundamental task in many fields, including urban planning, land management, and urban model generation. With the widespread adoption of Open Street Maps (OSM), utilizing historical building tags to assist in updating and extracting building information from the latest remote sensing images has become an efficient and feasible method. Historical building tags typically include historical base location markings and historical roof location markings, which can be used to pinpoint the building's exact location.

[0003] However, for each building, there is often a positional offset between its historical building label and the most recent remote sensing image of that building. This offset is particularly complex in remote sensing images taken at an angle. In vertically captured remote sensing images (i.e., images taken directly above the building), the roof outline of the building largely overlaps with its base outline, so the roof outline features in the remote sensing image can be directly used to align the historical base location labels and historical roof location labels. However, in remote sensing images taken at an angle (i.e., images not taken directly above the building), due to the imaging angle, there is a significant spatial displacement between the roof outline and the base outline of the building, and the displacement varies for buildings of different heights. This makes it difficult to directly align the historical base location labels and historical roof location labels using roof outline features, increasing the difficulty of aligning and updating historical building labels.

[0004] In some related technologies, models can be used to extract the roof and base contours of buildings in remote sensing images, calculate the spatial offset between the roof and base contours and the historical building labels, and finally align the historical building labels based on the spatial offset.

[0005] However, most existing algorithms rely on parameter-sensitive post-processing operations to identify roof and base contours. This results in low accuracy in building identification, leading to low precision in roof and base contour extraction, which in turn affects the alignment accuracy of subsequent historical building labels. Furthermore, during the model inference stage, most existing algorithms employ single-step prediction methods to align historical building labels. However, single-step prediction methods suffer from insufficient accuracy and are primarily used in near-satellite imaging, making them difficult to apply directly to remote sensing image processing.

[0006] Therefore, existing building label alignment methods suffer from low label alignment accuracy and are difficult to apply directly to remote sensing image processing. Summary of the Invention

[0007] This invention provides a building label alignment method, apparatus, device, storage medium, and program product to solve the problem of low label alignment accuracy in existing building label alignment methods, which makes them difficult to apply directly to remote sensing image processing.

[0008] This invention provides a building label alignment method, comprising: acquiring a remote sensing image and labels to be aligned; the remote sensing image is obtained by photographing an area containing buildings, and the labels to be aligned are historical building labels of the buildings, including historical base location labels and historical roof location labels of the buildings; inputting the remote sensing image and the labels to be aligned into a pre-trained label alignment model, performing multi-step iterative inference to obtain target base location labels and target roof location labels output by the label alignment model; aligning the target base location labels with the base contour of the buildings, and aligning the target roof location labels with the roof contour of the buildings; wherein, the label alignment model is obtained by training an initial model based on the DragOSM algorithm with noise based on sample remote sensing images, sample building labels corresponding to the sample remote sensing images, and sample correction vectors corresponding to the sample building labels.

[0009] According to the building label alignment method provided by the present invention, the target base location label and the target roof location label are generated by the label alignment model based on the following steps: based on the mapping function and the scaling factor, the label to be aligned is subjected to multi-step iterative reasoning to determine the base outline of the building and generate the target base location label; the mapping function is used to inversely map the label to be aligned to the base outline to generate the target base location label, and the scaling factor is used to control the single-step correction amount in the multi-step iterative reasoning process; based on the base outline, the roof outline of the building is determined according to the fixed offset relationship between the base and the roof and the target roof location label is generated.

[0010] According to the present invention, a building label alignment method is provided, which performs multi-step iterative reasoning on the labels to be aligned based on a mapping function and a scaling factor to determine the base outline of the building and generate a target base location label. The method includes: determining the basic reasoning step count; performing iterative reasoning on the labels to be aligned according to the basic reasoning step count based on the mapping function and scaling factor to generate a basic convergence result; using the basic convergence result as the current convergence result; adding random noise to the current convergence result and performing iterative reasoning on the current convergence result according to the basic reasoning step count based on the mapping function and scaling factor to obtain an updated convergence result; using the updated convergence result as the current convergence result and returning to the step of adding random noise to the current convergence result and performing iterative reasoning on the current convergence result according to the basic reasoning step count based on the mapping function and scaling factor, until multiple updated convergence results are obtained; and determining the base outline of the building based on the multiple updated convergence results to generate a target base location label.

[0011] According to the present invention, a building label alignment method is provided, which performs multi-step iterative reasoning on the labels to be aligned based on a mapping function and a scaling factor to determine the base outline of the building and generate a target base position label. The method includes: determining the basic reasoning step number and the additional reasoning step number; performing iterative reasoning on the labels to be aligned according to the basic reasoning step number based on the mapping function and the scaling factor to generate a basic convergence result; performing iterative reasoning on the basic convergence result according to the additional reasoning step number to obtain a target convergence result; and determining the base outline of the building based on the mean center of the target convergence result to generate a target base position label.

[0012] According to the present invention, a building label alignment method is provided, wherein the label alignment model is trained based on the following steps: obtaining sample remote sensing images and the original building labels corresponding to the sample remote sensing images; the sample remote sensing images are obtained by taking pictures of the areas containing sample buildings; determining sample correction vectors based on the original building labels; adding Gaussian noise to the original building labels to generate sample building labels; and training an initial model based on the DragOSM algorithm based on the sample remote sensing images, sample building labels, and sample correction vectors to generate a label alignment model.

[0013] According to a building label alignment method provided by the present invention, the sample correction vector includes a first correction vector, a second correction vector, and a third correction vector; wherein, the first correction vector is determined based on the original building label and the actual base position of the sample building; the second correction vector is determined based on the original building label and the actual roof position of the sample building; and the third correction vector is determined based on the actual base position and the actual roof position of the sample building.

[0014] This invention also provides a building label alignment device, comprising: an acquisition module for acquiring remote sensing images and labels to be aligned; the remote sensing images are obtained by photographing areas containing buildings, and the labels to be aligned are historical building labels of the buildings, including historical base location labels and historical roof location labels of the buildings; a label alignment module for inputting the remote sensing images and labels to be aligned into a pre-trained label alignment model, performing multi-step iterative inference, and obtaining target base location labels and target roof location labels output by the label alignment model; the target base location labels are aligned with the base contour of the building, and the target roof location labels are aligned with the roof contour of the building; wherein, the label alignment model is obtained by training an initial model based on the DragOSM algorithm with noise based on sample remote sensing images, sample building labels corresponding to the sample remote sensing images, and sample correction vectors corresponding to the sample building labels.

[0015] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any of the building label alignment methods described above.

[0016] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the building label alignment methods described above.

[0017] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements any of the building label alignment methods described above.

[0018] The building label alignment method, apparatus, device, storage medium, and program product provided by this invention do not employ a single-step prediction method in the model inference stage. Instead, they introduce a multi-step iterative inference method. After inputting remote sensing images and labels to be aligned into the label alignment model, the label alignment model, constructed based on the DragOSM algorithm, can perform multi-step iterative inference on the remote sensing images and labels to be aligned. By iteratively reducing the inference error caused by single-step prediction, the model achieves stable convergence from unaligned labels to aligned labels. It outputs target base position labels aligned with the building's base contour and target roof position labels aligned with the building's roof contour, effectively improving the accuracy of label alignment. Simultaneously, noise-based training is introduced in the model training stage. Unlike traditional methods that use noise-free or denoised samples for model training, this invention utilizes sample data for noise training during the model training phase. Since the spatial offset of buildings caused by shooting angle deviations in real-world application scenarios can be regarded as noise by the model, and the convergence process from unaligned labels to aligned labels can be regarded as a label denoising process by the model, introducing noise during the model training phase can simulate the spatial offset of buildings in real-world application scenarios. By using sample remote sensing images, sample building labels corresponding to the sample remote sensing images, and sample correction vectors corresponding to the sample building labels to train the model with noise, the model can have good label correction and denoising capabilities, and is more adaptable to real-world remote sensing image application scenarios. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0020] Figure 1 This is one of the flowcharts illustrating the building label alignment method provided by the present invention.

[0021] Figure 2 This is the second flowchart illustrating the building label alignment method provided by this invention.

[0022] Figure 3 This is the third flowchart illustrating the building label alignment method provided by this invention.

[0023] Figure 4 This is a comparative schematic diagram of the enhancement strategies provided by the present invention during testing.

[0024] Figure 5 This is a comparison chart showing the implementation effects of the building label alignment method provided by this invention and traditional methods on typical remote sensing images.

[0025] Figure 6 This is a structural schematic diagram of the building label alignment device provided by the present invention.

[0026] Figure 7 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0028] Please see Figures 1 to 4 , Figure 1 This is one of the flowcharts illustrating the building label alignment method provided by the present invention. Figure 2 This is the second flowchart illustrating the building label alignment method provided by the present invention. Figure 3 This is the third flowchart illustrating the building label alignment method provided by this invention. Figure 4 This is a comparative schematic diagram of the enhancement strategies provided by the present invention during testing.

[0029] like Figure 1 As shown, in this embodiment, the building label alignment method includes steps S110 to S120, and the specific steps are as follows: S110: Obtain remote sensing images and labels to be aligned.

[0030] Remote sensing images are obtained by taking pictures of areas containing buildings. The labels to be aligned are the historical building labels of the buildings, which include the location of the historical base and the location of the historical roof.

[0031] S120: Input the remote sensing image and the label to be aligned into the pre-trained label alignment model, perform multi-step iterative inference, and obtain the target base location label and the target roof location label output by the label alignment model.

[0032] The target base location is aligned with the building's base outline, and the target roof location is aligned with the building's roof outline.

[0033] The label alignment model is obtained by training an initial model based on the DragOSM algorithm with noise, based on the sample remote sensing image, the sample building label corresponding to the sample remote sensing image, and the sample correction vector corresponding to the sample building label.

[0034] Specifically, such as Figure 2As shown and Figure 3 During the model training phase, publicly available remote sensing images can be obtained as sample remote sensing images. These sample remote sensing images are obtained by taking pictures of areas containing sample buildings. At the same time, the original building labels corresponding to the sample remote sensing images are obtained, and the sample correction vectors used for model training are obtained through alignment.

[0035] Optionally, the original building labels corresponding to the sample remote sensing images can be determined by an OpenStreetMap (OSM), where the original building labels are the real historical building labels corresponding to the sample buildings in the sample remote sensing images.

[0036] Furthermore, random Gaussian noise is dynamically injected into the original building labels corresponding to the sample remote sensing images to simulate the spatial location shift of buildings in the real world, generating sample building labels carrying noise. Figure 2 (Noised Polygon Labels).

[0037] Furthermore, based on sample remote sensing images, sample building labels, and sample correction vectors, the initial model constructed using the DragOSM algorithm is trained with noise. Figure 3 (Add noise training to the left image in the middle). The initial model completes the training process by learning to recover the sample correction vector as the ground truth from the building labels of noisy samples, and finally generates a label alignment model.

[0038] The core idea of ​​the DragOSM algorithm model is to regard the label alignment process as an iterative drag-and-drop process from noisy labels to precise positions. Through dynamic noise training and multi-step denoising inference, high-precision label alignment is achieved.

[0039] Furthermore, after completing model training, the model inference stage (i.e. model application stage) is entered. The model inference process can be modeled as a multi-step iterative denoising process. By iterating through multiple steps, the prediction error of the model is gradually reduced, thereby achieving convergence from noise labels to the precise location of buildings.

[0040] Specifically, the process begins by acquiring remote sensing images and labels to be aligned. The remote sensing images are obtained by taking pictures of the area containing the buildings, and the labels to be aligned are the historical building labels of the buildings, which include the location markings of the historical base and the location markings of the historical roof.

[0041] Furthermore, the remote sensing image and the label to be aligned are input into the label alignment model. The label alignment model can perform multi-step iterative reasoning based on the remote sensing image and the label to be aligned to achieve multi-step iterative denoising of the label to be aligned and gradually correct the position of the label to be aligned in the remote sensing image, thereby outputting the target base location label and the target roof location label.

[0042] Specifically, the target base location is aligned with the base outline of the building in the remote sensing image, and the target roof location is aligned with the roof outline of the building in the remote sensing image.

[0043] The building label alignment method provided in this embodiment does not employ a single-step prediction method during the model inference stage. Instead, it introduces a multi-step iterative inference method. After inputting the remote sensing image and the label to be aligned into the label alignment model, the label alignment model, constructed based on the DragOSM algorithm, can perform multi-step iterative inference on the remote sensing image and the label to be aligned. By iteratively reducing the inference error caused by single-step prediction, it achieves stable convergence from unaligned labels to aligned labels, outputting target base position labels aligned with the building's base outline and target roof position labels aligned with the building's roof outline, effectively improving the accuracy of label alignment. Simultaneously, a noise-based training mechanism is introduced during the model training stage, which differs from traditional methods. Unlike other methods that use noise-free or denoised samples for model training, this invention uses sample data for noise training during the model training phase. Since the spatial offset of buildings caused by shooting angle deviations in real-world application scenarios can be regarded as noise by the model, and the convergence process from unaligned labels to aligned labels can be regarded as the label denoising process by the model, introducing noise during the model training phase can simulate the spatial offset of buildings in real-world application scenarios. By using sample remote sensing images, sample building labels corresponding to the sample remote sensing images, and sample correction vectors corresponding to the sample building labels to train the model with noise, the model can have good label correction and denoising capabilities, and is more adaptable to real-world remote sensing image application scenarios.

[0044] In some embodiments, the target base location label and the target roof location label are generated by the label alignment model based on the following steps: performing multi-step iterative reasoning on the label to be aligned based on a mapping function and a scaling factor to determine the base profile of the building and generate the target base location label; the mapping function is used to inversely map the label to be aligned to the base profile to generate the target base location label, and the scaling factor is used to control the single-step correction amount in the multi-step iterative reasoning process; based on the base profile, determining the roof profile of the building according to the fixed offset relationship between the base and the roof, and generating the target roof location label.

[0045] In this embodiment, the model inference process can be modeled as a multi-step iterative denoising process. By iteratively reducing the model's prediction error step by step, the convergence from noise labels to the precise location of buildings can be achieved.

[0046] Specifically, during the model training phase, the label alignment model can learn a mapping function using training samples. .

[0047] Mapping function It can be used to inversely map historical building labels, including historical base location markings and historical roof location markings, to the actual base outline of the building in a remote sensing image. The target base position label is obtained by aligning it with the actual base outline of the building in the remote sensing image.

[0048] Understandably, since the base and roof outlines of buildings are typically polygonal, the historical base location markers and historical roof location markers are also typically polygonal, and the mapping function... The inverse mapping process can be viewed as transforming a noisy polygon into a non-noisy polygon. Inverse mapping to the actual base profile of the building .

[0049] Furthermore, in the model inference phase, a discrete sequence of states is defined, denoted as... ,in, This indicates the historical building labels that need to be aligned. This represents the target base location label output by the label alignment model. Since the target base location label is the actual base contour of the building in the remote sensing image, Aligned, therefore It can also be viewed as the actual base profile of the building ultimately predicted by the label alignment model. .

[0050] Specifically, after inputting the remote sensing image and the label to be aligned into the label alignment model, the label alignment model can be based on a mapping function. and scaling factor The system performs multi-step iterative reasoning on the alignment labels to determine the true base outline of the building. Generate target base location annotations.

[0051] like Figure 3 As shown in the middle image, the process of generating target base location labels is essentially the process of decoding the base outline sequence of buildings, that is, the label alignment model performs denoising inference and gradually drags the historical building labels to the base outline.

[0052] Alternatively, the base contour sequence decoding process can be represented by the following formula: ; in, Indicates the first The result of the iterative reasoning step, i.e. the first step The building labels generated by the next iteration of reasoning are not yet fully aligned. hour, This indicates the historical building labels to be aligned in the newly input label alignment model. hour, This indicates the alignment of the target base position. This represents the total number of steps (i.e., the total number of times) in the label alignment model inference. It is a set of scaling factors used to control the single-step correction amount in a multi-step iterative inference process; Represents the mapping function, formula Indicates from arrive The inverse mapping process, For model parameters related to base profile alignment, This represents the input remote sensing image.

[0053] Optionally, scaling factor It can be defined in exponential form, and its calculation formula is as follows: ; in, It is a constant.

[0054] Understandably, scaling factor sequence The constant has a significant impact on the convergence speed and final accuracy of the label alignment model, therefore... Total number of steps Multiple parameter combinations can be designed according to actual needs.

[0055] For example, in addition to using The convergence scaling factor sequence can also be obtained by using or The scaling factor sequence; when even At that time, the label alignment model can still converge and achieve good results within a limited number of steps.

[0056] For example, you can choose ( , ), ( , ), ( , Different combinations of parameters, such as ), are used to strike a balance between reasoning speed and reasoning accuracy.

[0057] Furthermore, in determining the true base outline of buildings in remotely sensed images... Subsequently, the label alignment model can utilize the fixed offset relationship between the base and the roof learned during the model training phase to decode the roof sequence and determine the true roof outline of the building. Generate the target roof location marker.

[0058] like Figure 3 As shown in the image on the right, the process of generating the target roof location label is essentially the process of the label alignment model inferring the roof outline from the base outline.

[0059] Optionally, the actual roof outline of the building. The decoding process can be represented by the following formula: ; Wherein, formula Indicates from arrive The inverse mapping process, These are the model parameters related to alignment with the roof profile.

[0060] The building label alignment method provided in this embodiment introduces a multi-step iterative denoising inference mechanism. This mechanism models the denoising process as the accumulation of the location noise derivative field. This mechanism differs from the single-step prediction method in the prior art. It gradually reduces the inference error of the model through multi-step iterative inference. That is, each incompletely aligned label generated in each step can be corrected by the model in the next round. By controlling the step size and number of iterations, stable convergence from any noise label to the precise location of the building can be achieved, thereby significantly improving the final label alignment accuracy.

[0061] In some embodiments, based on a mapping function and a scaling factor, multi-step iterative reasoning is performed on the labels to be aligned to determine the base outline of the building and generate target base location annotations. This includes: determining the basic inference step count; performing iterative reasoning on the labels to be aligned according to the basic inference step count based on the mapping function and scaling factor to generate a basic convergence result; using the basic convergence result as the current convergence result; adding random noise to the current convergence result and performing iterative reasoning on the current convergence result according to the basic inference step count based on the mapping function and scaling factor to obtain an updated convergence result; using the updated convergence result as the current convergence result and returning to the step of adding random noise to the current convergence result and performing iterative reasoning on the current convergence result according to the basic inference step count based on the mapping function and scaling factor, until multiple updated convergence results are obtained; and determining the base outline of the building based on the multiple updated convergence results to generate target base location annotations.

[0062] like Figure 4As shown, to further improve the accuracy of the model's output, this embodiment introduces a test-time augmentation strategy. After the model completes the basic T-step (e.g., 5-step) inference, additional inference steps can be performed using two different strategies: the DragOSM-t1 strategy and the DragOSM-t1.5 strategy. Figure 4 The diagram shows two different inference paths generated by DragOSM-t1 and DragOSM-t1.5 on the same remote sensing image, denoted as t1 Path and t1.5 Path, respectively.

[0063] If the DragOSM-t1 strategy is adopted, random noise can be added every few steps (e.g., 5 steps) during the model inference process, so that the current inference process can jump out of the current convergence region to explore a wider solution space, and finally the average of multiple convergence results is used as the output.

[0064] Specifically, if the DragOSM-t1 strategy is adopted, the basic inference steps are determined first.

[0065] Optionally, the basic reasoning steps must be at least 5.

[0066] Furthermore, based on the mapping function and scaling factor, basic iterative inference is performed on the labels to be aligned according to the basic inference steps to generate basic convergence results.

[0067] Furthermore, the basic convergence result is used as the current convergence result. Random noise is added to the current convergence result, and based on the mapping function and scaling factor, iterative reasoning is performed on the current convergence result according to the basic inference steps to obtain the updated convergence result.

[0068] Furthermore, retain the current convergence result, use the updated convergence result as the current convergence result, and return to the step of adding random noise to the current convergence result and iteratively reasoning on the current convergence result based on the mapping function and scaling factor according to the basic inference steps, until multiple updated convergence results are obtained and retained.

[0069] Furthermore, based on the mean of multiple update convergence results, the base outline of the building is determined, and the target base location is marked.

[0070] In some embodiments, based on a mapping function and a scaling factor, multi-step iterative reasoning is performed on the labels to be aligned to determine the base contour of the building and generate target base location annotations. This includes: determining the basic reasoning steps and the additional reasoning steps; performing iterative reasoning on the labels to be aligned according to the basic reasoning steps based on the mapping function and the scaling factor to generate a basic convergence result; performing iterative reasoning on the basic convergence result according to the additional reasoning steps to obtain a target convergence result; and determining the base contour of the building based on the mean center of the target convergence result to generate target base location annotations.

[0071] If the DragOSM-t1.5 strategy is adopted, during the model inference process, after the model has completed several basic inference steps, several additional inference steps need to be performed directly, and the mean center of these additional inference results is calculated as the final output.

[0072] Specifically, if the DragOSM-t1.5 strategy is adopted, the base inference steps and the additional inference steps are determined.

[0073] Furthermore, based on the mapping function and scaling factor, iterative reasoning is performed on the labels to be aligned according to the basic reasoning steps to generate basic convergence results.

[0074] Furthermore, based on the existing basic convergence results, iterative reasoning is performed on the basic convergence results according to the additional reasoning steps to obtain multiple target convergence results.

[0075] For example, if the basic inference steps are 5 and the additional inference steps are 10, then first, based on the mapping function and scaling factor, perform 5 iterative inference steps on the labels to be aligned to generate a basic convergence result; then perform 10 iterative inference steps on the basic convergence result. Each additional inference step will generate a corresponding target convergence result, so 10 target convergence results can be obtained.

[0076] Furthermore, based on the mean center of multiple convergence results, the base outline of the building is determined, and the target base location is marked.

[0077] Optionally, since the prediction results of the model may oscillate around a local optimum after a certain number of inference steps, instead of directly taking the inference result of the last step, the mean center method can be introduced. That is, take all the intermediate results of the last few steps in the extra inference steps (e.g., from step 5 to step 10) and calculate the geometric center point of these results as the final output.

[0078] Optionally, since the prediction result may oscillate around a local optimum after the model has inferred to a certain number of steps, instead of directly taking the inference result of the last step, a noise perturbation method can be introduced, that is, a small amount of random noise is periodically injected during the additional inference process to help the model escape the local optimum. Then, the results of multiple independent inferences are averaged to obtain a more robust final result.

[0079] The building label alignment method provided in this embodiment can further improve the alignment accuracy of labels by introducing a test-time enhancement strategy to extend the inference process of the model.

[0080] In some embodiments, the label alignment model is trained based on the following steps: obtaining sample remote sensing images and their corresponding original building labels; the sample remote sensing images are obtained by taking pictures of the areas containing sample buildings; determining sample correction vectors based on the original building labels; adding Gaussian noise to the original building labels to generate sample building labels; and training an initial model based on the DragOSM algorithm based on the sample remote sensing images, sample building labels, and sample correction vectors to generate the label alignment model.

[0081] Understandably, model training is required before using a label alignment model for label alignment.

[0082] Specifically, during the model training phase, publicly available remote sensing images can be obtained as sample remote sensing images, which are obtained by taking pictures of areas containing sample buildings. At the same time, the original building labels corresponding to the sample remote sensing images are obtained, and the sample correction vectors used for model training are obtained through alignment.

[0083] Optionally, aerial images with a resolution of 0.5m can be obtained from existing open mapping software as sample remote sensing images.

[0084] Optionally, the original building labels corresponding to the sample remote sensing images can be determined by an OpenStreetMap (OSM), where the original building labels are the real historical building labels corresponding to the sample buildings in the sample remote sensing images.

[0085] Optionally, in each training iteration of the model, the original building labels should be randomly sampled and preprocessed such as matrix expansion and mask filtering to convert them into a standardized data format so that the model can process them efficiently.

[0086] Furthermore, to improve the robustness and generalization ability of the model, this embodiment adopts a dynamic noise injection training mechanism, that is, to dynamically inject random Gaussian noise into the original building labels corresponding to the sample remote sensing images, so as to simulate the spatial position offset of buildings in the real world and generate sample building labels carrying noise.

[0087] Specifically, the offset between the original building label and the actual roof and base locations of the sample building is modeled as a Gaussian distribution centered on the actual location, that is, Gaussian noise is independently applied to the coordinates of each key point of the actual base contour of the sample building. This dynamically generates a sample building label with perturbations. ,in, For a fixed standard deviation, It is the identity matrix. Given a Gaussian noise field, the process can be expressed by the following formula: ; in, For the generated Gaussian noise field, " represents a vector; It is a mask matrix; This is a matrix constructed based on sample correction vectors; the training objective of the model is to build labels from noisy input samples. From the sample remote sensing images, a noise vector, i.e., a Gaussian noise field, is learned and predicted for correction. .

[0088] Furthermore, based on sample remote sensing images, sample building labels, and sample correction vectors, the initial model constructed using the DragOSM algorithm is trained with noise. Figure 3 (Add noise training to the left image in the middle). The initial model completes the training process by learning to recover the sample correction vector as the ground truth from the building labels of noisy samples, and finally generates a label alignment model.

[0089] The core idea of ​​the DragOSM algorithm model is to regard the label alignment process as an iterative drag-and-drop process from noisy labels to precise positions. Through dynamic noise training and multi-step denoising inference, high-precision label alignment is achieved.

[0090] The building label alignment method provided in this embodiment introduces a training mechanism based on dynamic noise injection. The positional offset of the original building labels is modeled as a Gaussian distribution centered on the actual building location. During the training process, Gaussian perturbations are dynamically added to the accurate real labels to simulate various complex and diverse labeling errors. This is different from the practice of injecting fixed offset noise in the prior art. By simulating a more realistic noise distribution, the model's understanding of the building structure can be effectively enhanced, making the model more robust and with higher correction accuracy.

[0091] In some embodiments, the sample correction vector includes a first correction vector, a second correction vector, and a third correction vector; wherein the first correction vector is determined based on the original building label and the actual base position of the sample building; the second correction vector is determined based on the original building label and the actual roof position of the sample building; and the third correction vector is determined based on the actual base position and the actual roof position of the sample building.

[0092] Specifically, after obtaining the sample remote sensing image and its corresponding original building labels containing sample base location annotations and sample roof location annotations, three sets of core sample correction vectors are generated for the sample buildings in the sample remote sensing image through manual comparison. These are the first correction vector and the second correction vector. Second correction vector and the third correction vector .

[0093] Wherein, the first correction vector This is a correction vector pointing from the original building label to the actual base location of the sample building.

[0094] Second correction vector This is a correction vector pointing from the original building label to the actual roof location of the sample building.

[0095] Third correction vector This is a correction vector pointing from the true base position of the sample building to the true roof position of the sample building, reflecting the parallax effect caused by oblique photography.

[0096] First correction vector Second correction vector and the third correction vector The following geometric relations must be satisfied: in, and Used for two-stage alignment.

[0097] Understandably, a sample remote sensing image may contain multiple sample buildings simultaneously, and each sample building has its corresponding first correction vector. Second correction vector and the third correction vector The first correction vector corresponding to multiple sample buildings Can form a matrix .

[0098] The building label alignment method provided in this embodiment is a correction vector encoding method that takes into account both physical interpretability and spatiotemporal continuity. By introducing the concept of "alignment token", the correction vector pointing from the label position to the ground truth position is explicitly encoded. This method differs from traditional image segmentation-based extraction algorithms. Instead of generating pixel masks, it directly regresses a correction vector with clear physical meaning. It makes full use of the spatiotemporal relationship between historical labels and the latest remote sensing images, ensures the interpretability of the correction operation, and improves the reuse value of historical annotation data.

[0099] Please see Figure 5 And Table 1, Figure 5 Table 1 shows a comparison of the implementation effects of the building label alignment method provided by this invention and the traditional method on typical remote sensing images. Table 1 is a comparison table of the accuracy evaluation of the building label alignment method provided by this invention and the traditional method.

[0100] like Figure 5 As shown in Table 1, the building label alignment method provided by the present invention has the following technical advantages: (1) Enhanced physical interpretability and spatiotemporal continuity: The present invention explicitly models and corrects the spatial offset vector through "alignment mark", so that each step of the correction has a clear physical meaning and maximizes the value of historical label data. Compared with the method that relies on semantic segmentation, this method ensures the consistency of the label topology before and after correction, thereby enhancing the spatiotemporal continuity of the label and effectively avoiding the building missed detection or re-detection caused by semantic segmentation error; (2) Improved robustness and accuracy of the model: Through the training strategy of dynamically injecting Gaussian noise, the present invention simulates more diverse and realistic label offset situations in the real world, enabling the model to learn more essential building structure features, thereby showing stronger robustness when dealing with various complex noise data.

[0101] As shown in Table 1, the test results on the ReBO remote sensing dataset show that the multi-step inference model of this invention is significantly better than other methods in terms of F1 score and IoU index for roof and base profiles. For example, its comprehensive MF index reaches 91.55% and comprehensive MI index reaches 84.83%, which far exceeds other models.

[0102] Table 1

[0103] Therefore, this invention achieves effective convergence from noise tags to the precise location of buildings: by adopting a multi-step iterative denoising inference mechanism, the model can gradually correct the position error by dynamically adjusting the number and step size of iterations. Compared with the single-step prediction method, this progressive optimization process enables the tags to converge to the true location more stably, effectively avoiding the correction failure problem caused by the large error of single-step prediction, and can significantly improve the final tag alignment accuracy.

[0104] The present invention also provides a building label alignment device. See also: Figure 6 , Figure 6 This is a schematic diagram of the building label alignment device provided by the present invention. In this embodiment, the building label alignment device includes an acquisition module 610 and a label alignment module 620.

[0105] The acquisition module 610 is used to acquire remote sensing images and labels to be aligned.

[0106] Remote sensing images are obtained by taking pictures of areas containing buildings. The labels to be aligned are the historical building labels of the buildings, which include the location of the historical base and the location of the historical roof.

[0107] The label alignment module 620 is used to input remote sensing images and labels to be aligned into a pre-trained label alignment model, perform multi-step iterative inference, and obtain the target base location label and target roof location label output by the label alignment model.

[0108] The target base location is aligned with the building's base outline, and the target roof location is aligned with the building's roof outline.

[0109] The label alignment model is obtained by training an initial model based on the DragOSM algorithm with noise, based on the sample remote sensing image, the sample building label corresponding to the sample remote sensing image, and the sample correction vector corresponding to the sample building label.

[0110] In some embodiments, the target base location label and the target roof location label are generated by the label alignment model based on the following steps: performing multi-step iterative reasoning on the label to be aligned based on a mapping function and a scaling factor to determine the base profile of the building and generate the target base location label; the mapping function is used to inversely map the label to be aligned to the base profile to generate the target base location label, and the scaling factor is used to control the single-step correction amount in the multi-step iterative reasoning process; based on the base profile, determining the roof profile of the building according to the fixed offset relationship between the base and the roof, and generating the target roof location label.

[0111] In some embodiments, based on a mapping function and a scaling factor, multi-step iterative reasoning is performed on the labels to be aligned to determine the base outline of the building and generate target base location annotations. This includes: determining the basic inference step count; performing iterative reasoning on the labels to be aligned according to the basic inference step count based on the mapping function and scaling factor to generate a basic convergence result; using the basic convergence result as the current convergence result; adding random noise to the current convergence result and performing iterative reasoning on the current convergence result according to the basic inference step count based on the mapping function and scaling factor to obtain an updated convergence result; using the updated convergence result as the current convergence result and returning to the step of adding random noise to the current convergence result and performing iterative reasoning on the current convergence result according to the basic inference step count based on the mapping function and scaling factor, until multiple updated convergence results are obtained; and determining the base outline of the building based on the multiple updated convergence results to generate target base location annotations.

[0112] In some embodiments, based on a mapping function and a scaling factor, multi-step iterative reasoning is performed on the labels to be aligned to determine the base contour of the building and generate target base location annotations. This includes: determining the basic reasoning steps and the additional reasoning steps; performing iterative reasoning on the labels to be aligned according to the basic reasoning steps based on the mapping function and the scaling factor to generate a basic convergence result; performing iterative reasoning on the basic convergence result according to the additional reasoning steps to obtain a target convergence result; and determining the base contour of the building based on the mean center of the target convergence result to generate target base location annotations.

[0113] In some embodiments, the label alignment model is trained based on the following steps: obtaining sample remote sensing images and their corresponding original building labels; the sample remote sensing images are obtained by taking pictures of the areas containing sample buildings; determining sample correction vectors based on the original building labels; adding Gaussian noise to the original building labels to generate sample building labels; and training an initial model based on the DragOSM algorithm based on the sample remote sensing images, sample building labels, and sample correction vectors to generate the label alignment model.

[0114] In some embodiments, the sample correction vector includes a first correction vector, a second correction vector, and a third correction vector; wherein the first correction vector is determined based on the original building label and the actual base position of the sample building; the second correction vector is determined based on the original building label and the actual roof position of the sample building; and the third correction vector is determined based on the actual base position and the actual roof position of the sample building.

[0115] The present invention also provides an electronic device. Figure 7 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 7As shown, the electronic device may include a processor 710, a communications interface 720, a memory 730, and a communication bus 740, wherein the processor 710, communications interface 720, and memory 730 communicate with each other via the communication bus 740. The processor 710 can call logical instructions in the memory 730 to execute the building label alignment method.

[0116] Furthermore, the logical instructions in the aforementioned memory 730 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0117] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the building label alignment method provided by the above methods.

[0118] The present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the building label alignment method provided by the above methods.

[0119] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0120] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0121] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for aligning building labels, characterized in that, include: Acquire remote sensing images and labels to be aligned; The remote sensing image is obtained by taking pictures of the area containing buildings. The label to be aligned is the historical building label of the building. The historical building label includes the historical base location mark and the historical roof location mark of the building. The remote sensing image and the label to be aligned are input into a pre-trained label alignment model, and multi-step iterative reasoning is performed to obtain the target base location label and the target roof location label output by the label alignment model. The target base location marker is aligned with the base outline of the building, and the target roof location marker is aligned with the roof outline of the building; The label alignment model is obtained by training an initial model based on the DragOSM algorithm with noise, based on the sample remote sensing image, the sample building label corresponding to the sample remote sensing image, and the sample correction vector corresponding to the sample building label.

2. The building label alignment method according to claim 1, characterized in that, The target base location label and the target roof location label are generated by the label alignment model based on the following steps: Based on the mapping function and scaling factor, the labels to be aligned are subjected to multi-step iterative reasoning to determine the base outline of the building and generate the target base position label; The mapping function is used to inversely map the label to be aligned to the base contour to generate the target base position label, and the scaling factor is used to control the single-step correction amount in the multi-step iterative inference process; Based on the base profile, and according to the fixed offset relationship between the base and the roof, the roof profile of the building is determined, and the target roof position label is generated.

3. The building label alignment method according to claim 2, characterized in that, The process of performing multi-step iterative reasoning on the labels to be aligned based on a mapping function and a scaling factor to determine the base outline of the building and generate the target base location annotation includes: Determine the basic number of reasoning steps; Based on the mapping function and the scaling factor, the labels to be aligned are iteratively inferred according to the basic inference steps to generate basic convergence results; Use the aforementioned basic convergence result as the current convergence result; Random noise is added to the current convergence result, and based on the mapping function and the scaling factor, iterative reasoning is performed on the current convergence result according to the basic inference steps to obtain an updated convergence result; The updated convergence result is taken as the current convergence result, and the process returns to the step of adding random noise to the current convergence result and iteratively reasoning on the current convergence result according to the basic inference steps based on the mapping function and the scaling factor, until multiple updated convergence results are obtained; Based on multiple update convergence results, the base contour of the building is determined, and the target base location annotation is generated.

4. The building label alignment method according to claim 2, characterized in that, The process of performing multi-step iterative reasoning on the labels to be aligned based on a mapping function and a scaling factor to determine the base outline of the building and generate the target base location annotation includes: Determine the basic number of reasoning steps and the additional reasoning steps; Based on the mapping function and the scaling factor, the labels to be aligned are iteratively inferred according to the basic inference steps to generate basic convergence results; The target convergence result is obtained by iteratively reasoning on the basic convergence result according to the additional inference steps. Based on the mean center of the target convergence result, the base outline of the building is determined, and the target base location label is generated.

5. The building label alignment method according to claim 1, characterized in that, The label alignment model is trained based on the following steps: Obtain the sample remote sensing image and the corresponding original building labels; the sample remote sensing image is obtained by taking pictures of the area containing the sample building; Based on the original building labels, the sample correction vector is determined; Gaussian noise is added to the original building labels to generate the sample building labels; Based on the sample remote sensing images, the sample building labels, and the sample correction vectors, the initial model constructed based on the DragOSM algorithm is trained to generate the label alignment model.

6. The building label alignment method according to claim 5, characterized in that, The sample correction vector includes a first correction vector, a second correction vector, and a third correction vector; The first correction vector is determined based on the original building label and the actual base location of the sample building; The second correction vector is determined based on the original building label and the actual roof position of the sample building; The third correction vector is determined based on the actual base location and the actual roof location of the sample building.

7. A building label alignment device, characterized in that, include: The acquisition module is used to acquire remote sensing images and labels to be aligned; The remote sensing image is obtained by taking pictures of the area containing buildings. The label to be aligned is the historical building label of the building. The historical building label includes the historical base location mark and the historical roof location mark of the building. The label alignment module is used to input the remote sensing image and the label to be aligned into a pre-trained label alignment model, perform multi-step iterative inference, and obtain the target base location label and the target roof location label output by the label alignment model. The target base location marker is aligned with the base outline of the building, and the target roof location marker is aligned with the roof outline of the building; The label alignment model is obtained by training an initial model based on the DragOSM algorithm with noise, based on the sample remote sensing image, the sample building label corresponding to the sample remote sensing image, and the sample correction vector corresponding to the sample building label.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the building label alignment method as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the building label alignment method as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the building label alignment method as described in any one of claims 1 to 6.

Images