A method for monitoring semantic changes of urban buildings based on multi-task learning

Abstract

The present invention discloses a method for monitoring semantic changes in urban buildings based on multi-task learning, which relates to the field of remote sensing image technology. The method comprises the following steps: S1, remote sensing image input; S2, geographic location acquisition; S3, construction information access; S4, multi-task engineering drawing import; S5, building change diagram insertion; and S6, information annotation. The method uses early and late remote sensing images as the starting and ending points, and based on engineering information within an urban construction information database and geographic location acquisition, inserts an evolution diagram, i.e., a building change diagram, for the changed areas in the images between the starting and ending points based on the project completion date, thereby visualizing the process of semantic changes in urban buildings. This facilitates providing more advantageous assistance for urban planning, environmental monitoring, and other aspects.

Description

Technical Field

[0001] The present invention relates to the field of remote sensing image technology, and in particular to a method for monitoring semantic changes of urban buildings based on multi-task learning. Background Art

[0002] Semantic change monitoring of urban buildings is to obtain information on changes in surface building types by processing and comparing remote sensing images acquired at different times in a region. This information on changes in surface urban buildings is of great significance in urban planning, environmental monitoring and other aspects.

[0003] Existing methods for monitoring the semantic changes of urban buildings can process and compare remote sensing images of an area at different times, but cannot provide the evolution process of the area during this time period. If the change process in the time period between remote sensing images can be provided, the semantic change process of buildings can be more detailed, providing more favorable assistance for urban planning, environmental monitoring and other aspects. Summary of the Invention

[0004] In view of the shortcomings of the existing technology, the present invention provides a method for monitoring semantic changes of urban buildings based on multi-task learning, which solves the problems raised in the above background technology.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solutions: a method for monitoring semantic changes of urban buildings based on multi-task learning, the method comprising the following steps:

[0006] S1. Remote sensing image input:

[0007] Two remote sensing images of the same area but different time periods were input into a semantic change monitoring model to obtain object classification results for the two remote sensing images. A large number of drone-captured images were used as a dataset for training, algorithm model construction, and debugging. In the proposed multi-task network, consistency constraints across three tasks of building information were considered and constructed. The consistency constraints exploited the duality between mask prediction and two shape-related information predictions, further improving building segmentation performance.

[0008] S2. Geographic location acquisition:

[0009] When acquiring remote sensing images, the geographical location of the area is recorded, and then the changed areas are marked on the remote sensing images based on the ground object classification results. Then, the name information of the changed areas is obtained on the map software based on the geographical location;

[0010] S3. Construction information access:

[0011] The semantic change monitoring model is connected to the urban construction information database and searches for corresponding project information in the urban construction information database based on the geographical location and name of the changed area;

[0012] S4. Multi-task engineering drawing import:

[0013] The completion date of each project is used as the date to import the engineering drawings of each project into the deep learning network in turn. The deep learning network arranges the buildings reasonably according to the geographical information in each engineering drawing and the size of each building to form a building change map. Before the deep learning network is officially used, it is necessary to use the completed building change map and its corresponding engineering drawing. The engineering drawing is input into the deep learning network for learning until the output building change map is consistent with the completed building change map. Figure 1 To;

[0014] S5. Insert building change diagram:

[0015] Copy the older remote sensing image A between two remote sensing images with completed object classification results. On remote sensing image A, overlay the corresponding building change map to the corresponding area on remote sensing image A according to the geographical location to form remote sensing image B. Based on the completion time of each project, overlay the subsequent building change map to the corresponding area on remote sensing image B to form remote sensing image C. Repeat the operation until the final remote sensing image is obtained, thereby obtaining a semantic change set of urban buildings with an evolutionary process.

[0016] S6. Information Notes:

[0017] The changed areas on each remote sensing image in the urban building semantic change set are annotated according to engineering information.

[0018] Furthermore, in the step S1, the semantic change monitoring model is trained using remote sensing images that have been classified.

[0019] Furthermore, in step S2, the geographical location is the longitude and latitude coordinates of the area.

[0020] Furthermore, in the step S3, the urban construction information database is used to store engineering information of various types of buildings in the city.

[0021] Furthermore, the engineering information of the building includes architectural drawings and completion drawings, as well as the completion date of each project, and the geographical location of the building represented by each project.

[0022] Furthermore, in the step S4, the basis for reasonable layout of buildings is to implement large-area coverage of small-area coverage for multiple buildings located in the same geographical location, and high buildings covering low buildings.

[0023] Furthermore, in step S4, the number of building change diagrams is consistent with the number of projects.

[0024] Furthermore, in step S6, the information annotation of the changed region includes the name, function, and introduction of the changed region.

[0025] Furthermore, the urban building semantic change monitoring method based on multi-task learning is applied to the field of remote sensing image processing for urban planning and environmental monitoring.

[0026] The present invention provides a method for monitoring semantic changes of urban buildings based on multi-task learning, which has the following beneficial effects:

[0027] 1. This method for monitoring the semantic changes of urban buildings based on multi-task learning uses a semantic change monitoring model that is connected to an urban construction information database. The previous and later remote sensing images are used as the starting and ending points. Based on the engineering information within the urban construction information database and the acquisition of geographic location, the changed areas in the images between the starting and ending points are sequentially inserted into an evolution graph, i.e., a building change graph, based on the project completion date. This visualizes the process of semantic changes of urban buildings, thereby providing more favorable assistance for urban planning, environmental monitoring, and other aspects.

[0028] 2. This method for monitoring the semantic changes of urban buildings based on multi-task learning visualizes the process of semantic changes of urban buildings and annotates the building changes on each inserted evolution diagram based on engineering drawing information, so that personnel can clearly understand the building information in the changed area. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 This is a schematic diagram of the operation flow of a method for monitoring semantic changes of urban buildings based on multi-task learning in the present invention. DETAILED DESCRIPTION

[0030] The following embodiments of the present invention are described in further detail with reference to the accompanying drawings and examples. The following examples are used to illustrate the present invention but are not intended to limit the scope of the present invention.

[0031] like Figure 1 As shown, the present invention provides a technical solution: a method for monitoring semantic changes of urban buildings based on multi-task learning, the method for monitoring semantic changes of urban buildings based on multi-task learning includes the following steps:

[0032] S1. Remote sensing image input:

[0033] Two remote sensing images from the same area but different time periods were input into the semantic change monitoring model to obtain the object classification results of the two remote sensing images. A large number of drone-captured images were used as a training dataset for the algorithm model construction and debugging. In the proposed multi-task network, consistency constraints across three tasks of building information were considered and constructed. The consistency constraints exploited the duality between mask prediction and two shape-related information predictions, further improving the building segmentation performance. The semantic change monitoring model was trained using remote sensing images that had already been classified.

[0034] S2. Geographic location acquisition:

[0035] When acquiring remote sensing images, the geographical location of the area is recorded, and then the changed areas are marked on the remote sensing images based on the ground object classification results. Then, the name information of the changed areas is obtained on the map software based on the geographical location;

[0036] The geographical location is the longitude and latitude coordinates of the area, which can be further divided into the longitude and latitude coordinates of each building in the area;

[0037] S3. Construction information access:

[0038] The semantic change monitoring model is connected to the urban construction information database and searches for corresponding project information in the urban construction information database based on the geographical location and name of the changed area;

[0039] The urban construction information database is used to store the engineering information of various buildings in the city. The engineering information of buildings includes architectural drawings and completion drawings, as well as the completion date of each project and the geographical location of the building represented by each project.

[0040] S4. Multi-task engineering drawing import:

[0041] The completion date of each project is used as the date to import the engineering drawings of each project into the deep learning network in turn. The deep learning network arranges the buildings reasonably according to the geographical information in each engineering drawing and the size of each building to form a building change map. Before the deep learning network is officially used, it is necessary to use the completed building change map and its corresponding engineering drawing. The engineering drawing is input into the deep learning network for learning until the output building change map is consistent with the completed building change map. Figure 1 To;

[0042] Among them, the benchmark for reasonable layout of buildings is to implement large-area coverage of small-area coverage for multiple buildings located in the same geographical location, high buildings covering low buildings, and the number of building change maps is consistent with the number of projects;

[0043] S5. Insert building change diagram:

[0044] Copy the older remote sensing image A between two remote sensing images with completed object classification results. On remote sensing image A, overlay the corresponding building change map to the corresponding area on remote sensing image A according to the geographical location to form remote sensing image B. Based on the completion time of each project, overlay the subsequent building change map to the corresponding area on remote sensing image B to form remote sensing image C. Repeat the operation until the final remote sensing image is obtained, thereby obtaining a semantic change set of urban buildings with an evolutionary process.

[0045] S6. Information Notes:

[0046] In the urban building semantic change collection, each changed area on the remote sensing image is annotated with information based on engineering information;

[0047] The information annotation of the changed area includes the name, function and introduction of the changed area.

[0048] In steps S1-S5, the semantic change monitoring model is connected to the urban construction information database, with the previous and later remote sensing images as the starting and ending points. Based on the project information within the urban construction information database and the acquisition of geographic location, the changed areas in the images between the starting and ending points are sequentially inserted into the evolution graph, i.e., the building change graph, based on the project completion date. This realizes the visualization of the semantic change process of urban buildings, which is conducive to providing more favorable assistance for urban planning, environmental monitoring and other aspects.

[0049] In step S6, the process of semantic change of urban buildings is visualized and the building changes on each inserted evolution diagram are annotated based on the engineering drawing information to facilitate personnel to clearly understand the building information of the changed area.

[0050] The urban building semantic change monitoring method based on multi-task learning is applied in the field of remote sensing image processing for urban planning and environmental monitoring.

[0051] In summary, if Figure 1 As shown in the figure, the urban building semantic change monitoring method based on multi-task learning is used. First, two remote sensing images of the same area but different time periods are input into the semantic change monitoring model to obtain the object classification results of the two remote sensing images. The semantic change monitoring model is trained using the remote sensing images that have been classified.

[0052] When acquiring remote sensing images, the geographical location of the area is recorded. The geographical location is the longitude and latitude coordinates of the area, which can be further subdivided into the longitude and latitude coordinates of each building in the area. Based on the results of the object classification, the changed areas are marked on the remote sensing images. Based on the geographical location, the name information of the changed areas is obtained on the map software;

[0053] The semantic change monitoring model is then connected to the urban construction information database, which stores engineering information for various types of buildings in the city. This information includes architectural drawings and as-built drawings, as well as the completion date of each project and the geographic location of the building represented by each project. Based on the geographic location and name of the changed area, the corresponding engineering information is searched in the urban construction information database.

[0054] The completion date of each project is used as the date to import the engineering drawings of each project into the deep learning network in turn. The deep learning network arranges the buildings reasonably according to the geographical information in each engineering drawing and the size of each building to form a building change map. Among them, the basis for reasonable layout of buildings is to implement large-area coverage of small-area coverage for multiple buildings located in the same geographical location, and high buildings cover low buildings. The number of building change maps is consistent with the number of projects. Before the deep learning network is officially used, it is necessary to use the completed building change map and its corresponding engineering drawing. The engineering drawing is input into the deep learning network for learning until the output building change map is consistent with the completed building change map. Figure 1 Zhi, indicating that learning is completed;

[0055] Copy the older remote sensing image A between two remote sensing images with completed object classification results. On remote sensing image A, overlay the corresponding building change map to the corresponding area on remote sensing image A according to the geographical location to form remote sensing image B. Based on the completion time of each project, overlay the subsequent building change map to the corresponding area on remote sensing image B to form remote sensing image C. Repeat the operation until the final remote sensing image is obtained, thereby obtaining a semantic change set of urban buildings with an evolutionary process.

[0056] Finally, the changed areas on each remote sensing image in the urban building semantic change set are annotated according to the engineering information. The information annotations of the changed areas include the name, function and introduction of the changed areas.

[0057] The embodiments of the present invention are presented for purposes of illustration and description and are not intended to be exhaustive or to limit the invention to the disclosed forms. Many modifications and variations will be apparent to those skilled in the art. The embodiments are chosen and described in order to better illustrate the principles of the invention and its practical application and to enable those skilled in the art to understand the invention and design various embodiments with various modifications as suited for specific applications.

Claims

1. A method for monitoring semantic changes in urban buildings based on multi-task learning, characterized by: The method for monitoring semantic changes of urban buildings based on multi-task learning includes the following steps: S1. Remote sensing image input: Two remote sensing images of the same area but different time periods were input into a semantic change monitoring model to obtain object classification results for the two remote sensing images. A large number of drone-captured images were used as a dataset for training, algorithm model construction, and debugging. In the proposed multi-task network, consistency constraints across three tasks of building information were considered and constructed. The consistency constraints exploited the duality between mask prediction and two shape-related information predictions, further improving building segmentation performance. S2. Geographic location acquisition: When acquiring remote sensing images, the geographical location of the area is recorded, and then the changed areas are marked on the remote sensing images based on the ground object classification results. Then, the name information of the changed areas is obtained on the map software based on the geographical location; S3. Construction information access: The semantic change monitoring model is connected to the urban construction information database and searches for corresponding project information in the urban construction information database based on the geographical location and name of the changed area; S4. Multi-task engineering drawing import: The engineering drawings of each project are sequentially imported into the deep learning network based on the completion date of each project. The deep learning network arranges the buildings reasonably according to the geographical information in each engineering drawing and the size of each building, forming a building change map. Before the deep learning network is officially used, it needs to use the completed building change map and its corresponding engineering drawing. The engineering drawing is input into the deep learning network for learning until the output building change map is consistent with the completed building change map; S5. Insert building change diagram: Copy the older remote sensing image A between two remote sensing images with completed object classification results. On remote sensing image A, overlay the corresponding building change map to the corresponding area on remote sensing image A according to the geographical location to form remote sensing image B. Based on the completion time of each project, overlay the subsequent building change map to the corresponding area on remote sensing image B to form remote sensing image C. Repeat the operation until the final remote sensing image is obtained, thereby obtaining a semantic change set of urban buildings with an evolutionary process. S6. Information Notes: The changed areas on each remote sensing image in the urban building semantic change set are annotated according to engineering information.

2. The method for monitoring semantic changes of urban buildings based on multi-task learning according to claim 1, characterized in that: In step S1, the semantic change monitoring model is trained using remote sensing images that have been classified.

3. The method for monitoring semantic changes of urban buildings based on multi-task learning according to claim 1, characterized in that: In step S2, the geographical location is the longitude and latitude coordinates of the area.

4. The method for monitoring semantic changes of urban buildings based on multi-task learning according to claim 1, characterized in that: In the step S3, the urban construction information database is used to store engineering information of various types of buildings in the city.

5. The method for monitoring semantic changes of urban buildings based on multi-task learning according to claim 4 is characterized by: The engineering information of the building includes architectural drawings and as-built drawings, as well as the completion date of each project, and the geographical location of the building represented by each project.

6. The method for monitoring semantic changes of urban buildings based on multi-task learning according to claim 1, characterized in that: In the step S4, the basis for reasonable layout of buildings is to implement a large area covering a small area and a high building covering a low building for multiple buildings located in the same geographical location.

7. The method for monitoring semantic changes of urban buildings based on multi-task learning according to claim 1, characterized in that: In the step S4, the number of building change drawings is consistent with the number of projects.

8. The method for monitoring semantic changes of urban buildings based on multi-task learning according to claim 1, characterized in that: In step S6, the information annotation of the changed region includes the name, function, and introduction of the changed region.

9. The method for monitoring semantic changes of urban buildings based on multi-task learning according to any one of claims 1 to 8, characterized in that: The urban building semantic change monitoring method based on multi-task learning is applied in the field of remote sensing image processing for urban planning and environmental monitoring.

Images