A method and device for detecting illegal buildings used in urban management supervision

By using building identification models in urban management supervision, combined with prospect enhancement feature maps and differential feature maps, the problem of inaccurate detection results of illegal buildings in the existing technology is solved, and more efficient and accurate detection results are achieved.

CN114463624BActive Publication Date: 2024-11-29AEROSPACE INFORMATION RES INST CAS
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202111598817.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-12-24
Publication Date
2024-11-29
Estimated Expiration
2041-12-24

AI Technical Summary

Technical Problem

It is difficult for the existing technology to accurately obtain the detection results of illegal buildings, mainly because the image data content is complex, with large differences within and small differences between classes.

Method used

A method of illegal buildings used for urban management supervision is adopted. By obtaining multiple remote sensing images to be identified in different phases of the target area, inputting them into the trained building recognition model, and using the foreground enhancement feature map and differential feature map Conduct building identification and finally obtain the detection results of illegal buildings based on the identification results.

Benefits of technology

More accurate inspection of illegal buildings has been achieved, the inspection process has been simplified, the investment in manpower and material resources has been reduced, and the inspection cost has been reduced.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114463624B_ABST
    Figure CN114463624B_ABST
Patent Text Reader

Abstract

The present invention provides a method and device for detecting illegal buildings applied to urban management supervision, the method comprising: inputting each remote sensing image to be identified into a building recognition model, obtaining the building recognition results of each remote sensing image to be identified output by the building recognition model; obtaining the illegal building detection results of the target area based on the building recognition results of each remote sensing image to be identified; wherein the building recognition model is used to obtain and perform building recognition on each remote sensing image to be identified based on the foreground enhancement feature map of the target scale of each remote sensing image to be identified and the differential feature map of the target scale corresponding to each two remote sensing images to be identified, and the differential feature map is obtained based on the result of image subtraction of the foreground enhancement feature map of the target scale of each two remote sensing images to be identified. The method and device for detecting illegal buildings applied to urban management supervision provided by the present invention can detect illegal buildings in the target area more accurately and efficiently.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method and device for detecting illegal buildings used in urban management supervision. Background Art

[0002] With the development of social economy, illegal construction has emerged in an endless stream. Illegal buildings have eroded public land resources and posed a serious threat to people's living environment, safety environment, ecological environment, public security environment and social environment. Based on the development and application needs of our country's high-resolution satellite remote sensing data series, timely and accurate detection of illegal buildings and providing effective data support for urban management and supervision industry departments are of great significance.

[0003] In the existing illegal building detection methods, image data can be obtained based on satellite remote sensing, aerial remote sensing, drone remote sensing and other methods, and then the deep learning method can be used to detect illegal buildings. However, due to the complexity of the image data content and the large intra-class differences and small inter-class differences of the above image data, it is difficult to accurately obtain the detection results of illegal buildings based on the existing illegal building detection methods. Summary of the invention

[0004] The present invention provides an illegal building detection method and device applied to urban management supervision, so as to solve the defect that it is difficult to accurately obtain the illegal building detection result in the prior art, and realize more accurate acquisition of the illegal building detection result.

[0005] The present invention provides a method for detecting illegal buildings for urban management and supervision, comprising:

[0006] Acquire multiple remote sensing images of the target area at different phases to be identified;

[0007] Inputting each of the remote sensing images to be identified into a building recognition model, and obtaining a building recognition result of each of the remote sensing images to be identified output by the building recognition model;

[0008] Based on the building recognition results of each of the remote sensing images to be identified, obtaining the detection results of illegal buildings in the target area;

[0009] The building recognition model is obtained after training based on a plurality of sample images of different time phases of the sample area and the building recognition result labels of each of the sample images;

[0010] The building recognition model is used to obtain and perform building recognition on each remote sensing image to be identified based on the foreground enhancement feature map of the target scale of each remote sensing image to be identified and the differential feature map of the target scale corresponding to each two remote sensing images to be identified. The foreground enhancement feature map of the target scale of the remote sensing image to be identified is obtained by performing foreground feature enhancement on the feature map of the target scale of the remote sensing image to be identified, and the differential feature map of the target scale corresponding to each two remote sensing images to be identified is obtained based on the result of image subtraction of the foreground enhancement feature map of the target scale of each two remote sensing images to be identified.

[0011] According to a method for detecting illegal buildings applied to urban management supervision provided by the present invention, the building recognition model comprises: a feature extraction layer, a semantic segmentation layer, a change detection layer and a result output layer;

[0012] Correspondingly, the step of inputting each of the remote sensing images to be identified into a building recognition model and obtaining a building recognition result of each of the remote sensing images to be identified output by the building recognition model specifically includes:

[0013] Input each of the remote sensing images to be identified into the feature extraction layer, and obtain a first feature map and a second feature map of the target scale of each of the remote sensing images to be identified output by the feature extraction layer;

[0014] Inputting the first feature map and the second feature map of each to-be-identified remote sensing image target scale into the semantic segmentation layer, and obtaining the foreground enhancement feature map of each to-be-identified remote sensing image target scale output by the semantic segmentation layer;

[0015] Inputting the foreground enhancement feature map of the target scale of each remote sensing image to be identified into the change detection layer, the change detection layer performs image subtraction based on the foreground enhancement feature map of the target scale of each two remote sensing images to be identified, and generates a differential feature map of the target scale corresponding to each two remote sensing images to be identified, thereby obtaining the differential feature map of the target scale corresponding to each two remote sensing images to be identified output by the change detection layer;

[0016] The foreground enhancement feature map of the target scale of each remote sensing image to be identified and the differential feature map of the target scale corresponding to each two remote sensing images to be identified are input into the result output layer to obtain the building recognition results of each remote sensing image to be identified output by the result output layer.

[0017] According to a method for detecting illegal buildings applied to urban management and supervision provided by the present invention, the loss function of the building recognition model includes a dynamic weighted loss function, a semantic segmentation loss function and a change detection loss function;

[0018] The dynamic weighted loss function is determined based on a preset maximum number of iterations and a current number of iterations of the building recognition model in training, a predetermined maximum weight and a predetermined minimum weight, the semantic segmentation loss function, and the change detection loss function;

[0019] The semantic segmentation loss function and the change detection loss function are determined based on the predicted building recognition results of each of the sample images and the building recognition result labels of each of the sample images, and the predicted building recognition results of each of the sample images are input into the building recognition model under training and output by the result output layer of the building recognition model under training;

[0020] The dynamic weighted loss function is used to adjust the first weight corresponding to the semantic segmentation loss function to decrease as the number of iterations of the building recognition model under training increases, and the second weight corresponding to the change detection loss function to increase as the number of iterations of the building recognition model under training increases.

[0021] According to a method for detecting illegal buildings applied to urban management supervision provided by the present invention, judging whether there are illegal buildings in the target area based on the building recognition result specifically includes:

[0022] For every two remote sensing images to be identified, based on the building identification results and the GIS spatial analysis method, it is determined whether there is a new building in the remote sensing image to be identified after the comparison between the two remote sensing images to be identified;

[0023] In the case where a new building is added to the remote sensing image to be identified after the comparison between each two remote sensing images to be identified, judging whether each new building is an illegal building based on the functional area element layer corresponding to the target area;

[0024] The functional area element layer is obtained by performing buffer analysis based on the original functional area element layer obtained in advance.

[0025] According to a method for detecting illegal buildings applied to urban management and supervision provided by the present invention, judging whether each newly added building is an illegal building based on the functional area element layer corresponding to the target area specifically includes:

[0026] Based on the functional area element layer corresponding to the target area, obtaining the probability that each of the newly added buildings is an illegal building;

[0027] When the probability that any newly added building is an illegal building is greater than a preset threshold, the newly added building is determined to be an illegal building.

[0028] According to a method for detecting illegal buildings applied to urban management supervision provided by the present invention, the semantic segmentation layer includes: a foreground feature enhancement layer and a spatial detail restoration layer;

[0029] Correspondingly, the step of inputting the first feature map and the second feature map of each to-be-identified remote sensing image target scale into the semantic segmentation layer, and obtaining the foreground enhancement feature map of each to-be-identified remote sensing image target scale output by the semantic segmentation layer, specifically includes:

[0030] Inputting the first feature map and the second feature map of each target scale of the remote sensing image to be identified into the foreground feature enhancement layer, the foreground feature enhancement layer generates a target vector corresponding to each remote sensing image to be identified based on the first feature map of each remote sensing image to be identified, obtaining the inner product of the target vector and the second feature map corresponding to the target scale of the remote sensing image to be identified as the semantic segmentation feature map of the target scale of the remote sensing image to be identified, and then obtaining the semantic segmentation feature map of each target scale of the remote sensing image to be identified output by the foreground feature enhancement layer;

[0031] The semantic segmentation feature map of each target scale of the remote sensing image to be identified is input into the spatial detail recovery layer, and the spatial detail recovery layer generates a foreground enhancement feature map of each target scale of the remote sensing image to be identified by recovering the spatial details of the semantic segmentation feature map of each target scale of the remote sensing image to be identified, and then the foreground enhancement feature map of each remote sensing image to be identified output by the spatial detail recovery layer is obtained.

[0032] According to the illegal building detection method applied to urban management supervision provided by the present invention, the target scale includes multiple scales.

[0033] The present invention also provides an illegal building detection device for urban management supervision, comprising:

[0034] A data acquisition module is used to acquire multiple remote sensing images to be identified in different phases of the target area;

[0035] An image recognition module, used for inputting each of the remote sensing images to be recognized into a building recognition model, and obtaining a building recognition result of each of the remote sensing images to be recognized output by the building recognition model;

[0036] An illegal building detection module, used to obtain the illegal building detection results of the target area based on the building recognition results of each of the remote sensing images to be identified;

[0037] The building recognition model is obtained after training based on a plurality of sample images of different time phases of the sample area and the building recognition result labels of each of the sample images;

[0038] The building recognition model is used to obtain and perform building recognition on each remote sensing image to be identified based on the foreground enhancement feature map of the target scale of each remote sensing image to be identified and the differential feature map of the target scale corresponding to each two remote sensing images to be identified. The foreground enhancement feature map of the target scale of the remote sensing image to be identified is obtained by performing foreground feature enhancement on the feature map of the target scale of the remote sensing image to be identified, and the differential feature map of the target scale corresponding to each two remote sensing images to be identified is obtained based on the result of image subtraction of the foreground enhancement feature map of the target scale of each two remote sensing images to be identified.

[0039] The present invention also provides an electronic device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of any of the above-described methods for detecting illegal buildings applied to urban management supervision are implemented.

[0040] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned illegal building detection methods applied to urban management supervision are implemented.

[0041] The present invention also provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of any of the above-mentioned illegal building detection methods applied to urban management supervision.

[0042] The present invention provides an illegal building detection method and device for urban management supervision. Based on a trained building recognition model, a foreground enhancement feature map of the target scale of each remote sensing image to be identified and a differential feature map of the target scale corresponding to each two remote sensing images to be identified are obtained, and building recognition is performed on each of the remote sensing images to be identified respectively. After the building recognition results of each remote sensing image to be identified including the building recognition results of each two remote sensing images to be identified are obtained, the illegal building detection results of the target area are obtained based on the building recognition results of each remote sensing image to be identified. The method and device can realize automatic extraction of buildings in the remote sensing images to be identified in the previous and next phases and buildings that have changed, and can more accurately perform illegal building detection on the target area, making the illegal building detection process simpler and more efficient, reducing the investment in manpower and material resources, and reducing the cost investment in illegal building detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0044] Figure 1 This is one of the flow charts of a method for detecting illegal buildings for urban management supervision provided by the present invention;

[0045] Figure 2 This is the second flow chart of a method for detecting illegal buildings for urban management supervision provided by the present invention;

[0046] Figure 3 One of the schematic diagrams of the building recognition results of each remote sensing image to be identified in the illegal building detection method for urban management supervision provided by the present invention;

[0047] Figure 4 A second schematic diagram of building recognition results of each remote sensing image to be recognized in a method for detecting illegal buildings in urban management and supervision provided by the present invention;

[0048] Figure 5 It is a structural schematic diagram of a building recognition model in a method for detecting illegal buildings for urban management supervision provided by the present invention;

[0049] Figure 6 It is a structural schematic diagram of a building recognition model in the prior art;

[0050] Figure 7 It is a structural schematic diagram of an illegal building detection device applied to urban management supervision provided by the present invention;

[0051] Figure 8 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] In the description of the invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0054] Generally speaking, traditional illegal building detection methods can include the following:

[0055] 1. Regular manual inspections. Special personnel will conduct regular manual inspections at designated locations, and if any illegal construction is found, on-site records will be made;

[0056] 2. Use satellite remote sensing technology to assist visual identification. Regularly obtain high-resolution remote sensing images, and based on the remote sensing image display platform, perform manual visual identification on two remote sensing images from different periods on the same screen, and vectorize the changed buildings as suspected illegal buildings, generate suspected illegal building spots and upload them to the server simultaneously;

[0057] 3. Use drone technology to assist visual identification. Use drone technology to obtain real-time images or videos, and use manual visual identification to determine whether there are suspected illegal buildings in the images or videos. If there are any, automatically take screenshots and upload them to the server.

[0058] Fourth, using traditional remote sensing interpretation methods to extract buildings from remote sensing images of previous and next phases, and after obtaining the changed building patches through manual visual inspection or superposition and subtraction, compare the remote sensing images of previous and next phases to determine the newly added buildings in the later phase, and determine whether the newly added buildings are illegal buildings;

[0059] Fifth, using traditional deep learning methods to extract buildings from remote sensing images of previous and next phases, and obtaining the changed building patches through manual visual inspection or superposition and subtraction, GIS spatial analysis is performed on the buildings in the extracted remote sensing images of previous and next phases to obtain the newly added buildings in the changed building patches, and determine whether the newly added buildings are illegal buildings;

[0060] 6. Other methods, such as encouraging the public to report, using drones for visual comparison, etc.

[0061] However, the above method has the following defects:

[0062] 1. The above-mentioned manual regular inspections and other technologies consume a lot of manpower and material resources, and the workload is huge;

[0063] Second, using remote sensing satellite technology or drone technology to assist visual identification reduces the workload of staff, but still requires visual interpretation by staff, which takes a lot of time to discover the new buildings in the above images;

[0064] Third, the traditional remote sensing interpretation method has low accuracy in extracting buildings from remote sensing images of previous and subsequent phases, and it is difficult to further automatically compare the changes between different buildings, so it cannot be applied in practice;

[0065] 4. Although traditional deep learning methods and GIS spatial analysis can automatically compare and acquire the changed buildings in the remote sensing images of the previous and next phases, on the one hand, the content of remote sensing images is complex, the intra-class differences are large, the inter-class differences are small, the false detection phenomenon is serious, and the explicit enhancement of foreground features and the differences between foreground features in different phases are not considered; on the other hand, the recognition results are greatly affected by factors such as registration, lighting, viewing angle, antenna and sensor differences, and the "pseudo-changes" of buildings in the remote sensing images of the previous and next phases may appear in the recognition results.

[0066] In this regard, the present invention provides a method for detecting illegal buildings applied to urban management supervision, which can explicitly enhance the foreground features of remote sensing images in previous and next phases and highlight the differences between the foreground features of remote sensing images in previous and next phases, thereby more accurately obtaining detection results of illegal buildings.

[0067] Figure 1 This is one of the flow charts of a method for detecting illegal buildings for urban management and supervision provided by the present invention. Figure 1 The present invention describes a method for detecting illegal buildings used in urban management supervision. Figure 1 As shown, the method includes: step 101, obtaining a plurality of remote sensing images to be identified of a target area in different phases.

[0068] Specifically, the target area is the detection object of the illegal building detection method applied to urban management supervision provided by the present invention. Based on the illegal building detection method applied to urban management supervision provided by the present invention, whether illegal buildings appear in the target area is detected, and the illegal building detection result of the target area is obtained.

[0069] Alternatively, remote sensing technology has many advantages such as wide observation range, large amount of information, fast information acquisition, short update cycle, saving manpower and material resources, and less human interference factors, which can solve the problem of difficulty in obtaining evidence of historical illegal construction and improve the timeliness and accuracy of discovering illegal buildings. The remote sensing image to be identified in the embodiment of the present invention can be a high-resolution remote sensing image with a long time sequence and high revisit.

[0070] Optionally, at least two remote sensing images of the target area to be identified at different phases can be obtained through a high-resolution satellite. The high-resolution satellite can be: my country's domestic "Gaofen" series satellites, etc., or IKONOS satellites, GeoEye satellites, QuickBird satellites, WorldView series of foreign commercial satellites, etc.

[0071] Step 102: input each remote sensing image to be identified into a building recognition model, and obtain the building recognition results of each remote sensing image to be identified output by the building recognition model; wherein the building recognition model is obtained after training based on multiple sample images of different phases of the sample area and the building recognition result labels of each sample image; the building recognition model is used to obtain and perform building recognition on each remote sensing image to be identified based on the foreground enhancement feature map of the target scale of each remote sensing image to be identified and the differential feature map of the target scale corresponding to each two remote sensing images to be identified, wherein the foreground enhancement feature map of the target scale of the remote sensing image to be identified is obtained by performing foreground feature enhancement on the feature map of the target scale of the remote sensing image to be identified, and the differential feature map of the target scale corresponding to each two remote sensing images to be identified is obtained based on the result of image subtraction of the foreground enhancement feature map of the target scale of each two remote sensing images to be identified.

[0072] It should be noted that, when the number of remote sensing images to be identified is greater than two, the building recognition model can obtain the building recognition results of every two remote sensing images to be identified, and output them as the building recognition results of each remote sensing image to be identified.

[0073] For every two remote sensing images to be identified, the one with an earlier time phase may be referred to as the first remote sensing image to be identified, and the one with a later time phase may be referred to as the second remote sensing image to be identified.

[0074] Figure 2 This is the second flow chart of a method for detecting illegal buildings for urban management and supervision provided by the present invention. Figure 2 As shown, before inputting each remote sensing image to be identified into the building recognition model and obtaining the building recognition results of each remote sensing image to be identified output by the building recognition model, the building recognition model can be trained in advance to obtain a trained building recognition model.

[0075] Optionally, the building recognition model can be trained in the following manner: first, a plurality of high-resolution remote sensing images of the sample area at different phases are obtained, and the resolution of the high-resolution remote sensing images is the same. After processing the high-resolution remote sensing images, a plurality of sample images of the sample area at different phases and the building recognition result labels of the sample images are obtained to construct a building sample library. Among them, the changed buildings may include expanded / renovated buildings, newly added buildings, and disappeared buildings. Secondly, the building recognition model can be trained based on the sample images of the sample area at different phases and the building recognition result labels of the sample images, so as to obtain a trained building recognition model.

[0076] It should be noted that the building recognition model can be constructed based on a Siamese neural network. The building recognition model is a U-shaped structure.

[0077] Optionally, the high-resolution remote sensing image of the sample area is processed, and the specific process of obtaining the sample image may include: image preprocessing and sample production. Image preprocessing may include geometric correction, fusion, registration, mosaicking and cropping of the above-mentioned high-resolution remote sensing image. Sample production may include marking the building vector layer and the changed building vector layer in the above-mentioned preprocessed high-resolution remote sensing image, and cropping the above-mentioned marked high-resolution remote sensing image to the same size as the unmarked high-resolution remote sensing image. Preferably, the cropping size may be 512 pixels * 512 pixels. The high-resolution remote sensing image with better quality may be selected from the above-mentioned high-resolution remote sensing images for cleaning, and the cleaned high-resolution remote sensing images may be horizontally flipped, vertically flipped, rotated, transposed, randomly cropped and resampled, random brightness, random contrast, Gaussian noise, Gaussian blurring, etc., to obtain multiple sample images of different phases of the sample area and the building recognition result labels of the above-mentioned sample images.

[0078] After obtaining the trained building recognition model, each remote sensing image to be identified can be input into the trained building recognition model, and the trained building recognition model can output the building recognition results of each remote sensing image to be identified in an end-to-end manner.

[0079] For every two remote sensing images to be identified, the trained building recognition model can be used to obtain a feature map of the target scale of the first remote sensing image to be identified and a feature map of the target scale of the second remote sensing image to be identified, and obtain a foreground enhanced feature map of the target scale of the first remote sensing image to be identified by performing foreground feature enhancement on the feature map of the target scale of the first remote sensing image to be identified, and obtain a foreground enhanced feature map of the target scale of the second remote sensing image to be identified by performing foreground feature enhancement on the feature map of the target scale of the second remote sensing image to be identified. It is also possible to obtain a differential feature map of the target scale corresponding to the above two remote sensing images to be identified based on the result of image subtraction between the foreground enhanced feature map of the target scale of the first remote sensing image to be identified and the foreground enhanced feature map of the target scale of the second remote sensing image to be identified, and then, based on the foreground enhanced feature map of the target scale of the first remote sensing image to be identified, the foreground enhanced feature map of the target scale of the second remote sensing image to be identified and the differential feature map of the target scale corresponding to the above two remote sensing images to be identified, perform building recognition on the above two remote sensing images to be identified, and obtain and output the building recognition results of the above two remote sensing images to be identified.

[0080] It should be noted that the target scale may include one or more scales. When the target scale includes multiple scales, the scales of the feature map of the remote sensing image to be identified, the foreground feature enhancement feature map, and the differential feature map corresponding to every two remote sensing images to be identified have a corresponding relationship.

[0081] It should be noted that the building recognition results of each remote sensing image to be identified may include the building recognition results of every two remote sensing images to be identified. The building recognition results of every two remote sensing images to be identified may include the shape and position information of each building in the first remote sensing image to be identified, the shape and position information of each building in the second remote sensing image to be identified, and the shape and position information of the buildings that have changed between the first remote sensing image to be identified and the second remote sensing image to be identified.

[0082] It can be understood that the building recognition results of every two remote sensing images to be identified output by the building recognition model may include building spots in the first remote sensing image to be identified, building spots in the second remote sensing image to be identified, and building spots that have changed in the first remote sensing image to be identified and the second remote sensing image to be identified. The building spots in the first remote sensing image to be identified can indicate the shape and location information of each building in the first remote sensing image to be identified, the building spots in the second remote sensing image to be identified can be used to indicate the shape and location information of each building in the second remote sensing image to be identified, and the building spots that have changed in the first remote sensing image to be identified and the second remote sensing image to be identified can be used to indicate the shape and location information of the buildings that have changed in the first remote sensing image to be identified and the second remote sensing image to be identified.

[0083] Figure 3 This is one of the schematic diagrams of the building recognition results of each remote sensing image to be identified in the illegal building detection method for urban management supervision provided by the present invention. Figure 3 As shown in the figure, white spots can represent buildings. Figure (a) shows the building spots in the first remote sensing image to be identified, Figure (b) shows the building spots in the second remote sensing image to be identified, and Figure (c) shows the building spots that have changed in the first remote sensing image to be identified and the second remote sensing image to be identified. Figure 3 As shown, compared with the first remote sensing image to be identified, there is no changed building in the second remote sensing image to be identified.

[0084] Figure 4 The second schematic diagram of the building recognition results of each remote sensing image to be recognized in the illegal building detection method for urban management supervision provided by the present invention. Figure 4 As shown in the figure, white spots can represent buildings. Figure (a) shows the building spots in the first remote sensing image to be identified, Figure (b) shows the building spots in the second remote sensing image to be identified, and Figure (c) shows the building spots that have changed in the first remote sensing image to be identified and the second remote sensing image to be identified. Figure 4 As shown, compared with the first remote sensing image to be identified, there are more changed buildings in the second remote sensing image to be identified.

[0085] Step 103: Based on the building recognition results of each remote sensing image to be recognized, the detection results of illegal buildings in the target area are obtained.

[0086] Specifically, the illegal building detection results in the target area can be obtained based on the building recognition results of each remote sensing image to be identified through a variety of methods. For example, the illegal building detection results in the target area can be obtained based on the building recognition results of each remote sensing image to be identified through a GIS spatial analysis method.

[0087] The embodiment of the present invention obtains and performs building recognition on each of the two remote sensing images to be identified based on a trained building recognition model, based on a foreground enhancement feature map of the target scale of each remote sensing image to be identified and a differential feature map of the target scale corresponding to each two remote sensing images to be identified, and obtains the building recognition results of each remote sensing image to be identified including the building recognition results of each two remote sensing images to be identified. Then, based on the building recognition results of each remote sensing image to be identified, the illegal building detection results of the target area are obtained, and the buildings in the remote sensing images to be identified in the previous and next phases and the changed buildings can be automatically extracted, and the illegal building detection in the target area can be performed more accurately. The process of illegal building detection is simpler and more efficient, and the investment in manpower and material resources can be reduced, and the cost investment in illegal building detection can be reduced.

[0088] Based on the contents of the above embodiments, the target scale includes multiple scales.

[0089] Specifically, the remote sensing image to be identified may include multiple buildings of different scales, for example, the building targets are low buildings and a row of large factories closely arranged. In order to improve the accuracy of building identification for each remote sensing image to be identified, the target scale in the embodiment of the present invention may include multiple scales, and the trained building identification model may obtain feature maps of multiple scales for each remote sensing image to be identified, obtain foreground enhancement feature maps of corresponding scales for each remote sensing image to be identified, and then obtain differential feature maps of corresponding scales corresponding to each two remote sensing images to be identified, so as to achieve more accurate building identification for each remote sensing image to be identified.

[0090] The embodiment of the present invention obtains feature maps of multiple scales for each remote sensing image to be identified based on a trained building recognition model, obtains a foreground enhancement feature map of the corresponding scale of each remote sensing image to be identified, and then obtains a differential feature map of the corresponding scale corresponding to every two remote sensing images to be identified, so as to avoid losing small-scale buildings or failing to identify large-scale buildings when performing building recognition on each remote sensing image to be identified, thereby achieving more accurate building recognition for each remote sensing image to be identified.

[0091] Figure 5 Schematic diagram of the structure of a building recognition model in a method for detecting illegal buildings for urban management and supervision provided by the present invention. Figure 5 As shown, the building recognition model includes: feature extraction layer, semantic segmentation layer, change detection layer and result output layer.

[0092] Correspondingly, each remote sensing image to be identified is input into the building recognition model, and the building recognition results of each remote sensing image to be identified output by the building recognition model are obtained, which specifically includes: inputting each remote sensing image to be identified into the feature extraction layer, and obtaining the first feature map and the second feature map of the target scale of each remote sensing image to be identified output by the feature extraction layer.

[0093] It should be noted that, in the embodiment of the present invention, the target scale includes five scales as an example for description. j can be used as an identifier of the target scale, j=1, 2, 3, 4, 5.

[0094] Specifically, for each two remote sensing images to be identified, for buildings of different scales in the two remote sensing images to be identified, the feature extraction layer in the embodiment of the present invention can use the top-down path of the Feature Pyramid Network (FPN) and the lateral connection method to generate and output the first pyramid-shaped feature map of each remote sensing image to be identified. The first feature map can also be enhanced by shallow spatial detail information and deep semantic information, so as to better integrate spatial information, semantic information and multi-scale context information, and obtain the second pyramid-shaped feature map of each remote sensing image to be identified, and then the building recognition results of the two remote sensing images to be identified can be more accurately obtained based on the first feature maps and the second feature maps.

[0095] Optionally, the feature extraction layer may include a first encoder and a second encoder. The first encoder and the second encoder have the same structure. Compared with the commonly used Resnet network, the first feature extraction layer and the second feature extraction layer in the embodiment of the present invention can use the Resnest-50 network as an encoder, so that the calculation accuracy of the first encoder and the second encoder can be improved by introducing an attention mechanism in the first encoder and the second encoder.

[0096] In the case where the target scale includes five scales, the first encoder and the second encoder each include a layer of semantic head and four layers of residual blocks composed of Conv-Relu-BatchNormalization.

[0097] Optionally, for every two remote sensing images to be identified, I i represents the above two remote sensing images to be identified, wherein i represents the identifier of the remote sensing image to be identified, i=1,2, I1 represents the first remote sensing image to be identified, and I2 represents the second remote sensing image to be identified.

[0098] The first remote sensing image to be identified I1 is input into the first encoder, and the first feature map of the target scale of the first remote sensing image to be identified I1 output by the first encoder can be obtained. (1,j) |j=1,2,3,4,5}. Among them, {C (1,j) |j=1,2,3,4,5} is output by a layer of semantic head and four layers of residual blocks in the first encoder.

[0099] The second remote sensing image to be identified I2 is input into the second encoder, and the first feature map of the target scale of the second remote sensing image to be identified I2 output by the second encoder is obtained. (2,j) |j=1,2,3,4,5}. Among them, {C (2,j) |j=1,2,3,4,5} is output by a layer of semantic head and four layers of residual blocks in the second encoder.

[0100] It should be noted that since the feature graphs extracted by the first layer semantic heads of the first encoder and the second encoder have weak expression capabilities, the first feature graph C extracted by the first layer semantic head of the first encoder is not used. (1,1) As the input of the first decoder, the first feature map C extracted by the first layer semantic head of the second encoder is not (2,1) As input to the second decoder.

[0101] Optionally, the feature extraction layer may include a first decoder and a second decoder. The first decoder and the second decoder have the same structure. The first decoder and the second decoder may adopt the top-down fusion context information of the FPN network, and may adopt the jumper connection method of the Unet network to restore {C (1,j) |j=2,3,4,5} and {C (2,j) |j=2,3,4,5} spatial details.

[0102] {C (1,j) |j=2,3,4,5} is input into the first decoder, and the second feature map of the target scale of the first remote sensing image I1 to be identified output by the first decoder can be obtained {P (1,j) |j=2,3,4,5}.

[0103] {C (2,j) |j=2,3,4,5} is input into the second decoder, and the second feature map of the target scale of the second remote sensing image I2 to be identified output by the second decoder can be obtained. (2,j) |j=2,3,4,5}.

[0104] P (i,j) It can be expressed by the following formula:

[0105]

[0106] Among them, ζ represents the skip-line lateral connection realized by the 1*1 convolutional layer; Γ represents the bilinear interpolation with an upsampling factor of 2.

[0107] The first feature map and the second feature map of each to-be-identified remote sensing image target scale are input into the semantic segmentation layer, and the foreground enhancement feature map of each to-be-identified remote sensing image target scale output by the semantic segmentation layer is obtained.

[0108] Specifically, for each two remote sensing images to be identified, the semantic segmentation layer in the building recognition model can use high-level semantic information to enhance the foreground features, and improve the discriminability of foreground features by associating the context related to the geographic space scene, based on the first feature map of the target scale of the two remote sensing images to be identified. (i,j)|i=1,2j=2,3,4,5} and the second feature map {P (i,j) |i=1,2j=2,3,4,5} to enhance the foreground features, enhance the difference between the foreground features and the background features of the above two remote sensing images to be identified, generate and output the foreground enhancement feature map of the target scale of the above two remote sensing images to be identified {Z (i,j) |i=1,2j=2,3,4,5}.

[0109] The foreground enhancement feature map of the target scale of each remote sensing image to be identified is input into the change detection layer, and the change detection layer generates a differential feature map of the target scale corresponding to each two remote sensing images to be identified based on the result of image subtraction of the foreground enhancement feature map of the target scale of each two remote sensing images to be identified, and then obtains the differential feature map of the target scale corresponding to each two remote sensing images to be identified output by the change detection layer.

[0110] Specifically, for each two remote sensing images to be identified, in order to highlight the difference between the buildings in the first remote sensing image to be identified and the second remote sensing image to be identified, the change detection layer can use the Sigmoid function to enhance the foreground feature map {Z (i,j) |i=1,2j=2,3,4,5} are normalized, and the normalized {Z (1,j) |j=2,3,4,5} and the normalized {Z (2,j) The result of image subtraction of |j=2,3,4,5} is used as the differential feature map of the above two remote sensing images to be identified at each target scale {D j |j=2,3,4,5}. D j It can be expressed by the following formula:

[0111] D j =S[ζ(Z (1,j) )]-S[ζ(Z (2,j) )]j=2,3,4,5

[0112] Where S represents the Sigmoid function.

[0113] Figure 6 The figure is a schematic diagram of the structure of a building recognition model in the prior art. Figure 6 Figures (a), (b), (c) and (d) in the figure are single-task learning architectures, and figure (e) is a multi-task learning framework. Figure 6As shown, the building recognition model in the prior art does not consider the explicit enhancement of foreground features and the differences between foreground features in different phases, while the building recognition model in the present application can explicitly enhance the foreground features of each remote sensing image to be identified in the previous and next phases based on the semantic segmentation layer and the change detection layer, and highlight the differences between the foreground features of each remote sensing image to be identified in the previous and next phases, thereby achieving more accurate building recognition for each remote sensing image to be identified, and further, based on the more accurate building recognition results of each remote sensing image to be identified, the illegal building detection results in the target area can be obtained.

[0114] The foreground enhancement feature map of the target scale of each remote sensing image to be identified and the differential feature map of the target scale corresponding to every two remote sensing images to be identified are input into the result output layer to obtain the building recognition results of each remote sensing image to be identified output by the result output layer.

[0115] Specifically, for every two remote sensing images to be identified, the result output layer can use the strategy of dynamic weighted pyramid to realize the foreground enhancement feature map {Z (i,j) |i=1,2j=2,3,4,5} and the differential feature map corresponding to the above two remote sensing images to be identified {D j The feature fusion of |j=2,3,4,5} can avoid the loss of smaller buildings in the above two remote sensing images to be identified and the failure to identify larger buildings.

[0116] Optionally, the result output layer can also use Conv-Relu-BatchNormalization-Upsample (denoted as Υ j ) to restore step by step (i,j) |i=1,2j=2,3,4,5} and {D j |j=2,3,4,5} with a spatial resolution of Z (1,2) and Z (2,2) After the same, use 1*1 convolution kernel {Z (i,j) |i=1,2j=2,3,4,5} and {D j |j=2,3,4,5} for convolution, and feature fusion based on the weight corresponding to each target scale to obtain the fusion feature O1 of the first remote sensing image to be identified, the fusion feature O2 of the second remote sensing image to be identified, and the fusion feature O of the differential feature map of the target scale corresponding to the above two remote sensing images to be identified 1 / 2 . Fusion features of remote sensing images to be identified O i It can be expressed by the following formula:

[0117]

[0118] The fusion feature O of the differential feature map of the target scale corresponding to each two remote sensing images to be identified 1 / 2 It can be expressed by the following formula:

[0119]

[0120] Among them, ω j represents the weight corresponding to the target scale j; J represents the number of scales included in the target scale. When the target scale includes 5 scales, J=5.

[0121] Optionally, the result output layer can also use Conv-Relu-BatchNormalization to construct a classifier to convert O i and O 1 / 2 The number of channels is converted into two categories, which represent buildings and non-buildings in the semantic segmentation layer and buildings that have changed and buildings that have not changed in the change detection layer. The bilinear interpolation with an upsampling factor of 4 is used to convert O i and O 1 / 2 After restoring to the same spatial resolution as I1 or I2, the building recognition results of the above two remote sensing images to be identified are output.

[0122] It should be noted that the result output layer can also post-process the building recognition results of the above-mentioned remote sensing images to be identified, and obtain and output the post-processed building recognition results of each remote sensing image to be identified, thereby avoiding the occurrence of the "salt and pepper phenomenon" and reducing the false detection rate and missed detection rate of the building recognition model of each remote sensing image to be identified.

[0123] The process of post-processing the building recognition results of each remote sensing image to be identified in the result output layer may include: using fully connected conditional random fields and hole filling to optimize the boundaries of each identified building outline and fill the holes with smaller areas in the foreground to reduce the missed detection rate and false detection rate of the building recognition model.

[0124] The embodiment of the present invention obtains the first feature map and the second feature map of the target scale of each remote sensing image to be identified through the feature extraction layer in the building recognition model, obtains the foreground enhancement feature map of the target scale of each remote sensing image to be identified based on the first feature map and the second feature map of the target scale of each remote sensing image to be identified through the semantic segmentation layer in the building recognition model, obtains the differential feature map of the target scale corresponding to each two remote sensing images to be identified through the change detection layer in the building recognition model, obtains the building recognition result of the remote sensing image to be identified based on the foreground enhancement feature map of the target scale of each remote sensing image to be identified and the differential feature map of the target scale corresponding to each two remote sensing images to be identified through the result output layer in the building recognition model, can more effectively focus on the foreground feature difference in the remote sensing image to be identified, ignore the background feature difference in the remote sensing image to be identified, can better consider the context information, improve the accuracy of the building recognition results of each remote sensing image to be identified, and further improve the accuracy of illegal building detection in the target area.

[0125] Based on the contents of the above embodiments, the semantic segmentation layer includes: a foreground feature enhancement layer and a spatial detail restoration layer.

[0126] Correspondingly, the first feature map and the second feature map of each target scale of the remote sensing image to be identified are input into the semantic segmentation layer, and the foreground enhancement feature map of each target scale of the remote sensing image to be identified output by the semantic segmentation layer is obtained, which specifically includes: inputting the first feature map and the second feature map of each target scale of the remote sensing image to be identified into the foreground feature enhancement layer, generating a target vector corresponding to each remote sensing image to be identified based on the first feature map of each remote sensing image to be identified by the foreground feature enhancement layer, obtaining the inner product of the target vector and the second feature map corresponding to the target scale of the remote sensing image to be identified as the semantic segmentation feature map of the target scale of the remote sensing image to be identified, and then obtaining the semantic segmentation feature map of each target scale of the remote sensing image to be identified output by the foreground feature enhancement layer.

[0127] Specifically, for every two remote sensing images to be identified, the first feature map {C (i,j) |i=1,2j=2,3,4,5} and the second feature map {P (i,j) |i=1,2j=2,3,4,5} is input into the foreground feature enhancement layer, which uses global average pooling to convert {C (i,5) |i=1,2} is transformed into the target vector {C (i,6) |i=1,2}.

[0128] {P (i,j) |i=1,2j=2,3,4,5} is converted to the geographic scene space through 1*1 convolution to obtain {P′(i,j) |i=1,2j=2,3,4,5}, we can convert {P′ (i,j) |i=1,2j=2,3,4,5} and {C (i,6) |i=1,2} as the semantic segmentation feature map {R (i,j) |i=1,2j=2,3,4,5}, and then obtain the semantic segmentation feature map {R (i,j) |i=1,2j=2,3,4,5}. (i,j) It can be expressed by the following formula:

[0129] R (i,j) =C (i,6) ⊙ζP′ (i,j) i=1,2j=2,3,4,5

[0130] Where ⊙ represents the inner product.

[0131] The semantic segmentation feature map of each target scale of the remote sensing image to be identified is input into the spatial detail recovery layer, and the spatial detail recovery layer generates a foreground enhancement feature map of each target scale of the remote sensing image to be identified by recovering the spatial details of the semantic segmentation feature map of each target scale of the remote sensing image to be identified, and then obtains the foreground enhancement feature map of each target scale of the remote sensing image to be identified output by the spatial detail recovery layer.

[0132] Specifically, the semantic segmentation feature map {R (i,j) |i=1,2j=2,3,4,5} is input into the spatial detail recovery layer, and the spatial detail recovery layer converts {R (i,j) |i=1,2j=2,3,4,5} is converted from the geographic scene space to the feature space through 1*1 convolution to obtain {R′ (i,j) |i=1,2j=2,3,4,5}, use Sigmoid function to solve {R′ (i,j) |i=1,2j=2,3,4,5} to obtain the normalized {R′ (i,j) |i=1,2j=2,3,4,5}, and use Sigmoid function to {C (i,j) |i=1,2j=2,3,4,5} for normalization, and change the normalized {C (i,j) |i=1,2j=2,3,4,5} channels and normalized {R′ (i,j) |i=1,2j=2,3,4,5}, we can restore {C (i,j) |i=1,2j=2,3,4,5} spatial details of each target scale, and obtain the foreground enhancement map of each remote sensing image target scale to be identified {Z(i,j) |i=1,2j=2,3,4,5}. (i,j) It can be expressed by the following formula:

[0133] Z (i,j) =S[ζ(R (i,j) )]+S[ζ(C (i,j) )]

[0134] Where S represents the Sigmoid function.

[0135] In the embodiment of the present invention, after obtaining the target vector corresponding to each remote sensing image to be identified based on the first feature map of the target scale of each remote sensing image to be identified through the foreground feature enhancement layer in the semantic segmentation layer, the inner product of the target vector and the second feature map corresponding to the target scale of the remote sensing image to be identified is obtained as the semantic segmentation feature map of the target scale of the remote sensing image to be identified, and the spatial details in the semantic segmentation feature map of the target scale of each remote sensing image to be identified are restored through the spatial detail recovery layer in the semantic segmentation layer to obtain the foreground enhancement feature map of the target scale of each remote sensing image to be identified. The foreground enhancement feature map of the target scale of each remote sensing image to be identified can be obtained more efficiently and accurately, and the accuracy of the building recognition results of each remote sensing image to be identified output by the building recognition model can be further improved.

[0136] Based on the contents of the above embodiments, the loss function of the building recognition model includes a dynamic weighted loss function.

[0137] The dynamic weighted loss function is determined based on the preset maximum number of iterations and the current number of iterations of the building recognition model under training, the predetermined highest weight and the lowest weight, the semantic segmentation loss function, and the change detection loss function.

[0138] The semantic segmentation loss function and the change detection loss function are determined based on the predicted building recognition results of each sample image and the building recognition result labels of each sample image. The predicted building recognition results of each sample image are input into the building recognition model under training and output by the result output layer of the building recognition model under training.

[0139] The dynamic weighted loss function is used to adjust the first weight corresponding to the semantic segmentation loss function to decrease as the number of iterations of the building recognition model in training increases, and the second weight corresponding to the change detection loss function to increase as the number of iterations of the building recognition model in training increases.

[0140] Specifically, in order to improve the accuracy of the building recognition results of each remote sensing image to be recognized output by the building recognition model, the building recognition model can be trained based on a dynamic weighted loss function, and the training goal is to minimize the dynamic weighted loss function.

[0141] The dynamic weighted loss function is determined based on the semantic segmentation loss function, the change detection loss function, the predetermined highest weight and lowest weight, and the maximum number of iterations and the current number of iterations of the building recognition model under training.

[0142] Optionally, the semantic segmentation loss function and the change detection loss function are constructed based on the DiceLoss function and the FocalLoss function. The DiceLoss function can be expressed by the following formula:

[0143]

[0144] Wherein, X represents the predicted building recognition result of each sample image; Y represents the building recognition result label of each sample image. The predicted building recognition result of each sample image is output by the result output layer of the building recognition model in training after inputting each sample image into the building recognition model in training.

[0145] The FocalLoss function can be expressed by the following formula:

[0146] L focal = -α(1-p t ) γ log(p t )

[0147] Among them, α represents the weight of controlling positive and negative samples; γ represents the weight of controlling difficult and easy samples; p t Represents probability.

[0148] Semantic segmentation loss function L seg It can be expressed by the following formula:

[0149] L seg =L dice +L focal

[0150] Change detection loss function L cd It can be expressed by the following formula:

[0151] L cd =L dice +L focal

[0152] Dynamic weighted loss function L dwm It can be expressed by the following formula:

[0153] L dwm =Ψ seg L cd +Ψ cd L seg

[0154]

[0155]

[0156] Among them, seg Represents the semantic segmentation loss function L seg The corresponding first weight; Ψ cd Represents the dynamic weighted loss function L dwm The corresponding second weight; epoch max Indicates the maximum number of iterations of the building recognition model under training; epoch indicates the current number of iterations of the building recognition model under training; Ψ max represents the predetermined highest weight; Ψ min represents the predetermined minimum weight. Preferably, Ψ max =1.5,Ψ min =0.5.

[0157] In the process of training the building recognition model based on the dynamic weighted loss function, as the number of epochs of the current iteration of the building recognition model in training increases, the semantic segmentation loss function L seg The corresponding first weight Ψ seg From the highest weight max Gradually reduce to the minimum weight Ψ min , change detection loss function L cd The corresponding second weight Ψ cd From the lowest weight Ψ min Gradually increase to the highest weight Ψ max The building recognition model is trained based on the dynamic weighted loss function, which can gradually shift the focus of training from the semantic segmentation loss function L seg Biased towards change detection loss function L cd , which can improve the performance of the building recognition model.

[0158] The embodiment of the present invention trains a building recognition model based on a dynamic weighted loss function, which can ensure that the training focus is biased towards the semantic segmentation loss function in the early stage of training the building recognition model, and ensure that the training focus is biased towards the change detection loss function in the training candidates of the building recognition model, thereby improving the performance of the building recognition model and effectively avoiding the multi-task learning model formed by integrating multiple single-task learning models, and reducing the accuracy of the above-mentioned single-task learning models.

[0159] Based on the contents of the above embodiments, based on the building recognition results, it is judged whether there are illegal buildings in the target area, specifically including: for every two remote sensing images to be identified, based on the building recognition results and the GIS spatial analysis method, it is judged whether there are new buildings in the remote sensing images to be identified after the comparison between the two remote sensing images to be identified; wherein, the functional area element layer is obtained by performing buffer analysis based on the original functional area element layer obtained in advance.

[0160] like Figure 2 As shown, the original functional area element layer may include but is not limited to a cultivated land protection area element layer, a scenic spot protection area element layer, a river and lake protection area element layer, and an ecological protection area element layer, etc. The above original functional area element layer may be pre-acquired from a relevant database.

[0161] By performing buffer analysis on the original functional area feature layer, different types of functional area feature layers can be obtained, such as cultivated land, scenic spots, rivers and lakes, and ecological protection areas. Based on the type of the target area, the functional area feature layer corresponding to the target area can be determined.

[0162] The buffer analysis of the original functional area feature layer may include determining the buffer distance based on the level of the functional area in the original functional area feature layer, and then performing a buffer analysis on the original functional area feature layer based on the above cache distance. For example, when performing a buffer analysis on the river and lake protection area feature layer, the buffer distance may be set according to the level of the river, and the buffer analysis may be performed on the river and lake protection area feature layer based on the above cache distance; or, when performing a buffer analysis on the scenic and historic protection area feature layer, the buffer distance may be set according to the level of the scenic and historic protection area, and the buffer analysis may be performed on the scenic and historic protection area feature layer based on the above cache distance.

[0163] After obtaining the building recognition results of each remote sensing image to be identified output by the building recognition model, for the building recognition results of every two remote sensing images to be identified, based on the intersection analysis in the overlay analysis in the GIS spatial analysis, each element in the building patches that have changed in the first remote sensing image to be identified and the second remote sensing image to be identified can be traversed, and an intersection analysis is performed on each of the above elements with the building patches in the second remote sensing image to be identified, and the elements whose intersection area is greater than the preset first area threshold are added to the newly added building layer, and the elements whose intersection area is less than the preset second area threshold are added to the disappeared building layer.

[0164] After all elements are traversed, a new building layer and a missing building layer can be obtained. Based on the new building layer, it can be determined whether there are new buildings in the second remote sensing image to be identified, and when it is determined that there are new buildings in the second remote sensing image to be identified, the shape and location information of the new buildings can be obtained.

[0165] It should be noted that the first area threshold and the second area threshold can be determined according to actual conditions. In the embodiment of the present invention, the first area threshold and the second area threshold are not specifically limited.

[0166] When determining the newly added buildings in the remote sensing image to be identified after the comparison between every two remote sensing images to be identified, it is determined whether each newly added building is an illegal building based on the functional area feature layer corresponding to the target area.

[0167] Specifically, when a new building is determined in the second remote sensing image to be identified, the new building layer can be overlaid and analyzed with the functional area element layer corresponding to the target area to determine whether the new building in the second remote sensing image to be identified is an illegal building.

[0168] The embodiment of the present invention determines, based on the building recognition results of each remote sensing image to be identified, the number of newly added buildings in the remote sensing image to be identified after the time phase is compared between each two remote sensing images to be identified, and then judges whether the newly added buildings are illegal buildings based on the functional area element layer corresponding to the target area. From the perspective of the functional area, a scientific basis can be provided for the detection of illegal buildings in the target area. Whether there are newly added buildings in the remote sensing image to be identified after the time phase is compared between each two remote sensing images to be identified can be determined only through one GIS spatial analysis, the influence of the cumulative errors caused by multiple GIS spatial analyses can be avoided, and the accuracy of illegal building detection in the target area can be further improved.

[0169] Based on the contents of the above embodiments, based on the functional area element layer corresponding to the target area, judging whether the newly added building is an illegal building specifically includes: based on the functional area element layer corresponding to the target area, obtaining the probability that each newly added building is an illegal building.

[0170] Specifically, based on the overlay analysis result of the newly added building layer and the functional area element layer corresponding to the target area, the probability that each newly added building in the second remote sensing image to be identified is an illegal building can be obtained.

[0171] When the probability that any newly added building is an illegal building is greater than a preset threshold, the newly added building is determined to be an illegal building.

[0172] Specifically, after obtaining the probability that each newly added building in the second remote sensing image to be identified is an illegal building, for any newly added building, the probability that the newly added building is an illegal building may be compared with a preset probability threshold.

[0173] When it is determined that the above probability is greater than a preset probability threshold, it is determined that the newly added building is an illegal building.

[0174] It should be noted that the probability threshold in the embodiment of the present invention can be determined according to actual conditions. The embodiment of the present invention does not limit the specific value of the probability threshold.

[0175] The embodiment of the present invention obtains the probability that each newly added building in the remote sensing image to be identified after comparison between every two remote sensing images to be identified is an illegal building based on the functional area element layer corresponding to the target area, and determines that the newly added building is an illegal building when the probability of any of the above-mentioned newly added buildings being an illegal building is greater than a preset probability threshold. The illegal building detection results in the target area can be quantified, the illegal building detection results in the target area can be more intuitively represented, and the staff can focus on the newly added buildings with higher probabilities, thereby improving the management efficiency of the staff on illegal construction.

[0176] Figure 7 This is a schematic diagram of the structure of an illegal building detection device for urban management supervision provided by the present invention. Figure 7 The present invention provides a device for detecting illegal buildings for use in urban management and supervision. The device for detecting illegal buildings for use in urban management and supervision described below and the method for detecting illegal buildings for use in urban management and supervision described above provided by the present invention can be referred to each other. Figure 7 As shown, the device includes: a data acquisition module 701, an image recognition module 702 and an illegal construction detection module 703.

[0177] The data acquisition module 701 is used to acquire a plurality of remote sensing images to be identified in different phases of the target area.

[0178] The image recognition module 702 is used to input each remote sensing image to be recognized into the building recognition model, and obtain the building recognition results of each remote sensing image to be recognized output by the building recognition model.

[0179] The illegal building detection module 703 is used to obtain the illegal building detection results in the target area based on the building recognition results of each remote sensing image to be recognized.

[0180] The building recognition model is obtained after training based on multiple sample images of different phases of the sample area and the building recognition result labels of each sample image.

[0181] The building recognition model is used to obtain and perform building recognition on each remote sensing image to be identified based on the foreground enhancement feature map of the target scale of each remote sensing image to be identified and the differential feature map of the target scale corresponding to each two remote sensing images to be identified. The foreground enhancement feature map of the target scale of the remote sensing image to be identified is obtained by performing foreground feature enhancement on the feature map of the target scale of the remote sensing image to be identified, and the differential feature map of the target scale corresponding to each two remote sensing images to be identified is obtained based on the result of image subtraction of the foreground enhancement feature map of the target scale of each two remote sensing images to be identified.

[0182] Specifically, the data acquisition module 701, the image recognition module 702 and the illegal construction detection module 703 are electrically connected.

[0183] The embodiment of the present invention obtains and performs building recognition on each of the two remote sensing images to be identified based on a trained building recognition model, based on a foreground enhancement feature map of the target scale of each remote sensing image to be identified and a differential feature map of the target scale corresponding to each two remote sensing images to be identified, and obtains the building recognition results of each remote sensing image to be identified including the building recognition results of each two remote sensing images to be identified. Then, based on the building recognition results of each remote sensing image to be identified, the illegal building detection results of the target area are obtained, and the buildings in the remote sensing images to be identified in the previous and next phases and the changed buildings can be automatically extracted, and the illegal building detection in the target area can be performed more accurately. The process of illegal building detection is simpler and more efficient, and the investment in manpower and material resources can be reduced, thereby reducing the cost investment in illegal building detection.

[0184] Figure 8 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 8As shown, the electronic device may include: a processor (processor) 810, a communication interface (Communications Interface) 820, a memory (memory) 830 and a communication bus 840, wherein the processor 810, the communication interface 820, and the memory 830 communicate with each other through the communication bus 840. The processor 810 can call the logic instructions in the memory 830 to execute a method for detecting illegal buildings applied to urban management supervision, the method comprising: obtaining a plurality of remote sensing images to be identified at different phases of the target area; inputting each remote sensing image to be identified into a building recognition model, and obtaining the building recognition results of each remote sensing image to be identified output by the building recognition model; obtaining the detection results of illegal buildings in the target area based on the building recognition results of each remote sensing image to be identified; wherein the building recognition model is obtained after training based on a plurality of sample images of different phases of the sample area and the building recognition result labels of each sample image; the building recognition model is used to obtain and perform building recognition on each remote sensing image to be identified based on the foreground enhancement feature map of the target scale of each remote sensing image to be identified and the differential feature map of the target scale corresponding to each two remote sensing images to be identified, the foreground enhancement feature map of the target scale of the remote sensing image to be identified is obtained by performing foreground feature enhancement on the feature map of the target scale of the remote sensing image to be identified, and the differential feature map of the target scale corresponding to each two remote sensing images to be identified is obtained based on the result of image subtraction of the foreground enhancement feature map of the target scale of each two remote sensing images to be identified.

[0185] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0186] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute a method for detecting illegal buildings applied to urban management supervision provided by the above methods. The method includes: obtaining multiple remote sensing images to be identified in different phases of a target area; inputting each remote sensing image to be identified into a building recognition model, and obtaining a building recognition result of each remote sensing image to be identified output by the building recognition model; based on the building recognition results of each remote sensing image to be identified, obtaining a detection result of illegal buildings in the target area; wherein the building recognition The model is obtained after training based on multiple sample images of different phases in the sample area and the building recognition result labels of each sample image; the building recognition model is used to obtain and recognize buildings for each remote sensing image to be identified based on the foreground enhancement feature map of the target scale of each remote sensing image to be identified and the differential feature map of the target scale corresponding to each two remote sensing images to be identified. The foreground enhancement feature map of the target scale of the remote sensing image to be identified is obtained by enhancing the foreground features of the feature map of the target scale of the remote sensing image to be identified, and the differential feature map of the target scale corresponding to each two remote sensing images to be identified is obtained based on the result of image subtraction of the foreground enhancement feature map of the target scale of each two remote sensing images to be identified.

[0187] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute a method for detecting illegal buildings applied to urban management supervision provided by the above-mentioned methods, the method comprising: obtaining a plurality of remote sensing images to be identified of a target area at different time phases; inputting each remote sensing image to be identified into a building recognition model, and obtaining a building recognition result of each remote sensing image to be identified output by the building recognition model; based on the building recognition result of each remote sensing image to be identified, obtaining a detection result of illegal buildings in the target area; wherein the building recognition model is based on a plurality of sample areas at different time phases. The building recognition model is used to obtain and perform building recognition on each remote sensing image to be identified based on the foreground enhancement feature map of the target scale of each remote sensing image to be identified and the differential feature map of the target scale corresponding to each two remote sensing images to be identified. The foreground enhancement feature map of the target scale of the remote sensing image to be identified is obtained by performing foreground feature enhancement on the feature map of the target scale of the remote sensing image to be identified, and the differential feature map of the target scale corresponding to each two remote sensing images to be identified is obtained based on the result of image subtraction of the foreground enhancement feature map of the target scale of each two remote sensing images to be identified.

[0188] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0189] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0190] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting illegal buildings for urban management supervision, characterized in that: include: Acquire multiple remote sensing images of the target area at different phases to be identified; Inputting each of the remote sensing images to be identified into a building recognition model, and obtaining a building recognition result of each of the remote sensing images to be identified output by the building recognition model; Based on the building recognition results of each of the remote sensing images to be identified, obtaining the detection results of illegal buildings in the target area; The building recognition model is obtained after training based on a plurality of sample images of different time phases of the sample area and the building recognition result labels of each of the sample images; The building recognition model is used to obtain and perform building recognition on each of the remote sensing images to be recognized based on a foreground enhancement feature map of the target scale of each remote sensing image to be recognized and a differential feature map of the target scale corresponding to each two remote sensing images to be recognized, wherein the foreground enhancement feature map of the target scale of the remote sensing image to be recognized is obtained by performing foreground feature enhancement on the feature map of the target scale of the remote sensing image to be recognized, and the differential feature map of the target scale corresponding to each two remote sensing images to be recognized is obtained based on a result of image subtraction of the foreground enhancement feature map of the target scale of each two remote sensing images to be recognized; The building recognition model includes: a feature extraction layer, a semantic segmentation layer, a change detection layer and a result output layer; Correspondingly, the step of inputting each of the remote sensing images to be identified into a building recognition model and obtaining a building recognition result of each of the remote sensing images to be identified output by the building recognition model specifically includes: Input each of the remote sensing images to be identified into the feature extraction layer, and obtain a first feature map and a second feature map of the target scale of each of the remote sensing images to be identified output by the feature extraction layer; Inputting the first feature map and the second feature map of each to-be-identified remote sensing image target scale into the semantic segmentation layer, and obtaining the foreground enhancement feature map of each to-be-identified remote sensing image target scale output by the semantic segmentation layer; Inputting the foreground enhancement feature map of the target scale of each remote sensing image to be identified into the change detection layer, the change detection layer performs image subtraction based on the foreground enhancement feature map of the target scale of each two remote sensing images to be identified, and generates a differential feature map of the target scale corresponding to each two remote sensing images to be identified, thereby obtaining the differential feature map of the target scale corresponding to each two remote sensing images to be identified output by the change detection layer; The foreground enhancement feature map of the target scale of each remote sensing image to be identified and the differential feature map of the target scale corresponding to each two remote sensing images to be identified are input into the result output layer to obtain the building recognition results of each remote sensing image to be identified output by the result output layer.

2. The method for detecting illegal buildings used in urban management supervision according to claim 1, characterized in that: The loss functions of the building recognition model include a dynamic weighted loss function, a semantic segmentation loss function, and a change detection loss function; The dynamic weighted loss function is determined based on a preset maximum number of iterations and a current number of iterations of the building recognition model in training, a predetermined maximum weight and a predetermined minimum weight, the semantic segmentation loss function, and the change detection loss function; The semantic segmentation loss function and the change detection loss function are determined based on the predicted building recognition results of each of the sample images and the building recognition result labels of each of the sample images, and the predicted building recognition results of each of the sample images are input into the building recognition model under training and output by the result output layer of the building recognition model under training; The dynamic weighted loss function is used to adjust the first weight corresponding to the semantic segmentation loss function to decrease as the number of iterations of the building recognition model under training increases, and the second weight corresponding to the change detection loss function to increase as the number of iterations of the building recognition model under training increases.

3. The method for detecting illegal buildings used in urban management supervision according to claim 1, characterized in that: The determining whether there is an illegal building in the target area based on the building recognition result specifically includes: For every two remote sensing images to be identified, based on the building identification results and the GIS spatial analysis method, it is determined whether there is a new building in the remote sensing image to be identified after the comparison between the two remote sensing images to be identified; In the case where a new building is added to the remote sensing image to be identified after the comparison between each two remote sensing images to be identified, judging whether each new building is an illegal building based on the functional area element layer corresponding to the target area; The functional area element layer is obtained by performing buffer analysis based on the original functional area element layer obtained in advance.

4. The method for detecting illegal buildings used in urban management supervision according to claim 3 is characterized in that: The determining whether each newly added building is an illegal building based on the functional area element layer corresponding to the target area specifically includes: Based on the functional area element layer corresponding to the target area, obtaining the probability that each of the newly added buildings is an illegal building; When the probability that any newly added building is an illegal building is greater than a preset threshold, the newly added building is determined to be an illegal building.

5. The method for detecting illegal buildings used in urban management supervision according to claim 1, characterized in that: The semantic segmentation layer includes: a foreground feature enhancement layer and a spatial detail restoration layer; Correspondingly, the step of inputting the first feature map and the second feature map of each to-be-identified remote sensing image target scale into the semantic segmentation layer, and obtaining the foreground enhancement feature map of each to-be-identified remote sensing image target scale output by the semantic segmentation layer, specifically includes: Inputting the first feature map and the second feature map of each target scale of the remote sensing image to be identified into the foreground feature enhancement layer, the foreground feature enhancement layer generates a target vector corresponding to each remote sensing image to be identified based on the first feature map of each remote sensing image to be identified, obtaining the inner product of the target vector and the second feature map corresponding to the target scale of the remote sensing image to be identified as the semantic segmentation feature map of the target scale of the remote sensing image to be identified, and then obtaining the semantic segmentation feature map of each target scale of the remote sensing image to be identified output by the foreground feature enhancement layer; The semantic segmentation feature map of each target scale of the remote sensing image to be identified is input into the spatial detail recovery layer, and the spatial detail recovery layer generates a foreground enhancement feature map of each target scale of the remote sensing image to be identified by recovering the spatial details of the semantic segmentation feature map of each target scale of the remote sensing image to be identified, and then the foreground enhancement feature map of each remote sensing image to be identified output by the spatial detail recovery layer is obtained.

6. A method for detecting illegal buildings used in urban management supervision according to any one of claims 1 to 5, characterized in that: The target scale includes multiple scales.

7. A device for detecting illegal buildings used in urban management supervision, characterized in that: include: A data acquisition module is used to acquire multiple remote sensing images to be identified in different phases of the target area; An image recognition module, used for inputting each of the remote sensing images to be recognized into a building recognition model, and obtaining a building recognition result of each of the remote sensing images to be recognized output by the building recognition model; An illegal building detection module, used to obtain the illegal building detection results of the target area based on the building recognition results of each of the remote sensing images to be identified; The building recognition model is obtained after training based on a plurality of sample images of different time phases of the sample area and the building recognition result labels of each of the sample images; The building recognition model is used to obtain and perform building recognition on each of the remote sensing images to be recognized based on a foreground enhancement feature map of the target scale of each remote sensing image to be recognized and a differential feature map of the target scale corresponding to each two remote sensing images to be recognized, wherein the foreground enhancement feature map of the target scale of the remote sensing image to be recognized is obtained by performing foreground feature enhancement on the feature map of the target scale of the remote sensing image to be recognized, and the differential feature map of the target scale corresponding to each two remote sensing images to be recognized is obtained based on a result of image subtraction of the foreground enhancement feature map of the target scale of each two remote sensing images to be recognized; The building recognition model includes: a feature extraction layer, a semantic segmentation layer, a change detection layer and a result output layer; Correspondingly, the image recognition module inputs each of the remote sensing images to be recognized into a building recognition model, and obtains the building recognition results of each of the remote sensing images to be recognized output by the building recognition model, specifically including: Input each of the remote sensing images to be identified into the feature extraction layer, and obtain a first feature map and a second feature map of the target scale of each of the remote sensing images to be identified output by the feature extraction layer; Inputting the first feature map and the second feature map of each to-be-identified remote sensing image target scale into the semantic segmentation layer, and obtaining the foreground enhancement feature map of each to-be-identified remote sensing image target scale output by the semantic segmentation layer; Inputting the foreground enhancement feature map of the target scale of each remote sensing image to be identified into the change detection layer, the change detection layer performs image subtraction based on the foreground enhancement feature map of the target scale of each two remote sensing images to be identified, and generates a differential feature map of the target scale corresponding to each two remote sensing images to be identified, thereby obtaining the differential feature map of the target scale corresponding to each two remote sensing images to be identified output by the change detection layer; The foreground enhancement feature map of the target scale of each remote sensing image to be identified and the differential feature map of the target scale corresponding to each two remote sensing images to be identified are input into the result output layer to obtain the building recognition results of each remote sensing image to be identified output by the result output layer.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the illegal building detection method applied to urban management supervision as described in any one of claims 1 to 6 are implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the illegal building detection method applied to urban management supervision as described in any one of claims 1 to 6 are implemented.

Citation Information

Patent Citations

  • Urban illegal building detection method based on high-resolution remote sensing image

    CN112287832A

  • Power transmission line illegal building identification method and related device

    CN112633142A