A method and system for automatically updating building spots in airport clear area

Through the combination of deep learning and GIS technology, a scene-level change detection and pixel-level extraction network of buildings in airport clearance areas has been built, which has solved the problem of automatic map spot update in traditional monitoring methods and improved the safety monitoring efficiency and accuracy of airport clearance areas.

CN118918047BActive Publication Date: 2024-12-06WUHAN UNIV
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202411404501.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-10-10
Publication Date
2024-12-06
Estimated Expiration
2044-10-10

AI Technical Summary

Technical Problem

Traditional airport clearance building monitoring relies on manual field surveys and traditional remote sensing technology, and cannot achieve automated updates of building maps and is difficult to meet the needs of airport clearance safety monitoring.

Method used

Using deep learning methods based on twin neural networks and encoding-decoding structures, building scene-level change detection networks and pixel-level extraction networks are built. Combined with GIS technology, the map spectra of the airport clearance area is automatically updated to realize automatic identification of map spectra and Dynamic monitoring.

Benefits of technology

It realizes the automatic update of building maps in the airport clearance area, improves the efficiency of safety monitoring in the airport clearance area, reduces sample production and calculation costs, and optimizes the accuracy and efficiency of updates.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118918047B_ABST
    Figure CN118918047B_ABST
Patent Text Reader

Abstract

The present invention belongs to the field of image processing technology, and discloses a method and system for automatically updating building spots in an airport clear area. The present invention collects and obtains dual-phase images of an airport clear area and performs preprocessing, makes a sample set, constructs a building scene-level change detection network and a building pixel-level extraction network, trains the network using the sample set, detects the preprocessed dual-phase images using the trained building scene-level change detection network, and determines the regional changes; retains the building spots in the unchanged area; processes the post-phase images of the changed area using the trained building pixel-level extraction network to obtain the building spots in the changed area; updates the building spots in the changed area using GIS, merges the updated building spots in the changed area with the building spots in the unchanged area, and completes the overall automatic update of the spots. The present invention can meet the application requirements of airport clear area building monitoring.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and more specifically, relates to a method and system for automatically updating building patches in an airport clear area. Background Art

[0002] The airport clear zone is a safety guarantee for aircraft takeoff, landing and flight. The relevant airport safety standards clearly stipulate the restricted heights of various landforms in the airport clear zone. With the development of the airport's airside economic zone, the contradiction between airport clear zone safety and the expansion of urban construction land has become increasingly intensified, and the complexity of the management of airport clear zones has increased dramatically. In addition, with the proposal and promotion of the concept of clear zone integration, the clear zone protection area has far exceeded the scope of about 1,000 square kilometers stipulated in the "Technical Standards for Civil Airport Airfields" and increased to about 10,000 square kilometers. Traditional airport clear zone building monitoring is mainly based on manual field surveys, which has a large workload and a long cycle and can no longer meet actual needs.

[0003] Satellite remote sensing images have a large coverage area, a short re-entry period, and a spatial resolution of sub-meter level, providing reliable data support for timely acquisition and monitoring of surface information. However, although the current deep learning research in the field of remote sensing image processing is becoming more and more abundant, the update of building patches is still mainly based on manual means such as visual interpretation. In the later improvement research, the patch update scheme mainly relies on traditional remote sensing technical means and professional software. Although it has improved work efficiency to a certain extent, the accuracy is limited by the artificially designed middle and low-level features, and it is a semi-automatic update, which is difficult to meet the application requirements of airport clear area building monitoring. How to realize the automatic update of airport clear area building patches, improve the efficiency of airport clear area safety monitoring, and meet the application requirements of airport clear area building monitoring is a major issue that this field pays attention to and needs to be solved. Summary of the invention

[0004] The present invention provides a method and system for automatically updating a map of building spots in an airport clear area, thereby solving the problem that the prior art cannot realize automatic updating of a map of building spots in an airport clear area and is difficult to meet the application requirements of monitoring buildings in an airport clear area.

[0005] The present invention provides a method for automatically updating a building pattern in an airport clearance area, comprising the following steps:

[0006] Acquire dual-temporal images of the airport clear area and perform preprocessing;

[0007] Producing a sample set, including producing a number of building scene-level change detection samples and a number of building pixel-level samples;

[0008] Based on the twin neural network structure, a building scene-level change detection network guided by cumulative difference features is constructed; based on the encoding-decoding structure, a building pixel-level extraction network guided by multi-level features is constructed;

[0009] Using the building scene-level change detection samples to train the building scene-level change detection network to obtain a trained building scene-level change detection network; using the building pixel-level samples to train the building pixel-level extraction network to obtain a trained building pixel-level extraction network;

[0010] The preprocessed dual-phase images are detected using the trained building scene-level change detection network to determine the regional changes; if it is determined to be an unchanged area, the building patches in the unchanged area are retained; if it is determined to be a changed area, the post-phase images of the changed area are processed using the trained building pixel-level extraction network to extract the building patches in the changed area; the building patches in the changed area are updated using GIS, and the updated building patches in the changed area are merged with the building patches in the unchanged area to complete the overall automatic update of the patches.

[0011] Preferably, making the building scene-level change detection sample comprises: cropping and marking the dual-phase images; when marking, marking the image pairs as "changed" or "unchanged" according to whether the building has changed before and after the phase;

[0012] The production of the building pixel-level samples includes: marking and cropping the dual-phase images; when marking, marking clusters of buildings in a continuous manner and marking single-building buildings separately.

[0013] Preferably, the building scene-level change detection network includes a building feature extraction unit, a difference feature extraction module and a cumulative difference classifier;

[0014] The building feature extraction unit is used to extract dual-phase image features;

[0015] The difference feature extraction module is used to convert the dual-phase image features into image difference features and image intensity features, taking the image difference features as the features to be optimized and the image intensity features as the supplementary features to obtain the difference features;

[0016] The cumulative difference classifier is used to classify "changed" or "unchanged" according to the difference characteristics.

[0017] Preferably, the difference feature extraction module takes the dual-phase image features of the same layer and the difference features extracted from the previous layer as input, and converts the dual-phase image features into image difference features and image intensity features by matrix subtraction and taking absolute values ​​and matrix addition; if there is a difference feature of the previous layer, it is cascaded and fused with the image difference feature, and input 1×1 deep convolution and 1×1 convolution operations to obtain a comprehensive difference feature; the comprehensive difference feature is input into a spatial attention mechanism composed of 1×1 convolution and Sigmoid activation function, and the image intensity feature is added as a constraint, and the filtered intensity feature is input into 1×1 deep convolution and 1×1 convolution operations to obtain a comprehensive intensity feature; the comprehensive difference feature and the comprehensive intensity feature are cascaded and fused and input into the channel attention mechanism, split into optimized comprehensive difference features and optimized comprehensive intensity features according to the original channel, and then the two are added and fused to obtain a first comprehensive feature; the first comprehensive feature is input into two consecutive 3×3 convolution operations, and is residually connected with the comprehensive difference feature to obtain the difference feature of this layer;

[0018] The cumulative difference classifier inputs the difference features of different layers into the global feature fusion module to obtain the multi-level change features of the building; the difference features of the highest layer are input into the multi-scale feature fusion module to obtain the multi-scale change features of the building; the multi-level change features and the multi-scale change features are cascaded and fused and a convolution operation is performed to achieve classification.

[0019] Preferably, the comprehensive differential feature and the comprehensive intensity feature are cascaded and fused and input into an improved channel attention mechanism. The improved channel attention mechanism is a dual-branch structure. The first branch is used to focus on the global maximum feature, and the second branch is used to focus on the global average feature. The first branch and the second branch learn the task-related discriminative information and intensity information in the features respectively through corresponding adaptive pooling and multi-layer perceptrons. Finally, the channel attention weight is obtained by adding and fusion and inputting the Sigmoid activation function.

[0020] Preferably, the multi-scale feature fusion module is improved based on the dilated space pyramid pooling, global maximum pooling is introduced in the extraction of global features, and the improved multi-head attention mechanism and the improved feedforward network are used to extract long-distance features;

[0021] The improved multi-head attention mechanism first divides the input feature into multiple sub-features according to the spatial range, learns the Q matrix through a 1×1 convolution operation after downsampling, and comprehensively extracts the V matrix through a 1×1 convolution operation and a 3×3 deep convolution, and the K matrix is ​​obtained by inputting the V matrix into a 1×1 convolution operation; the correlation matrix is ​​calculated using the Q matrix and the K matrix, and after introducing the position code, it is input into a 1×1 deep convolution, and then into a Softmax activation function to obtain the multi-head attention weight; the multi-head attention weight mask is assigned to the V matrix, and after upsampling, it is spliced ​​according to the original spatial range, and then a nonlinear feature is introduced through a Relu activation function, and a 1×1 convolution operation is input for channel conversion, and a residual connection is performed with the original feature to obtain the output feature optimized by the improved multi-head attention mechanism; the output feature optimized by the improved multi-head attention mechanism is input into the improved feedforward network for nonlinear conversion to obtain long-distance features; wherein the improved feedforward network adopts an inverted residual structure, uses a 1×1 convolution operation for channel dimension increase, uses the middle 3×3 deep convolution to supplement the detail information, and then inputs a 1×1 convolution operation for channel dimension reduction.

[0022] Preferably, the building pixel-level extraction network includes a lightweight encoding unit, a residual pyramid pooling module and a multi-level feature fusion module;

[0023] The lightweight coding unit is used to extract building features, where the building features include detailed features of the building;

[0024] The residual pyramid pooling module is used to divide the building features into local features and global features, divide the local features into short-distance features and long-distance features, and fuse and convolve the global features, the short-distance features and the long-distance features through a cascade operation to extract context features;

[0025] The multi-level feature fusion module is used to use the semantic features in the decoding process as the features to be optimized, and the detail features and the context features as supplementary features to obtain semantic features that take both details and context into consideration; and is used to extract building boundary range information based on the deepest semantic features that take both details and context into consideration to obtain building patches.

[0026] Preferably, the residual pyramid pooling module extracts short-range features through a 1×1 convolution operation and compresses the channel; the short-range features are input into a dilated convolution with different expansion rates connected by a residual structure to extract long-range features; in the process of extracting global features, adaptive average pooling is used to compress the spatial dimension to 1×1, and then the channel is compressed through a 1×1 convolution operation, and restored to the original size through bilinear interpolation;

[0027] The multi-level feature fusion module adjusts the size of the semantic feature to keep its size consistent with the detail feature and the context feature, and then merges them in the latter stage, inputs a 1×1 deep convolution, performs batch normalization, and then performs two consecutive 3×3 convolution operations to obtain a second comprehensive feature; performs a 1×1 convolution operation on the semantic feature to obtain a semantic feature after alignment of semantics; then inputs it into the channel attention mechanism, and finally fuses the second comprehensive feature with the semantic feature after alignment of semantics through an addition operation to obtain a semantic feature that takes into account both details and context; the deepest semantic feature that takes into account both details and context is input into a classifier composed of 1×1 convolution to complete the extraction of building boundary range information.

[0028] Preferably, the updating of the building spots of the changed area by using GIS includes: using GIS to analyze and process the pseudo-change caused by the displacement of high-rise buildings caused by different satellite azimuths, using GIS to process the situation that the original spots include multiple extracted spots, and using GIS to process the situation that the extracted spots include multiple original spots due to the expansion of the original spots, and the extracted spots overflow the range of the original spots;

[0029] Among them, the use of GIS to analyze and process the pseudo-changes caused by the displacement of high-rise buildings due to different satellite azimuths includes:

[0030] Calculate the geometric area of ​​the building patch in the previous phase and record it as the first area ; Calculate the geometric area of ​​the building patch in the later phase, recorded as the second area ; Jointly analyze the building spots before and after the time phase and retain their attributes to obtain the joint spots ; Calculate the joint pattern The area of According to the first area and the second area Calculate the first area ratio , according to the area of ​​the joint patch and the second area Calculate the second area ratio ; Set a first selection condition, the first selection condition includes the first area ratio and the second area ratio Numerical conditions that need to be met; combined with joint patches , through attribute selection, we get the joint patches that meet the first selection condition ;

[0031] Among them, the use of GIS to process the situation where the original map includes multiple extracted maps includes:

[0032] According to the area of ​​the joint patch and the first area Calculate the third area ratio ; Based on the first area and the second area All need to be greater than 0 as a screening condition for the joint pattern Screening is performed to obtain the filtered joint patch ; Unique identification field of the building patch in the previous phase As a grouping field, the third area ratio As the field to be counted, combined with the filtered joint spots Perform summary statistics to obtain the first summary statistics value ; Unique identification field for building patches in the future phase As a grouping field, the second area ratio As the field to be counted, combined with the filtered joint spots Perform summary statistics to obtain the second summary statistics value ; Set a second selection condition, the second selection condition includes the first summary statistical value and the second summary statistic The numerical conditions that need to be met; combined with the joint patches that meet the first selection condition , through attribute selection, we get the joint patches that meet the second selection condition ; According to the previous phase pattern and the joint patches that meet the second selection criteria , through position selection, filter out the retained previous phase spots ;

[0033] Among them, the use of GIS to deal with the situation where the original map spots are expanded, resulting in the extracted map spots including multiple original map spots, and the extracted map spots overflow the original map spots include:

[0034] Phase pattern after joint analysis The preserved previous phase pattern , get the new joint patch ; Calculate the new joint patch The area of ; According to the area of ​​the new joint pattern and the second area The fourth area ratio is calculated ; Set a third selection condition, the third selection condition includes the fourth area ratio The first area and the area of ​​the new joint patch Numerical conditions that need to be met; combined with the new joint pattern , through attribute selection, a new joint patch that meets the third selection condition is obtained , complete the patch update of the changed area.

[0035] In another aspect, the present invention provides an airport clear area building pattern automatic updating system, comprising:

[0036] A data acquisition unit, used for acquiring dual-phase images of the airport clear area and performing pre-processing;

[0037] A sample making unit, used for making a sample set, including making a number of building scene-level change detection samples and a number of building pixel-level samples;

[0038] A network construction unit, used to construct a building scene-level change detection network guided by cumulative difference features based on a twin neural network structure, and used to construct a building pixel-level extraction network guided by multi-level features based on an encoding-decoding structure;

[0039] A network training unit is used to train the building scene-level change detection network using the building scene-level change detection samples to obtain a trained building scene-level change detection network; and to train the building pixel-level extraction network using the building pixel-level samples to obtain a trained building pixel-level extraction network;

[0040] A patch updating unit is used to detect the preprocessed dual-phase image using the trained building scene-level change detection network to determine the regional change; if it is determined to be an unchanged area, the building patches in the unchanged area are retained; if it is determined to be a changed area, the post-phase image of the changed area is processed using the trained building pixel-level extraction network to extract the building patches in the changed area; the building patches in the changed area are updated using GIS, and the updated building patches in the changed area are merged with the building patches in the unchanged area to complete the overall automatic patch update;

[0041] The airport clear zone building map automatic updating system is used to execute the steps in the above-mentioned airport clear zone building map automatic updating method.

[0042] One or more technical solutions provided in the present invention have at least the following technical effects or advantages:

[0043] (1) The present invention collects and obtains dual-phase images of an airport clear area and performs preprocessing to prepare a sample set, constructs a building scene-level change detection network and a building pixel-level extraction network, and uses the sample set to train the network. Afterwards, the preprocessed dual-phase images are detected using the trained building scene-level change detection network to determine the regional change situation; if it is determined to be an unchanged area, the building patches of the unchanged area are retained; if it is determined to be a changed area, the post-phase images of the changed area are processed using the trained building pixel-level extraction network to extract the building patches of the changed area; GIS (Geographic Information System) is used to update the building patches of the changed area, and the updated building patches of the changed area are merged with the building patches of the unchanged area to complete the overall patch automatic update. The present invention introduces deep learning and GIS technology on the basis of remote sensing image processing. Deep learning can automatically learn features from image samples, mine deep features for qualitative analysis, and realize automatic identification and dynamic monitoring of buildings; while GIS can conduct quantitative research from aspects such as spatial position, spatial distribution, and spatial relationship, extract its potential information, and assist in relevant decision-making. Both can provide effective technical support for clear area building monitoring. The present invention proposes an automatic update method for airport clear area building spots combining remote sensing, deep learning, and GIS technology, which can realize the automation and intelligence of spot update, can timely grasp the current clear area building distribution information, improve the efficiency of airport clear area safety monitoring, and meet the application needs of airport clear area building monitoring.

[0044] (2) On the one hand, deep learning relies on a large number of samples and computing resources. The basic model usually learns and distinguishes image features from a certain scale and level, and lacks comprehensive consideration of multi-scale and multi-level features. The buildings in the airport clear zone are of different sizes and types, and the background information is complex, making the basic model difficult to apply. The present invention proposes a deep learning method for scene-level change detection and pixel-level extraction of buildings in an airport clearance zone. Both methods make full use of the attention mechanism and improved convolutions such as deep separable convolution and dilated convolution, and build network models based on residual structures, pyramid structures, etc. The former is a scene-level method, and the latter has small sample learning capabilities. Both can reduce sample production costs and computational costs, and balance network efficiency and accuracy. The present invention proposes a multi-feature fusion scheme based on main feature constraints. The scheme uses the feature to be optimized as the main feature and the supplementary feature as the secondary feature. The secondary features are screened based on the channel attention weight or spatial attention weight of the main feature as a constraint, and then the residual structure is used to fuse the secondary features with the main features, so as to ensure the advantages of the main features while effectively supplementing other feature information, so that the network can fully learn and utilize different features and adapt to the diverse building information and complex background environment in the airport clearance zone. The present invention can fully learn and utilize different features while saving samples and computational costs, is suitable for the airport clearance zone, and achieves a balance between method cost, efficiency and accuracy. On the other hand, the update of building patches in the clear area involves GIS analysis of building patches in different phases. Affected by image quality and patch accuracy, updating patches through simple differences will result in limited accuracy. The present invention formulates a GIS analysis scheme for building change areas, which can alleviate problems caused by data differences, blurred patch boundaries, etc. The present invention combines the above deep learning methods with GIS methods of different detection granularities to realize the automatic update of building patches in the airport clear area, which can reduce sample costs and computational costs, and further optimize the efficiency and accuracy of the update. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 is an example diagram of a building scene-level change detection sample in an automatic updating method for building patches in an airport clear area provided by Embodiment 1 of the present invention. Figure 1 (a) in the equation is a variation sample. Figure 1 (b) in the figure is the unchanged sample.

[0046] Figure 2 is an example diagram of pixel-level samples of buildings in an automatic updating method for building patches in an airport clear area provided in Embodiment 1 of the present invention. Figure 2 (a) in the figure is the factory. Figure 2 (b) in the figure is a contiguous village. Figure 2 (c) is a single-family house. Figure 2 (d) in the table stands for others.

[0047] Figure 3 It is a framework diagram of a building scene-level change detection network (DAFS-Net) in an automated updating method for airport clear area building patches provided in Example 1 of the present invention.

[0048] Figure 4 Schematic diagram of the lightweight encoding block in DAFS-Net.

[0049] Figure 5 It is a schematic diagram of the difference feature extraction module in DAFS-Net.

[0050] Figure 6 Schematic diagram of the cumulative difference classifier in DAFS-Net.

[0051] Figure 7 It is a schematic diagram of the multi-scale feature fusion module in DAFS-Net.

[0052] Figure 8 It is a framework diagram of a building pixel-level extraction network (MLFR-Net) in an automated updating method for airport clear area building patches provided in Example 1 of the present invention.

[0053] Fig. 9 It is a schematic diagram of the lightweight encoding block in MLFR-Net.

[0054] Fig.10 It is a schematic diagram of the residual pyramid pooling module in MLFR-Net.

[0055] Fig.11 It is a schematic diagram of the multi-level feature fusion module in MLFR-Net.

[0056] Fig.12 It is a flow chart of GIS analysis and map update of building change areas in an automatic update method of airport clearance area building maps provided by Example 1 of the present invention.

[0057] Fig.13 It is the experimental area image and the corresponding update result diagram in Example 1 of the present invention.

[0058] Fig.14 It is a schematic diagram of the flow chart of the automatic updating framework of the airport clear area building pattern in Example 1 of the present invention. DETAILED DESCRIPTION

[0059] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0060] Embodiment 1:

[0061] Embodiment 1 provides a method for automatically updating a building pattern in an airport clear area, which mainly includes the following steps:

[0062] Acquire dual-temporal images of the airport clear area and perform preprocessing;

[0063] Producing a sample set, including producing a number of building scene-level change detection samples and a number of building pixel-level samples;

[0064] Based on the twin neural network structure, a building scene-level change detection network guided by cumulative difference features is constructed; based on the encoding-decoding structure, a building pixel-level extraction network guided by multi-level features is constructed;

[0065] Using the building scene-level change detection samples to train the building scene-level change detection network to obtain a trained building scene-level change detection network; using the building pixel-level samples to train the building pixel-level extraction network to obtain a trained building pixel-level extraction network;

[0066] The preprocessed dual-phase images are detected using the trained building scene-level change detection network to determine the regional changes; if it is determined to be an unchanged area, the building patches in the unchanged area are retained; if it is determined to be a changed area, the post-phase images of the changed area are processed using the trained building pixel-level extraction network to extract the building patches in the changed area; the building patches in the changed area are updated using GIS, and the updated building patches in the changed area are merged with the building patches in the unchanged area to complete the overall automatic update of the patches.

[0067] The following is a detailed explanation in the form of split steps combined with relevant parameters and design details.

[0068] Embodiment 1 provides a method for automatically updating a building pattern in an airport clear area, comprising the following steps:

[0069] Step 1: Acquire dual-temporal images of the airport clear area and perform preprocessing.

[0070] Step 1 is image data acquisition and preprocessing, that is, collecting dual-temporal airport clear zone high spatial resolution remote sensing images and performing image preprocessing operations.

[0071] The specific method of step 1 is: for the dual-phase high-spatial-resolution remote sensing images of the airport clear area, use remote sensing processing software to perform pre-processing operations such as image fusion, radiation correction, atmospheric correction, and registration.

[0072] For example, high-spatial-resolution remote sensing images of the Yinchuan Airport airspace area in 2017 and 2021 were collected, and remote sensing processing software was used to perform pre-processing operations such as image fusion, radiation correction, atmospheric correction, and registration.

[0073] Step 2: Make a sample set.

[0074] This includes producing a number of scene-level change detection samples of buildings in the airport's clear space and a number of pixel-level samples of buildings in the airport's clear space. On this basis, the sample set is divided into a training set and a validation set in a certain proportion.

[0075] Specifically, step 2 includes the following sub-steps:

[0076] Step 2.1: On the one hand, making the building scene-level change detection sample includes: cropping and labeling the dual-phase images; for the building scene-level change detection task, the image needs to be cropped into a suitable image block size, and the image pairs are labeled as "changed" or "unchanged" according to whether the before and after phases of the building have changed.

[0077] For example, step 2.1: for the building scene-level change detection task, the dual-phase image needs to be cropped into 256×256 image blocks, and the image pairs are marked as "changed" or "unchanged" according to whether the building has changed before and after the phase. The sample examples prepared in this embodiment are as follows: Figure 1 As shown, Figure 1 (a) in the equation is a variation sample. Figure 1 (b) in the figure is the unchanged sample.

[0078] On the other hand, making the pixel-level samples of the buildings includes: marking and cropping the dual-phase images; for the task of pixel-level extraction of buildings, considering the diversity of airport buildings, clusters of buildings are marked in a continuous manner, and single buildings are marked individually.

[0079] For example, for the task of pixel-level extraction of buildings, we selected images from 2017 for annotation. Considering the diversity of airport buildings, clusters of buildings were marked together, and single buildings were marked separately. Then, the images and samples were cropped into 256×256 size one by one. The sample examples are as follows: Figure 2 As shown, Figure 2 (a) in the figure is the factory. Figure 2 (b) in the figure is a contiguous village. Figure 2 (c) is a single-family house. Figure 2 (d) in the table means others.

[0080] Step 2.2: After completing sample labeling, screen the samples to ensure a balance between positive and negative samples, so as to avoid limiting the accuracy of the network due to uneven samples.

[0081] Step 2.3: After screening, the sample set needs to be divided into a training set and a validation set according to a ratio (for example, 8:2), and the training set needs to be augmented by rotation, flipping, scaling, and other operations for model training.

[0082] Step 3: Based on the twin neural network structure, construct a cumulative difference feature guided building scene-level change detection network (DAFS-Net). The network framework is shown in the figure below: Figure 3 shown.

[0083] The building scene-level change detection network includes a building feature extraction unit, a difference feature extraction module and a cumulative difference classifier; the building feature extraction unit is used to extract dual-phase image features; the difference feature extraction module is used to convert the dual-phase image features into image difference features and image intensity features, and use the image difference features as the features to be optimized and the image intensity features as the supplementary features to obtain difference features; the cumulative difference classifier is used to classify "change" or "unchanged" according to the difference features.

[0084] Specifically, step 3 includes the following sub-steps:

[0085] Step 3.1: Construct a building feature extraction part (which can be recorded as a building feature extraction unit) composed of depthwise separable convolution, dilated convolution, channel attention, pooling operation, etc. according to the residual structure. The network structure of the building feature extraction part is shown in Table 1. This part adopts a twin neural network structure with weight sharing. The lightweight coding block is as follows Figure 4 As shown, it is used to extract building features from dual-temporal images (which can be recorded as dual-temporal image features).

[0086] Table 1 Basic structure of building feature extraction

[0087]

[0088] Step 3.2: Construct a difference feature extraction module, such as Figure 5 As shown. This module uses image differential features as the features to be optimized and image intensity features as the supplementary features. First, the dual-branch image features (i.e., dual-phase image features) of the same layer and the differential features extracted from the previous layer are used as input. The dual-branch features are converted into image differential features and image intensity features by matrix subtraction, taking absolute values, and matrix addition. If there are differential features of the previous layer, they are cascaded and fused with the image differential features, and 1×1 deep convolution and 1×1 convolution operations are input to learn standardized feature information and obtain comprehensive differential features. The process can be expressed as:

[0089] (1)

[0090] In the formula, represents the weights learned by 1×1 depth convolution and 1×1 convolution kernel, Represents the bias of 1×1 convolution kernel learning; It represents batch normalization. Here, it is specially indicated for its role of information standardization. The batch normalization in other operations is omitted. represents the Relu activation function, represents the difference operation, Indicates cascade operation, , and Represent the dual-branch image features and the front-layer difference features, that is, Indicates the image features of the previous phase of the same layer, Indicates the characteristics of the later phase images of the same layer, represents the difference feature of the previous layer, Represents the comprehensive differential feature.

[0091] The comprehensive differential features are input into the spatial attention mechanism composed of 1×1 convolution and Sigmoid activation function, and the image intensity features are added as constraints. The filtered intensity features are also input into 1×1 deep convolution and 1×1 convolution operations to optimize the channel dimension features, further learn feature information, and extract comprehensive intensity features. The process can be expressed as:

[0092] (2)

[0093] In the formula, and Respectively represent the weights of 1×1 depth convolution and 1×1 convolution kernel learning, represents the bias of 1×1 convolution kernel learning, represents the Relu activation function, represents the spatial attention mechanism, , represents the dual-branch image feature, represents the comprehensive differential feature, Represents comprehensive strength characteristics.

[0094] Subsequently, the comprehensive differential features and the comprehensive intensity features will be cascaded and fused and input into the improved channel attention mechanism, and split into optimized differential and intensity features according to the original channel, and then the two will be added and fused to obtain the comprehensive features, which are recorded as the first comprehensive features. Finally, in order to ensure the effectiveness of the features, the first comprehensive feature will be input into two consecutive 3×3 convolution operations to fully learn the difference information, and connected with the comprehensive differential feature residual to obtain the difference feature. The process can be expressed as:

[0095] (3)

[0096] In the formula, represents the weights learned by two consecutive 3×3 convolution operations and a 1×1 convolution kernel, represents the bias of two consecutive 3×3 convolution operations and 1×1 convolution kernel learning, Indicates cascade operation, and Represent the comprehensive difference feature and comprehensive intensity feature respectively, and They represent the optimized comprehensive differential features and the optimized comprehensive intensity features respectively. Indicates differential characteristics. represents the improved channel attention mechanism, and its weight acquisition process can be expressed as:

[0097] (4)

[0098] In the formula, and Represent global average and maximum pooling respectively, and Represent the weights of the two branch multi-layer perceptron learning, and Respectively represent the bias of the two branch multi-layer perceptron learning, represents the Sigmoid activation function, represents the input features, represents the channel attention weight.

[0099] That is, the improved channel attention mechanism is a dual-branch structure, the first branch is used to focus on the global maximum feature, and the second branch is used to focus on the global average feature. The first branch and the second branch respectively learn the discriminative information and intensity information related to the task in the features through corresponding adaptive pooling and multi-layer perceptrons, and finally add and fusion them, and input the Sigmoid activation function to obtain the channel attention weight.

[0100] Step 3.3: Construct a cumulative difference classifier, such as Figure 6 First, we obtain the difference features of buildings at different levels, and input them into the global feature fusion device formed by different types of adaptive pooling and other residual connections to extract the multi-level change features of the buildings. The process can be expressed as:

[0101] (5)

[0102] In the formula, and Respectively represent the weights learned by two 1×1 convolution kernels, and Respectively represent the bias of two 1×1 convolution kernel learning, and Represent global average and maximum pooling respectively, represents the Relu activation function, represents the difference feature of layer l, Indicates the changing characteristics of layer l; represents the bias of 1×1 point-wise convolution learning, [ ] represents the cascade fusion features of the change features of each layer, Indicates multi-level change characteristics.

[0103] Secondly, the difference features of the highest-level buildings are input into the multi-scale feature fusion module to extract the multi-scale variation features of the buildings, such as Figure 7 As shown in the figure, this fusion module is improved based on Atrous Spatial Pyramid Pooling (ASPP), and global maximum pooling is introduced in the extraction of global features to obtain discriminative information; in order to learn and weigh long feature dependencies and fully mine contextual information, long-distance features are extracted by an improved multi-head attention mechanism and a feedforward network.

[0104] The improved multi-head attention mechanism first divides the input features into 4 sub-features according to the spatial range, and then learns the Q matrix through a 1×1 convolution operation after downsampling; the information of adjacent positions is usually highly correlated with the current position information, so the V matrix is ​​extracted through a 1×1 convolution operation and a 3×3 deep convolution to enrich the information; considering the corresponding relationship between the K matrix and the V matrix, the K matrix is ​​obtained by inputting the V matrix into a 1×1 convolution operation. The process can be expressed as:

[0105] (6)

[0106] In the formula, Respectively represent the weights of the corresponding 1×1 convolution kernel learning, They represent the corresponding 1×1 convolution kernel learning bias, and Respectively represent the weights and biases of 3×3 deep convolution learning, represents the sub-feature of the input, , , Denotes Q, V, K matrices.

[0107] Secondly, the correlation matrix is ​​calculated using the Q and K matrices. After the position encoding is introduced, it is input into a 1×1 deep convolution to weigh the importance of each subspace, and then input into the Softmax activation function to obtain the multi-head attention weight. The process can be expressed as:

[0108] (7)

[0109] In the formula, and represents the weights and biases of 1×1 deep convolution learning, represents the Softmax activation function, represents the dimension of the attention mechanism, represents the position code, represents the Q matrix, represents the transpose of the K matrix, Represents the multi-head attention weight.

[0110] Finally, the spatial attention weight mask is assigned to the V matrix, and after upsampling, it is spliced ​​according to the original spatial range, and then the nonlinear features are introduced through the Relu activation function, and the 1×1 convolution operation is input for channel conversion, and the residual connection is performed with the original features. The process can be expressed as:

[0111] (8)

[0112] In the formula, represents the multi-head attention weight, represents bilinear interpolation upsampling, Indicates spatial restoration of the four sub-features. Respectively represent the weights learned by two 1×1 convolution kernels, Respectively represent the bias of two 1×1 convolution kernel learning, represents the Relu activation function, represents the V matrix, and Represent the input features and the optimized input features respectively, Represents the output features optimized by the improved multi-head attention mechanism.

[0113] Since the multi-head attention mechanism is mainly based on matrix operations and lacks nonlinear operations, in order to fully learn long-distance features, it is also necessary to input the features into the feedforward network for nonlinear transformation to enrich the feature information. The improved feedforward network adopts an inverted residual structure, uses a 1×1 convolution operation to increase the channel dimension, allows the middle 3×3 deep convolution to add more detailed information, and then inputs a 1×1 convolution operation to reduce the channel dimension and extract long-distance features.

[0114] In the final classification stage, multi-level and multi-scale change features are cascaded and fused and input into a 1×1 deep convolution to weigh the importance of the two. Two consecutive 1×1 convolution operations are then input to first expand the channel dimension to twice the original dimension and then restore it to fully integrate and learn the change features. Finally, a 1×1 convolution kernel is input to reduce the channel dimension to two dimensions of "change" and "invariant" to complete the prediction of the change.

[0115] Step 4: Based on the encoding-decoding structure, a multi-level feature-guided airport clear zone building pixel-level extraction network (MLFR-Net) is constructed. The network framework is shown in the figure below: Figure 8 shown.

[0116] The building pixel-level extraction network comprises a lightweight coding unit, a residual pyramid pooling module and a multi-level feature fusion module; the lightweight coding unit is used to extract building features, and the building features include detail features of the building; the residual pyramid pooling module is used to divide the building features into local features and global features, divide the local features into short-distance features and long-distance features, and fuse the global features, the short-distance features and the long-distance features through cascade operations and convolution operations to extract context features; the multi-level feature fusion module is used to use the semantic features in the decoding process as features to be optimized, and the detail features and the context features as supplementary features to obtain semantic features that take both details and context into consideration; and is used to extract building boundary range information based on the deepest semantic features that take both details and context into consideration to obtain building patches.

[0117] Specifically, step 4 includes the following sub-steps:

[0118] Step 4.1: Construct a lightweight coding part (which can be recorded as a lightweight coding unit) composed of depth-separable convolution, channel attention, etc. according to the inverted residual structure. The network structure of the lightweight coding part is shown in Table 2, where the lightweight coding block is as follows Fig. 9 As shown, it is used to extract building features, where the building features include detailed features of the building.

[0119] Table 2 Basic structure of lightweight coding part

[0120]

[0121] Step 4.2: Construct the residual pyramid pooling module, such as Fig.10 As shown. This module divides the features to be fused into local features and global features, where local features are divided into short-distance features and long-distance features, a total of 3 features to be fused. First, the short-distance features are refined and extracted through a 1×1 convolution operation, and the channel is compressed to 1 / 2 of the original feature. The process can be expressed as:

[0122] (9)

[0123] In the formula, and Respectively represent the weight and bias of 1×1 convolution kernel learning, represents the Relu activation function, and Represent the input features and short-distance features respectively.

[0124] Secondly, the short-distance features are input into the dilated convolution with different dilation rates connected by the residual structure to extract the long-distance features. The process can be expressed as:

[0125] (10)

[0126] In the formula, and They represent the weights and biases learned by the 3×3 convolution kernel with a dilation rate of r, represents the Relu activation function, and Respectively represent the features of the previous layer and the features extracted by the convolution operation, and the final long-distance features That is, it is the extraction result of a 3×3 convolution operation with a dilation rate of n, where n is the maximum dilation rate.

[0127] For example, the expansion rate can be set to 2, 3, 7, and 15 respectively, and the final long-distance features This is the extraction result of a 3×3 convolution operation with a dilation rate of 15.

[0128] In the process of global feature extraction, adaptive average pooling is used to compress the spatial dimension to 1×1, and then the channel is compressed to 1 / 2 of the original feature through 1×1 convolution operation, and restored to the original size through bilinear interpolation. The process can be expressed as:

[0129] (11)

[0130] In the formula, represents global average pooling, and Respectively represent the weight and bias of 1×1 convolution kernel learning, represents the Relu activation function, represents bilinear interpolation upsampling, and Represent the input features and global features respectively.

[0131] Finally, the features will be fused through a cascade operation and input into a 3×3 convolution to extract contextual features.

[0132] Step 4.3: Construct a multi-level feature fusion module, such as Fig.11As shown. This module uses the semantic features in the decoding process as the features to be optimized, and the detail features in the encoding process and the extracted context features as the supplementary features. First, the semantic features are enlarged to twice the original size through bilinear interpolation, and the size is kept consistent with the detail features and context features, and then merged in the latter stage; considering the differences between the three features, direct input convolution may introduce noise, so first input 1×1 depth convolution, and perform batch normalization to standardize the feature information while assigning weights to each channel; then perform two consecutive 3×3 convolution operations to learn effective comprehensive features, which are recorded as the second comprehensive features. The process can be expressed as:

[0133] (12)

[0134] In the formula, represents the weight of 1×1 deep convolution learning, and Respectively represent the weights learned by two 3×3 convolution kernels, and Respectively represent the bias of two 3×3 convolution kernel learning, It represents batch normalization. Here, it is specially indicated for its role of information standardization. The batch normalization in other operations is omitted. represents the Relu activation function, Represents the features formed by the cascade fusion of semantic features, detail features and context features, where Represents semantic features, Represents detailed features, Represents contextual features; Represents the second comprehensive feature.

[0135] Secondly, the semantic features are subjected to 1×1 convolution operation to obtain semantic features after semantic alignment to improve the semantic gap between them and the second comprehensive features. Then, the channel attention mechanism is input and the second comprehensive features are selected based on this condition. Finally, the second comprehensive features and the semantic features after semantic alignment are fused through addition operation to obtain semantic features that take into account both details and context. The process is shown in formula (13). The deepest improved semantic features (i.e., semantic features that take into account both details and context) are input into the classifier composed of 1×1 convolution to complete the extraction of building boundary range information.

[0136] (13)

[0137] In the formula, and Respectively represent the weight and bias of 1×1 convolution kernel learning, represents the Relu activation function, represents the channel attention mechanism, the same as formula (4), and Respectively represent the semantic features after semantic alignment and the original semantic features, represents the second comprehensive feature, Represents semantic features that take into account both details and context.

[0138] It should be noted that the order of step 3 and step 4 can be adjusted, that is, the order of building the above two networks can be adjusted, or the above two networks can be built at the same time.

[0139] Step 5: Network training.

[0140] The building scene-level change detection network is trained using the building scene-level change detection samples to obtain a trained building scene-level change detection network; and the building pixel-level extraction network is trained using the building pixel-level samples to obtain a trained building pixel-level extraction network.

[0141] That is, the scene-level change detection samples of buildings in the airport clear area are input in batches into the network built in step 3, and the pixel-level samples of buildings in the airport clear area are input in batches into the network built in step 4. Through multiple rounds of iterative training, the network parameters are continuously optimized to achieve network convergence.

[0142] Specifically, step 5 includes the following two parts:

[0143] (5.1) Take N samples as a batch, input the samples into the network for training, use the loss function to calculate the error between the network prediction result and the true value and perform reverse derivation. The calculation formula of the cross-entropy loss function is as follows:

[0144] (14)

[0145] In the formula, is the sample size, is the number of categories, and are the predicted value and the true value respectively.

[0146] For example, 32 samples of scene-level change detection of airport clear zone buildings are batched and input into the network built in step 3; 16 samples of pixel-level samples of airport clear zone buildings are batched and input into the network built in step 4; the error between the network prediction result and the true value is calculated using the loss function and reverse derivation is performed.

[0147] (5.2) Update the weights and bias parameters of each layer through the optimizer.

[0148] For example, the Adam optimizer is used to update the weights and bias parameters of each layer. The parameter update formula is as follows:

[0149] (15)

[0150] In the formula, is the learning rate, and are the hyperparameters controlling the first-order momentum and the second-order momentum, is the first-order momentum of the previous update, is the second-order momentum of the previous update, is the second-order momentum of this update, , are the parameters before and after the update.

[0151] Step 6: Network prediction.

[0152] The preprocessed dual-phase image is detected using the trained building scene-level change detection network to determine the regional changes; if it is determined to be an unchanged area, the building patches in the unchanged area are retained; if it is determined to be a changed area, the post-phase image of the changed area is processed using the trained building pixel-level extraction network to extract the building patches in the changed area.

[0153] That is, the trained scene-level change detection network is used to judge the changes in regional buildings, and the trained pixel-level building extraction network is used to extract the later-phase buildings in the changed area.

[0154] The specific method of step 6 is as follows: the dual-phase image pair of the airport clearance area preprocessed in step 1 is input into the trained building scene-level change detection network to determine the regional change situation; the post-phase image of the changed area is input into the building pixel-level extraction network to extract the specific boundary range of the building, and the building patch of the changed area is obtained.

[0155] Afterwards, the present invention uses GIS to update the building spots in the changed area, merges the updated building spots in the changed area with the building spots in the unchanged area, and completes the overall automatic update of the spots. The following is an expanded description of steps 7 and 8.

[0156] Step 7: Use GIS to update the building map of the changed area.

[0157] Based on the spatial and temporal relationship of buildings in the airport clear area, and considering the impact of image data and network accuracy, a GIS analysis plan for building change areas is formulated. The analysis process is as follows: Fig.12 shown.

[0158] Specifically, step 7 includes the following sub-steps:

[0159] Step 7.1: To address the spurious changes caused by the displacement of high-rise buildings due to different satellite azimuth angles, the following GIS analysis scheme is proposed.

[0160] First, the geometric areas of the building patches in the previous and next phases are calculated respectively, including: calculating the geometric area of ​​the building patches in the previous phase, recorded as the first area ; Calculate the geometric area of ​​the building patch in the later phase, recorded as the second area .

[0161] Secondly, the joint analysis of the previous and next phase spots and retaining their attributes to obtain the joint spots , and then calculate the joint patch The area of Calculate the area ratio by combining the area of ​​the previous and next phase spots, including: and the second area Calculate the first area ratio , according to the area of ​​the joint patch and the second area Calculate the second area ratio ;

[0162] Finally, a first selection condition is set, wherein the first selection condition includes the first area ratio and the second area ratio Numerical conditions that need to be met; combined with joint patches , through attribute selection, we get the joint patches that meet the first selection condition That is, through attribute selection, the joint patches that meet the conditions are found for subsequent screening of the previous phase patches.

[0163] For example, combined with parameter design, the attribute selection process can be expressed as:

[0164] (16)

[0165] In the formula, and They represent the areas of the image patches in the preceding and following phases, which are recorded as the first area and the second area respectively; Represents the area of ​​the joint patch, and the units of the above areas are all m²; Represents the first area ratio, which is used to evaluate the similarity between the front and back patches; It represents the second area ratio, which is used to evaluate the degree of deviation of the previous and next spots, so as to avoid excessive size difference or excessive deviation of the previous and next phase spots. Excessive deviation is regarded as a changing spot and needs to be updated. Indicates an ArcPy attribute selection method with selection type "NEW_SELECTION", Represents the attribute relationship. For the joint patch, To select the result.

[0166] Step 7.2: To address the problem that the original image includes multiple extracted images, the following GIS analysis solution is proposed.

[0167] First, the area ratio of the joint pattern to the previous phase pattern is calculated. and the first area Calculate the third area ratio .

[0168] Secondly, select the joint spots where Area1 and Area2 are both greater than 0 to obtain the intersection of the front and back spots. and the second area All need to be greater than 0 as a screening condition for the joint pattern Screening is performed to obtain the filtered joint patch .

[0169] The unique identifier of the previous phase image is used as the grouping field, and the statistics ; The unique identifier of the subsequent phase map is used as the grouping field, and Rate2 is counted; the two statistical attributes are connected to the joint map. Finally, the joint map is supplemented by attribute selection, and then the previous phase map to be retained is selected by location selection. That is, the unique identifier field of the previous phase building map As a grouping field, the third area ratio As the field to be counted, combined with the filtered joint spots Perform summary statistics to obtain the first summary statistics value ; Unique identification field for building patches in the future phase As a grouping field, the second area ratio As the field to be counted, combined with the filtered joint spots Perform summary statistics to obtain the second summary statistics value ; Set a second selection condition, the second selection condition includes the first summary statistical value and the second summary statistic The numerical conditions that need to be met; combined with the joint patches that meet the first selection condition , through attribute selection, we get the joint patches that meet the second selection condition ; According to the previous phase pattern and the joint patches that meet the second selection criteria , through position selection, filter out the retained previous phase spots .

[0170] For example, combined with parameter design, the main steps in the process can be expressed as:

[0171] (17)

[0172] In the formula, and Respectively represent the area of ​​the previous phase patch and the joint patch; Represents the third ratio. Represents the ArcPy Summary Statistics tool, is the joint patch after screening, that is, the intersection of the front and back patches. and is the field to be counted. and The unique identification fields representing the previous and next phase spots are used as grouping fields. Recorded as the first identification field, Recorded as the second identification field; and To summarize the statistics for the intersection, connect to the attribute table of the joint patch and use Recorded as the first summary statistic, Recorded as the second summary statistic.

[0173] (18)

[0174] In the formula, Indicates the ArcPy attribute selection method with selection type "ADD_TO_SELECTION", is the joint patch selected by formula (16), To add the result after selection; Indicates attribute relationship, in parallel conditions It is used to determine whether a single pre-phase image spot contains multiple post-phase image spots. It is used to judge whether most of the later phase spots are in the previous phase spots. The two conditions can be combined to judge whether the original spots include multiple extracted spots. If they are met, it means that the previous phase spots remain unchanged. Indicates an ArcPy attribute selection method with selection type "NEW_SELECTION", Indicates that the input needs to satisfy the intersection relationship, For the previous phase, This is the preserved previous phase pattern.

[0175] Step 7.3: To solve the problem that the original map is expanded, resulting in the extracted map including multiple original map patches and the extracted map overflowing the original map, the following GIS analysis scheme is proposed.

[0176] First, the phase spots after joint analysis The preserved previous phase pattern , get the new joint patch .

[0177] Secondly, the area and area ratio of the patch are calculated; that is, the new joint patch is calculated The area of ; According to the area of ​​the new joint pattern and the second area The fourth area ratio is calculated .

[0178] Finally, the new joint image patch that meets the conditions is selected through the attributes to complete the image patch update of the changed area. That is, the third selection condition is set, and the third selection condition includes the fourth area ratio The first area and the area of ​​the new joint patch Numerical conditions that need to be met; combined with the new joint pattern , through attribute selection, a new joint patch that meets the third selection condition is obtained , complete the patch update of the changed area.

[0179] For example, combined with parameter design, the main steps in the process can be expressed as:

[0180] (19)

[0181] In the formula, Represents ArcPy's joint analysis method, which combines the phase spots The preserved previous phase pattern , get the new joint patch . and Respectively represent the area of ​​the before and after phases, Represents the area of ​​the new joint patch. Indicates the ArcPy attribute selection method with selection type "SWITCH_SELECTION", Indicates attribute relationship, in parallel conditions It is used to extract the later phase spots outside the retained earlier phase spots, and set the area threshold condition to determine whether the exceeding part needs to be retained, so as to solve the problem that the extracted spots include multiple original spots and the original spots are expanded.

[0182] Step 8: Update the image.

[0183] The building patches before and after the changed area will be input into the GIS analysis process formulated in step 7 to update the building patches in the changed area, and then complete the overall patch automatic update.

[0184] The specific practice of step 8 is as follows: the building spots in the unchanged area in step 6 will be directly retained, and the building spots in the changed area will be superimposed with the original distribution spots of the buildings, and the GIS analysis process formulated in step 7 will be input to update the building spots in the changed area, and then merged with the spots in the unchanged area to complete the overall automatic update of the spots.

[0185] Among them, the experimental area image and the corresponding update result are shown in the figure Fig.13 The combination of step 6 and step 8 is the complete process of automatic updating of airport clear area building spots. The flowchart of the automatic updating framework of airport clear area building spots is shown in Fig.14 shown.

[0186] Embodiment 2:

[0187] Embodiment 2 provides an airport clear area building spot automatic updating system, including:

[0188] A data acquisition unit, used for acquiring dual-phase images of the airport clear area and performing pre-processing;

[0189] A sample making unit, used for making a sample set, including making a number of building scene-level change detection samples and a number of building pixel-level samples;

[0190] A network construction unit, used to construct a building scene-level change detection network guided by cumulative difference features based on a twin neural network structure, and used to construct a building pixel-level extraction network guided by multi-level features based on an encoding-decoding structure;

[0191] A network training unit is used to train the building scene-level change detection network using the building scene-level change detection samples to obtain a trained building scene-level change detection network; and to train the building pixel-level extraction network using the building pixel-level samples to obtain a trained building pixel-level extraction network;

[0192] A patch updating unit is used to detect the preprocessed dual-phase image using the trained building scene-level change detection network to determine the regional change; if it is determined to be an unchanged area, the building patches in the unchanged area are retained; if it is determined to be a changed area, the post-phase image of the changed area is processed using the trained building pixel-level extraction network to extract the building patches in the changed area; the building patches in the changed area are updated using GIS, and the updated building patches in the changed area are merged with the building patches in the unchanged area to complete the overall automatic patch update;

[0193] The airport clear zone building map automatic updating system is used to execute the steps in the airport clear zone building map automatic updating method as described in Example 1.

[0194] Since the functions of each unit in the system provided in Example 2 correspond to the steps in the method provided in Example 1, Example 2 can be understood by referring to Example 1, and will not be repeated here.

[0195] Finally, it should be noted that the above specific implementation methods are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to examples, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention, which should be included in the scope of the claims of the present invention.

Claims

1. A method for automatically updating building patches in an airport clear area, characterized in that: The following steps are involved: Acquire dual-temporal images of the airport clear area and perform pre-processing; Producing a sample set, including producing a number of building scene-level change detection samples and a number of building pixel-level samples; Based on the twin neural network structure, a building scene-level change detection network guided by cumulative difference features is constructed; based on the encoding-decoding structure, a building pixel-level extraction network guided by multi-level features is constructed; The building scene-level change detection network includes a building feature extraction unit, a difference feature extraction module and a cumulative difference classifier; the building feature extraction unit is used to extract dual-temporal image features; The difference feature extraction module is used to convert the dual-phase image features into image difference features and image intensity features, using the image difference features as the features to be optimized and the image intensity features as the supplementary features to obtain difference features; the cumulative difference classifier is used to classify "changed" or "unchanged" according to the difference features; The difference feature extraction module takes the dual-phase image features of the same layer and the difference features extracted from the previous layer as input, and converts the dual-phase image features into image difference features and image intensity features by matrix subtraction and absolute value taking and matrix addition; If there is a difference feature in the previous layer, it is cascaded and fused with the image difference feature, and input into the 1×1 depth convolution and 1×1 convolution operation to obtain the comprehensive difference feature; The comprehensive differential feature is input into the spatial attention mechanism composed of 1×1 convolution and Sigmoid activation function, and the image intensity feature is added based on this as a constraint. The filtered intensity feature is input into 1×1 depth convolution and 1×1 convolution operations to obtain a comprehensive intensity feature; the comprehensive differential feature and the comprehensive intensity feature are cascaded and fused and input into the channel attention mechanism, and split into the optimized comprehensive differential feature and the optimized comprehensive intensity feature according to the original channel, and then the two are added and fused to obtain the first comprehensive feature; Input the first comprehensive feature into two consecutive 3×3 convolution operations and connect it with the comprehensive differential feature residual to obtain the differential feature of this layer; The cumulative difference classifier inputs the difference features of different layers into the global feature fusion module to obtain the multi-level change features of the building; the difference features of the highest layer are input into the multi-scale feature fusion module to obtain the multi-scale change features of the building; Cascading and fusing the multi-level variation features and the multi-scale variation features and performing a convolution operation to achieve classification; Using the building scene level change detection samples to train the building scene level change detection network to obtain a trained building scene level change detection network; Using the building pixel-level samples to train the building pixel-level extraction network to obtain a trained building pixel-level extraction network; The trained building scene-level change detection network is used to detect the pre-processed dual-phase images to determine the regional changes; if it is determined to be an unchanged area, the building patches in the unchanged area are retained; if it is determined to be a changed area, the trained building pixel-level extraction network is used to process the post-phase images of the changed area to extract the building patches in the changed area; GIS is used to update the building patches in the changed area, and the updated building patches in the changed area are merged with the building patches in the unchanged area to complete the overall automatic patch update.

2. The method for automatically updating the airport clear area building pattern according to claim 1, characterized in that: The production of the building scene-level change detection sample includes: cropping and labeling the dual-phase image; when labeling, labeling the image pair as "changed" or "unchanged" according to whether the building has changed before and after the phase; The production of the pixel-level sample of the building includes: marking and cropping the dual-temporal image; when marking, marking clusters of buildings in a continuous manner and marking single-building buildings individually.

3. The method for automatically updating the airport clear area building pattern according to claim 1, characterized in that: The comprehensive differential feature and the comprehensive intensity feature are cascaded and fused and input into an improved channel attention mechanism. The improved channel attention mechanism has a dual-branch structure. The first branch is used to focus on the global maximum feature, and the second branch is used to focus on the global average feature. The first branch and the second branch learn the task-related discriminative information and intensity information in the features respectively through corresponding adaptive pooling and multi-layer perceptrons. Finally, they are added and fused and input into the Sigmoid activation function to obtain the channel attention weight.

4. The method for automatically updating the airport clear area building pattern according to claim 1, characterized in that: The multi-scale feature fusion module is improved based on the dilated spatial pyramid pooling, introduces global maximum pooling in the extraction of global features, and uses the improved multi-head attention mechanism and the improved feedforward network to extract long-range features; The improved multi-head attention mechanism first divides the input features into multiple sub-features according to the spatial range, and then learns the Q matrix through a 1×1 convolution operation after downsampling. The V matrix is ​​extracted by combining the 1×1 convolution operation with the 3×3 depthwise convolution. The K matrix is ​​obtained by inputting the V matrix into the 1×1 convolution operation. The correlation matrix is ​​calculated using the Q matrix and the K matrix. After introducing the position code, it is input into a 1×1 depthwise convolution and then into the Softmax activation function to obtain the multi-head attention weights. The multi-head attention weight mask is assigned to the V matrix, and after upsampling, it is spliced ​​according to the original spatial range. Then, nonlinear features are introduced through the ReLU activation function, and a 1×1 convolution operation is input for channel conversion. The residual connection is performed with the original features to obtain the output features optimized by the improved multi-head attention mechanism; The output features optimized by the improved multi-head attention mechanism are input into the improved feedforward network for nonlinear transformation to obtain long-distance features; wherein, the improved feedforward network adopts an inverted residual structure, uses a 1×1 convolution operation to perform channel dimensionality increase, uses the middle 3×3 depth convolution to supplement the detail information, and then inputs a 1×1 convolution operation for channel dimensionality reduction.

5. The method for automatically updating the airport clear area building pattern according to claim 1 is characterized in that: The building pixel-level extraction network includes a lightweight encoding unit, a residual pyramid pooling module and a multi-level feature fusion module; The lightweight coding unit is used to extract building features, where the building features include detailed features of the building; The residual pyramid pooling module is used to divide the building features into local features and global features, divide the local features into short-distance features and long-distance features, and fuse and convolve the global features, the short-distance features, and the long-distance features through a cascade operation to extract context features; The multi-level feature fusion module is used to use the semantic features in the decoding process as the features to be optimized, and the detail features and the context features as supplementary features to obtain semantic features that take into account both details and context; and is used to extract building boundary range information based on the deepest semantic features that take into account both details and context to obtain building patches.

6. The method for automatically updating the airport clear area building pattern according to claim 5, characterized in that: The residual pyramid pooling module extracts short-range features through 1×1 convolution operations and compresses the channels; the short-range features are input into the dilated convolution with different expansion rates connected by the residual structure to extract long-range features; In the process of extracting global features, adaptive average pooling is used to compress the spatial dimension to 1×1, and then the channel is compressed through a 1×1 convolution operation and restored to its original size through bilinear interpolation; The multi-level feature fusion module adjusts the size of the semantic feature to keep its size consistent with the detail feature and context feature, and then merges them in the next stage, inputs them into a 1×1 depthwise convolution, performs batch normalization, and then performs two consecutive 3×3 convolution operations to obtain the second comprehensive feature; Perform 1×1 convolution operation on the semantic features to obtain semantic features after semantic alignment; Then it is input into the channel attention mechanism, and finally the second comprehensive feature is fused with the semantic feature after the semantic alignment through the addition operation to obtain the semantic feature that takes into account both details and context; the deepest semantic feature that takes into account both details and context is input into the classifier composed of 1×1 convolution to complete the extraction of the building boundary range information.

7. The method for automatically updating the airport clear area building pattern according to claim 1, characterized in that: The updating of the building patches of the changed area by using GIS includes: using GIS to analyze and process pseudo-changes caused by displacement of high-rise buildings due to different satellite azimuth angles, using GIS to process a situation where the original patch includes multiple extracted patches, and using GIS to process a situation where the extracted patch includes multiple original patches due to expansion of the original patch, or the extracted patch overflows the original patch range; Among them, the use of GIS to analyze and process the pseudo-changes caused by the displacement of high-rise buildings due to different satellite azimuth angles includes: Calculate the geometric area of ​​the building patch in the previous phase and record it as the first area ; Calculate the geometric area of ​​the building patch in the later phase and record it as the second area ; Jointly analyze the building patches before and after the time phase and retain their attributes to obtain the joint patches ; Calculate joint patches The area of According to the first area and the second area Calculate the first area ratio , according to the area of ​​the joint patch and the second area Calculate the second area ratio ; Set a first selection condition, the first selection condition includes the first area ratio and the second area ratio Numerical conditions that need to be met; combined with joint patches , through attribute selection, obtain the joint patch that meets the first selection condition ; Among them, the use of GIS to process the situation where the original map includes multiple extracted maps includes: According to the area of ​​the joint patch and the first area Calculate the third area ratio ; With the first area and the second area All need to be greater than 0 as a screening condition for the joint patch Filter and obtain the filtered joint patch ; Unique identification field of the building patch in the previous phase As a grouping field, the third area ratio As the field to be counted, combined with the filtered joint spots Perform summary statistics to obtain the first summary statistics value ; Unique identification field for building patches in the future phase As a grouping field, the second area ratio As the field to be counted, combined with the filtered joint spots Perform summary statistics to obtain the second summary statistics value ; Set a second selection condition, the second selection condition includes the first summary statistical value and the second summary statistic Numerical conditions that need to be met; combined with the joint patches that meet the first selection conditions , through attribute selection, obtain the joint patch that meets the second selection condition ; According to the previous phase pattern and the joint patch that meets the second selection criteria , by selecting the location, filter out the retained previous phase spots ; Among them, the use of GIS to deal with the situation where the original map is expanded, resulting in the extracted map including multiple original map spots, and the extracted map overflows the original map range includes: Phase pattern after joint analysis With the retained previous phase pattern , get the new joint patch ; Calculate the new joint patch The area of According to the area of ​​the new joint patch and the second area Calculate the fourth area ratio ; Set a third selection condition, the third selection condition includes the fourth area ratio 、The first area and the area of ​​the new joint patch Numerical conditions that need to be met; combined with the new joint pattern , through attribute selection, a new joint patch that meets the third selection condition is obtained , complete the patch update of the changed area.

8. An automated updating system for airport clear zone building patches, characterized in that: include: A data acquisition unit is used to acquire dual-temporal images of the airport clear area and perform pre-processing; A sample production unit, used to produce a sample set, including producing a number of building scene-level change detection samples and a number of building pixel-level samples; A network construction unit is used to construct a building scene-level change detection network guided by cumulative difference features based on a twin neural network structure, and to construct a building pixel-level extraction network guided by multi-level features based on an encoding-decoding structure; a network training unit, configured to train the building scene level change detection network using the building scene level change detection samples to obtain a trained building scene level change detection network; Using the building pixel-level samples to train the building pixel-level extraction network to obtain a trained building pixel-level extraction network; A patch updating unit is configured to detect the pre-processed dual-temporal image using the trained building scene-level change detection network to determine regional changes; if the area is determined to be unchanged, the building patches of the unchanged area are retained; if the area is determined to be a changed area, the post-temporal image of the changed area is processed using the trained building pixel-level extraction network to extract the building patches of the changed area; the building patches of the changed area are updated using GIS, and the updated building patches of the changed area are merged with the building patches of the unchanged area to complete the overall automatic patch update; The airport clear zone building pattern automatic updating system is used to execute the steps in the airport clear zone building pattern automatic updating method according to any one of claims 1 to 7.

Citation Information

Patent Citations

  • Method for semi-automatically extracting buildings with different shapes from high-resolution remote sensing image

    CN111325684A

  • System and method of feature detection in satellite images using neural networks

    WO2021205424A2