Remote sensing image building change detection method based on ResNet34

Abstract

The invention relates to a remote sensing image building change detection method based on ResNet34, and belongs to the technical field of remote sensing image information change detection. The method comprises the following steps: carrying out preprocessing and data enhancement on a training data set; inputting the two images into a ResNet34 encoder to obtain feature maps of different scales of each stage of the two images; inputting the images before and after change into a GFSN module, and extracting building related texture features in multiple scales and directions; features extracted from the GFSN module are fused with a layered encoder, and the fused features are integrated into a Siamese network composed of four pairs of multi-stage down-sampling-transformation blocks; the feature weights of the fused multi-layer features are redistributed through a ratio attention module, and meanwhile an image difference value is obtained through a difference module; and fusing up-sampling operation layer by layer through a decoder to obtain a final prediction map. According to the method, the problems of building boundary fuzziness in building transformation detection and cavities in a detection result are solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of remote sensing image information change detection, and relates to a remote sensing image building change detection method based on ResNet34. Background Art

[0002] The purpose of change detection is to detect changes in remote sensing images at different stages in the same area, and to achieve change detection of remote sensing images by means of binarization. Remote sensing image change detection can be divided into land change detection, building change detection, water body change detection, etc. Remote sensing images at different phases of the same area are affected by factors such as lighting, contrast, and imaging angle. In addition, the complex scene relationships and background complexity of the remote sensing images themselves lead to limited detection accuracy of building changes.

[0003] Since the early days of remote sensing datasets, pixel-based data collection methods have been widely studied. These methods usually use pixels as the basic unit of image processing and are widely used in low- and medium-resolution remote sensing images. However, with the continuous development of earth observation technology, large-scale high-resolution multispectral images have challenged previous traditional research methods. The main reason is that with the continuous improvement of spatial resolution, the increase in internal variability of objects and the increase in complexity of object boundaries have become the main obstacles. With the rapid development of convolutional neural networks in deep learning in recent years, these obstacles have also been greatly overcome. Image features are extracted through convolutional neural networks, and then the similarity of these features is calculated to obtain a change map. The more popular convolutional neural networks currently include end-to-end full convolutional neural networks, such as full convolutional networks, U-Net and its variants. Inspired by DenseNet and NestedNet, Siamese Nested U-Net for Change Detection (SNUNet) uses dense skip connections between the encoder and decoder to achieve high-resolution and fine-grained representation. Since then, in order to better utilize contextual information, researchers have begun to add attention mechanisms to change detection networks, such as channel attention, spatial attention, and self-attention.

[0004] In addition to the above research on network structure optimization, some scholars have also studied semi-supervised and weakly supervised change detection methods. Peng et al. designed a CD semi-supervised convolutional network based on generative adversarial networks (GANs). Some scholars have studied unsupervised change detection, such as pre-training convolutional neural networks to extract features from multi-phase images, unsupervised deep learning frameworks to extract difference representations of pairs of images from the same or different sources, or using the results of traditional methods to train deep neural networks, so that the entire change detection process does not require any human intervention.

[0005] Although the above models have made progress in detecting building changes in high-resolution remote sensing images, there are still problems that need to be solved, such as poor extraction of edge information of building changes, blurred building boundaries in the results, weak ability to retain global context information, and occasional holes in the extraction results. Summary of the invention

[0006] In view of this, the purpose of the present invention is to provide a remote sensing image building change detection method based on ResNet34 to solve the problem of blurred building boundaries in complex urban scenes and the problem of holes in the extraction results of existing building change detection methods.

[0007] In order to achieve the above object, the present invention provides the following technical solutions:

[0008] A remote sensing image building change detection method based on ResNet34, the method includes the following steps:

[0009] S1: The acquired remote sensing building dataset images are divided into training set, validation set and test set, the training set images are cropped using a sliding window method, the annotated images are processed accordingly, and then the training set images are data enhanced;

[0010] S2: Input the data-enhanced training set images in S1 into an extraction model with a residual network. The extraction model extracts features from two input remote sensing images at different times, and obtains feature maps of different scales in four stages from the first stage to the fourth stage respectively.

[0011] S3: Introduce multi-scale Gabor filters in the S2 hierarchical encoder, and introduce a pair of Gabor filters with different scales and kernels in each stage, so as to obtain 4 pairs of Gabor feature maps with different size resolutions;

[0012] S4: The feature extraction module DAM aims to combine the downsampled Gabor feature maps extracted by GFSN in S3 with the features of different layers in the ResNet encoder. Here, a pair of Gabor features are selected for mapping. Here, we connect the low-frequency Gabor features of the larger patterns in the captured image in GFSN with the ResNet features of smaller resolution in order to retain the meaningful information after the downsampling operation.

[0013] S5: The fusion features of multiple stages obtained in S4 and the feature maps of different scales in the four stages are upsampled and the ratio attention module is used to obtain a more refined feature result. The resulting feature map reduces useless information and highlights the features of the changed part. The feature maps of different layers are re-weighted to obtain feature maps with rich semantic information and spatial information at different resolutions in each stage of fusion. Finally, the feature difference of different layers of the two images is calculated to obtain the difference feature.

[0014] S6: The difference features of the first to fourth stages obtained in S5 are fused and up-sampled by the decoder to obtain a final prediction map, and the final prediction map is used as the final building extraction result.

[0015] S7: Put the predicted data set into the model trained by the training set in S6 for testing, and finally obtain the relevant evaluation indicators of the entire model.

[0016] Furthermore, in S1, the labeled image is processed, specifically, the pixels in the non-building area in the label image are set to 0, and the pixels in the building area are set to 1.

[0017] Furthermore, in S1, data enhancement specifically includes: random image resizing, random horizontal flipping, vertical flipping, random Gaussian noise, contrast-constrained adaptive histogram equalization, and random cropping.

[0018] Furthermore, the random size scaling uses a scaling ratio of 0.5-2.0, the random horizontal flipping, vertical flipping, random Gaussian noise and limited contrast adaptive histogram equalization use a probability of 0.5 to perform data enhancement on the image, and the random cropping has a cropping size of 256×256.

[0019] Furthermore, in S2, the deep residual neural network extraction model is a stage with four stages, each stage contains several residual blocks, of which the first stage contains 3, the second stage includes 4, the third stage includes 6, and the fourth stage includes 3. These residual blocks are connected to each other through jump connections, allowing the network to directly learn residual mapping.

[0020] Furthermore, in S2, the Gabor filter has the ability to mimic the receptive field of the mammalian visual cortex and is widely used in image analysis and texture feature extraction, especially to capture repetitive visual patterns in images. We believe that this ability is suitable for our building change detection task because we need to perceive the repetitive texture of building clusters in an area.

[0021] Therefore, a Gabor filter feature extraction network (GFSN) is introduced in each stage. Each stage includes a pair of filters of different scales. The input images before and after the change are convolved with multiple filters to generate a set of feature maps of different frequencies and directions. Here, the size and direction scale of each layer of filters need to be considered in order to capture the repetitive textures generated by the building. Here, the direction scale is set to 4, which can cover the horizontal, vertical and diagonal directions. For the filter size, it is experimentally found that when the size is 1, smaller buildings can be clearly captured. When the frequency is increased, the filter will respond to larger objects. When the size is greater than 6, large buildings will also be very blurred, and no meaningful texture of the building can be extracted. Therefore, the maximum filter size is set to 6.

[0022] Furthermore, in S2 and S3, different feature maps are obtained through the filter network and the ResNet encoder. Through the feature fusion module (DAM), a pair of Gabor feature maps are selected to be fused with the features of different layers of ResNet. When fusion is performed here, we choose to fuse the low-frequency features of the larger module of the captured image in Gabor with the ResNet features of smaller resolution, so as to retain the meaningful information after the downsampling operation. For example, when the filter size is 6, only larger buildings can be captured, while small buildings are ignored. At this time, it is fused with the minimum spatial resolution H / 32xW / 32 in the ResNet feature, so as to retain more complete information.

[0023] Furthermore, in S5, the fused multi-layer features are redistributed through the ratio attention module to redistribute the feature map weights, thereby reducing the attention to useless information and making the module pay more attention to useful information.

[0024] The ratio attention module performs ratio processing on the data of the two features respectively and obtains a ratio data in the range of (0, 1). It adds a large ratio to the data with a small ratio and proposes T(x) = 1-ratio. The smaller the ratio, the more important the original data is. For the same ratio, the larger the ratio, the more 1-ratio is used, so that the model pays less attention to these contents.

[0025] Perform a ratio operation on the two input images, that is, divide the two matrices and take the absolute value to obtain a feature map with a value range of 0-1; make two branches for this feature map, the left branch uses the channel attention mechanism to take the importance of different features and allocates different tasks according to the features; the right branch uses the spatial attention mechanism to process the most important part of the network; the final attention representation of this layer is;

[0026] T i =Div1 i +Ti (1)

[0027] T i =Div2 i +T i (2)

[0028] Diff i =|X i -Y i |(3)

[0029] Where: T i T represents x and y in the input image; i represents the current layer; Div1 i , Div2 i Respectively represent the attention weights obtained at the current i-th layer of the two feature maps; Diff i represents the difference between the two feature images of the i-th layer; X i , Y i They represent the feature maps obtained by formula (1) and formula (2) respectively.

[0030] Furthermore, S4 is specifically as follows: the multi-scale feature maps after fusion and refinement of the first to fourth stages output by the decoder in S3 are densely upsampled to the same size as the feature map of the first stage, and the shapes are all 64×64×256; then the four feature maps are spliced ​​in the channel dimension using a cascade operation, and the shape after splicing is 64×64×1024, and then the spliced ​​feature maps are fused using 3×3 convolution to obtain a 64×64×256 feature map; the fused result is classified using 1×1 convolution, and upsampled to the original image size, and finally a predicted image is obtained.

[0031] The beneficial effects of the present invention are:

[0032] First, the present invention proposes a high-resolution optical remote sensing image building change detection framework based on multi-scale fusion, which can perform multi-scale context feature extraction, fully capture the local and long-range dependencies between features in the image, and improve the extraction capability of large buildings.

[0033] Second, the present invention proposes to introduce the traditional Gabor filter into the convolutional network to generate a filter network, which helps the module to better capture the repetitive texture of building clusters and solves the problem of blurred boundaries that may occur in building change detection.

[0034] Third, the present invention proposes a feature fusion module (DAM) to fuse the features of the four Stage layers of the traditional residual network ResNet34 with the features of the Gabor filter network to achieve high- and low-frequency feature fusion, thereby retaining the global contextual information and solving the void problem in building change detection to a certain extent.

[0035] Fourth, the present invention proposes to combine the ratio method in the traditional method into channel attention and spatial attention to form a new attention mechanism, so that the model pays more attention to the characteristics of the changing part, and further solves the problem of blurred boundaries in building change detection compared with the second one.

[0036] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0038] Figure 1 It is a flow chart of the specific steps of the present invention;

[0039] Figure 2 It is a schematic diagram of the model structure in the present invention;

[0040] Figure 3 is the diff attention module;

[0041] Figure 4 Schematic diagram of building extraction effect. DETAILED DESCRIPTION

[0042] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0043] Among them, the drawings are only used for illustrative explanations, and they only represent schematic diagrams rather than actual pictures, and should not be understood as limitations on the present invention. In order to better illustrate the embodiments of the present invention, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0044] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if the terms "upper", "lower", "left", "right", "front", "rear", etc. indicate the orientation or position relationship, they are based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation. Therefore, the terms describing the position relationship in the drawings are only used for illustrative purposes and cannot be understood as limiting the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0045] See also Figure 1 to Figure 4 Taking the LEVIR-CD dataset building change detection as an example, the remote sensing image building change detection method based on ResNet34 is as follows:

[0046] S1: The acquired remote sensing building dataset images are cropped using a sliding window method, and the annotated images are processed. The dataset is divided into a training set, a validation set, and a test set, and data enhancement is performed on the training set images.

[0047] Specifically, the pixels in the non-building area in the label map of the WHU and LEVIR-CD datasets are set to 0, and the pixels in the building area are set to 1. The dataset is divided into three parts, with 137 images in the training set, 4 images in the validation set, and 10 images in the test set. The original images are cropped into images of size 256×256 with an overlap ratio of 0.5%.

[0048] During the training process, data enhancement is performed on the training set, including horizontal flipping, vertical flipping, random resizing, random Gaussian noise, limited contrast adaptive histogram equalization, and random cropping. Random resizing uses a scaling ratio of 0.5-2.0 by default. Horizontal flipping, vertical flipping, random Gaussian noise, and limited contrast adaptive histogram equalization use a probability of 0.5 to enhance the image data. Finally, random cropping is performed, and the default crop size is 256×256.

[0049] All methods are combined to enhance the original image.

[0050] S2: Input the training set data into the ResNet34 encoder to obtain feature maps of different scales in four stages from the first stage to the fourth stage.

[0051] S3: In the S2 hierarchical encoder, a Gabor filter feature extraction network (GFSN) is introduced in each stage. Each stage includes a pair of filters of different scales. The input image before and after the change is convolved with multiple filters to generate a set of feature maps of different frequencies and directions. Here, the size and direction scale of each layer of filters need to be considered in order to capture the repetitive textures generated by the building. Here, the direction scale is set to 4, which can cover the horizontal, vertical and diagonal directions. For the filter size, it is experimentally found that when the size is 1, smaller buildings can be clearly captured. When the frequency is increased, the filter will respond to larger objects. When the size is greater than 6, large buildings will also be very blurred, and any meaningful texture of the building cannot be extracted. Therefore, the maximum filter size is set to 6.

[0052] S4: In S2 and S3, different feature maps are obtained through the filter network and ResNet encoder. Through the feature fusion module (DAM), a pair of Gabor feature maps are selected to be fused with the features of different layers of ResNet. When fusion is performed here, we choose to fuse the low-frequency features of the larger module of the captured image in Gabor with the ResNet features of smaller resolution, so as to retain the meaningful information after the downsampling operation. For example, when the filter size is 6, only larger buildings can be captured, while small buildings are ignored. At this time, it is fused with the minimum spatial resolution H / 32x W / 32 in the ResNet feature, so as to retain more complete information.

[0053] S5: In S5, the fused multi-layer features are redistributed through the ratio attention module to redistribute the feature map weights, reduce the attention to useless information, and make the module pay more attention to useful information.

[0054] The ratio attention module mainly performs ratio processing on the data of two features respectively, and obtains a ratio data range of (0, 1). We know that the smaller the ratio, the greater the change may be, so we need to add a large proportion to the data with a small ratio, so that it is easier for the model to pay attention to this part of the content. Based on this idea, T(x) = 1-ratio is proposed, that is, the smaller the ratio, we use 1 minus the ratio to multiply the result by the original data to make it more important. The larger the ratio, the smaller the change may be. We also use 1-ratio to make the model pay less attention to these contents.

[0055] Here we perform a ratio operation on the two input images, that is, divide the two matrices and take the absolute value to obtain a feature map with a value range of 0-1. We make two branches for this feature map. The left branch mainly uses the channel attention mechanism to take the importance of different features and allocate different tasks according to the features. The right branch uses the spatial attention mechanism to process the most important part of the network. The final attention representation of this layer is obtained.

[0056] T i =Div1 i +T i (1)

[0057] T i =Div2 i +T i (2)

[0058] Diff i =|X i -Y i |(3)

[0059] Where: T i T represents x and y in the input image; i represents the current layer; Div1 i , Div2 i Respectively represent the attention weights obtained at the current i-th layer of the two feature maps; Diff i represents the difference between the two feature images of the i-th layer; X i , Y i They represent the feature maps obtained by formula (1) and formula (2) respectively;

[0060] S6: The first to fourth stages of the decoder output in S5 are processed and the results after the feature maps of each stage are fused to obtain a final prediction map, and the final prediction map is used as the final building extraction result;

[0061] Specifically, the multi-scale feature maps of the first to fourth stages output by the decoder in S5 are densely upsampled to the same size as the feature map of the first stage, and the shape is 64×64×256;

[0062] Then, the four feature maps are concatenated in the channel dimension using a cascade operation. The concatenated shape is 64×64×1024. Then, a 3×3 convolution is used to fuse the concatenated feature maps to obtain a 64×64×256 feature map.

[0063] The fused result is classified using a 1×1 convolution and upsampled to the original image size to finally obtain the predicted image.

[0064] S7: Input the test data image into the trained model for testing, and select IOU, OA, F1-Score, Precision, and recall as evaluation indicators.

[0065] Specifically: the evaluation index OA is the global accuracy, and the equation is: (TP+TN) / (TP+TN+FP+FN);

[0066] The evaluation index Precision is the precision rate, and the equation is: (TP) / (TP+FP);

[0067] The evaluation index Recall is the recall rate, and the equation is: (TP) / (TP+FN);

[0068] The evaluation index F1-Score is the harmonic mean of precision and recall, and the equation is: (2×Presicion×Recall) / (Precision+Recall);

[0069] The evaluation index Iou is the intersection-over-union ratio, and the equation is: (TP) / (TP+FP+FN);

[0070] Wherein, TP, TN, FP, and FN in the equation are respectively expressed as:

[0071] TP is the number of samples correctly predicted as positive;

[0072] TN is the number of samples correctly predicted as negative;

[0073] FP is the number of samples that are incorrectly predicted as positive;

[0074] FN is the number of samples that are incorrectly predicted as negative.

[0075] Use the test set data for testing and select a picture to export the final building extraction results, such as Figure 4 shown.

[0076] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A remote sensing image building change detection method based on ResNet34, characterized by: The method comprises the following steps: S1: The acquired remote sensing building dataset images are divided into training set, validation set and test set, the training set images are cropped using a sliding window method, the annotated images are processed accordingly, and then the training set images are data enhanced; S2: Input the data-enhanced training set images in S1 into an extraction model with a residual network. The extraction model extracts features from two input remote sensing images at different times, and obtains feature maps of different scales in four stages from the first stage to the fourth stage respectively. S3: Introduce multi-scale Gabor filters in the S2 hierarchical encoder, introduce a pair of Gabor filters with different scales and kernels in each stage, and obtain 4 pairs of Gabor feature maps with different size resolutions; S4: The feature extraction module DAM combines the downsampled Gabor feature maps extracted by GFSN in S3 with the features of different layers in the ResNet encoder, selects a pair of Gabor features for mapping, and connects the low-frequency Gabor features of the larger pattern in the captured image in GFSN with the ResNet features of smaller resolution in order to retain the meaningful information after the downsampling operation; S5: The fusion features of multiple stages obtained in S4 and the feature maps of different scales in the four stages are upsampled and the ratio attention module is used to obtain a more refined feature result. The resulting feature map reduces useless information and highlights the features of the changed part. The feature maps of different layers are re-weighted to obtain feature maps with rich semantic information and spatial information at different resolutions in each stage of fusion. Finally, the feature difference of different layers of the two images is calculated to obtain the difference feature; S6: The difference features from the first stage to the fourth stage obtained in S5 are fused and up-sampled by the decoder to obtain a final prediction map, and the final prediction map is used as the final building extraction result; S7: Put the predicted data set into the model trained by the training set in S6 for testing, and finally obtain the relevant evaluation indicators of the entire model.

2. The remote sensing image building change detection method based on ResNet34 according to claim 1 is characterized in that: In S1, the labeled image is processed, specifically: the pixels in the non-building area in the label image are set to 0, and the pixels in the building area are set to 1.

3. The remote sensing image building change detection method based on ResNet34 according to claim 1 is characterized in that: In S1, data enhancement specifically includes: performing random resizing, random horizontal flipping, vertical flipping, random Gaussian noise, contrast-limited adaptive histogram equalization, and random cropping on the image; The random size scaling uses a scaling ratio of 0.5-2.0, the random horizontal flipping, vertical flipping, random Gaussian noise and limited contrast adaptive histogram equalization use a probability of 0.5 to perform data enhancement on the image, and the random cropping has a cropping size of 256×256.

4. The method for detecting building changes in remote sensing images based on ResNet34 according to claim 3 is characterized in that: In S2, the deep residual neural network extraction model is a stage with four stages, each stage contains several residual blocks, of which the first stage contains 3, the second stage includes 4, the third stage includes 6, and the fourth stage includes 3. These residual blocks are connected to each other through jump connections, allowing the network to directly learn residual mapping.

5. The method for detecting building changes in remote sensing images based on ResNet34 according to claim 4 is characterized in that: In S2, a Gabor filter feature extraction network GFSN is introduced in each stage, each stage includes a pair of filters of different scales, the input images before and after the change are convolved with multiple filters to generate a set of feature maps of different frequencies and directions, taking into account the size and direction scale of each layer of filters to capture the repetitive texture generated by the building; the direction scale is set to 4, which can cover the horizontal, vertical and diagonal directions; the maximum filter size is set to 6.

6. The remote sensing image building change detection method based on ResNet34 according to claim 5, characterized in that: In S2 and S3, the high-frequency and low-frequency features of the filter network and the encoder are fused through a feature fusion module to enhance contextual semantic information and solve the problem of internal voids in buildings; Different feature maps are obtained through the filter network and ResNet encoder. Through the feature fusion module DAM, a pair of Gabor feature maps are selected to be fused with the features of different layers of ResNet. When fusion is performed, the low-frequency features of the larger module of the captured image in Gabor are selected to be fused with the ResNet features of smaller resolution to retain the information after the downsampling operation.

7. The method for detecting building changes in remote sensing images based on ResNet34 according to claim 3, characterized in that: In S4, the fused multi-layer features are redistributed through the ratio attention module to redistribute the feature map weights, reduce the attention to useless information, make the module pay more attention to useful information, and further enhance the model's ability to extract building boundaries; The ratio attention module performs ratio processing on the data of the two features respectively and obtains a ratio data in the range of (0, 1). It adds a large ratio to the data with a small ratio and proposes T(x) = 1-ratio. The smaller the ratio, the more important the original data is. For the same ratio, the larger the ratio, the more 1-ratio is used, so that the model pays less attention to these contents. Perform a ratio operation on the two input images, that is, divide the two matrices and take the absolute value to obtain a feature map with a value range of 0-1; divide this feature map into two branches, the left branch uses the channel attention mechanism to take the importance of different features and allocate them to different tasks according to the features; The right branch processes the most important part of the network through the spatial attention mechanism; the final attention representation of this layer is: T i =Div1 i +T i (1) T i =Div2 i +T i (2) Diff i =|X i -Y i |(3) Where: T i T represents x and y in the input image; i represents the current layer; Div1 i , Div2 i Respectively represent the attention weights obtained at the current i-th layer of the two feature maps; Diff i represents the difference between the two feature images of the i-th layer; X i , Y i They represent the feature maps obtained by formula (1) and formula (2) respectively.

8. The method for detecting building changes in remote sensing images based on ResNet34 according to claim 7, characterized in that: Specifically, S6 comprises: densely upsampling the multi-scale feature maps after fusion and refinement of the first to fourth stages output by the decoder in S3 to the same size as the feature map of the first stage, and the shapes are all 64×64×256; then, a cascade operation is used to splice the four feature maps in the channel dimension, and the shape after splicing is 64×64×1024; then, a 3×3 convolution is used to fuse the spliced ​​feature maps to obtain a 64×64×256 feature map; the fused result is classified using a 1×1 convolution, and upsampled to the original image size, and finally a predicted image is obtained.

Images