A semantic segmentation optimization method for large-scale remote sensing images

Abstract

The present invention relates to a large-scale remote sensing image semantic segmentation optimization method, comprising the following steps: step S1, determining the size of the cut block, the edge distance and the reserved area size; step S2, defining the segmentation result matrix, dividing the large-scale original image to be segmented into an edge area and an internal area according to the edge distance; step S3, defining the size of the window based on the size of the reserved area, sliding in sequence in the internal area, and completing the block positioning; step S4, expanding the reserved area block range of each position according to the positioning result and cutting the reserved area block, and inputting the cut block into the semantic segmentation network to obtain the block reasoning result; step S5, cutting the edge of the block reasoning result corresponding to each position and pasting it back to the corresponding position of the segmentation result matrix; step S6, cutting the edge of the segmentation result matrix to obtain the final segmentation result. The present invention can effectively solve the problem of partial regional information error or missing in the traditional large-scale remote sensing image segmentation method.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing images, and in particular to a semantic segmentation optimization method for large-scale remote sensing images. Background Art

[0002] In the field of remote sensing image processing technology, the fine segmentation of typical scenes in large-scale optical remote sensing images is a key basic technology. The semantic segmentation technology of remote sensing images is a method that uses a deep learning architecture to extract high-order scene semantic features and accurately classify and label image pixels. This technology deeply explores and analyzes the inherent multi-dimensional information structure of remote sensing images to achieve a deep understanding of complex remote sensing scenes, and is the core technology for intelligent interpretation of remote sensing images. By performing high-precision semantic segmentation of typical landforms such as waters, roads, and buildings in large-scale remote sensing images, it can generate important application value in topographic mapping, urban planning, and disaster emergency response. In recent years, remote sensing image segmentation technology based on deep convolutional neural networks (DCNNs) has performed excellently. It can adaptively learn and extract feature information at different levels, including underlying object detail features and high-level abstract semantic features.

[0003] However, the current large-scale remote sensing image segmentation technology based on DCNN generally faces a core challenge, namely the "chessboard-like" distortion effect in the segmentation results. This phenomenon mainly stems from the inherent contradiction between large-scale remote sensing image data and existing processor storage and computing resources, as well as the limitations of the inherent mechanism of DCNN convolution operations. Specifically, since large remote sensing images cannot be adapted as a whole to the DCNN model for real-time processing, they need to be divided into multiple sub-image blocks for block input. Each sub-block is independently inferred by the segmentation network and then spliced ​​back to the original image layout. The resulting splicing interface often shows significant transition discontinuity, forming a "checkers board"-like texture defect.

[0004] In-depth exploration of its technical principles shows that the structural design of the convolution layer in the DCNN semantic segmentation network and its unique processing method are the root causes of the above problems. When building such a network, the "filling" strategy of the feature map is usually adopted to maintain the consistency of the spatial dimension, that is, a certain amount of zero-value padding is added to the boundary of the original image according to the size of the convolution kernel. Despite this, the "filling" operation inevitably introduces non-effective information at the edge of the feature map, reducing the prediction credibility of these edge pixels. As the number of network layers increases, the edge effect gradually expands inward during the convolution process, resulting in the overall quality of the feature map showing a decreasing trend from the edge to the center, which further causes the recognition accuracy gap between the edge area and the center area of ​​the segmentation result to increase.

[0005] In addition, traditional segmentation methods for large-scale remote sensing images lack effective edge information integration measures when processing small blocks after cutting, which easily causes information missing or omission in some areas of the segmentation results. Summary of the invention

[0006] In order to solve the technical problems existing in the above-mentioned prior art, the purpose of the present invention is to provide a large-scale remote sensing image semantic segmentation optimization method to solve the problem that the segmentation results of traditional large-scale remote sensing image segmentation methods have some regional information errors or missing.

[0007] To achieve the above-mentioned object of the invention, the present invention provides a large-scale remote sensing image semantic segmentation optimization method, comprising the following steps:

[0008] Step S1, determining the size of the cut block, the edge distance and the size of the reserved area;

[0009] Step S2, defining a segmentation result matrix, and dividing the large-scale original image to be segmented into an edge region and an internal region according to the edge distance;

[0010] Step S3, defining the size of the window based on the size of the reserved area, and sliding in the inner area in sequence to complete the positioning of the reserved area block;

[0011] Step S4: according to the positioning result, the reserved area block range of each position is expanded, the reserved area block is cut according to the expanded size to obtain a cut block, and the cut block is input into the semantic segmentation network to obtain a block reasoning result;

[0012] Step S5, performing edge cropping on the image block reasoning result corresponding to each position, and pasting the image block reasoning result after edge cropping back to the corresponding position of the segmentation result matrix;

[0013] Step S6: cutting the edge of the segmentation result matrix to obtain a final segmentation result.

[0014] According to a technical solution of the present invention, in step S1, it specifically includes:

[0015] Step S11, defining the size of the cut block as w×h, wherein w represents the width of the cut block, and h represents the height of the cut block;

[0016] Step S12, defining the edge distance n as the distance from the edge of the reserved area to the edge of the cut block;

[0017] Step S13: Determine the size of the reserved area as (wn)×(hn) according to the size of the cut image block and the edge distance.

[0018] According to a technical solution of the present invention, the width w of the cut block is equal to the height h of the cut block.

[0019] According to a technical solution of the present invention, the edge distance n is not less than 20 pixels.

[0020] According to a technical solution of the present invention, in the step S2, it specifically includes:

[0021] The segmentation result matrix M is defined to store the semantic segmentation result of the large-scale original image to be segmented, the size of the segmentation result matrix M is equal to the size of the large-scale original image to be segmented, and the internal area is the remaining central area of ​​the large-scale original image to be segmented after removing the edge area;

[0022] The size of the large-scale original image to be segmented is set to W×H, and according to the edge distance n, the size of the internal area is (Wn)×(Hn).

[0023] According to a technical solution of the present invention, in step S3, it specifically includes:

[0024] Defining the size of the window based on the size of the reserved area, sliding the window within the internal area, and recording the range of each tile covered by the window and the coordinates of the tile;

[0025] The sliding of the window starts from the upper left corner of the inner area and slides row by row and column by column in sequence without overlap.

[0026] According to a technical solution of the present invention, the sliding of the window further includes:

[0027] a) for each row-wise sliding, when the window slides to the rightmost end of the large-scale original image to be segmented, if the window completely covers the internal area, no processing is required; if the window covers part of the internal area, the window is moved left until the window completely covers the internal area;

[0028] b) for each column-wise sliding, when the window slides to the bottom of the large-scale original image to be segmented, if the window completely covers the internal area, no processing is required; if the window covers part of the internal area, the window is moved up until the window completely covers the internal area;

[0029] c) When the window slides to the lower right corner, if the window completely covers the internal area, no processing is required; if the window covers part of the internal area, the window is moved left and up at the same time until the window completely covers the internal area.

[0030] According to a technical solution of the present invention, in step S4, it specifically includes:

[0031] Step S41, setting the expansion distance as the edge distance n, expanding the reserved area block to the surrounding areas, and the size of the cut block obtained after the expansion is w×h;

[0032] Step S42: input the cropped image block into the semantic segmentation network for end-to-end mode reasoning, and the size of the output image block reasoning result remains unchanged, which is still w×h.

[0033] According to a technical solution of the present invention, in step S5, it specifically includes:

[0034] Perform edge cropping on the block inference result according to the edge distance n, and the size of the block inference result after cropping is (wn)×(hn);

[0035] According to the coordinates of the block reasoning results in the segmentation result matrix corresponding to the block reasoning results, the block reasoning results are pasted back to the segmentation result matrix.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] The present invention can effectively improve the transition continuity between tiles by flexibly configuring segmentation parameters such as the size of the cut tiles, effectively alleviate or even solve the "checkers board" phenomenon; by adjusting parameters such as the size of the cut tiles and the edge, a more optimized semantic segmentation result can be output.

[0038] The present invention can maximize the segmentation of the original image and reduce information extraction loopholes. The present invention only removes a small number of peripheral pixels of the large-scale image to be segmented, and completes the semantic segmentation of all remaining pixels, thereby maintaining the integrity of information extraction to the greatest extent possible. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0040] Figure 1 A flowchart schematically showing a large-scale remote sensing image semantic segmentation optimization method provided in accordance with one embodiment of the present invention;

[0041] Figure 2 A schematic diagram schematically shows the positional relationship between the edge of a cut image block and a reserved area according to an embodiment of the present invention;

[0042] Figure 3 The figure schematically shows the positional relationship between the edge area and the internal area of ​​the large-scale original image to be segmented and the sliding window according to one embodiment of the present invention. DETAILED DESCRIPTION

[0043] The description of the embodiments of this specification should be combined with the corresponding drawings, which should be considered as part of the complete specification. In the drawings, the shape or thickness of the embodiments may be enlarged and indicated for simplification or convenience. Furthermore, the parts of each structure in the drawings will be described separately. It is worth noting that the elements not shown in the drawings or not described in words are in a form known to ordinary technicians in the relevant technical field.

[0044] The description of the embodiments herein and any reference to directions and orientations are only for the convenience of description and are not to be construed as any limitation on the scope of protection of the present invention. The following description of the preferred embodiments may involve combinations of features, which may exist independently or in combination, and the present invention is not particularly limited to the preferred embodiments. The scope of the present invention is defined by the claims.

[0045] like Figure 1 As shown, a large-scale remote sensing image semantic segmentation optimization method of the present invention comprises the following steps:

[0046] Step S1, determining the size of the cut block, the edge distance and the size of the reserved area.

[0047] like Figure 2 As shown, the size of the cropped image block is determined by setting the width w and the height h. To obtain a square cropped image block, w and h are usually set to the same value.

[0048] Preferably, the width w and height h need to be determined in combination with the processor storage performance, and the larger the value, the better the segmentation effect. In this embodiment, when the processor storage condition performance is sufficient, when the size of the cropped image block is set to be equal to the size of the large-scale original image to be segmented, the "checkers board" problem of the segmentation result can be eliminated to the maximum extent.

[0049] The edge distance n represents the distance from the edge of the reserved area to the edge of the cut image block. Preferably, the edge width n is generally not less than 20 pixels.

[0050] Therefore, by cutting the tile size w×h and the edge distance n, the size of the reserved area can be determined as: (wn)×(hn).

[0051] Step S2: define a segmentation result matrix, and divide the large-scale original image to be segmented into an edge region and an internal region according to the edge distance.

[0052] The segmentation result matrix M is used to store the semantic segmentation result of the large-scale original image to be segmented, and the size of the matrix M is equal to the size of the large-scale original image to be segmented.

[0053] like Figure 3 As shown in FIG. 1 , the distance between the edge region and the boundary is n, and the internal region is the remaining central region after removing the edge region from the large-scale original image to be segmented. Assuming that the size of the large-scale original image to be segmented is W×H, the size of the internal region is: (Wn)×(Hn).

[0054] Step S3, defining the size of the window based on the size of the reserved area, and sliding in the inner area in sequence to complete the positioning of the reserved area block.

[0055] The window size is defined according to the size of the reserved area determined in step S1, and then the window is slid in the inner area of ​​the large-scale original image to be segmented, and the range of each window covering the image block is recorded.

[0056] The window sliding starts from the upper left corner of the inner area of ​​the large-scale image and slides row by row and column by column in sequence without overlap.

[0057] In order to ensure full-pixel segmentation of large-scale images, special processing is required for the edge (right and bottom edges of the image) and corner (lower right corner of the image) areas.

[0058] It is divided into the following three different situations:

[0059] a) For each row-wise sliding, when the window slides to the rightmost end of the image, if the window completely covers the internal area, no processing is required; if the window covers part of the internal area, the window is moved left until the window completely covers the internal area.

[0060] b) For each column-wise sliding, when the window slides to the bottom, if the window completely covers the internal area, no processing is required; if the window covers part of the internal area, the window is moved up until the window completely covers the internal area.

[0061] c) When the window slides to the lower right corner, if the window completely covers the internal area, no processing is required; if the window covers part of the internal area, the window is moved left and up at the same time until the window completely covers the internal area.

[0062] Step S4: according to the positioning result, the reserved area block range of each position is expanded, the reserved area block is cropped according to the expanded size to obtain a cropped block, and the cropped block is input into the semantic segmentation network to obtain the block reasoning result.

[0063] The peripheral expansion distance is set as the edge distance n. According to the reserved area size specified in step S1, the size of the cut image block after the range expansion is w×h.

[0064] According to the end-to-end reasoning mode of semantic segmentation, the size of the output tile reasoning result remains unchanged and is still w×h.

[0065] Step S5: perform edge cropping on the image block reasoning result corresponding to each position, and paste the edge cropped image block reasoning result back to the corresponding position of the segmentation result matrix.

[0066] The cutting edge distance is n. The size of the retained area after cutting is (wn)×(hn).

[0067] Since the size of the segmentation result matrix M is equal to the size of the large-scale original image to be segmented, the corresponding coordinate position of the pasted matrix M is the same as the coordinate position of the large-scale original image to be segmented when the block is cut.

[0068] Step S6: cutting the edge of the segmentation result matrix to obtain the final segmentation result.

[0069] The cutting edge distance is n. The size of the remaining area after cutting is (Wn)×(Hn).

[0070] It should be noted that the above is a preferred embodiment of the present invention. It should be pointed out that although the preferred embodiment of the present invention has been described, once the basic creative concept of the present invention is known, a number of improvements and modifications can be made by a person skilled in the art without departing from the principles of the present invention, and these improvements and modifications should also be regarded as the protection scope of the present invention. Therefore, the attached claims are intended to be interpreted as including the preferred embodiment and all changes and modifications that fall within the scope of the embodiments of the present invention.

Claims

1. A large-scale remote sensing image semantic segmentation optimization method, characterized in that: The following steps are involved: Step S1, determining the size of the cut block, the edge distance and the reserved area size; Step S2, defining a segmentation result matrix, and dividing the large-scale original image to be segmented into an edge region and an internal region according to the edge distance; Step S3, defining the size of the window based on the size of the reserved area, and sliding in the inner area in sequence to complete the positioning of the reserved area block; Step S4: according to the positioning result, the reserved area block range of each position is expanded, the reserved area block is cut according to the expanded size to obtain a cut block, and the cut block is input into the semantic segmentation network to obtain a block reasoning result; Step S5, performing edge cropping on the image block reasoning result corresponding to each position, and pasting the image block reasoning result after edge cropping back to the corresponding position of the segmentation result matrix; Step S6: cutting the edge of the segmentation result matrix to obtain a final segmentation result.

2. The method according to claim 1, characterized in that In the step S1, it specifically includes: Define the size of the crop block as w×h, where w represents the width of the crop block and h represents the height of the crop block; The edge distance n is defined as the distance from the edge of the reserved area to the edge of the cut block; According to the size of the cut image block and the edge distance, the size of the reserved area is determined to be (wn)×(hn).

3. The method according to claim 2, characterized in that The width w of the cropped image block is equal to the height h of the cropped image block.

4. The method according to claim 2, characterized in that: The distance n from the edge is not less than 20 pixels.

5. The method according to claim 2, characterized in that: In the step S2, it specifically includes: The segmentation result matrix M is defined to store the semantic segmentation result of the large-scale original image to be segmented, the size of the segmentation result matrix M is equal to the size of the large-scale original image to be segmented, and the internal area is the remaining central area of ​​the large-scale original image to be segmented after removing the edge area; The size of the large-scale original image to be segmented is set to W×H, and according to the edge distance n, the size of the internal area is (Wn)×(Hn).

6. The method according to claim 2, characterized in that In the step S3, it specifically includes: Defining the size of the window based on the size of the reserved area, sliding the window within the internal area, and recording the range of each tile covered by the window and the coordinates of the tile; The sliding of the window starts from the upper left corner of the inner area and slides row by row and column by column in sequence without overlap.

7. The method according to claim 6, characterized in that The sliding of the window also includes: a) for each row-wise sliding, when the window slides to the rightmost end of the large-scale original image to be segmented, if the window completely covers the internal area, no processing is required; if the window covers part of the internal area, the window is moved left until the window completely covers the internal area; b) for each column-wise sliding, when the window slides to the bottom of the large-scale original image to be segmented, if the window completely covers the internal area, no processing is required; if the window covers part of the internal area, the window is moved up until the window completely covers the internal area; c) When the window slides to the lower right corner, if the window completely covers the internal area, no processing is required; if the window covers part of the internal area, the window is moved left and up at the same time until the window completely covers the internal area.

8. The method according to claim 2, characterized in that: In the step S4, it specifically includes: Step S41, setting the expansion distance as the edge distance n, expanding the reserved area block to the surrounding areas, and the size of the cut block obtained after the expansion is w×h; Step S42: input the cropped image block into the semantic segmentation network for end-to-end mode reasoning, and the size of the output image block reasoning result remains unchanged, which is still w×h.

9. The method according to claim 2, characterized in that: The step S5 specifically includes: Perform edge cropping on the block inference result according to the edge distance n, and the size of the block inference result after cropping is (wn)×(hn); According to the coordinates of the block reasoning results in the segmentation result matrix corresponding to the block reasoning results, the block reasoning results are pasted back to the segmentation result matrix.

Images