Target segmentation method and related equipment for remote sensing images

Abstract

This application discloses a target segmentation method for remote sensing images, an electronic device, and a computer-readable storage medium. The method includes: acquiring a first remote sensing image; extracting a first feature map from the first remote sensing image; performing boundary restoration on targets in the first feature map to obtain a second feature map; and predicting the second feature map to obtain a target segmentation result. This approach improves the accuracy of the target segmentation result.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to a method for segmenting a remote sensing image target, an electronic device, and a computer-readable storage medium. Background Art

[0002] Remote sensing images are obtained by detecting objects on the earth's surface through sensors from different working platforms far away from the ground, such as towers, balloons, airplanes, rockets, artificial satellites, spacecraft and space shuttles. The bands used by sensors to detect objects can be visible light bands, ultraviolet bands, infrared bands and microwave bands, etc. Remote sensing images are widely used in many fields (electronic map production, urban planning, transportation and road facilities, environmental assessment, precision agriculture, forestry measurement).

[0003] By performing target segmentation on remote sensing images, we can obtain the target area required in the remote sensing image for subsequent applications. However, the current target segmentation method has low accuracy. Summary of the invention

[0004] The present application provides a remote sensing image target segmentation method, electronic equipment and computer-readable storage medium, which can solve the problem of low accuracy of current target segmentation methods.

[0005] In order to solve the above technical problems, a technical solution adopted by the present application is: to provide a method for target segmentation of remote sensing images. The method includes: obtaining a first remote sensing image; extracting a first feature map of the first remote sensing image; repairing the boundary of the target in the first feature map to obtain a second feature map; and predicting the second feature map to obtain a target segmentation result.

[0006] To solve the above technical problems, another technical solution adopted in the present application is: to provide an electronic device, which includes a processor and a memory connected to the processor, wherein the memory stores program instructions; the processor is used to execute the program instructions stored in the memory to implement the above method.

[0007] In order to solve the above technical problems, another technical solution adopted by the present application is: providing a computer-readable storage medium storing program instructions, which can implement the above method when executed.

[0008] In the above manner, after obtaining the first feature map, the present application does not directly use the first feature map for prediction, but repairs the boundary of the target in the first feature map to obtain a second feature map, and uses the second feature map for prediction. Since the second feature map after boundary repair has a stronger ability to express the target, it can improve the accuracy of the target segmentation result. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 It is a flowchart of an embodiment of a method for object segmentation of remote sensing images of the present application;

[0010] Figure 2 yes Figure 1 Specific process diagram of S12;

[0011] Figure 3 It is a flowchart of another embodiment of the method for object segmentation of remote sensing images of the present application;

[0012] Figure 4 It is a schematic diagram of the structure of the target segmentation network;

[0013] Figure 5 It is a flowchart of another embodiment of the method for object segmentation of remote sensing images of the present application;

[0014] Figure 6 It is a flowchart of another embodiment of the method for object segmentation of remote sensing images of the present application;

[0015] Figure 7 It is a flowchart of another embodiment of the method for object segmentation of remote sensing images of the present application;

[0016] Figure 8 It is a flowchart of another embodiment of the method for object segmentation of remote sensing images of the present application;

[0017] Fig. 9 It is a flowchart of another embodiment of the method for object segmentation of remote sensing images of the present application;

[0018] Fig.10 It is a flowchart of another embodiment of the method for object segmentation of remote sensing images of the present application;

[0019] Fig.11 It is a schematic diagram of multiple sub-images obtained by continuous cropping;

[0020] Fig.12 It is a schematic diagram of multiple sub-images obtained by overlapping cropping;

[0021] Fig.13 It is a structural schematic diagram of an embodiment of the electronic device of the present application;

[0022] Fig.14 It is a structural schematic diagram of an embodiment of a computer-readable storage medium of the present application; DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0024] The terms "first", "second", and "third" in this application are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first", "second", and "third" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "multiple" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0025] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments without conflict.

[0026] Figure 1 FIG. 1 is a flow chart of an embodiment of the method for segmenting a remote sensing image. It should be noted that if there are substantially the same results, this embodiment is not limited to the above. Figure 1 The process sequence shown is limited. Figure 1 As shown, this embodiment may include:

[0027] S11: Acquire a first remote sensing image.

[0028] The first remote sensing image can be a single-spectral band / single-channel remote sensing image, or a multi-spectral band / multi-channel remote sensing image. For example, the first remote sensing image can be a three-band RGB remote sensing image, a four-band remote sensing image with near infrared, etc. The size of the first remote sensing image is H×W×C, where H is the height, W is the width, and C is the channel.

[0029] S12: Extracting a first feature map of the first remote sensing image.

[0030] In some embodiments, a feature map of a single scale of the first remote sensing image may be extracted as the first feature map.

[0031] In some embodiments, multiple third feature maps of different scales can be extracted from the first remote sensing image from top to bottom, and the multiple third feature maps are fused to obtain the first feature map. Figure 2 , S12 may include the following sub-steps:

[0032] S121: Encode the first remote sensing image to sequentially generate a plurality of third feature maps of different scales of the first remote sensing image.

[0033] Among them, the third feature map of the current scale is obtained based on the third feature map of the previous scale, and the current scale is smaller than the previous scale.

[0034] S122: Fusing multiple third feature maps of different scales to obtain a first feature map.

[0035] In some embodiments, multiple third feature maps of different scales may be directly fused to obtain a first feature map.

[0036] In some embodiments, feature extraction can be performed on the third feature map of the smallest scale to obtain a global feature map; the global feature map and multiple third feature maps of different scales are fused to obtain a first feature map. It can be understood that multiple third feature maps of different scales include underlying feature maps and high-level semantic feature maps, so the fused first feature map covers the underlying information and high-level semantic information of the first remote sensing image, and has a stronger ability to express the target.

[0037] S13: Perform boundary repair on the target in the first feature map to obtain a second feature map.

[0038] Boundary repair can be achieved through operations such as corrosion and / or dilation. Compared with the first feature map, the second feature map has a stronger ability to express the target because the target boundary has been repaired.

[0039] S14: Predict the second feature map to obtain a target segmentation result.

[0040] Prediction is to classify the first remote sensing image at the pixel level based on the second feature map. The target categories for prediction can be one or more, depending on the application scenario. For example, in the application scenario of electronic map production, the target categories include water bodies, roads, buildings, vegetation and others. For another example, in the application scenario of precision agriculture, the target categories for target segmentation include various crops.

[0041] The target segmentation result represents the location of each type of target in the first remote sensing image. Specifically, the target segmentation result may include the probability that each pixel belongs to each target category, and the target category corresponding to the maximum probability is the target category to which the pixel belongs.

[0042] Through the implementation of this embodiment, after obtaining the first feature map, the present application does not directly use the first feature map for prediction, but performs boundary repair on the target in the first feature map to obtain a second feature map, and uses the second feature map for prediction. Since the second feature map after boundary repair has a stronger ability to express the target, the accuracy of the target segmentation result can be improved.

[0043] Figure 3 FIG. 1 is a flow chart of another embodiment of the method for segmenting a remote sensing image. It should be noted that if the results are substantially the same, this embodiment is not limited to the above. Figure 3 The process sequence shown is limited. This embodiment is a further extension of S13. Figure 3 As shown, this embodiment may include:

[0044] S21: performing an erosion operation and an expansion operation on the first feature map respectively, and obtaining an erosion feature map and an expansion feature map respectively.

[0045] A convolution operation can be performed on the first feature map to obtain a feature map to be corroded and a feature map to be expanded; a corrosion displacement feature map corresponding to the feature map to be corroded is generated, and a dilation displacement feature map corresponding to the feature map to be expanded is generated, the corrosion displacement feature map represents the displacement of corrosion of each pixel point in the feature map to be corroded, and the dilation displacement feature map represents the displacement of dilation of each pixel point in the feature map to be expanded; the feature map to be corroded is processed based on the corrosion displacement feature map to obtain a corrosion feature map, and the feature map to be expanded is processed based on the dilation displacement feature map to obtain a dilation feature map.

[0046] S22: Fusing the corrosion feature map and the expansion feature map to obtain a repair feature map.

[0047] The repair feature map contains the repair information of the target boundary.

[0048] S23: Fusing the repaired feature map and the first feature map to obtain a second feature map.

[0049] Combination Figure 4 The target segmentation network shown is used to illustrate this embodiment by way of example:

[0050] like Figure 4 As shown in the figure, the target segmentation network adopts the CascadeEncoderDecoder framework, including the Encoder module, the Decoder module and the Refine module. The first remote sensing image (H×W×C) is sent to the target segmentation network, and the Encoder module, the Decoder module and the Refine module process the first remote sensing image in turn to obtain the target segmentation result.

[0051] 1) First, according to the Patch size of 4, a convolutional layer is used to encode the first remote sensing image into an original feature map of H / 4*W / 4*(16*C), and then it is sent to the Encoder module.

[0052] 2) The Encoder module adopts the MixVision Transformer structure, which consists of 4 stages. Each stage includes an Overlap Patch Merging module and a Transformer Block. In the Encoder module, the Overlap Patch Merging module and the Transformer Block module are used in sequence to process the original feature map.

[0053] The Overlap Patch Merging module mainly uses 2D convolution to scale the original feature map by changing the convolution kernel size and step size to form a hierarchical result of the feature map. From Stage 1 to Stage 4, the size of the feature map is 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the original feature map size, and the number of channels of the feature map is 64, 128, 320, and 512, respectively.

[0054] Transformer Block includes N EEfficient Self-Attention modules and Mix-FFN modules. N is 3 in Stage 1, 6 in Stage 2, 40 in Stage 3, and 3 in Stage 4.

[0055] The Efficient Self-Attention module performs channel pruning based on the self-attention structure of the qkv mode. The implementation process is as follows:

[0056] ① Based on the processing of the Overlap Patch Merging module, a 1*1 convolutional layer is used to process the original feature map to obtain three feature maps of N×C channels, namely Q, K, and V.

[0057] ② Convert the channels of K and V into N / R×(C·R) in turn; transform the (C·R) channel into C through a linear layer, and the values ​​of R are 8, 4, 2, and 1 in Stage 1 to Stage 4 respectively.

[0058] ③After matrix multiplication of Q and K, the Softmax function is used to normalize them to generate attention weights, which are multiplied by V to obtain the processing result of the Efficient Self-Attention module.

[0059] The Mix-FFN module adds a 3*3 convolutional layer (Conv 3×3 ). This processing can be achieved through the following calculation formula:

[0060] x out =MLP(GELU(Conv 3×3 (MLP(x in ))))+x in

[0061] Among them, x in is the output of the Efficient Self-Attention module, MLP is the multi-layer perceptron layer, GELU is the activation function, and x out It is the processing result of the Efficient Self-Attention module.

[0062] Among them, by processing the original feature map in four stages, four third feature maps of different scales are obtained. Among them, S1a~S4a are the third feature maps obtained by stage1~stage4 respectively. S1a~S4a are sent to the Decoder module.

[0063] 3) The decoder module uses a 1*1 convolution layer to process S1a~S4a respectively to obtain S1b~S4b. The PSP module is used to extract features from S4b to obtain the global feature map S5c. Specifically, an adaptive average pooling layer can be used to convert S4a into feature maps of 1*1, 2*2, 3*3, and 6*6 sizes, and a 1*1 convolution layer can be used to convert feature maps of 1*1, 2*2, 3*3, and 6*6 sizes to the same size as S4, and then spliced ​​together. Finally, a 3*3 convolution layer is used to convert the channels of the spliced ​​feature map to the same channels of S4b to obtain S5c. Furthermore, from S1b to S4b, S5c is converted to the current size in turn and then added to obtain S4c~S1c. Furthermore, a 3*3 convolutional layer is used to convert S1c~S5c into the original feature map size in turn, and S1d~S5d are obtained. S1d~S5d are concatenated together, and then a 3*3 convolutional layer is used to convert the channels of the concatenated feature map into C to obtain the first feature map F-num. F-num is sent to the Refine module.

[0064] 4) The Refine module uses the idea of ​​morphological processing to repair the boundary of the target in F-num. The implementation process is:

[0065] ① Use two 3*3 convolutional layers to process F-num respectively to generate the feature map to be eroded F-erosion and the feature map to be expanded F-dilation.

[0066] ② Use a 3*3 convolutional layer to process F-erosion and F-dilation respectively to obtain the erosion displacement feature map δ1 and the dilation displacement feature map δ2. The number of channels of δ1 and δ2 is 2*C, representing the displacement of each channel in the height and width directions respectively.

[0067] ③ According to δ1, F-erosion is processed to obtain the erosion feature map F-contraction; according to δ2, F-dilation is processed to obtain the expansion feature map F-expansion. The implementation method of this process is to warp the feature value of the original position P in F-erosion / F-dilation to the new position P' according to δ1 / δ2.

[0068] ④ F-contraction and F-dilation are fused, that is, F-contraction and F-expansion are added and then two consecutive 3*3 convolutional layers are used for feature re-extraction to obtain the repair feature map F-boundary, which contains the boundary repair information. In the training stage, the boundary prediction of F-boundary is further performed, the boundary prediction result is output, and the loss function (BCE and Dice loss) is used for supervision to improve the accuracy of the boundary repair information implied by the application node F-boundary.

[0069] ⑤ Use a 1*1 convolution layer to process F-boundary and add it to F-num, and use a 1*1 convolution layer to make predictions to get the target segmentation result.

[0070] It should be noted that the processing parameters of each module included in the above example 1 can be adjusted according to needs, and therefore do not constitute a limitation to the present application.

[0071] Figure 5 FIG. 1 is a flow chart of another embodiment of the method for segmenting a remote sensing image. It should be noted that if there are substantially the same results, this embodiment is not limited to the above. Figure 5 The process sequence shown is limited. This embodiment is a further extension of S11. Figure 5 As shown, this embodiment may include:

[0072] S31: Acquire a second remote sensing image.

[0073] S32: Determine whether the channel of the second remote sensing image is a preset channel.

[0074] The preset channels may include all channels of the remote sensing image, or may include commonly used channels of the remote sensing image. The channel of the first remote sensing image is a preset channel, which means that the channel type of the first remote sensing image is a preset channel type and the number of channels is a preset number of channels.

[0075] If yes, execute S33; otherwise, execute S34.

[0076] S33: Using the second remote sensing image as the first remote sensing image.

[0077] S34: performing channel expansion on the second remote sensing image to obtain the first remote sensing image.

[0078] The channel of the second remote sensing image is a preset channel.

[0079] For example, the preset channel types are visible light channel (3 channels) and near infrared channel (1 channel), the preset channel number is 4, and the remote sensing image supported by the segmentation network is a 4-channel visible light + near infrared remote sensing image (H×W×4). If the second remote sensing image is a 3-channel visible light remote sensing image (H×W×3), then an array of H×W×1 with a value of 0 is spliced ​​on the second remote sensing image to form a first remote sensing image of size H×W×4.

[0080] It is understandable that the generalization of the same target segmentation network is insufficient and cannot be compatible with the processing of remote sensing images of different channels. Therefore, the target segmentation network needs to adapt to the change of channels in order to achieve target segmentation of remote sensing images of different channels.

[0081] To this end, in this embodiment, the channel supported by the target segmentation network is set as the preset channel. When the channel of the second remote sensing image is not the preset channel, the channel of the second remote sensing image is expanded to obtain the first remote sensing image whose channel is the preset channel. Thus, the target segmentation of remote sensing images of different channels can be achieved through the same target segmentation network, reducing the complexity of target segmentation.

[0082] Furthermore, if the application scenario of target segmentation is electronic map production, the target segmentation is the segmentation of multiple categories of targets (for example, including water bodies, roads, buildings, vegetation and others). Because target segmentation follows the WYSIWYG principle, the prediction of the target segmentation result corresponds one-to-one to the category represented by the actual pixel. Roads blocked by vegetation, buildings, etc. will cause problems such as disconnected roads, uneven road edges, and the coexistence of roads of different levels (such as main roads and side roads). Buildings have problems such as insufficient prediction and adhesion between buildings. For this reason, after obtaining the target segmentation result, the target segmentation result can also be post-processed to optimize the target segmentation result. The post-processing of the target segmentation result is introduced as follows:

[0083] Figure 6 FIG. 1 is a flow chart of another embodiment of the method for segmenting a remote sensing image. It should be noted that if there are substantially the same results, this embodiment is not limited to the above. Figure 6 The process sequence shown is limited. In this embodiment, the target segmentation result includes the first segmentation result of the road. Figure 6 As shown, this embodiment may include:

[0084] S41: Based on the first segmentation result of the road, determine a plurality of first connected domains included in the road.

[0085] S42: taking each first connected domain as a current first connected domain in turn, and if the area of ​​the current first connected domain is smaller than the area threshold, removing the current first connected domain to update the first segmentation result of the road.

[0086] It can be understood that the road objects targeted for segmentation are roads of all levels (including main roads and side roads). The first connected domain with an area less than the area threshold represents the side road, which is not required when making an electronic map. Therefore, through the implementation of this embodiment, the first connected domain with an area less than the area threshold is removed, which can facilitate the production of electronic maps.

[0087] Figure 7 FIG. 1 is a flow chart of another embodiment of the method for segmenting a remote sensing image. It should be noted that if there are substantially the same results, this embodiment is not limited to the above. Figure 7 The process sequence shown is limited. In this embodiment, the target segmentation result is post-processed using the road network extraction network (road network post-processing). Figure 7 As shown, this embodiment may include:

[0088] S51: Perform road segmentation on the first remote sensing image to obtain a second road segmentation result.

[0089] S52: Superimpose the second segmentation result of the road onto the target segmentation result.

[0090] In some embodiments, the second segmentation result of the road may be directly superimposed on the target segmentation result.

[0091] In some embodiments, the second segmentation result of the road can be optimized and then superimposed on the target segmentation result. Specifically, based on the second segmentation result of the road, the road can be expanded according to the first expansion parameter; the road can be thinned (the center line of the road can be extracted); a number of second connected domains included in the road can be determined; each second connected domain can be used as the current second connected domain in turn, and if the area of ​​the current second connected domain is less than the area threshold, the current second connected domain can be removed (the disconnected side branch in the road can be removed); the road area can be expanded according to the second expansion parameter (the width of the road can be restored) to update the second segmentation result of the road.

[0092] Through the implementation of this embodiment, it is possible to optimize the first segmentation result of the road in the target segmentation result by using the second segmentation result of the road, so as to enhance the connectivity of the road and improve the visual effect of the road.

[0093] Figure 8 FIG. 1 is a flow chart of another embodiment of the method for segmenting a remote sensing image. It should be noted that if there are substantially the same results, this embodiment is not limited to the above. Figure 8 In this embodiment, the target segmentation result includes the first segmentation result of the building, and the target segmentation result is post-processed (building base post-processing) using the building base extraction network. Figure 8 As shown, this embodiment may include:

[0094] S61: Based on the first segmentation result of the building, determine a plurality of third connected domains included in the building.

[0095] S62: removing a plurality of third connected domains.

[0096] The method of removing the third connected domain is similar to the process of removing the current first connected domain, which is not described here in detail. The difference is that the area of ​​the second connected domain does not need to be compared with the area threshold and is directly removed.

[0097] S63: Perform building base segmentation on the first remote sensing image to obtain a second segmentation result of the building base.

[0098] The process of processing the first remote sensing image by the building base extraction network is similar to the process of processing the first remote sensing image by the road network extraction network, which will not be repeated here.

[0099] S64: Superimpose the second segmentation result of the building base onto the target segmentation result.

[0100] In some embodiments, the second segmentation result of the building substrate may be directly superimposed on the target segmentation result.

[0101] In some embodiments, the second segmentation result of the building base can be optimized first and then superimposed on the target segmentation result. Specifically, based on the second segmentation result of the building base, several fourth connected domains in the non-building base can be determined; each fourth connected domain is sequentially used as the current fourth connected domain, and if the area of ​​the current fourth connected domain is less than the area threshold, the current fourth connected domain is removed to update the second segmentation result of the building base.

[0102] The method of removing the current fourth connected domain is similar to the method of removing the current first connected domain, except that: the target category for building base segmentation is building base and others, and when removing the current fourth connected domain, there is no need to calculate the replacement category, but directly change the pixel category in the current fourth connected domain to the building base.

[0103] It is understandable that when making an electronic map, it is not necessary to use the parts of the building other than the building base. Therefore, in this embodiment, the first segmentation result of the building in the target segmentation result is first removed, and then the second segmentation result of the building base is superimposed on the target segmentation result, thereby replacing the first segmentation result of the building with the second segmentation result of the building base to facilitate subsequent electronic map production.

[0104] Through the implementation of this embodiment, it is possible to use the second segmentation result of the building base to correct the first segmentation result of the building, thereby removing the voids in the building base and improving the segmentation effect and visual effect of the building base.

[0105] Furthermore, after obtaining the target segmentation result, or the road network post-processing, or the building base post-processing, the target segmentation result can also be processed as follows to increase the connectivity of all types of targets. Specifically, it can be as follows:

[0106] Fig. 9 FIG. 1 is a flow chart of another embodiment of the method for segmenting a remote sensing image. It should be noted that if there are substantially the same results, this embodiment is not limited to the above. Fig. 9 The process sequence shown is limited. Fig. 9 As shown, this embodiment may include:

[0107] S71: sequentially taking the first segmentation result of each type of target in the target segmentation results as the first segmentation result of the current type of target, and determining a plurality of fifth connected domains of the current type of target based on the first segmentation result of the current type of target.

[0108] S72: taking each fifth connected domain as the current fifth connected domain in turn, and if the area of ​​the current fifth connected domain is smaller than the area threshold, removing the current fifth connected domain to update the target segmentation result.

[0109] The method of removing the current fifth connected domain is similar to the method of removing the current first connected domain, which will not be described in detail here.

[0110] Furthermore, considering that the high-resolution first remote sensing image occupies a large amount of memory space, directly using it for target segmentation will reduce processing efficiency, so after obtaining the first remote sensing image, the first remote sensing image is cropped into multiple sub-images, and the multiple sub-images are applied to target segmentation. Therefore, the above embodiment can be further expanded:

[0111] Fig.10 FIG. 1 is a flow chart of another embodiment of the method for segmenting a remote sensing image. It should be noted that if there are substantially the same results, this embodiment is not limited to the above. Fig.10 In this embodiment, S81 is a step that can be included before S11, and S82 to S85 are further expansions of S11. Fig.10 As shown, this embodiment may include:

[0112] S81: cropping the first remote sensing image according to a preset cropping method to obtain a plurality of sub-images of the first remote sensing image.

[0113] The geographical range covered by the multiple sub-images is equal to the geographical range covered by the first remote sensing image. The preset cropping method can be a continuous cropping method or an overlapping cropping method. In the continuous cropping method, there is no overlap between adjacent sub-images. In the overlapping cropping method, there is partial overlap between adjacent sub-images. For example, the size of the sub-image is 512×512, and the partial overlap ratio is 50%. See the example of multiple sub-images obtained by the continuous cropping method. Fig.11 ,like Fig.11 As shown in FIG. 1 , the first remote sensing image is cropped into four consecutive sub-images. An example of multiple sub-images obtained by overlapping cropping is shown in FIG. Fig.12 ,like Fig.12 As shown, the first remote sensing image is cut into 9 sub-images, and the overlapping ratio of adjacent sub-images is 50%.

[0114] S82: Extract first feature maps of multiple sub-maps respectively.

[0115] S83: Perform boundary repair on the targets in the first feature maps of the multiple sub-maps respectively to obtain second feature maps of the multiple sub-maps.

[0116] S84: Predict the second feature maps of the multiple sub-images respectively to obtain target segmentation results of the multiple sub-images.

[0117] The target segmentation result of the sub-image includes the probability that each pixel in the sub-image belongs to each target category, and the target category corresponding to the maximum probability is the target category to which the pixel belongs.

[0118] S85: splicing the target segmentation results of the multiple sub-images to obtain the target segmentation result of the first remote sensing image.

[0119] In the continuous cropping mode, the probability of the pixel points in each sub-image can be used as the probability of the corresponding pixel points in the first remote sensing image.

[0120] In the overlapping cropping mode, for the overlapping areas of adjacent sub-images, if the target segmentation results of the pixels corresponding to the same position are different, the target categories to which the pixels belong are different. Therefore, if the probabilities of the pixels in each sub-image are directly used as the probabilities of the corresponding pixels in the first remote sensing image, there may be a patchwork situation. Therefore, in this case, the probabilities of the corresponding pixels in the overlapping areas of adjacent sub-images can be added as the probability of the corresponding pixels in the first remote sensing image, and the probabilities of the pixels in each sub-image that do not belong to the overlapping areas can be used as the probability of the corresponding pixels in the first remote sensing image to obtain the target segmentation result of the first remote sensing image.

[0121] Fig.13 Schematic diagram of the structure of an embodiment of the electronic device of the present application. Fig.13 As shown, the electronic device includes a processor 21 and a memory 22 coupled to the processor 21 .

[0122] The memory 22 stores program instructions for implementing the method of any of the above embodiments; the processor 21 is used to execute the program instructions stored in the memory 22 to implement the steps of the above method embodiments. The processor 21 may also be referred to as a CPU (Central Processing Unit). The processor 21 may be an integrated circuit chip with signal processing capabilities. The processor 21 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0123] The electronic device mentioned in this embodiment may be an access management node, or may be other device connected to the access management node.

[0124] Fig.14 Schematic diagram of the structure of an embodiment of a computer-readable storage medium of the present application. Fig.14As shown, the computer-readable storage medium 30 of the embodiment of the present application stores program instructions 31, and the program instructions 31 are executed to implement the method provided in the above embodiment of the present application. Among them, the program instructions 31 can form a program file and be stored in the above computer-readable storage medium 30 in the form of a software product, so that a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) executes all or part of the steps of each implementation method of the present application. The aforementioned computer-readable storage medium 30 includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk, or a terminal device such as a computer, a server, a mobile phone, and a tablet.

[0125] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0126] In addition, each functional unit in each embodiment of the present application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional units. The above is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made by using the description and drawings of this application, or directly or indirectly used in other related technical fields, is also included in the patent protection scope of the present application.

Claims

1. A method for target segmentation of remote sensing images, characterized in that: include: Acquiring the first remote sensing image; Extracting a first feature map of the first remote sensing image; Perform boundary repair on the target in the first feature map to obtain a second feature map; Predicting the second feature map to obtain a target segmentation result; Wherein, before extracting the first feature map of the first remote sensing image, the method further includes: Cropping the first remote sensing image according to a preset cropping method to obtain a plurality of sub-images of the first remote sensing image; The step of extracting a first feature map of the first remote sensing image includes: Extracting first feature graphs of the multiple sub-graphs respectively; The performing boundary repair on the target in the first feature map to obtain a second feature map includes: Perform boundary repair on the targets in the first feature maps of the multiple sub-maps respectively to obtain second feature maps of the multiple sub-maps; The predicting the second feature map to obtain the target segmentation result includes: Predicting the second feature maps of the multiple sub-images respectively to obtain target segmentation results of the multiple sub-images; The target segmentation results of the multiple sub-images are spliced ​​to obtain the target segmentation result of the first remote sensing image.

2. The method according to claim 1, characterized in that The step of extracting a first feature map of the first remote sensing image includes: encoding the first remote sensing image, and sequentially generating a plurality of third feature maps of different scales of the first remote sensing image, wherein the third feature map of a current scale is obtained based on the third feature map of a previous scale, and the current scale is smaller than the previous scale; The plurality of third feature maps of different scales are fused to obtain the first feature map.

3. The method according to claim 2, characterized in that The fusing the plurality of third feature maps of different scales to obtain the first feature map includes: Perform feature extraction on the third feature map of the smallest scale to obtain a global feature map; The global feature map and the third feature maps of the multiple different scales are fused to obtain the first feature map.

4. The method according to claim 1, characterized in that: The performing boundary repair on the target in the first feature map to obtain a second feature map includes: Performing an erosion operation and an expansion operation on the first characteristic map respectively, to obtain an erosion characteristic map and an expansion characteristic map respectively; The corrosion feature map and the expansion feature map are merged to obtain a repair feature map; The repaired feature map and the first feature map are fused to obtain the second feature map.

5. The method according to claim 4, characterized in that The performing an erosion operation and an expansion operation on the first characteristic map respectively to obtain an erosion characteristic map and an expansion characteristic map respectively includes: Performing a convolution operation on the first feature map to obtain a feature map to be eroded and a feature map to be expanded; Generate an erosion displacement feature map corresponding to the feature map to be eroded, and generate an expansion displacement feature map corresponding to the feature map to be expanded, wherein the erosion displacement feature map represents the displacement of the corrosion of each pixel point in the feature map to be eroded, and the expansion displacement feature map represents the displacement of the expansion of each pixel point in the feature map to be expanded; The to-be-corroded characteristic map is processed based on the corrosion displacement characteristic map to obtain the corrosion characteristic map, and the to-be-expanded characteristic map is processed based on the expansion displacement characteristic map to obtain the expansion characteristic map.

6. The method according to claim 1, characterized in that The step of acquiring the first remote sensing image comprises: acquiring a second remote sensing image; In response to the channel of the second remote sensing image not being a preset channel, channel expansion is performed on the second remote sensing image to obtain the first remote sensing image, and the channel of the first remote sensing image is a preset channel.

7. The method according to claim 1, characterized in that The preset cropping method is a continuous cropping method, and there is no overlap between adjacent sub-images.

8. The method according to claim 7, characterized in that The preset cropping method is an overlapping cropping method, there is partial overlap between adjacent sub-images, and the target segmentation results of the sub-images include the probabilities that the pixels in the sub-images belong to each category; The step of splicing the target segmentation results of the plurality of sub-images to obtain the target segmentation result of the first remote sensing image includes: The probabilities of the corresponding pixel points in the overlapping areas of the adjacent sub-images are added as the probability of the corresponding pixel points in the first remote sensing image, and the probability of the pixel points in each sub-image that do not belong to the overlapping area is used as the probability of the corresponding pixel points in the first remote sensing image to obtain the target segmentation result of the first remote sensing image.

9. An electronic device, characterized in that: The method comprises a processor and a memory connected to the processor, wherein: The memory stores program instructions; The processor is configured to execute the program instructions stored in the memory to implement the method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The storage medium stores program instructions, and when the program instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images