Region segmentation method, device, apparatus and storage medium

Abstract

The present application provides a kind of regional segmentation method, device, equipment and storage medium, the method comprises: obtaining remote sensing image to be processed;The remote sensing image to be processed is input to offshore aquaculture area segmentation model, and the aquaculture area segmentation result output by the offshore aquaculture area segmentation model is obtained;Offshore aquaculture area segmentation model includes encoding module and decoding module of joint attention mechanism, is based on the image label of pre-collected remote sensing image sample and remote sensing image sample Iterative training is obtained.The present application extracts high-dimensional feature map containing hidden complex feature information through encoding module, and introduces joint attention mechanism in decoding process, which can highlight the spatial position and morphological characteristics of offshore aquaculture farm and other detailed information, solves the technical problem that manual extraction of SAR image imaging feature leads to lower accuracy of aquaculture area segmentation, improves the accuracy of aquaculture area segmentation.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a region segmentation method, apparatus, device, and storage medium. Background Technology

[0002] In recent years, in order to meet people's demand for marine products, nearshore aquaculture has developed rapidly. However, overly intensive net enclosure aquaculture is seriously damaging the nearshore ecological environment. Therefore, in order to control the number and scale of nearshore aquaculture farms, it is necessary to obtain the spatial distribution of nearshore aquaculture farms in a timely and accurate manner, so as to rationally plan the layout of nearshore aquaculture farms.

[0003] Currently, remote sensing imagery is a highly effective means of observing nearshore aquaculture farms. Due to the all-weather, all-day advantage of SAR imagery, extracting nearshore aquaculture areas using SAR imagery has become a recent research hotspot. Existing methods for extracting aquaculture areas from SAR imagery mainly involve manually extracting texture and color features, or features such as reflectivity and backscattering coefficients. Furthermore, during the segmentation process, the adaptability of the parameters is very limited, requiring manual adjustment of segmentation parameters under different conditions, resulting in low accuracy in aquaculture area segmentation. Summary of the Invention

[0004] This invention provides a method, apparatus, device, and storage medium for regional segmentation, aiming to improve the accuracy of aquaculture regional segmentation.

[0005] This invention provides a region segmentation method, comprising:

[0006] Acquire remote sensing images to be processed;

[0007] The remote sensing image to be processed is input into the nearshore aquaculture area segmentation model to obtain the aquaculture area segmentation result output by the nearshore aquaculture area segmentation model;

[0008] The nearshore aquaculture area segmentation model includes an encoding module and a decoding module with a joint attention mechanism; the nearshore aquaculture area segmentation model is obtained through iterative training based on pre-collected remote sensing image samples and the image labels of the remote sensing image samples.

[0009] Optionally, according to a region segmentation method provided by the present invention, the nearshore aquaculture area segmentation model further includes a region segmentation output module;

[0010] The encoding module includes a first convolutional layer and multiple cascaded encoding units, each of which is composed of a different number of stacked MBConv convolutional structures;

[0011] The decoding module includes multiple cascaded decoding units. Each decoding unit, except for the first decoding unit, includes a feature connection layer, an upsampling layer, a second convolutional layer, and a joint attention mechanism layer. The first decoding unit includes a feature connection layer, a second convolutional layer, and a joint attention mechanism layer.

[0012] The output of the last decoding module serves as the input to the region segmentation output module.

[0013] Optionally, according to a region segmentation method provided by the present invention, the step of inputting the remote sensing image to be processed into a nearshore aquaculture area segmentation model to obtain the aquaculture area segmentation result output by the nearshore aquaculture area segmentation model includes:

[0014] The remote sensing image to be processed is input into the first convolutional layer to perform convolution processing on the remote sensing image to be processed, so as to obtain a convolutional feature map.

[0015] The convolutional feature map is input into the first coding unit to obtain the coding feature map output by the first coding unit, and the coding feature map is used as the input of the next coding unit until the coding feature map output by the last coding unit is obtained;

[0016] The encoded feature map output by the last encoding unit and the encoded feature map output by the encoding unit at the same depth level as the first decoding unit are input into the feature connection layer in the first decoding unit to obtain the first connection feature map output by the feature connection layer in the first decoding unit.

[0017] The first connection feature map is convolved by the second convolutional layer and the joint attention mechanism layer in the first decoding unit to obtain the decoded feature map output by the first decoding unit, and the decoded feature map output by the first decoding unit is used as the input of the next decoding unit.

[0018] The upsampling layer in the next decoding unit upsamples the decoded feature map output by the first decoding unit to obtain an upsampled feature map.

[0019] The upsampled feature map and the encoded feature map output by the encoding unit at the same depth level as the next decoding unit are connected through the feature connection layer in the next decoding unit to obtain a second connected feature map;

[0020] The second connected feature map is convolved by the second convolutional layer and the joint attention mechanism layer in the next decoding unit to obtain the decoded feature map output by the next decoding unit.

[0021] Return to execution steps: Upsample the decoded feature map output by the first decoding unit through the upsampling layer in the next decoding unit to obtain an upsampled feature map; until the decoded feature map output by the last decoding module is obtained;

[0022] The decoded feature map of the last decoding module is input into the region segmentation output module to obtain the aquaculture region segmentation result output by the region segmentation output module.

[0023] Optionally, according to the region segmentation method provided by the present invention, the nearshore aquaculture area segmentation model is obtained by training based on the following steps:

[0024] Acquire a remote sensing image sample set;

[0025] Data augmentation processing is performed on the remote sensing image sample set to obtain multiple augmented remote sensing images;

[0026] The augmented remote sensing images are stitched together to obtain a stitched remote sensing image.

[0027] The synthesized remote sensing image is cropped to obtain several remote sensing image slices, wherein there is no overlap between the remote sensing image slices;

[0028] Based on each remote sensing image slice and its corresponding image label, the initial model to be trained is iteratively trained to obtain the nearshore aquaculture area segmentation model.

[0029] Optionally, according to a region segmentation method provided by the present invention, the step of iteratively training an initial model to be trained based on each of the remote sensing image slices and the image labels corresponding to each of the remote sensing image slices to obtain the nearshore aquaculture area segmentation model includes:

[0030] For any remote sensing image slice, the remote sensing image slice is input into the initial model to be trained, and the prediction result output by the initial model to be trained is obtained.

[0031] Based on the prediction results and the image labels corresponding to the remote sensing image slices, the model loss value is calculated.

[0032] Based on the model loss value obtained in each iteration, the parameters of the initial model to be trained are updated to obtain the nearshore aquaculture area segmentation model.

[0033] Optionally, according to a region segmentation method provided by the present invention, the step of calculating the model loss value based on the prediction result and the image label corresponding to the remote sensing image tile includes:

[0034] Based on the prediction results, the image labels corresponding to the positions of each pixel in the remote sensing image slice, and the pre-set weight coefficients, the first loss value is calculated by the pre-set boundary cross-entropy loss function.

[0035] Based on the prediction results and the image labels corresponding to the positions of each pixel in the remote sensing image slice, a second loss value is calculated using a pre-set similarity loss function.

[0036] The model loss value is calculated based on the first loss value and the second loss value.

[0037] Optionally, according to a region segmentation method provided by the present invention, the expression for the boundary cross-entropy loss function is:

[0038]

[0039] The expression for the similarity loss function is:

[0040]

[0041] Where Loss1 represents the first loss value, Loss2 represents the second loss value, W represents the weight coefficient, N represents the number of pixels, and y i pi represents the image label corresponding to pixel i, and pi represents the prediction result corresponding to pixel i.

[0042] The present invention also provides a region segmentation device, comprising:

[0043] The acquisition module is used to acquire remote sensing images to be processed;

[0044] The segmentation module is used to input the remote sensing image to be processed into the nearshore aquaculture area segmentation model to obtain the aquaculture area segmentation result output by the nearshore aquaculture area segmentation model;

[0045] The nearshore aquaculture area segmentation model includes an encoding module and a decoding module with a joint attention mechanism; the nearshore aquaculture area segmentation model is obtained through iterative training based on pre-collected remote sensing image samples and the image labels of the remote sensing image samples.

[0046] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement any of the region segmentation methods described above.

[0047] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the region segmentation method as described above.

[0048] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements any of the region segmentation methods described above.

[0049] The region segmentation method, apparatus, device, and storage medium provided by this invention extract high-dimensional feature maps containing hidden complex feature information through an encoding module constructed by the EfficientNet feature extraction network. Furthermore, a joint attention mechanism is introduced during the decoding process to highlight detailed information such as the spatial location and morphological features of nearshore aquaculture farms. This solves the technical problem of low accuracy in aquaculture region segmentation caused by manually extracting features from SAR images, and effectively improves the accuracy of aquaculture region segmentation. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0051] Figure 1 This is one of the flowcharts illustrating the region segmentation method provided by the present invention;

[0052] Figure 2 This is a schematic diagram of the nearshore aquaculture zone segmentation model provided by the present invention;

[0053] Figure 3 This is a schematic diagram of the coding module in the nearshore aquaculture zone segmentation model provided by the present invention;

[0054] Figure 4 A schematic diagram of the region segmentation device provided by the present invention;

[0055] Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0057] The terminology used in one or more embodiments of the present invention is for the purpose of describing particular embodiments only and is not intended to limit the scope of the invention. The singular forms “a,” “the,” and “the” used in one or more embodiments of the invention are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” used in one or more embodiments of the invention refers to and includes any or all possible combinations of one or more associated listed items.

[0058] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of the present invention, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of the present invention, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0059] The following is combined Figures 1-3 The exemplary embodiments of the present invention will be described in detail below.

[0060] Figure 1 This is a flowchart of a region segmentation method according to an embodiment of the present invention, such as... Figure 1 As shown, the region segmentation method includes:

[0061] Step 11: Acquire the remote sensing image to be processed;

[0062] It should be noted that the remote sensing image to be processed is a remote sensing image that includes the width, height and number of channels of the image. The remote sensing image can be obtained from a public remote sensing image set, or it can be obtained by crawling remote sensing image tiles from an existing geographic information system and stitching them together, or by obtaining remote sensing images corresponding to the Gaofen-3 satellite and the Maritime Silk Road-1 satellite through other online acquisition methods. For example, to obtain 512×512×3 image data to be identified, that is, the height is 512, the width is 512, and the number of channels is 3. The channels may include red, green and blue channels.

[0063] Step 12: Input the remote sensing image to be processed into the nearshore aquaculture area segmentation model to obtain the aquaculture area segmentation result output by the nearshore aquaculture area segmentation model;

[0064] The nearshore aquaculture area segmentation model includes an encoding module and a decoding module with a joint attention mechanism; the nearshore aquaculture area segmentation model is obtained through iterative training based on pre-collected remote sensing image samples and the image labels of the remote sensing image samples.

[0065] Specifically, the remote sensing image to be processed is input into the nearshore aquaculture area segmentation model. Based on the output of the nearshore aquaculture area segmentation model, the segmentation result of the aquaculture area is obtained. The nearshore aquaculture area segmentation model is obtained through iterative training based on pre-collected remote sensing image samples and their image labels. It can be understood that after training, the nearshore aquaculture area segmentation model can effectively segment remote sensing images to obtain the aquaculture area segmentation result of the remote sensing image to be processed.

[0066] In this embodiment of the invention, the nearshore aquaculture area segmentation model is a segmentation model based on the encoder-decoder architecture of the U-net model. The nearshore aquaculture area segmentation model includes an encoding module, a decoding module, and a region segmentation output module. The encoding module is constructed using the EfficientNet feature extraction network. The EfficientNet-b0 to b7 series of deep convolutional neural networks are selected based on the depth, width, and resolution of EfficientNet-b0, searching for the most suitable model construction parameters among other networks in this series. The basic principle is that increasing the network depth yields richer and more complex features, increasing the width captures more morphological features of nearshore aquaculture farms, and a larger resolution captures detailed features of nearshore aquaculture farms. Considering the time consumption and segmentation efficiency of the model, this embodiment preferably uses EfficientNet-b0 as the encoding module of the nearshore aquaculture area segmentation model.

[0067] The encoding module includes a first convolutional layer and multiple cascaded encoding units. Each encoding unit is composed of a different number of stacked MBConv convolutional structures. The first convolutional layer is a convolutional layer with a kernel size of 3×3. The scaling factor and kernel size of each MBConv convolutional structure are not the same.

[0068] The decoding module includes multiple cascaded decoding units. Each decoding unit, except for the first decoding unit, includes a feature connection layer, an upsampling layer, a second convolutional layer, and a joint attention mechanism layer. The first decoding unit includes a feature connection layer, a second convolutional layer, and a joint attention mechanism layer. The joint attention mechanism layer includes a channel attention mechanism and a spatial attention mechanism. The output of the last decoding module is used as the input of the region segmentation output module.

[0069] Figure 2 This is a schematic diagram of the nearshore aquaculture zone segmentation model provided by the present invention, as shown below. Figure 2As shown, the encoding module includes 5 encoding units, and the decoding module includes 4 decoding units. The encoded feature map output by the fifth encoding unit is directly used as an input to the feature connection layer in the first decoding unit. The encoding units are connected to the decoding units at the same depth level. Specifically, the fourth encoding unit and the first decoding unit are at the same depth level, the third encoding unit and the second decoding unit are at the same depth level, the second encoding unit and the third decoding unit are at the same depth level, and the first encoding unit and the fourth decoding unit are at the same depth level.

[0070] Specifically, the remote sensing image to be processed is input into the first convolutional layer to perform convolution processing on the remote sensing image, obtaining a convolutional feature map. This convolutional feature map is then input into the coding unit to obtain the coding feature map output by the coding unit. The output of the previous coding unit serves as the input to the next coding unit, until the coding feature map output by the last coding unit is obtained. Further, the coding feature map output by the last coding unit and the coding feature map output by the coding unit at the same depth level as the first decoding unit are used as feature connections in the first decoding unit. The input to the layer, except for the first decoding unit, is the feature map of the output of the previous decoding unit after upsampling and the encoded feature map output by the encoding unit at the same depth level as the corresponding decoding unit. After obtaining the connection feature map output by the feature connection layer, the connection feature map is convolved by the second convolutional layer and the joint attention mechanism layer in the decoding unit to obtain the channel and spatial intelligent correction decoding feature map. Then, the decoding feature map of the last decoding module is input to the region segmentation output module to obtain the aquaculture region segmentation result output by the region segmentation output module.

[0071] This invention, through the above-described scheme, involves acquiring a remote sensing image to be processed; inputting the image into a nearshore aquaculture area segmentation model to obtain aquaculture area segmentation results output by the model; wherein the nearshore aquaculture area segmentation model includes an encoding module and a decoding module with a joint attention mechanism; the model is obtained through iterative training based on pre-collected remote sensing image samples and their image labels. This achieves the extraction of high-dimensional feature maps containing hidden complex features through an encoding module constructed using the EfficientNet-B0 network, and the introduction of a joint attention mechanism during decoding highlights detailed information such as the spatial location and morphological features of nearshore aquaculture farms, effectively improving the accuracy of aquaculture area segmentation.

[0072] In one embodiment, step 12 above: inputting the remote sensing image to be processed into the nearshore aquaculture area segmentation model to obtain the aquaculture area segmentation result output by the nearshore aquaculture area segmentation model, including:

[0073] Step 121: Input the remote sensing image to be processed into the first convolutional layer to perform convolution processing on the remote sensing image to be processed using the first convolutional layer to obtain a convolutional feature map.

[0074] It should be noted that the first convolution is a convolutional layer with a kernel size of 3×3.

[0075] For example, if the input image size of the remote sensing image to be processed is 512×512×3, the first convolutional layer is used to perform convolution processing on the remote sensing image to be processed to obtain a convolutional feature map with a size of 512×512×1.

[0076] Step 122: Input the convolutional feature map into the first coding unit to obtain the coding feature map output by the first coding unit, and use the coding feature map as the input of the next coding unit until the coding feature map output by the last coding unit is obtained;

[0077] Figure 3 This is a schematic diagram of the coding module in the nearshore aquaculture zone segmentation model provided by the present invention, as shown below. Figure 3 As shown, the first encoding unit includes one MBConv1, k3×3 network unit and two MBConv6, k3×3 network units; the second encoding unit includes two MBConv6, k5×5 network units; the third encoding unit includes three MBConv6, k3×3 network units; the fourth encoding unit includes three MBConv6, k3×3 network units and three MBConv6, k5×5 network units; and the fifth encoding unit includes one MBConv6, k3×3 network unit. Here, 1 and 6 represent scaling factors, and k3×3 and k3×3 represent the size of the convolutional kernel. The MBConv structure in the encoding unit is constructed by expanding coefficients, depthwise separable convolutional layers, and Squeeze-and-Excitation modules.

[0078] Specifically, the convolutional feature map is input to the first coding unit to perform convolution processing using each MBConv6 convolutional structure in the first coding unit, thereby obtaining the coding feature map output by the first coding unit. The coding feature map is then used as the input to the next coding unit until the coding feature map output by the last coding unit is obtained.

[0079] Understandably, the size of the input convolutional feature map is 512×512×1, and the size of the encoded feature map obtained after processing by the first encoding unit is 256×256×32. The 256×256×32 feature map is then used as the input of the next encoding unit, and so on, until the encoded feature map output by the last encoding unit is obtained.

[0080] Step 123: Input the encoded feature map output by the last encoding unit and the encoded feature map output by the encoding unit at the same depth level as the first decoding unit into the feature connection layer in the first decoding unit to obtain the first connection feature map output by the feature connection layer in the first decoding unit;

[0081] Specifically, the coding feature map output by the coding unit at the same depth level as the current decoding unit is determined, and the coding feature map output by the last coding unit and the coding feature map output by the coding unit at the same depth level are connected through the feature connection layer in the first decoding unit to obtain a first connection feature map.

[0082] Understandably, understandably, refer to Figure 2 The fourth encoding unit is at the same depth level as the first decoding unit. Therefore, the output of the first convolutional layer in the fourth encoding unit is directly connected to the input of the feature connection layer in the first decoding unit. The last encoding unit outputs an encoded feature map with a size of 32×32×320, and the fourth encoding unit outputs an encoded feature map with a size of 32×32×112. After feature concatenation, a feature map with a size of 32×32×432 is obtained.

[0083] Step 124: The first connection feature map is convolved by the second convolutional layer and the joint attention mechanism layer in the first decoding unit to obtain the decoded feature map output by the first decoding unit, and the decoded feature map output by the first decoding unit is used as the input of the next decoding unit.

[0084] It should be noted that the joint attention mechanism layer includes a channel attention module and a spatial attention module. The channel attention module contains two channel attention methods: global max pooling and global average pooling. These methods are passed through two fully connected layers with shared weights to obtain two adjusted vectors, i.e., the channel attention vectors. They first undergo element-wise addition and finally pass through a sigmoid activation function to obtain the channel attention feature map.

[0085] Spatial Attention Module: This module compresses spatial activation channels by performing max pooling and average pooling along the channel dimension to obtain two spatial attention points. These two spatial attention points are then connected to form a feature map with a channel dimension of 2. The convolutional module then compresses this feature map into a single channel, and finally, it passes through the sigmoid activation function to obtain the spatial attention feature map.

[0086] Specifically, the first connection feature map is input into the second convolutional layer in the first decoding unit to obtain the target convolutional feature map output by the second convolutional layer. Then, the target convolutional feature map is input into the channel attention module in the joint attention mechanism layer to obtain the channel attention feature map output by the channel attention module. Finally, the channel attention feature map is multiplied by the target convolutional feature map to obtain the channel-corrected feature map. Further, the channel-corrected feature map is input into the spatial attention module to obtain the spatial attention feature map output by the spatial attention module. The spatial attention feature map is multiplied by the channel-corrected feature map to obtain the channel- and spatially corrected feature maps. Even further, the second connection feature map is convolved by the second convolutional layer and the joint attention mechanism layer in the first decoding unit, and the channel- and spatially corrected feature maps are convolved again to obtain the decoded feature map.

[0087] Step 125: The decoded feature map output by the first decoding unit is upsampled through the upsampling layer in the next decoding unit to obtain an upsampled feature map;

[0088] Specifically, the decoded feature map output by the first decoding unit is upsampled by the upsampling layer in the next decoding unit to obtain an upsampled feature map.

[0089] Understandably, following the example of 123 above, the size of the decoded feature map output by the first decoding unit is 32×32×432, and after upsampling, a feature map with a size of 64×64×432 is obtained.

[0090] Step 126: The upsampled feature map and the encoded feature map output by the encoding unit at the same depth level as the next decoding unit are connected through the feature connection layer in the next decoding unit to obtain a second connected feature map;

[0091] It should be noted that the process of feature connection in this embodiment is basically the same as the process of feature connection in step 121 above, and will not be repeated here.

[0092] Step 127: Perform convolution processing on the second connected feature map through the second convolutional layer and the joint attention mechanism layer in the next decoding unit to obtain the decoded feature map output by the next decoding unit; return to the execution step: perform upsampling processing on the decoded feature map output by the first decoding unit through the upsampling layer in the next decoding unit to obtain the upsampled feature map; until the decoded feature map output by the last decoding module is obtained;

[0093] It should be noted that the convolution processing in this embodiment, using the second convolutional layer and the joint attention mechanism layer, is essentially the same as the convolution processing in step 124 above, and will not be described again here.

[0094] Step 128: Input the decoded feature map of the last decoding module into the region segmentation output module to obtain the aquaculture region segmentation result output by the region segmentation output module.

[0095] Specifically, the decoded feature map of the last decoding module is input to the region segmentation output module, so that the convolutional layer in the region segmentation output module performs a convolution operation on the decoded feature map of the last decoding module to obtain the aquaculture region segmentation result output by the region segmentation output module.

[0096] Table 1

[0097] method Feature extraction network Image segmentation evaluation metrics (%) PSPNet model resnet34 0.9478 Deeplabv3+ model resnet34 0.9527 Unet model resnet34 0.9528 Nearshore aquaculture zone segmentation model efficientnet-b0 0.9576

[0098] Referring to Table 1, the evaluation metrics of image segmentation using different models are compared. Among them, the evaluation metric of image segmentation using the nearshore aquaculture area segmentation model of this invention is the highest, proving that the nearshore aquaculture area segmentation model has higher accuracy in segmenting aquaculture areas.

[0099] The embodiments of the present invention, through the above-described scheme, achieve the extraction of high-dimensional feature maps containing hidden complex feature information by an encoding module constructed by an efficientnet-b0 network. Furthermore, by introducing channel attention and spatial attention mechanisms during the decoding process, detailed information such as the spatial location and morphological features of nearshore aquaculture farms can be highlighted, effectively improving the accuracy of aquaculture area segmentation.

[0100] In one embodiment, the nearshore aquaculture area segmentation model is trained based on the following steps:

[0101] Step A10: Obtain the remote sensing image sample set;

[0102] Step A20: Perform data augmentation processing on the remote sensing image sample set to obtain multiple augmented remote sensing images;

[0103] Step A30: The augmented remote sensing images are stitched together to obtain a stitched remote sensing image;

[0104] Step A40: The synthesized remote sensing image is cropped to obtain several remote sensing image slices, wherein there is no overlap between the remote sensing image slices;

[0105] Step A50: Based on each remote sensing image slice and the image label corresponding to each remote sensing image slice, the initial model to be trained is iteratively trained to obtain the nearshore aquaculture area segmentation model.

[0106] It should be noted that the remote sensing image sample set includes several remote sensing images with image tags. The image tags are information used to mark whether a pixel in the remote sensing image belongs to an aquaculture area. For example, a pixel belonging to an aquaculture area is marked as 1, and a pixel not belonging to an aquaculture area is marked as 0.

[0107] It should be further noted that data augmentation processing includes methods such as horizontal flipping, vertical flipping, rotation from 90° to 270°, and multi-scale resolution (1m-3m resolution images) image augmentation.

[0108] Specifically, a remote sensing image sample set is acquired, and the remote sensing image sample set is subjected to horizontal flipping, vertical flipping, rotation from 90° to 270°, and multi-scale resolution (1m-3m resolution images) image augmentation to obtain multiple augmented remote sensing images. It should be noted that, due to the overlapping parts between the acquired SAR remote sensing image sets, the training set and validation set obtained based on the SAR remote sensing image set contain the same images. This results in higher segmentation accuracy of the model on the validation set during the model validation stage, which may lead to the final nearshore aquaculture area segmentation model not being the optimal model, and consequently, lower image segmentation accuracy during the application stage. Therefore, in this embodiment, the augmented remote sensing images are stitched together to obtain a complete high-resolution stitched remote sensing image. The composite remote sensing image is then cropped to obtain several remote sensing image slices without overlapping parts. These slices are then divided into a training set and a validation set according to a preset ratio. This preset ratio can be set according to actual conditions; preferably, a 4:1 ratio is used. Further, based on the training set and its corresponding image labels, the initial model to be trained is iteratively trained to obtain a training model. The validation set is then input into the training model to verify its generalization ability. For example, the MSE (mean square error), MAE (mean absolute error), and RMSE (root mean square error) formulas can be used for verification. The training model with the best verification results is then selected as the nearshore aquaculture area segmentation model.

[0109] The embodiments of the present invention, through the above-described scheme, realize the method of cropping a remote sensing image sample set into several remote sensing image slices without overlapping parts, and then training and validating the model based on the remote sensing image slices without overlapping parts, thereby solving the problem of inaccurate accuracy evaluation during model training and improving the accuracy of model segmentation.

[0110] In one embodiment, step A50 above: based on each of the remote sensing image slices and the image labels corresponding to each of the remote sensing image slices, iteratively train the initial model to be trained to obtain the nearshore aquaculture area segmentation model, including:

[0111] Step A51: For any remote sensing image slice, input the remote sensing image slice into the initial model to be trained, and obtain the prediction result output by the initial model to be trained;

[0112] Step A52: Based on the prediction results and the image labels corresponding to the remote sensing image slices, calculate the model loss value;

[0113] Specifically, step A52 includes:

[0114] The calculation of the model loss value based on the prediction result and the image label corresponding to the remote sensing image tile includes:

[0115] Step A521: Based on the prediction result, the image label corresponding to each pixel position in the remote sensing image slice, and the pre-set weight coefficient, the first loss value is calculated by the pre-set boundary cross-entropy loss function.

[0116] Step A522: Based on the prediction result and the image label corresponding to each pixel position in the remote sensing image slice, a second loss value is calculated using a pre-set similarity loss function;

[0117] Step A523: Calculate the model loss value based on the first loss value and the second loss value.

[0118] Step A53: Based on the model loss value obtained in each iteration, update the parameters of the initial model to be trained to obtain the nearshore aquaculture area segmentation model.

[0119] It should be noted that, since the image tags are used to mark whether pixels in remote sensing images belong to aquaculture areas, the pixels corresponding to the boundary positions of aquaculture areas can be determined based on the image tags corresponding to each similar point.

[0120] Specifically, for any remote sensing image slice, the remote sensing image slice is input into the initial model to be trained to obtain the prediction result output by the initial model to be trained. Since the boundary sample features of near-shore aquaculture areas are complex and difficult to extract boundary region features, in this embodiment, based on the prediction result and the image labels corresponding to each pixel position in the remote sensing image slice, a boundary cross-entropy loss function and weight coefficients are designed to calculate the first loss value. Furthermore, to alleviate the problem of imbalanced positive and negative sample numbers, in this embodiment, a Dice similarity loss function is set to calculate the second loss value between the prediction result and the image labels corresponding to each pixel position in the remote sensing image slice. Then, the first loss value and the second loss value are added together, and the result is used as the model loss value. The expression for the boundary cross-entropy loss function is:

[0121]

[0122] The expression for the similarity loss function is:

[0123]

[0124] Loss = Loss1 + Loss2

[0125] Where Loss represents the model loss value, Loss1 represents the first loss value, Loss2 represents the second loss value, N represents the number of pixels, yi represents the image label corresponding to pixel i, and p i Let represent the prediction result corresponding to pixel i, and W represent the weight coefficient. The weight coefficient can be set according to actual needs. In this embodiment, preferably, the weight coefficient of the pixel corresponding to the boundary position is set to 4, and the weight coefficient of the pixel corresponding to the non-boundary position is set to 1.

[0126] Based on this, in other embodiments, the loss function can be set according to actual needs, and no specific limitation is made here. After calculating the model loss value, the current training process ends, and the model parameters in the initial model to be trained are updated using the error backpropagation algorithm, and then the next training is performed. During the training process, it is determined whether the updated initial model to be trained meets the preset training termination conditions. If it does, the updated initial model to be trained is used as the nearshore aquaculture area segmentation model; if it does not, the model training continues. The preset training termination conditions include loss convergence and reaching the maximum number of iterations threshold, etc.

[0127] The embodiments of the present invention, through the above-described method, achieve multi-task learning of nearshore aquaculture areas and boundary areas by using a joint loss function composed of boundary cross-entropy loss function and DICE loss function, and by increasing the boundary weight of nearshore aquaculture areas, thereby improving the extraction accuracy of the entire nearshore aquaculture area.

[0128] The region segmentation apparatus provided by the present invention is described below. The region segmentation apparatus described below and the region segmentation method described above can be referred to in correspondence.

[0129] Figure 4 A schematic diagram of the region segmentation device provided by the present invention is shown below. Figure 4 As shown, an embodiment of the present invention provides a region segmentation device, which includes:

[0130] Acquisition module 41 is used to acquire remote sensing images to be processed;

[0131] The segmentation module 42 is used to input the remote sensing image to be processed into the nearshore aquaculture area segmentation model to obtain the aquaculture area segmentation result output by the nearshore aquaculture area segmentation model;

[0132] The nearshore aquaculture area segmentation model includes an encoding module and a decoding module with a joint attention mechanism; the nearshore aquaculture area segmentation model is obtained through iterative training based on pre-collected remote sensing image samples and the image labels of the remote sensing image samples.

[0133] Optionally, the nearshore aquaculture area segmentation model further includes a region segmentation output module; the encoding module includes a first convolutional layer and multiple cascaded encoding units, each of which is composed of a different number of stacked MBConv convolutional structures; the decoding module includes multiple cascaded decoding units, each of which, except for the first decoding unit, includes a feature connection layer, an upsampling layer, a second convolutional layer, and a joint attention mechanism layer, the first decoding unit including a feature connection layer, a second convolutional layer, and a joint attention mechanism layer; the output of the last decoding module serves as the input of the region segmentation output module;

[0134] The segmentation module 42 is also used for:

[0135] The remote sensing image to be processed is input into the first convolutional layer to perform convolution processing on the remote sensing image to be processed, so as to obtain a convolutional feature map.

[0136] The convolutional feature map is input into the first coding unit to obtain the coding feature map output by the first coding unit, and the coding feature map is used as the input of the next coding unit until the coding feature map output by the last coding unit is obtained;

[0137] The encoded feature map output by the last encoding unit and the encoded feature map output by the encoding unit at the same depth level as the first decoding unit are input into the feature connection layer in the first decoding unit to obtain the first connection feature map output by the feature connection layer in the first decoding unit.

[0138] The first connection feature map is convolved by the second convolutional layer and the joint attention mechanism layer in the first decoding unit to obtain the decoded feature map output by the first decoding unit, and the decoded feature map output by the first decoding unit is used as the input of the next decoding unit.

[0139] The upsampling layer in the next decoding unit upsamples the decoded feature map output by the first decoding unit to obtain an upsampled feature map.

[0140] The upsampled feature map and the encoded feature map output by the encoding unit at the same depth level as the next decoding unit are connected through the feature connection layer in the next decoding unit to obtain a second connected feature map;

[0141] The second connected feature map is convolved by the second convolutional layer and the joint attention mechanism layer in the next decoding unit to obtain the decoded feature map output by the next decoding unit.

[0142] Return to execution steps: Upsample the decoded feature map output by the first decoding unit through the upsampling layer in the next decoding unit to obtain an upsampled feature map; until the decoded feature map output by the last decoding module is obtained;

[0143] The decoded feature map of the last decoding module is input into the region segmentation output module to obtain the aquaculture region segmentation result output by the region segmentation output module.

[0144] Optionally, the region segmentation device further includes:

[0145] Acquire a remote sensing image sample set;

[0146] Data augmentation processing is performed on the remote sensing image sample set to obtain multiple augmented remote sensing images;

[0147] The augmented remote sensing images are stitched together to obtain a stitched remote sensing image.

[0148] The synthesized remote sensing image is cropped to obtain several remote sensing image slices, wherein there is no overlap between the remote sensing image slices;

[0149] Based on each remote sensing image slice and its corresponding image label, the initial model to be trained is iteratively trained to obtain the nearshore aquaculture area segmentation model.

[0150] Optionally, the region segmentation device further includes:

[0151] For any remote sensing image slice, the remote sensing image slice is input into the initial model to be trained, and the prediction result output by the initial model to be trained is obtained.

[0152] Based on the prediction results and the image labels corresponding to the remote sensing image slices, the model loss value is calculated.

[0153] Based on the model loss value obtained in each iteration, the parameters of the initial model to be trained are updated to obtain the nearshore aquaculture area segmentation model.

[0154] Optionally, the region segmentation device further includes:

[0155] Based on the prediction results, the image labels corresponding to the positions of each pixel in the remote sensing image slice, and the pre-set weight coefficients, the first loss value is calculated by the pre-set boundary cross-entropy loss function.

[0156] Based on the prediction results and the image labels corresponding to the positions of each pixel in the remote sensing image slice, a second loss value is calculated using a pre-set similarity loss function.

[0157] The model loss value is calculated based on the first loss value and the second loss value.

[0158] Optionally, the region segmentation device further includes:

[0159] The expression for the boundary cross-entropy loss function is:

[0160]

[0161] The expression for the similarity loss function is:

[0162]

[0163] Where Loss1 represents the first loss value, Loss2 represents the second loss value, W represents the weight coefficient, N represents the number of pixels, and y i pi represents the image label corresponding to pixel i, and pi represents the prediction result corresponding to pixel i.

[0164] It should be noted that the apparatus provided in this embodiment of the invention can implement all the method steps implemented in the above method embodiment and can achieve the same technical effect. Therefore, the parts that are the same as those in the method embodiment and the beneficial effects will not be described in detail here.

[0165] Figure 5 This is a schematic diagram of the electronic device provided by the present invention. The electronic device may include: a processor 510, a memory 520, a communication interface 530, and a communication bus 540, wherein the processor 510, the memory 520, and the communication interface 530 communicate with each other through the communication bus 540. The processor 510 can call logical instructions in the memory 520 to execute a region segmentation method, which includes: acquiring a remote sensing image to be processed; inputting the remote sensing image to be processed into a nearshore aquaculture area segmentation model to obtain aquaculture area segmentation result output by the nearshore aquaculture area segmentation model; wherein the nearshore aquaculture area segmentation model includes an encoding module and a decoding module with a joint attention mechanism; the nearshore aquaculture area segmentation model is obtained through iterative training based on pre-collected remote sensing image samples and the image labels of the remote sensing image samples.

[0166] Furthermore, the logical instructions in the aforementioned memory 520 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0167] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the region segmentation method provided by the above methods. The method includes: acquiring a remote sensing image to be processed; inputting the remote sensing image to be processed into a nearshore aquaculture area segmentation model to obtain an aquaculture area segmentation result output by the nearshore aquaculture area segmentation model; wherein the nearshore aquaculture area segmentation model includes an encoding module and a decoding module with a joint attention mechanism; the nearshore aquaculture area segmentation model is obtained through iterative training based on pre-collected remote sensing image samples and image labels of the remote sensing image samples.

[0168] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the region segmentation method provided by the above methods. The method includes: acquiring a remote sensing image to be processed; inputting the remote sensing image to be processed into a nearshore aquaculture area segmentation model to obtain aquaculture area segmentation result output by the nearshore aquaculture area segmentation model; wherein the nearshore aquaculture area segmentation model includes an encoding module and a decoding module with a joint attention mechanism; the nearshore aquaculture area segmentation model is obtained by iterative training based on pre-collected remote sensing image samples and image labels of the remote sensing image samples.

[0169] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0170] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0171] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A region segmentation method characterized by, The method comprises: acquiring a to-be-processed remote sensing image; inputting the to-be-processed remote sensing image into a near-shore aquaculture area segmentation model to obtain an aquaculture area segmentation result output by the near-shore aquaculture area segmentation model; wherein the near-shore aquaculture area segmentation model comprises an encoding module and a decoding module with joint attention mechanism; the near-shore aquaculture area segmentation model is obtained by iterative training based on pre-collected remote sensing image samples and image labels of the remote sensing image samples; wherein the near-shore aquaculture area segmentation model further comprises an area segmentation output module; the encoding module comprises a first convolutional layer and a plurality of cascaded encoding units, each of the encoding units is composed of a different number of stacked MBConv convolutional structures; the decoding module comprises a plurality of cascaded decoding units, each decoding unit except the first decoding unit comprises a feature connection layer, an up-sampling layer, a second convolutional layer and a joint attention mechanism layer, and the first decoding unit comprises a feature connection layer, a second convolutional layer and a joint attention mechanism layer; the output of the last decoding module is input into the area segmentation output module; wherein the near-shore aquaculture area segmentation model is obtained by training based on the following steps: acquiring a set of remote sensing image samples; performing data enhancement processing on the set of remote sensing image samples to obtain a plurality of expanded remote sensing images; splicing each of the expanded remote sensing images to obtain a spliced remote sensing image; cropping the spliced remote sensing image to obtain a plurality of remote sensing image slices, wherein there is no overlapping part between each of the remote sensing image slices; based on each of the remote sensing image slices and the image labels corresponding to each of the remote sensing image slices, iteratively training a to-be-trained initial model to obtain the near-shore aquaculture area segmentation model.

2. The region segmentation method according to claim 1, characterized in that, the inputting of the to-be-processed remote sensing image into the near-shore aquaculture area segmentation model to obtain the aquaculture area segmentation result output by the near-shore aquaculture area segmentation model comprises: inputting the to-be-processed remote sensing image into the first convolutional layer to perform convolutional processing on the to-be-processed remote sensing image by using the first convolutional layer to obtain a convolutional feature map; inputting the convolutional feature map into the first encoding unit to obtain an encoding feature map output by the first encoding unit, and taking the encoding feature map as the input of the next encoding unit until the encoding feature map output by the last encoding unit is obtained; inputting the encoding feature map output by the last encoding unit and the encoding feature map output by the encoding unit at the same depth level as the first decoding unit into the feature connection layer in the first decoding unit to obtain a first connection feature map output by the feature connection layer in the first decoding unit; performing convolutional processing on the first connection feature map by using the second convolutional layer and the joint attention mechanism layer in the first decoding unit to obtain a decoding feature map output by the first decoding unit, and taking the decoding feature map output by the first decoding unit as the input of the next decoding unit; performing up-sampling processing on the decoding feature map output by the first decoding unit by using the up-sampling layer in the next decoding unit to obtain an up-sampled feature map; feature connection is performed between the up-sampling feature map and an encoded feature map output by an encoding unit at a same depth level as the next decoding unit through a feature connection layer in the next decoding unit, to obtain a second connection feature map; a decoding feature map output by the next decoding unit is obtained by performing convolution processing on the second connection feature map through a second convolution layer and a joint attention mechanism layer in the next decoding unit; the step of performing up-sampling processing on the decoding feature map output by the first decoding unit through an up-sampling layer in the next decoding unit is returned to be executed until a decoding feature map output by a last decoding module is obtained; the decoding feature map of the last decoding module is input into the region segmentation output module to obtain a mariculture region segmentation result output by the region segmentation output module.

3. The region segmentation method according to claim 2, characterized in that, The offshore mariculture area segmentation model is obtained by iteratively training the initial model to be trained based on each of the remote sensing image slices and the image labels corresponding to each of the remote sensing image slices, including: For any one remote sensing image slice, the remote sensing image slice is input into the initial model to be trained to obtain a prediction result output by the initial model to be trained; a model loss value is calculated based on the prediction result and the image label corresponding to the remote sensing image slice; The offshore mariculture area segmentation model is obtained by iteratively training the initial model to be trained based on each of the remote sensing image slices and the image labels corresponding to each of the remote sensing image slices, including:

4. The region segmentation method according to claim 3, characterized in that, A first loss value is calculated based on the prediction result, the image label corresponding to each pixel position in the remote sensing image slice, and a pre-set weight coefficient through a pre-set boundary cross-entropy loss function; A second loss value is calculated based on the prediction result and the image label corresponding to each pixel position in the remote sensing image slice through a pre-set similarity loss function; The model loss value is calculated based on the first loss value and the second loss value.

5. The region segmentation method according to claim 4, wherein The expression of the boundary cross-entropy loss function is: The expression of the similarity loss function is: ； including: ； wherein, represents a first loss value, represents a second loss value, represents a weight coefficient, N represents a number of pixel points, y i represents an image label corresponding to the i-th pixel point, p i represents a prediction result corresponding to the i-th pixel point.

6. A region dividing apparatus characterized by comprising: an acquisition module configured to acquire a remote sensing image to be processed; a segmentation module configured to input the remote sensing image to be processed into an offshore mariculture area segmentation model to obtain a mariculture region segmentation result output by the offshore mariculture area segmentation model; The offshore mariculture area segmentation model includes an encoding module and a decoding module with a joint attention mechanism, and is obtained by iteratively training based on pre-collected remote sensing image samples and image labels of the remote sensing image samples. ​ The offshore aquaculture area segmentation model further comprises a region segmentation output module; the encoding module comprises a first convolutional layer and a plurality of cascaded encoding units, each of the encoding units being composed of a stack of MBConv convolutional structures with different numbers; the decoding module comprises a plurality of cascaded decoding units, each of the decoding units except the first decoding unit comprising a feature connection layer, an up-sampling layer, a second convolutional layer and a joint attention mechanism layer, and the first decoding unit comprising a feature connection layer, a second convolutional layer and a joint attention mechanism layer; and an output of the last decoding module is taken as an input of the region segmentation output module. The region segmentation device is further configured to: obtain a remote sensing image sample set; perform data enhancement processing on the remote sensing image sample set to obtain a plurality of expanded remote sensing images; splice the expanded remote sensing images to obtain spliced remote sensing images; cut the spliced remote sensing images to obtain a plurality of remote sensing image slices, wherein there is no overlapping part between the remote sensing image slices; based on the remote sensing image slices and the image labels corresponding to the remote sensing image slices respectively, perform iterative training on the initial model to be trained to obtain the offshore aquaculture area segmentation model.

7. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the region segmentation method according to any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the region segmentation method according to any one of claims 1 to 5.

Images