Small farmland image segmentation method and device based on double attention mechanism

Abstract

The application relates to a small farmland image segmentation method and device based on a double attention mechanism. The method comprises the following steps: acquiring a farmland image to be segmented; uniformly cutting the size of a collected farmland image to be segmented; inputting the farmland image to be segmented with a uniform size into a reconstructed Mask R-CNN model to obtain an instance segmentation result of the farmland image; wherein the main structure and the double attention mechanism feature pyramid in the reconstructed Mask R-CNN model are used for image feature extraction, a region proposal network is used to generate a plurality of regions of interest, the regions of interest are mapped to generate a fixed-size feature map through RoIAlign, and the three branches of the head of the Mask R-CNN model are used for prediction to obtain an object category, a refined bounding box positioning and an instance segmentation result. The method can realize accurate identification and instance segmentation of small farmland in a remote sensing image.

Description

Technical Field

[0001] This application relates to the fields of deep learning and image processing, and in particular to a method and apparatus for segmenting small farmland images based on a dual attention mechanism. Background Technology

[0002] Statistics show that 80% of food production in Asia and sub-Saharan Africa comes from small-scale farms, and approximately 90% of the world's farmers are smallholders, owning less than two hectares of land. Small-scale farms play a crucial role in maintaining global food security. Therefore, accurately mapping and detecting the spatial distribution of small-scale farms is essential for increasing food production and ensuring global food security.

[0003] With the development of image detection and segmentation technologies, instance segmentation devices and methods for farmland remote sensing images have emerged. Instance segmentation of farmland remote sensing images can be divided into two methods: manual survey and intelligent segmentation. Manual survey is very time-consuming and inefficient, and prone to human error. Therefore, intelligent methods are often used for instance segmentation of farmland remote sensing images. Intelligent methods generally use deep learning models, such as the Mask R-CNN model and its partially modified versions, for image instance segmentation. However, currently, these methods are only suitable for relatively large farmland remote sensing images. Large farmland areas in remote sensing images correspond to large regions, while small farmland areas correspond to small regions. During feature extraction by deep learning models, feature information is easily lost, causing small farmland areas to disappear from the features extracted by the deep learning model, resulting in poor instance segmentation performance for small farmland areas using deep learning. Summary of the Invention

[0004] Therefore, it is necessary to provide a small-scale farmland image segmentation method and apparatus based on a dual attention mechanism that can improve the farmland segmentation effect, in order to address the above-mentioned technical problems.

[0005] A method for segmenting small farmland images based on a dual attention mechanism, the method comprising:

[0006] Obtain the image of the farmland to be segmented;

[0007] The collected farmland images to be segmented are cropped to a uniform size to obtain farmland images to be segmented of a uniform size;

[0008] The uniform-sized farmland images to be segmented are input into the modified Mask R-CNN model to obtain the instance segmentation results of the farmland images. Specifically, the modified Mask R-CNN model uses the backbone structure and the Dual Attention Feature Pyramid Network (DAFPN) to extract image features. Then, a number of Regions of Interest (ROIs) are generated through the Region Proposal Network (RPN). The RoIAlign is then used to map the ROIs to generate fixed-size feature maps. Finally, the three branches of the modified Mask R-CNN model head are used to predict the object category, the refined bounding box localization, and the segmentation mask. The segmentation mask is the instance segmentation result of the farmland image.

[0009] In one embodiment, the uniformly sized farmland image to be segmented is input into the modified Mask R-CNN model to obtain instance segmentation results of the farmland image, including:

[0010] After uniformly cropping the farmland images to be segmented, they are input into the backbone structure and dual attention mechanism feature pyramid of the modified Mask R-CNN model to extract feature maps;

[0011] The feature map is input into the region proposal network, which uses a sliding window method on the feature map to find regions where the target may exist, thereby obtaining the region of interest.

[0012] Input the region of interest into RoIAlign, and RoIAlign will use a bilinear interpolation algorithm to map the region of interest of any size in the features to generate a feature map of a fixed size.

[0013] Fixed-size feature maps are predicted using three branches of the modified Mask R-CNN model head to obtain object category, refined bounding box localization, and instance segmentation results.

[0014] In one embodiment, the dual-attention mechanism feature pyramid used to modify the Mask R-CNN model includes a spatial attention module and a channel attention module. The spatial attention module is used to emphasize features at the spatial scale, and the channel attention module is used to emphasize features at the channel scale.

[0015] The expression for the operation process of the spatial attention module is as follows:

[0016]

[0017]

[0018]

[0019]

[0020] in, , Given two adjacent feature maps; For connection operation; For channel compression Convolutional layer for Convolutional layer; This is an upsampling operation; , This represents the process quantity obtained after average pooling and max pooling operations, and , ; It is the sigmoid activation function; This is an average pooling operation; This is a max pooling operation; Spatial attention coefficient, This is the feature map after multiplying by the spatial attention coefficient. For spatial attention, Number the feature maps. This is element-wise multiplication;

[0021] The expression for the operation process of the channel attention module is as follows:

[0022]

[0023]

[0024]

[0025]

[0026] in, , Given two adjacent feature maps; Indicates a connection operation; Indicates compressed channel Convolutional layer express Convolutional layer with a stride of 2; Indicates a fully connected layer; , This refers to the process quantities obtained after average pooling and max pooling operations; This represents the sigmoid activation function; Channel attention coefficient; This is the feature map after multiplying by the channel attention coefficient; For spatial attention, Number the feature map.

[0027] In one embodiment, farmland images are acquired and cropped to construct a model training dataset; the model training dataset is then input into the initially modified Mask R-CNN model to train the initially modified Mask R-CNN model, resulting in a trained modified Mask R-CNN model.

[0028] The modified Mask R-CNN model uses ResNet50 as its backbone, consisting of five stages corresponding to feature maps at five different scales. Feature map The corresponding step size is Pixel; Feature Map The feature pyramids used to build the dual-attention mechanism feature pyramid network are used to obtain new features. The modified Mask R-CNN model replaces the feature pyramid network in the original model with a dual attention mechanism feature pyramid network.

[0029] In one embodiment, the loss function of the modified Mask R-CNN model is a combined loss function, which includes the classification loss of the candidate boxes, the location regression loss, and the object mask loss. The loss calculation formula is as follows:

[0030]

[0031] The classification loss of the candidate boxes is calculated using the following formula:

[0032]

[0033] in, The index of the region of interest. The serial number is The probability that the region of interest is predicted as a positive sample. Represents the normalization parameter. This indicates that the region of interest is a negative sample. The region of interest is considered a positive sample.

[0034] The location regression loss is calculated using the following formula:

[0035]

[0036]

[0037] in, For normalization parameters, It is the predicted offset parameter. This is the actual offset parameter. and These represent regions of interest as positive and negative samples, respectively. yes loss, Indicates smoothed ;when When the absolute value is less than 1, the value used is , The upper half of the piecewise function; when When the absolute value is greater than or equal to 1, the value used is Translation, The lower half of the piecewise function; This represents the numerical difference between the predicted bounding box and the ground truth bounding box;

[0038] The target mask loss is represented by the following formula:

[0039]

[0040] Where m represents the size of the feature map, k represents the number of feature maps, and v represents... Corresponding coordinate position , For the target true label value, These are the predicted values ​​from the modified Mask R-CNN model.

[0041] In one embodiment, the farmland image to be segmented is an ultra-high resolution farmland remote sensing satellite image;

[0042] The collected farmland images to be segmented are cropped to a uniform size to obtain farmland images to be segmented of a uniform size. This includes cropping ultra-high resolution farmland remote sensing satellite images to a uniform size to obtain farmland images to be segmented of a uniform size; wherein, the size of the farmland images to be segmented of a uniform size is 1024×1024 pixels.

[0043] In one embodiment, the modified Mask R-CNN model includes: a backbone structure and a dual attention mechanism feature pyramid for extracting features from the entire image and generating features at different scales; a region proposal network for generating regions of interest; RoIAlign for mapping regions of interest of arbitrary size from the features to a fixed-size feature map using bilinear interpolation; and three branches in the head of the modified Mask R-CNN model for performing predictions to obtain object category, bounding box localization, and instance segmentation results.

[0044] A small-scale farmland image segmentation device based on a dual attention mechanism, the device comprising:

[0045] The image acquisition module is used to acquire images of farmland to be segmented.

[0046] The preprocessing module for the images to be segmented crops the collected farmland images to be segmented to a uniform size, thereby obtaining farmland images to be segmented of a uniform size.

[0047] The target calculation module is used to input the uniform-sized farmland image to be segmented into the modified Mask R-CNN model to obtain the instance segmentation result of the farmland image. Specifically, the modified Mask R-CNN model uses the backbone structure and dual attention mechanism feature pyramid in the modified Mask R-CNN model to extract image features, then generates several regions of interest through the region proposal network, and then uses RoIAlign to map the regions of interest to generate a fixed-size feature map. Finally, the three branches of the modified Mask R-CNN model head are used to predict the object category, the refined bounding box localization, and the segmentation mask, where the segmentation mask is the instance segmentation result of the farmland image.

[0048] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program performing the following steps:

[0049] Obtain the image of the farmland to be segmented;

[0050] The collected farmland images to be segmented are cropped to a uniform size to obtain farmland images to be segmented of a uniform size;

[0051] The uniform-sized farmland images to be segmented are input into the modified Mask R-CNN model to obtain the instance segmentation results of the farmland images. Specifically, the modified Mask R-CNN model uses the backbone structure and dual attention mechanism feature pyramid to extract image features, then generates several regions of interest through a region proposal network, and then uses RoIAlign to map the regions of interest to generate fixed-size feature maps. Finally, the three branches of the modified Mask R-CNN model head are used for prediction to obtain the object category, the refined bounding box localization, and the instance segmentation results of the farmland images.

[0052] A computer-readable storage medium having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0053] Obtain the image of the farmland to be segmented;

[0054] The collected farmland images to be segmented are cropped to a uniform size to obtain farmland images to be segmented of a uniform size;

[0055] The uniform-sized farmland images to be segmented are input into the modified Mask R-CNN model to obtain the instance segmentation results of the farmland images. Specifically, the modified Mask R-CNN model uses the backbone structure and dual attention mechanism feature pyramid to extract image features, then generates several regions of interest through a region proposal network, and then uses RoIAlign to map the regions of interest to generate fixed-size feature maps. Finally, the three branches of the modified Mask R-CNN model head are used for prediction to obtain the object category, the refined bounding box localization, and the instance segmentation results of the farmland images.

[0056] The aforementioned method, apparatus, computer equipment, and storage medium for small farmland image segmentation based on a dual-attention mechanism, through a modified Mask R-CNN model, employs a dual-attention mechanism feature pyramid, which, compared to other network models, achieves better feature extraction and thus improves segmentation accuracy. It simultaneously enables instance segmentation of small farmland images. Furthermore, it uses the backbone structure and dual-attention mechanism feature pyramid in the modified Mask R-CNN model for image feature extraction, then generates several regions of interest (ROIs) through a region proposal network, and finally maps these ROIs to generate fixed-size feature maps using RoIAlign. Prediction is then performed through the three branches of the modified Mask R-CNN model head, accurately obtaining object categories, refined bounding box localization, and farmland image instance segmentation results. This processing method improves the detection and recognition rate of farmland with different shapes, increases the speed of farmland instance segmentation, and enhances the robustness of the farmland segmentation model by enabling farmland feature learning under different crop environments. Attached Figure Description

[0057] Figure 1 This is a flowchart illustrating a small-scale farmland image segmentation method based on a dual attention mechanism in one embodiment.

[0058] Figure 2 This is a schematic diagram of the modified Mask R-CNN model in one embodiment;

[0059] Figure 3 This is a schematic diagram of a dual-attention feature pyramid network structure in one embodiment;

[0060] Figure 4 This is a structural block diagram of a small farmland image segmentation device based on a dual attention mechanism in one embodiment;

[0061] Figure 5 This is a schematic diagram of the spatial attention module structure;

[0062] Figure 6 This is a schematic diagram of the channel attention module structure;

[0063] Figure 7 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0064] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0065] In one embodiment, such as Figure 1 As shown, a method for segmenting small farmland images based on a dual attention mechanism is provided, including the following steps:

[0066] S110, Obtain the farmland image to be segmented;

[0067] The farmland images to be segmented are acquired through satellite cameras and then sent to a remote server or a backend server for processing.

[0068] S120, The collected farmland images to be segmented are cropped to a uniform size to obtain farmland images to be segmented of a uniform size;

[0069] Among them, the original ultra-high resolution remote sensing satellite images were cropped to a uniform size of 1024×1024 pixels.

[0070] S130, the uniform-sized farmland image to be segmented is input into the modified Mask R-CNN model to obtain the instance segmentation result of the farmland image; wherein, the image features are extracted using the backbone structure and dual attention mechanism feature pyramid in the modified Mask R-CNN model, several regions of interest are generated through the region proposal network, and the regions of interest are mapped to generate fixed-size feature maps through RoIAlign, and then prediction is performed through the three branches of the modified Mask R-CNN model head to obtain the object category, the refined bounding box localization and the farmland image instance segmentation result.

[0071] The modified Mask R-CNN model is deployed on a remote server or backend server. This modified model processes the farmland image to be segmented, obtaining object category, bounding box localization, and instance segmentation results. In this example, the object category is either farmland or non-farmland, the bounding box localization is the circumscribed rectangle of each farmland patch in the image, and the instance segmentation result is the pixel block in the image predicted as farmland.

[0072] In the aforementioned small farmland image segmentation method based on the dual attention mechanism, the modified Mask R-CNN model, compared to other network models, employs a dual attention mechanism feature pyramid, resulting in better feature extraction and improved segmentation accuracy. It also achieves instance segmentation of small farmland images. Furthermore, the modified Mask R-CNN model utilizes its backbone structure and dual attention mechanism feature pyramid for image feature extraction. A region proposal network then generates several regions of interest (ROIs), and RoIAlign maps these ROIs to generate fixed-size feature maps. Finally, predictions are made using the three branches of the modified Mask R-CNN model's head, accurately obtaining object categories, refined bounding box localization, and farmland image instance segmentation results. This processing method improves the detection and recognition rate of farmland with different shapes, increases the speed of farmland instance segmentation, and enhances the robustness of the farmland segmentation model by enabling farmland feature learning under different crop environments.

[0073] In one embodiment, such as Figure 2As shown in the figure, the backbone is the main structure, DAFPN is a dual-attention feature pyramid network, Feature Maps are feature maps, RPN is a region proposal network, conv is a convolution operation, softmax is an activation function, bbox reg is bounding box regression, Proposal is the region of interest, ROIAlign is the region of interest alignment, three branches are three branches, head is the head, Fully Convolutional Nets are fully convolutional neural networks, Mask is a segmentation mask, FC layer is a fully connected layer, coordinates is bounding box localization, and category is the classification category. The process involves inputting the uniformly sized farmland images to be segmented into the modified Mask R-CNN model to obtain instance segmentation results. This includes: uniformly cropping the ultra-high-resolution farmland images to be segmented to a size of 1024×1024, and then inputting them into the backbone structure and dual-attention mechanism feature pyramid of the modified Mask R-CNN model to extract feature maps; inputting the feature maps into a region proposal network, which uses a sliding window method to find regions where objects may exist on the feature maps, thereby obtaining regions of interest; inputting the regions of interest into RoIAlign, which uses a bilinear interpolation algorithm to map regions of interest of arbitrary sizes in the features to generate fixed-size feature maps; and using the fixed-size feature maps to predict the object category, refined bounding box localization, and instance segmentation results through the three branches of the modified Mask R-CNN model head.

[0074] In this study, two attention modules—spatial attention module and channel attention module—were designed into the feature pyramid network of the existing Mask R-CNN model, forming a dual-attention mechanism feature pyramid network. The dual-attention mechanism feature pyramid network can be easily inserted into the existing feature pyramid network structure to enhance feature learning during training.

[0075] In one embodiment, such as Figure 3 As shown, the dual-attention mechanism feature pyramid includes a spatial attention module and a channel attention module. The spatial attention module is used to emphasize features at the spatial scale, and the channel attention module is used to emphasize features at the channel scale.

[0076] The expression for the operation process of the spatial attention module is as follows:

[0077]

[0078]

[0079]

[0080]

[0081] in, , Given two adjacent feature maps; For connection operation; For channel compression Convolutional layer for Convolutional layer; This is an upsampling operation; , This represents the process quantity obtained after average pooling and max pooling operations, and , ; It is the sigmoid activation function; This is an average pooling operation; This is a max pooling operation; Spatial attention coefficient, This is the feature map after multiplying by the spatial attention coefficient. For spatial attention, Number the feature maps. This is element-wise multiplication;

[0082] The expression for the operation process of the channel attention module is as follows:

[0083]

[0084]

[0085]

[0086] in, , Given two adjacent feature maps; Indicates a connection operation; Indicates compressed channel Convolutional layer express Convolutional layer with a stride of 2; Indicates a fully connected layer; , This refers to the process quantities obtained after average pooling and max pooling operations; This represents the sigmoid activation function; Channel attention coefficient; This is the feature map after multiplying by the channel attention coefficient; For spatial attention, Number the feature map.

[0087] In one embodiment, the spatial attention module structure is as follows: Figure 5 As shown in the figure , Given two adjacent feature maps, `conv` is the convolution operation, `upsample` is the upsampling operation, `Global Average Pool` is the global average pooling operation, `Global Max Pool` is the global max pooling operation, `Sigmoid` is the activation function, `Spital weight` is the spatial weight, `Concatenation` is the concatenation operation, and `Pixel-Wise Multiplication` is the element-wise multiplication operation. Then, the feature maps... use The convolutional layers are compressed to reduce computational cost, resulting in compressed feature maps. By upsampling and Same size, then and The concatenation is performed, and average pooling and max pooling operations are applied to the concatenated feature maps to obtain two 2D maps representing the average pooling features and max pooling features in the channels.

[0088] In one embodiment, the channel attention module structure is as follows: Figure 6 As shown in the figure , Given two adjacent feature maps, `conv` is the convolution operation, `stride` is the stride, `Global Average Pool` is global average pooling, `Global Max Pool` is global max pooling, `FC layer` is a fully connected layer, `Sigmoid` is the activation function, `Channelweight` is the spatial weights, `Concatenation` is the concatenation operation, `Pixel-Wise Sum` is element-wise addition, and `Pixel-Wise Multiplication` is element-wise multiplication. Then, the feature maps... use The convolutional layers are compressed to reduce computational cost, and then... Convolutional layers on feature maps Perform downsampling to Same size, then and The features are concatenated, and global average pooling and global max pooling are used to aggregate two different spatial context information to generate two different spatial context descriptors. These two descriptors are then fed into a fully connected layer, where channel weights, i.e., channel attention maps, are obtained by element-wise summation using the sigmoid function. Finally, the channel attention maps are multiplied by... Obtain feature map .

[0089] In one embodiment, the small farmland image segmentation method based on the dual attention mechanism further includes: acquiring and cropping farmland images to construct a model training dataset; inputting the model training dataset into the initially modified Mask R-CNN model to train the initially modified Mask R-CNN model, thereby obtaining a trained modified Mask R-CNN model; wherein the backbone structure of the modified Mask R-CNN model is ResNet50, consisting of 5 stages, corresponding to feature maps at 5 different scales. Feature map The corresponding step size is Pixel; Feature Map The feature pyramids used to build the dual-attention mechanism feature pyramid network are used to obtain new features. The modified Mask R-CNN model replaces the feature pyramid network in the original model with a dual attention mechanism feature pyramid network.

[0090] When constructing the dataset used to train the model, the cropped images were divided into a training set, a validation set, and a test set in a 7:2:1 ratio. The training set was used to train the model parameters, the validation set was used to verify the performance of the trained model during training, and the test set was used to evaluate the performance of the model after training was completed.

[0091] The overall framework of the Dual Attention Mechanism Feature Pyramid Network (DAFPN) is as follows: Figure 3 As shown, two attention modules were designed: Spatial Attention Module (SPA) and Channel Attention Module (CHA), which are used to optimize different problems. To reduce coupling between the two modules during joint optimization, spatial attention and channel fusion are optimized alternately. The dual-attention mechanism, the Feature Pyramid Network (DAFPN), extends a bottom-up path after the lowest level of the Feature Pyramid Network (FPN), then embeds SPA into the top-down path and CHA into the bottom-up path. The Pyramid Pooling Module (PPM) is used because of its powerful ability to capture contextual information. In the specific implementation, the output of PPM has the same resolution as the output of the last FPN level. Thus, PPM and the highest-level features extracted from the backbone are concatenated together as the input to the top-down path.

[0092] The process involves inputting the model training dataset into the initially modified Mask R-CNN model, training the initially modified Mask R-CNN model, and obtaining the trained modified Mask R-CNN model. Specific steps include:

[0093] S301: Input the model training data into the modified Mask R-CNN model in batches, and propagate forward based on the initial model parameters to obtain the forward propagation result; S302: Calculate the model loss using the label ground truth and the forward propagation result, and update the model parameters through algorithms such as backpropagation and gradient descent; S303: Repeat S301 and S302 continuously until the model loss reaches the required level or the model loss remains unchanged in the last 10 iterations, to obtain the trained modified Mask R-CNN model parameters.

[0094] In one embodiment, the loss function of the modified Mask R-CNN model is a combined loss function, which includes the classification loss of the candidate boxes, the location regression loss, and the object mask loss. The loss calculation formula is as follows:

[0095]

[0096] The classification loss of the candidate boxes is calculated using the following formula:

[0097]

[0098] in, The index of the region of interest. The serial number is The probability that the region of interest is predicted as a positive sample. Represents the normalization parameter. This indicates that the region of interest is a negative sample. The region of interest is considered a positive sample.

[0099] The location regression loss is calculated using the following formula:

[0100]

[0101]

[0102] in, For normalization parameters, It is the predicted offset parameter. This is the actual offset parameter. and These represent regions of interest as positive and negative samples, respectively. yes loss, Indicates smoothed ;when When the absolute value is less than 1, the value used is , The upper half of the piecewise function; when When the absolute value is greater than or equal to 1, the value used is Translation, The lower half of the piecewise function; This represents the numerical difference between the predicted bounding box and the ground truth bounding box.

[0103] The target mask loss is represented by the following formula:

[0104]

[0105] Where m represents the size of the feature map, k represents the number of feature maps, and v represents... Corresponding coordinate position , For the target true label value, These are the predicted values ​​from the modified Mask R-CNN model.

[0106] In one embodiment, the farmland image to be segmented is an ultra-high resolution farmland remote sensing satellite image; cropping the collected farmland image to be segmented to a uniform size to obtain a uniform-sized farmland image to be segmented includes: cropping the ultra-high resolution farmland remote sensing satellite image to a uniform size to obtain a uniform-sized farmland image to be segmented; wherein, the uniform-sized farmland image to be segmented has a size of 1024×1024 pixels.

[0107] In one embodiment, the modified Mask R-CNN model includes the following components: a backbone structure and a dual attention mechanism feature pyramid for extracting features from the entire image and generating features at different scales; a region proposal network for generating regions of interest; RoIAlign for mapping regions of interest of arbitrary size from the features to a fixed-size feature map using bilinear interpolation; and three branches in the head of the modified Mask R-CNN model for performing predictions to obtain object category, bounding box localization, and instance segmentation results.

[0108] In one embodiment, a pyramid pooling module is used to capture contextual information, and the output of the pyramid pooling module has the same resolution as the output of the last pyramid pooling module level. In this case, the highest-level features extracted by the pyramid pooling module and the backbone structure are concatenated together as input to the top-down path.

[0109] It should be understood that, although Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1 At least some of the steps in the process may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but may be executed at different times. The execution order of these steps or stages is not necessarily sequential, but may be executed in turn or alternately with other steps or at least some of the steps or stages in other steps.

[0110] In one embodiment, such as Figure 4 As shown, a small-scale farmland image segmentation device based on a dual attention mechanism is provided, comprising: an image acquisition module 210, an image preprocessing module 220, and a target calculation module 230, wherein:

[0111] The image acquisition module 210 is used to acquire the farmland image to be segmented.

[0112] The image preprocessing module 220 is used to crop the collected farmland images to be segmented to a uniform size to obtain farmland images to be segmented of a uniform size.

[0113] The target calculation module 230 is used to input the uniform-sized farmland image to be segmented into the modified Mask R-CNN model to obtain the instance segmentation result of the farmland image. Specifically, the modified Mask R-CNN model uses the backbone structure and dual attention mechanism feature pyramid in the modified Mask R-CNN model to extract image features, then generates several regions of interest through the region proposal network, and then maps the regions of interest through RoIAlign to generate a feature map of a fixed size. Finally, the three branches of the modified Mask R-CNN model head are used to predict the object category, the refined bounding box localization, and the segmentation mask, where the segmentation mask is the instance segmentation result of the farmland image.

[0114] In one embodiment, the target calculation module 230 includes: a feature extraction unit, used to input a uniform-sized image of farmland to be segmented into the backbone structure and dual-attention mechanism feature pyramid of the modified Mask R-CNN model to extract feature maps; a region of interest extraction unit, used to input the feature maps into a region proposal network, which uses a sliding window method on the feature maps to find regions where targets may exist, thereby obtaining regions of interest; a feature alignment unit, used to input the regions of interest into RoIAlign, which uses a bilinear interpolation algorithm to map regions of interest of arbitrary size in the features to generate fixed-size feature maps; and a prediction unit, which uses the fixed-size feature maps to predict the object category, the refined bounding box localization, and the instance segmentation results through the three branches of the modified Mask R-CNN model head.

[0115] In one embodiment, the dual-attention mechanism feature pyramid includes a spatial attention module and a channel attention module, wherein the spatial attention module is used to emphasize features at the spatial scale, and the channel attention module is used to emphasize features at the channel scale.

[0116] The expression for the operation process of the spatial attention module is as follows:

[0117]

[0118]

[0119]

[0120]

[0121] in, , Given two adjacent feature maps; For connection operation; For channel compression Convolutional layer for Convolutional layer; This is an upsampling operation; , This represents the process quantity obtained after average pooling and max pooling operations, and , ; It is the sigmoid activation function; This is an average pooling operation; This is a max pooling operation; Spatial attention coefficient, This is the feature map after multiplying by the spatial attention coefficient. For spatial attention, Number the feature maps. This is element-wise multiplication;

[0122] The expression for the operation process of the channel attention module is as follows:

[0123]

[0124]

[0125]

[0126]

[0127] in, , Given two adjacent feature maps; Indicates a connection operation; Indicates compressed channel Convolutional layer express Convolutional layer with a stride of 2; Indicates a fully connected layer; , This refers to the process quantities obtained after average pooling and max pooling operations; This represents the sigmoid activation function; Channel attention coefficient; This is the feature map after multiplying by the channel attention coefficient; For spatial attention, Number the feature map.

[0128] In one embodiment, the small-scale farmland image segmentation device based on the dual attention mechanism further includes: a preprocessing module for acquiring farmland images and cropping them to a uniform size of 1024×1024 pixels to construct a model training dataset; and a training module for inputting the model training dataset into the modified Mask R-CNN model to train the modified Mask R-CNN model and obtain a trained modified Mask R-CNN model; wherein the modified Mask R-CNN model replaces the feature pyramid network in the original Mask R-CNN model with a dual attention mechanism feature pyramid network, and the dual attention mechanism feature pyramid network is easily inserted into the original feature pyramid network structure to enhance feature learning during training.

[0129] In one embodiment, the modification process of the Mask R-CNN model includes: acquiring and cropping farmland images to construct a model training dataset; inputting the model training dataset into the initially modified Mask R-CNN model to train the initially modified Mask R-CNN model, thereby obtaining a trained modified Mask R-CNN model; wherein, the backbone structure of the modified Mask R-CNN model is ResNet50, consisting of 5 stages, corresponding to 5 feature maps at different scales. Feature map The corresponding step size is Pixel; Feature Map The feature pyramids used to build the dual-attention mechanism feature pyramid network are used to obtain new features. The modified Mask R-CNN model replaces the feature pyramid network in the original model with a dual attention mechanism feature pyramid network.

[0130] In one embodiment, the loss function of the modified Mask R-CNN model is a combined loss function, which includes the classification loss of the candidate boxes, the location regression loss, and the object mask loss. The loss calculation formula is as follows:

[0131]

[0132] The classification loss of the candidate boxes is calculated using the following formula:

[0133]

[0134] in, The index of the region of interest. The serial number is The probability that the region of interest is predicted as a positive sample. Represents the normalization parameter. This indicates that the region of interest is a negative sample. The region of interest is considered a positive sample.

[0135] The location regression loss is calculated using the following formula:

[0136]

[0137]

[0138] in, For normalization parameters, It is the predicted offset parameter. This is the actual offset parameter. and These represent regions of interest as positive and negative samples, respectively. yes loss, Indicates smoothed ;when When the absolute value is less than 1, the value used is , The upper half of the piecewise function; when When the absolute value is greater than or equal to 1, the value used is Translation, The lower half of the piecewise function; This represents the numerical difference between the predicted bounding box and the ground truth bounding box.

[0139] The target mask loss is represented by the following formula:

[0140]

[0141] Where m represents the size of the feature map, k represents the number of feature maps, and v represents... Corresponding coordinate position , For the target true label value, These are the predicted values ​​from the modified Mask R-CNN model.

[0142] In one embodiment, the modified Mask R-CNN model includes the following components: a backbone structure and a dual attention mechanism feature pyramid for extracting features from the entire image and generating features at different scales; a region proposal network for generating regions of interest; RoIAlign for mapping regions of interest of arbitrary size from the features to a fixed-size feature map using bilinear interpolation; and three branches in the head of the modified Mask R-CNN model for performing predictions to obtain object category, bounding box localization, and instance segmentation results.

[0143] Specific limitations regarding the small-scale farmland image segmentation device based on the dual-attention mechanism can be found in the limitations of the small-scale farmland image segmentation method based on the dual-attention mechanism described above, and will not be repeated here. Each module in the aforementioned small-scale farmland image segmentation device based on the dual-attention mechanism can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0144] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 7 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores modified Mask R-CNN model data. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer program implements a small-scale farmland image segmentation method based on a dual-attention mechanism.

[0145] Those skilled in the art will understand that Figure 7The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0146] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0147] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0148] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0149] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0150] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for segmenting small farmland images based on a dual attention mechanism, characterized in that, The method includes: Obtain the image of the farmland to be segmented; The collected farmland images to be segmented are cropped to a uniform size to obtain farmland images to be segmented of a uniform size; The uniform-sized farmland image to be segmented is input into the modified Mask R-CNN model to obtain the instance segmentation result of the farmland image. Specifically, the modified Mask R-CNN model uses the backbone structure and dual attention mechanism feature pyramid to extract image features, then generates several regions of interest through the region proposal network, and then uses RoIAlign to map the regions of interest to generate a fixed-size feature map. Finally, the three branches of the modified Mask R-CNN model head are used to predict the object category, the refined bounding box localization, and the segmentation mask, where the segmentation mask is the instance segmentation result of the farmland image. The dual-attention mechanism feature pyramid includes a spatial attention module and a channel attention module. The spatial attention module is used to emphasize features at the spatial scale, and the channel attention module is used to emphasize features at the channel scale. The expression for the operation process of the spatial attention module is as follows: in, , Given two adjacent feature maps; For connection operation; For channel compression Convolutional layer for Convolutional layer; This is an upsampling operation; , This represents the process quantity obtained after average pooling and max pooling operations, and , ; It is the sigmoid activation function; This is an average pooling operation; This is a max pooling operation; Spatial attention coefficient, This is the feature map after multiplying by the spatial attention coefficient. For spatial attention, Number the feature maps. This is element-wise multiplication; The expression for the operation process of the channel attention module is as follows: in, , Given two adjacent feature maps; Indicates a connection operation; Indicates compressed channel Convolutional layer express Convolutional layer with a stride of 2; Indicates a fully connected layer; , This refers to the process quantities obtained after average pooling and max pooling operations; This represents the sigmoid activation function; Channel attention coefficient; This is the feature map after multiplying by the channel attention coefficient; For spatial attention, Number the feature map.

2. The method according to claim 1, characterized in that, The uniformly sized farmland images to be segmented are input into the modified Mask R-CNN model to obtain instance segmentation results of the farmland images, including: After uniformly cropping the farmland images to be segmented, they are input into the backbone structure and dual attention mechanism feature pyramid of the modified Mask R-CNN model to extract feature maps; The feature map is input into the region proposal network, which uses a sliding window method on the feature map to find regions where the target may exist, thereby obtaining the region of interest. Input the region of interest into RoIAlign, and RoIAlign will use a bilinear interpolation algorithm to map the region of interest of any size in the features to generate a feature map of a fixed size. Fixed-size feature maps are predicted using three branches of the modified Mask R-CNN model head to obtain object category, refined bounding box localization, and instance segmentation results.

3. The method according to claim 1, characterized in that, Also includes: Collect and crop images of farmland to build a training dataset for the model; Input the model training dataset into the initially modified Mask R-CNN model, train the initially modified Mask R-CNN model, and obtain the trained modified Mask R-CNN model. The modified Mask R-CNN model uses ResNet50 as its backbone, consisting of five stages corresponding to feature maps at five different scales. Feature map The corresponding step size is Pixel; Feature Map The feature pyramids used to build the dual-attention mechanism feature pyramid network are used to obtain new features. The modified Mask R-CNN model replaces the feature pyramid network in the original model with a dual attention mechanism feature pyramid network.

4. The method according to claim 3, characterized in that, The modified Mask R-CNN model uses a combined loss function, which includes the candidate box classification loss, location regression loss, and object mask loss. The loss calculation formula is as follows: The classification loss of the candidate boxes is calculated using the following formula: in, The index of the region of interest. The serial number is The probability that the region of interest is predicted as a positive sample. Represents the normalization parameter. This indicates that the region of interest is a negative sample. The region of interest is considered a positive sample. The location regression loss is calculated using the following formula: in, For normalization parameters, It is the predicted offset parameter. This is the actual offset parameter. and These represent regions of interest as positive and negative samples, respectively. yes loss, Indicates smoothed ;when When the absolute value is less than 1, the value used is , The upper half of the piecewise function; when When the absolute value is greater than or equal to 1, the value used is Translation, The lower half of the piecewise function; This represents the numerical difference between the predicted bounding box and the ground truth bounding box; The target mask loss is represented by the following formula: Where m represents the size of the feature map, k represents the number of feature maps, and v represents... Corresponding coordinate position , For the target true label value, These are the predicted values ​​from the modified Mask R-CNN model.

5. The method according to claim 1, characterized in that, The farmland images to be segmented are ultra-high resolution remote sensing satellite images of farmland; The collected farmland images to be segmented are cropped to a uniform size to obtain farmland images to be segmented of a uniform size. This includes cropping ultra-high resolution farmland remote sensing satellite images to a uniform size to obtain farmland images to be segmented of a uniform size; wherein, the size of the farmland images to be segmented of a uniform size is 1024×1024 pixels.

6. The method according to claim 1, characterized in that, The modified Mask R-CNN model includes: The main structure and dual attention mechanism feature pyramid are used to extract features from the entire image and generate features at different scales. Region proposal network is used to generate regions of interest; RoIAlign uses bilinear interpolation to map regions of interest of arbitrary size in features onto a fixed-size feature map. The modified Mask R-CNN model has three branches in its head that perform predictions, yielding object category, bounding box localization, and instance segmentation results.

7. A small-scale farmland image segmentation device based on a dual attention mechanism, characterized in that, The device includes: The image acquisition module is used to acquire images of farmland to be segmented. The preprocessing module for the images to be segmented is used to crop the collected farmland images to be segmented to a uniform size to obtain farmland images to be segmented of a uniform size. The target calculation module is used to input the uniform-sized farmland image to be segmented into the modified Mask R-CNN model to obtain the instance segmentation result of the farmland image. Specifically, the modified Mask R-CNN model uses the backbone structure and dual attention mechanism feature pyramid to extract image features, then generates several regions of interest through the region proposal network, and then uses RoIAlign to map the regions of interest to generate a fixed-size feature map. Finally, the modified Mask R-CNN model uses three branches in the head to predict the object category, the refined bounding box localization, and the segmentation mask, where the segmentation mask is the instance segmentation result of the farmland image. The dual-attention mechanism feature pyramid includes a spatial attention module and a channel attention module. The spatial attention module is used to emphasize features at the spatial scale, and the channel attention module is used to emphasize features at the channel scale. The expression for the operation process of the spatial attention module is as follows: in, , Given two adjacent feature maps; For connection operation; For channel compression Convolutional layer for Convolutional layer; This is an upsampling operation; , This represents the process quantity obtained after average pooling and max pooling operations, and , ; It is the sigmoid activation function; This is an average pooling operation; This is a max pooling operation; Spatial attention coefficient, This is the feature map after multiplying by the spatial attention coefficient. For spatial attention, Number the feature maps. This is element-wise multiplication; The expression for the operation process of the channel attention module is as follows: in, , Given two adjacent feature maps; Indicates a connection operation; Indicates compressed channel Convolutional layer express Convolutional layer with a stride of 2; Indicates a fully connected layer; , This refers to the process quantities obtained after average pooling and max pooling operations; This represents the sigmoid activation function; Channel attention coefficient; This is the feature map after multiplying by the channel attention coefficient; For spatial attention, Number the feature map.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images