Image segmentation method, terminal device and computer-readable storage medium

Abstract

This application relates to the field of image segmentation technology, and provides an image segmentation method, terminal device, and computer-readable storage medium, comprising: acquiring a plurality of first images, wherein the first images are images obtained by slicing a target image; extracting features from each of the first images to obtain a plurality of first features for each first image; fusing the plurality of first features of each first image to obtain a second feature corresponding to each first image; and segmenting the target image according to the second features corresponding to each of the plurality of first images to obtain an image segmentation result. This method can improve the segmentation accuracy of farmland plot images.

Description

Technical Field

[0001] This application belongs to the field of image segmentation technology, and in particular relates to an image segmentation method, terminal device and computer-readable storage medium. Background Art

[0002] With the rapid development of deep learning technology and the widespread application of aerial imagery data, the technology for extracting farmland plots based on deep learning and aerial imagery data has attracted widespread attention in the agricultural field.

[0003] In some related technologies, such as the U-Net image segmentation algorithm based on Convolutional Neural Network (U-Net), the segmentation of aerial imagery data may not be able to accurately segment the target area for images with complex scenes or blurred boundaries, resulting in low segmentation accuracy and unsatisfactory segmentation results for farmland images. Summary of the Invention

[0004] This application provides an image segmentation method, a terminal device, and a computer-readable storage medium, which can improve the segmentation accuracy of farmland plot images.

[0005] In a first aspect, embodiments of this application provide an image segmentation method, including:

[0006] Acquire multiple first images, wherein the first image is an image obtained by slicing the target image;

[0007] Feature extraction is performed on each of the first images to obtain multiple first features for each of the first images;

[0008] Multiple first features of each first image are fused to obtain a second feature corresponding to each first image;

[0009] The target image is segmented based on the second features corresponding to each of the multiple first images to obtain the image segmentation result.

[0010] In this embodiment, the target image is preprocessed and divided into multiple first images. Each first image undergoes a multi-scale transformation to generate multiple first features. These first features are then fused to generate second features. Finally, the target image is segmented based on the fused second features. The second features provided by each slice image may help to more accurately locate and segment the target within a local region. This method can improve the segmentation accuracy of the target image.

[0011] In one possible implementation of the first aspect, the step of extracting features from each of the first images to obtain a plurality of first features of the first images includes:

[0012] m second images are generated based on the target image, each second image containing the farmland shape in the target image, where m is a positive integer;

[0013] Generate n scale factors based on m of the second images, where n is an integer greater than 1;

[0014] Feature extraction is performed on each of the first images based on the n scale factors to obtain multiple first features for each first image.

[0015] In this embodiment, by generating images at different scales and performing multi-scale feature extraction on the first image, the system can more comprehensively capture the details and overall shape of land features, improving its ability to understand land features. Generating a second image containing the shape of cultivated land in the target image helps to obtain information about the distribution and shape of cultivated land, which has potential value for land use monitoring in the agricultural sector. Using a scale factor to extract features from the first image allows the system to adapt to changes in terrain features at different scales, improving its generalization ability for land features.

[0016] In one possible implementation of the first aspect, the step of extracting features from each of the first images based on the n scale factors to obtain a plurality of first features for each of the first images includes:

[0017] For each first image, scale the first image according to the n scale factors to generate n first sub-images corresponding to the first image;

[0018] Feature extraction is performed on each of the n first sub-images to obtain the first feature of each first sub-image.

[0019] In this embodiment, by generating sub-images at multiple scales for each first image and extracting their features, the system can more comprehensively understand the information changes of the image at different scales, enhancing its multi-scale perception capability. The expansion operation enables the system to adapt to images at different scales, improving its robustness to images and helping to cope with feature changes caused by scale variations. By extracting features at different scales, richer and more diverse feature representations can be obtained, which helps to improve the understanding and expression of image content.

[0020] In one possible implementation of the first aspect, the step of extracting features from n first sub-images to obtain a first feature for each first sub-image includes:

[0021] For each of the first sub-images, obtain the local feature information of each of the first sub-images;

[0022] Obtain the global feature information of the first sub-image;

[0023] The local feature information of the first sub-image is concatenated with the global feature information of the first sub-image to obtain the first feature of the first sub-image.

[0024] In the embodiments of this application, combining local and global feature information can provide a richer and more comprehensive image feature representation, which helps to better capture various aspects of the image. Fusing local and global information helps make the feature representation more robust to local changes and overall structural changes in the image, enhancing the understanding of the image content.

[0025] In one possible implementation of the first aspect, the step of fusing multiple first features of each first image to obtain a second feature corresponding to each first image includes:

[0026] For each of the first images, a first process is performed based on the first features of the first image to obtain a plurality of third features of the first image, wherein the first process is used to enhance the feature representation capability of the first features;

[0027] The first feature of the first image is processed in a second way to obtain multiple fourth features of the first image, wherein the second processing is used to improve the ability of the first feature to capture feature information at different scales.

[0028] The second feature of the first image is obtained by fusing the third and fourth features.

[0029] In this embodiment, the first processing aims to enhance the expressive power of the first feature, enabling it to better capture important information in the image and improve the discriminative power of the feature. The second processing focuses on improving the capture ability of the first feature at different scales, making the network more adaptable and generalizable. Through feature fusion, features obtained through different processing methods are comprehensively utilized to obtain a more comprehensive and representative second feature, which helps improve the performance of subsequent tasks.

[0030] In one possible implementation of the first aspect, the step of performing feature fusion based on the third feature and the fourth feature to obtain the second feature of the first image includes:

[0031] For each of the first images, the fifth feature of the first image is obtained by concatenating each of the third features of the first image with its corresponding fourth feature.

[0032] The fifth feature of the first image is weighted with multiple first features of the first image to obtain the second feature of the first image.

[0033] In this embodiment, the feature stitching stage helps integrate feature information from different processing methods, forming a more representative and richer feature representation. By using weighted operations to fuse the fifth feature with multiple first features, information from different levels and perspectives can be better utilized, improving the comprehensive understanding of image content. Weighted operations allow adjusting the importance of different features according to the specific task and data characteristics, enhancing the network's flexibility and adjustability.

[0034] In one possible implementation of the first aspect, the step of performing image segmentation on the target image based on the second features corresponding to each of the plurality of first images to obtain image segmentation results includes:

[0035] Multiple first images are merged to obtain a merged third image;

[0036] The target image is segmented based on the third image and the second feature to obtain multiple predicted segmentation regions;

[0037] Image segmentation results are obtained based on multiple predicted segmentation regions.

[0038] In this embodiment, multi-image merging helps integrate global information from multiple images into a single image, providing more comprehensive contextual information and facilitating a better understanding of the target image. Utilizing deep learning models for image segmentation can divide an image into semantically meaningful regions, providing a foundation for subsequent image understanding and analysis. The final post-processing steps help further optimize the segmentation results, improving accuracy and interpretability.

[0039] In one possible implementation of the first aspect, obtaining the image segmentation result based on the plurality of predicted segmentation regions includes:

[0040] The overlapping region of the multiple predicted segmentation regions is obtained based on the multiple predicted segmentation regions;

[0041] The mean of the overlapping region is calculated to obtain the result of the mean calculation of the overlapping region;

[0042] The image segmentation result of the target image is determined based on the calculation result of the mean of the overlapping region and the predicted segmentation region.

[0043] In this embodiment, fusing information from multiple predictions, particularly by calculating the mean in overlapping regions, helps reduce errors caused by the uncertainty of a single prediction. Calculating the mean in overlapping regions improves the stability of the results, making the final image segmentation result more reliable. Combining different prediction results, especially by calculating the mean in overlapping regions, enhances the system's robustness to different input variations and improves the generalization performance of image segmentation.

[0044] In a second aspect, embodiments of this application provide a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the image segmentation method as described in any one of the first aspects above.

[0045] Thirdly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the image segmentation method as described in any one of the first aspects above.

[0046] Fourthly, embodiments of this application provide a computer program product that, when run on a terminal device, causes the terminal device to execute the image segmentation method described in any one of the first aspects.

[0047] It is understood that the beneficial effects of the second to fourth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 This is a schematic block diagram of the segmentation model provided in the embodiments of this application;

[0050] Figure 2 This is a schematic flowchart of the image segmentation method provided in the embodiments of this application;

[0051] Figure 3 This is a schematic diagram of the process for obtaining the first feature provided in an embodiment of this application;

[0052] Figure 4 This is a schematic diagram of the feature fusion process provided in an embodiment of this application;

[0053] Figure 5This is a general schematic diagram of the segmentation method provided in the embodiments of this application;

[0054] Figure 6 This is a schematic diagram of the structure of the terminal device provided in the embodiments of this application. Detailed Implementation

[0055] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0056] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0057] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0058] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0059] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0060] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized.

[0061] In most geographic information systems, segmenting farmland plots on aerial imagery remains a repetitive, lengthy, and time-consuming labor-intensive task. With the rapid development of deep learning technology and the widespread application of aerial imagery data, farmland plot extraction technology based on deep learning and aerial imagery data has attracted widespread attention in the agricultural field. This technology offers advantages such as automation, efficiency, and high accuracy, and is of great significance for land use monitoring and agricultural decision-making.

[0062] In some related technologies, such as the U-Net model, the segmentation of aerial impact data may fail to accurately segment the target area for images with complex scenes or blurred boundaries, resulting in low segmentation accuracy and unsatisfactory segmentation results for farmland images.

[0063] To address the problems encountered in the aforementioned related technologies, this application provides an image segmentation method. In this embodiment, aerial image data is preprocessed and segmented. Then, a U-Net-based model is used to classify and segment the segmented aerial images at the pixel level, distinguishing farmland plots from other land types. To improve model performance, a Multi-Scale Fusion Attention Convolutional Neural Network (MFAU-Net) suitable for extracting farmland plots from aerial images is further designed. Considering the characteristics of farmland plots—small size, diverse shapes, and unclear boundary definitions—this application innovatively introduces a multi-scale module suitable for this task based on the original U-Net, enhancing the original U-Net network's ability to learn semantic information from images.

[0064] In addition, the original U-Net has problems such as weak boundary detection capability and easy loss of cross-layer semantic information. Farmland plot data has the problem of ambiguous boundary definition. Therefore, this invention further designed a fusion attention module to narrow the semantic gap between the low-level features and high-level features of the original U-Net network and enhance the network's ability to learn farmland plot boundary information.

[0065] The U-Net network architecture is U-shaped, consisting of an encoder and a symmetric decoder. The encoder captures contextual information from the image, while the decoder restores the resolution. The encoder, composed of a series of convolutional and pooling layers, progressively reduces the size of the input image and extracts high-level abstract features. This helps capture global information within the image. The decoder, composed of a series of upsampling and convolutional layers, progressively restores the resolution. The decoder's task is to restore the feature maps extracted by the encoder to the original input image resolution, providing more accurate segmentation results. To preserve richer low-level features, U-Net uses skip connections to connect the encoder's feature maps to the corresponding decoder layers. This helps mitigate information loss and improve model accuracy. The final layer typically uses a convolutional layer with an output channel number equal to the number of classes to be segmented in the task. Pixel-level classification is performed using activation functions.

[0066] Before discussing the farmland parcel extraction technology based on aerial imagery data provided in this application, it is necessary to first discuss the preparatory work for the scheme, see [link to relevant documentation]. Figure 1 This is a schematic block diagram of the segmentation model provided in the embodiments of this application, such as... Figure 1 As shown, the MFAU-Net model adds a multi-scale module and a fusion attention module to the U-Net framework, as detailed below:

[0067] 1. Multi-scale module

[0068] Farmland is often covered by various crops, and aerial imagery of farmland plots is characterized by significant differences between plots, small size, irregular shape, and ambiguous boundary definitions. To address these issues, this paper innovatively designs a multi-scale module suitable for farmland plot extraction tasks, enhancing the network's ability to learn multi-scale information from farmland plot edge data.

[0069] The scaling module primarily constructs a cascaded pyramid structure for feature images using bilinear interpolation. This maintains the image shape and size while enriching the network's receptive field, enabling the learning of more semantic information. Specifically, it first obtains the approximate outline of the image using a watershed algorithm. Then, based on the rule that the image must contain at least one complete farmland plot outline, it adaptively generates two multi-scale factors, thereby enhancing the model's semantic information learning. This also further enriches the data volume and improves the model's generalization ability.

[0070] 2. Integration of Attention Modules

[0071] The design of the fusion attention module is primarily inspired by the Squeeze-and-Excitation (SE) channel attention approach and the Selective Kernel (SK) receptive field adaptation approach. While the original U-Net has advantages such as simple structure, it suffers from poor boundary information learning ability and easy loss of cross-layer semantic information. To alleviate these problems, this paper designs a multi-scale fusion attention module based on SE channel attention and SK receptive field adaptation, which enriches semantic information and simultaneously achieves receptive field adaptation and channel weighting. The encoder's primary semantic features are fused with the decoder's high-level semantic features via a skip link that integrates the multi-scale module. This design not only alleviates U-Net's poor boundary information learning ability but also enhances the network's ability to learn discriminative features during feature extraction.

[0072] The following describes the segmentation process of the MFAU-Net model:

[0073] See Figure 2 This is a schematic flowchart of an image segmentation method provided in an embodiment of this application. It is intended as an example and not a limitation. The method may include the following steps:

[0074] S101, acquire multiple first images, wherein the first image is an image obtained by slicing the target image.

[0075] In this embodiment, the target image can be aerial imagery data, which is image data obtained by aircraft (usually drones, airplanes, or satellites). These images typically contain high-resolution views of the Earth's surface and can be used for applications such as geographic information systems, map making, land use planning, and environmental monitoring. After acquiring the target image containing geographic information, preprocessing operations are required.

[0076] The target image also serves as training data for the model, therefore preprocessing of the training data is necessary. Preprocessing operations can involve slicing the target image into smaller blocks, known as slices. Slicing can be done using fixed-size windows or sliding windows. The slice size typically depends on the task requirements. Preprocessing is then applied to each slice, i.e., the first image, to better adapt it to the model's input requirements. Preprocessing may include image normalization, denoising, and adjusting brightness and contrast. It is crucial to ensure consistent preprocessing for each slice to maintain input consistency.

[0077] S102, perform feature extraction on each of the first images to obtain multiple first features for each of the first images.

[0078] In this embodiment, each first image is passed as input to the Unet model. The input image passes through an encoder, a series of convolutional and pooling layers, gradually reducing its size and extracting high-level abstract features of global information. These features capture the overall structure and content of the image. Using a skip connection mechanism, the encoder's feature maps are connected to the corresponding layers of the decoder. This helps preserve low-level features, such as image details and boundary information. The decoder progressively restores the feature maps to the resolution of the original input image. At this stage, the model focuses on recovering more detailed information, including image details and structure, forming mappings corresponding to different features. Multiple features of the image can be obtained through the intermediate layers of this model. Multiple feature representations of each image can be extracted and stored in appropriate data structures, such as matrices or vectors. The multiple features of each image will form a feature set (first features).

[0079] The first feature mentioned above can also be called a feature map, which refers to a two-dimensional or three-dimensional data structure output from the intermediate layer of a convolutional neural network. It is obtained by performing a series of convolution and pooling operations on the input image (the first image). The feature map can be seen as the result of feature extraction from the input image, capturing feature information at different levels of abstraction in the image.

[0080] In one embodiment, see Figure 3 This is a schematic diagram of the process for obtaining the first feature provided in an embodiment of this application, such as... Figure 3 As shown, step S102 includes:

[0081] S201, Generate m second images based on the target image, the second images containing the farmland shape in the target image, where m is a positive integer.

[0082] In this embodiment, image processing techniques can be used to identify and extract the shape of farmland in a target image. This may involve methods such as image segmentation and feature extraction to ensure that the extracted shape is relevant to the farmland. Using the extracted farmland shape information, m second images can be generated as needed. These images can be variations of the original target image, where the farmland shape is highlighted or modified, or new images that preserve the spatial layout and features of the farmland.

[0083] For example, one way to obtain the second image is to obtain the approximate outline of the target image (the second image) using a watershed algorithm. The watershed algorithm is an image segmentation technique used to identify the boundaries and dividing lines of different regions in an image. This algorithm can help distinguish the boundaries of farmland from other features, thus enabling the acquisition of the second image. Specifically, the implementation involves: preprocessing the target image, including grayscale conversion, smoothing, and gradient calculation. This helps highlight edges and features in the image; based on gradient information, applying an appropriate thresholding method to divide the image into foreground and background. This step helps separate different regions in the target image; performing a distance transformation on the segmented image to calculate the distance from each pixel to the nearest background pixel. This step provides the basis for subsequent watershed transformation; finding peaks in the distance-transformed image, which represent the possible locations of watershed lines; performing a watershed transformation on the image based on the peak information, dividing the image into different regions and marking them. The boundaries of these regions constitute the approximate outline of the target image; and generating the second image based on the marked watershed regions, where the outline can be highlighted or extracted to form the approximate outline of the target image.

[0084] S202, Generate n scale factors based on m of the second images, where n is an integer greater than 1.

[0085] In this embodiment, the multi-scale module primarily constructs a cascaded feature image pyramid structure using bilinear interpolation to maintain the image shape and size while enriching the network's receptive field, thereby learning richer semantic information. Specifically, after obtaining the second image using the watershed algorithm, two multi-scale factors are adaptively generated based on the rule that it contains at least one complete farmland plot outline (the second image). The scale factor typically refers to the scale variation ratio in the image. In image processing, the scale factor can be used to represent the degree of image scaling. The process of generating the scale factor can be adjusted as needed. In the cascaded feature image pyramid structure, a series of images at different scales are generated by scaling the first image according to different scales. To maintain the consistency of image shape and size, bilinear interpolation is applied during the image scaling process at each scale. Through bilinear interpolation, the smoothness and continuity of the image can be maintained, avoiding jagged distortions.

[0086] One method for generating the aforementioned scale factor is as follows: First, image processing techniques or object detection algorithms are used to ensure that at least one complete farmland plot outline (second image) can be detected. This may include using segmentation algorithms or object detection models to identify farmland areas in the image; obtaining scale information from the detected farmland plots, which can be obtained by measuring indicators such as the width, height, or area of ​​the plots; and generating two multi-scale factors based on the obtained farmland plot scale information. One scale factor can be slightly smaller than the scale of the farmland plot, and the other scale factor can be slightly larger than the scale of the farmland plot. This ensures that the generated second image has scale differences while still containing complete farmland plots. The generated scale factors are then applied to the target image, reducing and enlarging the image respectively to generate two multi-scale second images. Here, the scale factor n = 2. The method in this application is illustrated below using a scale factor n = 2 as an example.

[0087] The methods described above enhance the learning of semantic information in the model by generating multi-scale factors. This also enriches the amount of data and improves the model's generalization ability.

[0088] S203, feature extraction is performed on each of the first images according to the n scale factors to obtain multiple first features for each of the first images.

[0089] In this application example, feature extraction for each first image based on n (example n=2) scale factors can be understood as the process of representing features for each image at different scales. This process can be divided into the following steps: For each first image, scaling or magnification is performed on the image according to the given n scale factors, generating n images at different scales. These images represent the same scene at different scales. For each image at each scale, features are extracted from the image using a UET model. These features may include edge information, texture, color distribution, or higher-level semantic features. For each image and each scale, a feature set is obtained, with corresponding feature representations at each scale. Therefore, for each first image, multiple feature sets at different scales are obtained based on the n scale factors, i.e., multiple first features for each first image.

[0090] In the above method, by generating images at different scales and performing multi-scale feature extraction on the first image, the system can more comprehensively capture the details and overall shape of land features, improving its ability to understand land features. Generating a second image containing the shape of cultivated land in the target image helps to obtain information about the distribution and shape of cultivated land, which has potential value for land use monitoring in the agricultural sector. Using scale factors to extract features from the first image allows the system to adapt to changes in terrain features at different scales, improving its generalization ability for land features.

[0091] In one embodiment, step S203 includes:

[0092] For each first image, scale the first image according to the n scale factors to generate n first sub-images corresponding to the first image;

[0093] Feature extraction is performed on each of the n first sub-images to obtain the first feature of each first sub-image.

[0094] In the embodiments of this application, for each first image, it is necessary to first set the initial scale factor, which is generally 1, that is, the same size as the original input image. Then, according to the n (in the example n=2) scale factors obtained in this application, the feature image of the current level is scaled respectively. Methods such as bilinear interpolation can be used to maintain the smoothness and continuity of the image.

[0095] For example, the first image is first divided into sub-images of the same size as the first image by a scale factor of 1. Then, the sub-images are downsampled by the scale factor to reduce their size, generating feature images of the next level, i.e., multiple sub-images. The downsampling operation can use methods such as pixel average or maximum values. These sub-images represent the original image at different scales. For each sub-image at each scale, feature extraction is performed using the Unet model. Unet is a convolutional neural network for image segmentation that can learn semantic information in an image and extract its feature representation. For each sub-image at each scale, the Unet model can obtain the first feature representation of each sub-image. These features may include edges, textures, semantic information, etc.

[0096] By following the above process, feature representations of each first image at different scales can be obtained, thus providing more comprehensive and diverse feature information. This method helps improve the model's ability to understand images, especially in feature learning and representation in multi-scale scenes.

[0097] In one embodiment, feature extraction is performed on n first sub-images to obtain a first feature for each first sub-image, including:

[0098] For each of the first sub-images, obtain the local feature information of each of the first sub-images;

[0099] Obtain the global feature information of the first sub-image;

[0100] The local feature information of the first sub-image is concatenated with the global feature information of the first sub-image to obtain the first feature of the first sub-image.

[0101] In this embodiment, the first sub-image is used as input to the Unet model, ensuring that the dimension of the input image matches the model's expected input dimension. In the Unet model, the encoder (downsampling) is responsible for extracting low-level and local features of the image. As the encoder progresses, the model gradually learns local information such as details and textures in the image. Skip connections in Unet allow information to be transferred between the encoder and decoder (upsampling), thus preserving more global contextual information. The decoder's integration of global information helps in understanding the overall structure and semantics. Typically, fusion can be achieved by concatenating or adding the feature map (local features) of a certain layer of the encoder with the feature map (global features) of the corresponding layer of the decoder. This allows the decoder to utilize both local and global information at each step, improving the model's understanding of the image. Further processing, such as applying pooling or convolutional layers, is then performed on the fused feature map to obtain the final first feature representation.

[0102] The methods described above, which combine local and global feature information, can provide richer and more comprehensive image feature representations, helping to better capture various aspects of the image. Fusing local and global information makes the feature representation more robust to local and overall structural changes in the image, enhancing the understanding of the image content.

[0103] S103, perform feature fusion on multiple first features of each first image to obtain a second feature corresponding to each first image.

[0104] In this embodiment, for each first image and the extracted first feature, a fusion strategy can be considered, such as stitching, summation, or averaging. This can merge multiple features into a more comprehensive feature representation. After multi-feature fusion, the resulting merged feature is the second feature corresponding to each first image. This feature can capture information from the original image at different scales, different local regions, and at the global level.

[0105] In the process of multi-feature fusion, this application considers introducing a fusion attention mechanism (FAM), which is an attention mechanism that combines SE channel attention and SK receptive field adaptation. It aims to enrich semantic information and achieve receptive field adaptation and channel weighting. In the image encoder-decoder architecture, the FAM module is used to fuse the encoder's primary semantic features and the decoder's high-level semantic features.

[0106] For example, the implementation of fusing multiple features using a fusion attention mechanism can be as follows: Prepare multiple features to be fused: these features can be features from different levels, scales, or sources; Introduce an attention mechanism: introduce attention calculations for each pair of features to be fused, which can learn and determine the weights of the features; Calculate attention weights: use different methods (such as fully connected layers, convolutional layers, etc.) to calculate the attention weights between each pair of features, these weights determine the importance of the features in the fusion; Feature fusion: use the calculated attention weights to fuse the features, commonly using weighted averaging or weighted summation, assigning weights to the features according to the attention weights; Obtain the fused features: the fused features regulated by the attention mechanism will contain information from multiple features and take into account their importance in the fusion process.

[0107] In one embodiment, see Figure 4 This is a schematic diagram of the feature fusion process provided in the embodiments of this application, such as... Figure 4 As shown, step S103 includes:

[0108] S301, for each of the first images, a first processing is performed based on the first feature of the first image to obtain a plurality of third features of the first image, wherein the first processing is used to enhance the feature representation capability of the first feature.

[0109] In this embodiment, SE channel attention based on a fusion attention mechanism is used to process the first feature. For each first feature, the SE channel attention mechanism is used to perform channel weighting on the primary semantic feature (first feature). First, the average value of each channel is calculated through a global average pooling operation. Then, the weight of each channel is generated through a fully connected layer and an activation function. Finally, the weight is multiplied by the feature map (first feature) of the corresponding channel to achieve channel weighting. This process generates multiple third features, which fuse information from the enhanced features and the original first features.

[0110] S302, perform a second processing based on the first feature of the first image to obtain a plurality of fourth features of the first image, wherein the second processing is used to improve the ability of the first feature to capture feature information at different scales.

[0111] In this embodiment, the first feature is processed using an SK receptive field based on a fusion attention mechanism to improve its ability to capture feature information at different scales. This processing may include operations such as convolution, pooling, and dilated convolution to enhance the expressive power and receptive field of the feature. During the second processing of the first feature, a fusion attention mechanism is introduced. This means that for each processed first feature, the SK receptive field combined with the fusion attention mechanism is used to learn and determine its weight and contribution in the fusion. For the second processing result of the first feature, the fusion attention mechanism is used to calculate the attention weight between it and the original first feature. These weights reflect the correlation and importance between the processed feature and the original feature. Based on the calculated attention weights, the second processing result of the first feature is fused with the original first feature. This process generates multiple fourth features, which fuse information from the enhanced processed features and the original first feature.

[0112] S303, perform feature fusion based on the third feature and the fourth feature to obtain the second feature of the first image.

[0113] In this embodiment, the basic structure of the Unet model has been established, including an encoder (downsampling path) and a decoder (upsampling path). In the Unet encoder, an SE channel attention module is introduced for each encoder module. The SE channel attention module is used to learn the importance of each channel in order to more accurately capture feature representations. In the Unet encoder, an SK receptive field module is introduced for each encoder module. The SK receptive field module can be used to process features and enhance its perception of information at different scales.

[0114] In the U-Net decoder process, the third and fourth features are fused. This can be done in each decoder module. The third and fourth features are obtained by processing the input SE channel attention module and SK receptive field module, respectively. The weights between the third and fourth features are calculated to reflect the contribution of different features in the fusion. Based on the calculated weights, the two features are weighted and fused to generate the fused feature, which is then passed to the output layer of U-Net as the second feature.

[0115] Through the above design, the U-Net model utilizes the introduced SE channel attention module and SK receptive field module, as well as the fusion attention mechanism, to achieve fine fusion of the third and fourth features, thereby generating a second feature that is more information-rich and has multi-scale perceptual capabilities. This helps improve the model's performance in image processing tasks.

[0116] In one embodiment, step S303 includes:

[0117] For each of the first images, the fifth feature of the first image is obtained by concatenating each of the third features of the first image with its corresponding fourth feature.

[0118] The fifth feature of the first image is weighted with multiple first features of the first image to obtain the second feature of the first image.

[0119] In this embodiment, for each first image, its corresponding third and fourth features are concatenated. This can be achieved by stacking the two features together along the channel dimension. After the concatenation operation, the resulting feature is the fifth feature of the first image. The fifth feature is then weighted with multiple first features of the first image. This can be done by introducing weight coefficients and summing each feature using weighted methods. The weight coefficients can be set according to the importance of the features or obtained through learning. After the weighting operation, the result is the second feature of the first image.

[0120] For example, the feature map (first feature) F TL The feature maps are divided into g groups based on channel dimension, and fusion attention learning is performed on each group separately. Specifically, to simultaneously achieve feature map attention and receptive field adaptation, the grouped feature maps are further divided into s branches based on the number of channels, resulting in a total of G = gs. The output of the grouped feature maps from the module input can be represented as:

[0121] F g =δ(U g )=δ(W1(F TL )), g∈{1,2} (1)

[0122] Where w1(.) represents the convolution transformation performed on the feature map, and δ(.) represents the attention calculation process.

[0123] The feature maps from multiple groups are combined by concatenation. The concatenated feature map can be represented as follows:

[0124] F′=[F1,F2](2)

[0125] Where [] represents the concatenation operation, F1 represents the feature map output of the first group, and F2 represents the feature map output of the second group.

[0126] The concatenated feature map, along with the original input feature map F, TL By performing an addition operation, the features of the original feature maps are weighted, and the final attention-based feature map output can be represented as:

[0127] F out =F TL +W2(F′) (3)

[0128] Here, w2(.) represents a 1x1 convolution operation.

[0129] The output attention-based feature map is the second feature, since activation functions are typically applied to the output of each convolutional layer. The role of the activation function is to introduce a non-linear transformation, enabling the network to learn non-linear feature mapping relationships. This is crucial for processing complex data such as images, as images contain rich non-linear structures and information. The activation functions used in this application are as follows:

[0130] This paper introduces the Mish activation function to improve the model's adaptability to images of small, poorly defined farmland plots. Mish is a self-gated activation function used for nonlinear transformations in neural networks. The definition of Mish is as follows:

[0131] f(x)=x·tanh(ω(x)) (4)

[0132] Where ω(x) is the soft activation function, equal to ln(1+e^x). x ).

[0133] As can be seen from Equation 4), the Mish activation function has no clearly defined upper or lower bounds within its input range, which allows the model to adapt more freely to different data distributions. The application of the Mish function in this paper effectively improves the segmentation capability for complex and diverse farmland plot images. This will significantly enhance the accuracy and applicability of farmland plot extraction technology, promoting intelligent and sustainable agricultural development.

[0134] Within the feature map group, an Attention Mechanism (AM) is used to acquire rich semantic information and feature map weight matrices. In the AM, a global average pooling operation is first used to model global contextual position information, achieving channel-wise feature compression. The compressed real values ​​correspond to the importance of each channel in the global feature map. The global average pooling operation assigns practical meaning to each parameter in the feature map. Next, a bottleneck-like mechanism is implemented through two fully connected layers with activation operations, fully learning the channel weights and completing the channel modeling. This step primarily enhances the non-linear representation capability through activation operations. Finally, the coefficient matrix generated by the multi-branch activation operation is multiplied back into the original feature map, thus achieving the modeling of weights between feature maps, i.e., feature map attention. The final attention feature map is generated by adding the different weight feature map matrices from multiple branches.

[0135] In the above method, weighted learning of convolutional channels and adaptive receptive fields are achieved through grouping, separating, and fusing attention computation units, which is beneficial to improving the segmentation effect of farmland plots. The fused attention module enhances the weight of important features in the original signal features to a certain extent, giving the model a stronger representation ability. This is very important for small target segmentation tasks such as farmland plot data extraction, which are small in size, have large shape differences, and unclear boundary definitions. In summary, the channel weighting technology embodied in the fused attention module effectively improves the performance of MFAU-Net in the farmland plot extraction task.

[0136] S104, perform image segmentation on the target image based on the second features corresponding to each of the multiple first images to obtain image segmentation results.

[0137] In this embodiment, the U-Net module automatically segments the target image based on the second feature. During image segmentation using the second feature, an appropriate loss function is defined, typically the cross-entropy loss function, to measure the difference between the model's prediction and the true segmentation label. The loss function can be as follows:

[0138] Semantic segmentation of farmland parcel images based on aerial imagery data is a binary classification problem, and the classic binary classification cross-entropy L0 method is employed. Bce As the loss function, its formula is as follows:

[0139]

[0140] Where N is the number of pixels in the image, y i p is the true label value of the i-th pixel. i Let be the predicted probability value of the i-th pixel.

[0141] After obtaining image segmentation results from the U-Net module, the aforementioned loss function is used to measure the difference between the model's predicted results and the true segmentation labels. The image segmentation model is then trained using the training set. Through backpropagation and optimization algorithms, network parameters are adjusted to minimize the loss function. The model performance is evaluated using the validation set, and adjustments are made as needed, such as adjusting the learning rate and employing data augmentation techniques, to improve the model's generalization ability.

[0142] In one embodiment, step S104 includes:

[0143] Multiple first images are merged to obtain a merged third image;

[0144] The target image is segmented based on the third image and the second feature to obtain multiple predicted segmentation regions;

[0145] Image segmentation results are obtained based on multiple predicted segmentation regions.

[0146] In this embodiment, multiple first images are merged to obtain a merged third image. This step can be performed by adding or averaging pixel values. Using the merged third image and the previously extracted second features, an appropriate deep learning model is used for image segmentation. This step yields multiple predicted segmentation regions. Different strategies can be employed for these multiple predicted segmentation regions, such as selecting the region with the highest confidence as the final result, or obtaining the final image segmentation result through further post-processing methods.

[0147] In the methods described above, multi-image merging helps integrate global information from multiple images into a single image, providing more comprehensive contextual information and facilitating a better understanding of the target image. Utilizing deep learning models for image segmentation can divide an image into semantically meaningful regions, providing a foundation for subsequent image understanding and analysis. The final post-processing steps help further optimize the segmentation results, improving accuracy and interpretability.

[0148] In one embodiment, obtaining an image segmentation result based on a plurality of predicted segmentation regions includes:

[0149] The overlapping region of the multiple predicted segmentation regions is obtained based on the multiple predicted segmentation regions;

[0150] The mean of the overlapping region is calculated to obtain the result of the mean calculation of the overlapping region;

[0151] The image segmentation result of the target image is determined based on the calculation result of the mean of the overlapping region and the predicted segmentation region.

[0152] In this embodiment, for multiple predicted segmentation regions, their overlapping regions are determined. This can be achieved through pixel-level logical operations to obtain an overlapping region containing the common portion of all predicted segmentation regions. For the overlapping region, the mean of its pixel values ​​is calculated. This can be achieved by adding the pixel values ​​within the overlapping region and dividing by the number of pixels to obtain the mean of the overlapping region. The result of calculating the mean of the overlapping region is combined with the original predicted segmentation regions to determine the final image segmentation result. This can be done by assigning the mean pixel value of the overlapping region to the corresponding pixel position while maintaining the original predicted segmentation values ​​of other regions.

[0153] The above method comprehensively considers multiple predicted segmentation regions, especially when they overlap, to obtain more detailed and accurate image segmentation results.

[0154] See Figure 5 This is a general schematic diagram of the segmentation method provided in the embodiments of this application, such as... Figure 5 As shown, the target image, i.e., the aerial image data to be segmented, is first preprocessed by slicing the target image into multiple first images. These first images are then input into the trained MFAU-Net model for image segmentation. The multi-scale module in MFAU-Net extracts features from the image using three different feature extraction branches, achieving multi-scale feature extraction. The first features extracted from the multi-scale model are then fused using a fusion attention module to generate second features. Finally, these second features are used to segment the target image. After post-processing, the final segmentation result is generated.

[0155] The above methods take into account the characteristics of farmland plots in aerial images, such as small size, large shape differences and unclear edge definition. A farmland plot extraction technology based on aerial images is proposed, which effectively integrates deep learning, basic segmentation network, multi-scale module and fusion attention module, and can achieve good generalization effect and high segmentation accuracy in farmland plot images based on aerial images.

[0156] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0157] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0158] Figure 6 This is a schematic diagram of the structure of the terminal device provided in the embodiments of this application. For example... Figure 6 As shown, the terminal device 6 in this embodiment includes: at least one processor 60 ( Figure 6(Only one is shown) a processor, a memory 61, and a computer program 62 stored in the memory 61 and executable on the at least one processor 60, which, when executing the computer program 62, implements the steps in any of the above-described image segmentation method embodiments.

[0159] The terminal device may be a desktop computer, laptop, handheld computer, or cloud server, etc. This terminal device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that... Figure 6 This is merely an example of terminal device 6 and does not constitute a limitation on terminal device 6. It may include more or fewer components than shown in the figure, or combine certain components, or different components, such as input / output devices, network access devices, etc.

[0160] The processor 60 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0161] In some embodiments, the memory 61 may be an internal storage unit of the terminal device 6, such as a hard disk or memory of the terminal device 6. In other embodiments, the memory 61 may be an external storage device of the terminal device 6, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the terminal device 6. Furthermore, the memory 61 may include both internal and external storage units of the terminal device 6. The memory 61 is used to store the operating system, applications, boot loader, data, and other programs, such as the program code of the computer program. The memory 61 can also be used to temporarily store data that has been output or will be output.

[0162] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the steps in the above-described method embodiments.

[0163] This application provides a computer program product that, when run on a terminal device, enables the terminal device to implement the steps described in the various method embodiments above.

[0164] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a device / terminal equipment, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0165] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0166] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0167] In the embodiments provided in this application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0168] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0169] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. An image segmentation method, characterized in that, The method includes: Acquire multiple first images, wherein the first image is an image obtained by slicing the target image; Feature extraction is performed on each of the first images to obtain multiple first features for each of the first images; Multiple first features of each first image are fused to obtain a second feature corresponding to each first image; The target image is segmented based on the second features corresponding to each of the multiple first images to obtain the image segmentation result; The step of extracting features from each of the first images to obtain multiple first features of the first images includes: m second images are generated based on the target image, each second image containing the farmland shape in the target image, where m is a positive integer; Generate n scale factors based on m of the second images, where n is an integer greater than 1; Wherein, at least one of the m second images contains a third image, the third image containing the complete shape of the cultivated land, and n scale factors are adaptively generated based on the third image; Based on the n scale factors, feature extraction is performed on each of the first images to obtain multiple first features for each of the first images; The step of extracting features from each of the first images based on the n scale factors to obtain multiple first features for each first image includes: For each first image, scale the first image according to the n scale factors to generate n first sub-images corresponding to the first image; Feature extraction is performed on each of the n first sub-images to obtain the first feature of each first sub-image; The step of segmenting the target image based on the second features corresponding to each of the multiple first images to obtain the image segmentation result includes: Multiple first images are merged to obtain a merged third image; The target image is segmented based on the third image and the second feature to obtain multiple predicted segmentation regions; Image segmentation results are obtained based on multiple predicted segmentations; The overlapping region of the multiple predicted segmentation regions is obtained based on the multiple predicted segmentation regions; The mean of the overlapping region is calculated to obtain the result of the mean calculation of the overlapping region; The image segmentation result of the target image is determined based on the calculation result of the mean of the overlapping region and the predicted segmentation region.

2. The image segmentation method as described in claim 1, characterized in that, The step of extracting features from n first sub-images to obtain a first feature for each first sub-image includes: For each of the first sub-images, obtain the local feature information of each of the first sub-images; Obtain the global feature information of the first sub-image; The local feature information of the first sub-image is concatenated with the global feature information of the first sub-image to obtain the first feature of the first sub-image.

3. The image segmentation method as described in claim 1, characterized in that, The step of fusing multiple first features of each first image to obtain a second feature corresponding to each first image includes: For each of the first images, a first process is performed based on the first features of the first image to obtain a plurality of third features of the first image, wherein the first process is used to enhance the feature representation capability of the first features; The first feature of the first image is processed in a second way to obtain multiple fourth features of the first image, wherein the second processing is used to improve the ability of the first feature to capture feature information at different scales. The second feature of the first image is obtained by fusing the third and fourth features.

4. The image segmentation method as described in claim 3, characterized in that, The step of fusing features based on the third feature and the fourth feature to obtain the second feature of the first image includes: For each of the first images, the fifth feature of the first image is obtained by concatenating each of the third features of the first image with its corresponding fourth feature. The fifth feature of the first image is weighted with multiple first features of the first image to obtain the second feature of the first image.

5. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 4.

6. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 4.

Images