An image semantic segmentation method based on dynamic convolution attention

Abstract

The application discloses an image semantic segmentation method based on dynamic convolution attention, which realizes semantic segmentation of an original image, extracts features of the original image by using a ResNet101 network, generates high-level features and low-level features, generates four groups of standard convolution kernels with different sizes through adaptive pooling layers with different scales for the high-level features, generates an initial dynamic convolution kernel by linear combination of the standard convolution kernels through a convolution layer, enhances feature extraction capability of the initial dynamic convolution kernel by using a spatial attention mechanism, generates a final dynamic convolution kernel, realizes multi-scale processing by using the final dynamic convolution kernels with different sizes, generates channel weights of the low-level features by using semantic information of the high-level features, reweights channel information of the low-level features, and fuses the low-level features and the high-level features. The application can effectively extract multi-scale information, uses semantic information of the high-level features and detailed information of the low-level features, and improves accuracy of semantic segmentation.

Description

Technical Field

[0001] This invention relates to the field of image semantic segmentation, and more specifically, to an image semantic segmentation method based on dynamic convolutional attention. Background Technology

[0002] Image semantic segmentation aims to assign a category label to each pixel in an image. In the segmentation results, different colors are used to distinguish targets with different semantic meanings. As a dense prediction task in computer vision, semantic segmentation has wide applications in many fields. In the field of intelligent healthcare, semantic segmentation provides doctors with diagnostic information by analyzing different medical images. In the field of geographic information systems, it provides auxiliary decision-making information for tasks such as regional monitoring, urban planning, and road monitoring by segmenting areas such as roads and rivers in observed images. In the field of intelligent surveillance, semantic segmentation adds safety assurance to monitoring decisions by segmenting targets such as pedestrians and buildings. Especially in the rapidly developing field of autonomous driving, semantic segmentation plays an irreplaceable role.

[0003] Traditional image segmentation relies on low-level semantic information such as color, shape, and texture of objects in an image, and is achieved through threshold comparison, boundary detection, or region division. It underutilizes mid-level semantic information such as pixel contrast and orientation, as well as the spatial relationships between pixels. The segmentation results are insufficient to meet the real-world needs of image understanding, preventing machines from analyzing and understanding the meaning of objects in an image like humans. In 2006, Hinton et al. proposed using deep neural networks to automatically learn high-dimensional features from large amounts of training data. Compared to traditional semantic segmentation methods, deep learning-based semantic segmentation methods eliminate the tedious manual design work, automatically learning features from data using convolutional neural networks, thus saving time and improving efficiency. Furthermore, deep convolutional neural networks can also be used for end-to-end semantic segmentation prediction.

[0004] In image semantic segmentation methods based on deep convolutional neural networks, labeled images with pixel-level semantic labels and the original image are required. These images are then input into a semantic segmentation model for training, resulting in a trained segmentation model. Finally, a test image is input into the trained semantic segmentation model, and the corresponding test result is output. In image semantic segmentation technology, the input is information from a single image, and the output is a pixel-level classification of that single image.

[0005] Multi-scale problems are one of the challenges in image semantic segmentation. In an image, targets of varying sizes often appear, and using a fixed receptive field cannot segment all targets of different sizes. Small targets require smaller receptive fields; when the network's preset receptive field is large, the network tends to capture large-scale targets while ignoring small ones, making them difficult to segment. Conversely, large targets require larger receptive fields; when the network's preset receptive field is small, the segmentation results for large targets will be incomplete. To address the problem of accurate segmentation of targets of different sizes and their boundaries, Li et al. proposed a pyramid attention network (PANet), which uses standard convolutional layers of different sizes to extract multi-scale semantic information from high-level features during encoding. This extracted multi-scale semantic information is then used as attention and multiplied pixel-by-pixel with the original input features. He et al. generated multiple sets of standard convolutional kernels of different sizes using input features, and then linearly combined these kernels using convolutional layers to generate multiple sets of dynamic convolutional kernels of different sizes, thereby extracting multi-scale semantic information. Based on this, He et al. proposed the semantic segmentation network DMNet. DMNet utilizes dynamic convolutions of different sizes to extract multi-scale semantic information, enhancing the network's feature extraction and reasoning capabilities.

[0006] This invention discloses an image semantic segmentation method based on dynamic convolutional attention. In the encoder part, a feature extraction network is used to extract features from the original image, generating high-level and low-level features. The high-level features are used to generate four sets of standard convolutional kernels of different sizes through adaptive pooling operations at different scales. The standard convolutional kernels are linearly combined through convolutional layers to generate dynamic convolutional kernels. The convolutional kernel parameters in the convolutional layers are learned by backpropagation of the network. The standard convolutional kernels are generated from the input high-level features. Different input high-level features generate different standard convolutional kernels, which are then linearly combined to form different dynamic convolutional kernels. Convolutional kernel parameter weights are generated based on the global information of the dynamic convolutional kernels. The parameters of the dynamic convolutional kernels are readjusted using the convolutional kernel parameter weights to generate the final dynamic convolutional kernels, thereby extracting semantic information at different scales. In the decoder section, high-level features aggregate global information through global pooling. This global information is then input into multiple sets of one-dimensional convolutional layers at different scales, enabling information interaction between different channels and fully considering the influence of multiple adjacent channels on the global information. Subsequently, dimensionality reduction is used to generate channel weights for low-level features, and the channel information of the low-level features is reweighted. Simultaneously, the upsampled high-level and low-level features are fused to generate the final segmentation result. The method disclosed in this invention can fully utilize dynamic convolution for multi-scale feature extraction during encoding, and can use high-level features to select channels for low-level features during decoding. Furthermore, it utilizes the semantic information of high-level features and the detailed information of low-level features for semantic segmentation. Summary of the Invention

[0007] In view of this, embodiments of the present invention provide an image semantic segmentation method based on dynamic convolutional attention to achieve semantic segmentation of the original image.

[0008] To achieve the above objectives, the embodiments of the present invention provide the following solutions:

[0009] A semantic segmentation method for images based on dynamic convolutional attention, characterized by comprising the following steps:

[0010] Step 1. Feature extraction;

[0011] The backbone network of this invention is a ResNet101 residual network with 101 convolutional layers. This network has five sets of residual structures: the first set consists of a single convolutional layer; the second to fifth sets consist of 3, 4, 23, and 3 residual structures, respectively. Each residual structure comprises three convolutional layers with kernel sizes of 1×1, 3×3, and 1×1, respectively. In the fourth and fifth sets of residual structures, the stride is set to 1, and the convolutional layers are replaced with dilated convolutional layers with dilation rates of 2 and 4, respectively, to expand the receptive field.

[0012] Step 2. Extract multi-scale information using a multi-scale dynamic filters attention module (MDAM);

[0013] MDAM consists of four Dynamic Filters Attention Modules (DAMs). High-level features are aggregated with multi-scale semantic information through four parallel DAMs, and then connected with the original high-level features along the channels. The resulting data is then input to the decoder through a 1×1 convolutional layer.

[0014] Step 2.1 Generate dynamic convolutional kernels;

[0015] The high-level features extracted by the ResNet101 feature extraction network are used to obtain a set of standard convolutional kernels through adaptive average pooling. A 1×1 convolutional layer is then used to linearly combine these standard kernels along the channels to generate dynamic convolutional kernels of size k×k. To enhance the spatial contextual information interaction of the kernel parameters, a spatial attention mechanism is used to readjust the parameters of the generated initial dynamic convolutional kernels, as shown in the following formula:

[0016] F k =σ(f 3×3 ([AvgPool(f k MaxPool(f) k )]))

[0017] In the formula, f kThis represents a dynamic convolution kernel of size k×k, f 3×3 This represents a convolution operation with a kernel size of 3×3, where σ represents the activation operation using the sigmoid activation function, and F... k This represents the calculated kernel parameter weights of the dynamic convolution kernel.

[0018] The calculated kernel parameter weights are multiplied by the parameters at the corresponding positions of the initial convolution kernel to obtain the final dynamic convolution kernel.

[0019] Step 2.2 Extract multi-scale information;

[0020] Step 2.1 generates four sets of dynamic convolutional kernels of different sizes, where k∈[1,3,5,7], and the extracted high-level features are then dimensionality-reduced using a 1×1 convolutional layer. The generated dynamic convolutional kernels are then used to aggregate multi-scale semantic information from the high-level features through depthwise separable convolution.

[0021] The extracted multi-scale features and the original high-level features are concatenated along the channels, reduced in dimensionality by a 1×1 convolutional layer, and then input into the decoder.

[0022] Step 3. Use the Feature Guide Module (FGM) to guide the selection of channels for low-level features and perform feature fusion;

[0023] The output of step 2 is used as high-level features. A global pooling layer aggregates the global semantic information of these high-level features. To enhance information interaction between adjacent channels, four sets of one-dimensional convolutional layers of different sizes are used to extract channel information at different scales. Then, a 1×1 convolutional layer is used for dimensionality reduction to obtain the channel weights of the low-level features. Each channel of the low-level features is multiplied by its obtained channel weights, leveraging the rich semantic information in the high-level features to guide channel selection in the low-level features, enhancing channels beneficial to segmentation performance while suppressing others. Simultaneously, to fully utilize the detailed information of the low-level features, the upsampled high-level features and the low-level features adjusted for channel weights are fused.

[0024] Step 4. Update network parameters and check for convergence;

[0025] Through the above steps, an iterative solution is performed to obtain the image semantic segmentation result. After obtaining the segmentation result, the loss value of this iteration is calculated using the loss function, and the momentum gradient descent method is used for optimization, and the network parameters are updated. When the maximum number of iterations is reached or the iteration converges, the iteration stops and the final segmentation result is output; otherwise, steps 1, 2, and 3 are repeated.

[0026] Preferably, the number of images per batch in the algorithm is set to 16.

[0027] Preferably, the algorithm iterates for a total of 100 epochs.

[0028] Preferably, the algorithm uses a polynomial decay learning rate update strategy to update the learning rate, with an initial learning rate of 4e-3 and a polynomial exponent of 0.9.

[0029] Preferably, the optimization method of the algorithm is momentum gradient descent, where the momentum coefficient is 0.9.

[0030] This invention combines dynamic convolution and spatial attention mechanisms in the encoder section and utilizes high-level features to guide low-level features in channel selection in the decoder section, proposing an image semantic segmentation method based on dynamic convolutional attention. During encoding, a feature extraction network is used to extract features from the original image, obtaining high-level and low-level features. The high-level features are used to obtain four sets of standard convolutional kernels of different sizes through adaptive pooling operations at different scales. These standard convolutional kernels are linearly combined through 1×1 convolutional layers to generate dynamic convolutional kernels. To fully consider the influence of global information on the convolutional kernel parameters, a spatial attention mechanism is used to generate convolutional kernel parameter weights based on the global information of the initial dynamic convolutional kernel. The parameters of the initial dynamic convolutional kernel are then readjusted using these weights to obtain the final dynamic convolutional kernel, thereby extracting spatial information at different scales. During decoding, high-level features are processed through global pooling to obtain global information. To enhance information interaction between adjacent channels, this global information is input into multiple sets of one-dimensional convolutional layers of different sizes, fully considering the influence of multiple adjacent channels on the global information. Subsequently, dimensionality reduction is performed to obtain the channel weights of low-level features, and the channel information of low-level features is reweighted. Simultaneously, the upsampled high-level and low-level features are fused to obtain the final segmentation result. The method disclosed in this invention can utilize global information to generate dynamic convolutional kernels and extract multi-scale information during encoding, and can utilize the semantic information of high-level features to guide low-level features in channel selection for semantic segmentation during decoding. Attached Figure Description

[0031] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 This is a schematic diagram of the image semantic segmentation method based on dynamic convolutional attention provided in an embodiment of the present invention;

[0033] Figure 2 A schematic diagram of MDAM provided in an embodiment of the present invention;

[0034] Figure 3 A schematic diagram of DAM provided in an embodiment of the present invention;

[0035] Figure 4 This is a schematic diagram of the FGM provided in an embodiment of the present invention;

[0036] Figure 5 This is a flowchart of the image semantic segmentation method based on dynamic convolutional attention provided in an embodiment of the present invention; Detailed Implementation

[0037] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0038] In image semantic segmentation, scale context describes the possible scale range of targets in an image scene, corresponding to size relationships. Specifically, scale context manifests as the presence of targets of different sizes in an image. Using a fixed-size receptive field cannot segment all targets. Some small targets require smaller receptive fields, and a network with a pre-set large receptive field will have difficulty segmenting small targets. Conversely, large targets require larger receptive fields, and a network with a pre-set small receptive field will result in discontinuous segmentation of large targets.

[0039] High-level features have lower resolution but contain rich semantic information, primarily used for target localization and classification. Low-level features have higher resolution but contain more detailed information, such as texture, edge information, and features related to local appearance. Although low-level features are insufficient for classification, they help improve the details of target boundaries. Therefore, this invention utilizes MDAM to extract multi-scale information during encoding and uses FGM to reuse the semantic information of high-level features to guide low-level features in channel selection and feature fusion during decoding, constructing a... Figure 1 The image semantic segmentation model shown.

[0040] Using ResNet101 as the backbone network, standard convolutions are replaced with dilated convolutions with dilation rates of 2 and 4 in the fourth and fifth residual structures of the backbone network, respectively, to aggregate richer semantic features while preserving the feature map size. Figure 1 The feature map output by each residual structure is F nFor n∈[1,2,3,4], the feature map F1 output from the fifth residual structure is input into the MDAM module. Multiple sets of dynamic convolutions of different sizes are used to obtain feature maps of different scales. By aggregating feature maps of different scales, more semantic information is obtained, enabling more effective simultaneous segmentation of large and small targets in the image. Although the feature maps obtained through the MDAM module aggregate rich semantic and multi-scale information, details such as edges are lost due to the small size of the feature maps at this point. To recover the details in the image and preserve the semantic information in the feature maps, three FGM modules are used to continuously pass the inter-class correlation semantic information from high-level features to adjacent low-level features, helping low-level features to perform channel feature selection and feature fusion, enhancing the details in the feature maps. Finally, an upsampling layer restores the feature maps to the same size as the original image, enabling classification and obtaining the segmentation result.

[0041] To extract multi-scale information from feature maps and fully utilize semantic information in high-level features and detailed information in low-level features, this embodiment discloses an image semantic segmentation algorithm based on dynamic convolutional attention to achieve semantic segmentation of images. The above method includes at least the following steps.

[0042] Step 1. Feature extraction;

[0043] The backbone network constructed in this invention is a ResNet101 residual network with 101 convolutional layers. This network can be divided into five groups of residual structures: the first group consists of a single convolutional layer with a kernel size of 7×7 and a stride of 2. The second to fifth groups consist of 3, 4, 23, and 3 residual structures, respectively. Each residual structure consists of three convolutional layers with kernel sizes of 1×1, 3×3, and 1×1, respectively. The number of channels in the output feature maps of the residual structures in the first to fifth groups are 64, 256, 512, 1024, and 2048, respectively. In the fourth and fifth groups of residual structures, the stride is set to 1, and the convolutional layers are replaced with dilated convolutional layers with dilation rates of 2 and 4, respectively. This expands the receptive field while preserving the feature map size, aggregating rich semantic information, and obtaining the feature maps output by each group of residual structures. The model parameters of the backbone network are shown in Table 1.

[0044] Table 1 Backbone Network Model Parameters

[0045]

[0046] Step 2. Extract multi-scale information using MDAM;

[0047] MDAM consists of four DAMs, such as Figure 2As shown, in MDAM, the high-level feature F1 aggregates multi-scale semantic information through four parallel DAMs. Then, the generated multi-scale feature map and the high-level feature F1 are connected along the channel and input to the decoder part through a 1×1 convolutional layer.

[0048] Step 2.1 Generate dynamic convolutional kernels;

[0049] Using convolution kernels of different sizes can extract feature information at different scales. However, the kernel parameters of standard convolution are shared for different input features. When the distribution positions of important features in two feature maps are not the same, standard convolution is difficult to extract feature information that is beneficial to segmentation at the same time. At the same time, using multiple sets of standard convolutions to extract semantic features can easily lead to a sharp increase in the computational cost of the model.

[0050] Therefore, in DAM, the present invention, as Figure 3 As shown, the high-level feature F1 is processed through an adaptive pooling layer to aggregate local feature information, thereby generating the kernel parameters of the standard convolution. The high-level feature F1 has 2048 channels, so 2048 standard convolution kernels are generated, with a size of k×k, k∈[1,3,5,7]. To enhance the network's feature extraction capability, the generated standard convolutions are fed into a 1×1 convolutional layer. The 1×1 kernels are used to integrate cross-channel features of the standard convolutions, and then linearly combined into 512 k×k convolution kernels, i.e., dynamic convolution kernels.

[0051] The initial standard convolutional kernels are generated based on the input features. The parameters in the 1×1 convolutional layer are learned through backpropagation. Different input features result in different standard convolutional kernels. Furthermore, by integrating cross-channel features through 1×1 convolutional layers, different dynamic convolutional kernels can be generated. However, while the dynamic convolutional kernels generated in this stage integrate the channel characteristics of standard convolutions in the channel dimension, they lack contextual spatial information interaction between kernel parameters in the spatial dimension. Therefore, a spatial attention mechanism is used to weight the kernel parameters of the dynamic convolutional kernels, enhancing their spatial information interaction capabilities. The formula for calculating the kernel parameter weights of the dynamic convolutional kernels is as follows:

[0052] F k =σ(f 3×3 ([AvgPool(f k MaxPool(f) k (1)

[0053] In the formula, f k This represents a dynamic convolution kernel of size k×k, f 3×3 This represents a convolution operation with a kernel size of 3×3, where σ represents the activation operation using the sigmoid activation function, and F... kThis represents the calculated kernel parameter weights of the dynamic convolution kernel.

[0054] The specific process is as follows: the obtained dynamic convolution is input into a spatial max pooling layer and an average pooling layer respectively to obtain two feature maps of size k×k with 1 channel. Then, the two feature maps are concatenated along the channels and input into a convolutional layer with a kernel size of 3×3 to obtain a k×k feature map with 1 channel. Finally, the generated feature map is activated by the Sigmoid activation function to obtain the kernel parameter weights of the dynamic convolution kernel.

[0055] The calculated parameter weights are multiplied by the parameters at the corresponding positions of the dynamic convolution kernel, and the kernel parameters of the dynamic convolution kernel are readjusted to generate the final dynamic convolution kernel.

[0056] Each dynamic convolution is a linear combination of multiple standard convolutions. The kernel parameters of the standard convolutions are determined based on the input of the adaptive pooling layer. Then, a 1×1 convolutional layer is used to linearly combine the standard convolutions to generate a dynamic convolution kernel suitable for the input. The kernel parameters of the dynamic convolution are weighted and tuned using a spatial attention mechanism. Finally, only one convolution operation is needed on the input using the dynamic convolution to achieve a training effect comparable to multiple standard convolutions, thus improving the feature extraction capability of the network.

[0057] Step 2.2 Extract multi-scale information;

[0058] like Figure 3 As shown, while calculating the dynamic convolution kernel, the number of channels of the high-level feature F1 is reduced to 512 using a 1×1 convolutional layer. The generated dynamic convolution kernel is used to aggregate the semantic information of the high-level feature F1 in a depth-separable convolutional manner.

[0059] A DAM generates a set of dynamically convolutional layers to extract semantic information from high-level features. In this invention, four parallel DAMs are used to aggregate multi-scale semantic information, such as... Figure 2 As shown, the dynamic convolutional kernel sizes generated in the four DAMs are 1×1, 3×3, 5×5, and 7×7, respectively. Different sizes of dynamic convolutional kernels can extract semantic information of targets of different sizes, as well as important features of each target in multi-scale scenarios. Finally, the extracted multi-scale features and the original high-level features F1 are concatenated along the channels, reduced in dimensionality by a 1×1 convolutional layer, and then input into the decoder.

[0060] Step 3. Use FGM to guide channel selection for low-level features and perform feature fusion;

[0061] The output of MDAM is high-level features, which contain rich semantic information. The FGM module can pass the inter-class semantic information of the high-level features to the adjacent low-level features, thereby guiding the low-level features to select channel features.

[0062] Figure 4 This is a schematic diagram of FGM (Full Global Pooling). In FGM, high-level features are aggregated through a global pooling layer to gather global semantic information. To fully utilize the global information of channels at different scales, four sets of one-dimensional convolutional layers of different sizes are used in FGM to propagate the correlation information between different classes. In this invention, the sizes of the one-dimensional convolutions are 1×9, 1×17, 1×33, and 1×65. Subsequently, dimensionality reduction is achieved through a 1×1 convolutional layer to generate channel feature weights for low-level features. The channels of the low-level features and the obtained channel feature weights are multiplied accordingly. The rich semantic information in the high-level features is used to adjust the channel feature information of the low-level features. Simultaneously, the upsampled high-level features and the low-level features adjusted by the channel weights are fused to fully utilize the detailed information of the low-level features and the semantic information of the high-level features.

[0063] In the decoder section, this invention uses a total of three FGMs, such as... Figure 1 As shown, the high-level features in the first FGM are the aggregated features output by MDAM, and the low-level features are F2 output by the fourth set of residual structures. The high-level features in the second FGM are the output of the first FGM, and the low-level features are F3 output by the third set of residual structures. And so on. The high-level features in the third FGM are the output of the second FGM, and the low-level features are F4 output by the second set of residual structures. Because the feature maps F2, F3, and MDAM output are one-eighth the size of the original image, upsampling only occurs in the third FGM. In the first and second FGMs, the high-level features are directly fused with the low-level features after channel feature adjustment.

[0064] Three FGMs gradually propagate the inter-class correlation information in high-level features to adjacent low-level features, and continuously perform feature fusion to restore detailed information. Finally, the input is fed into the upsampling layer, where bilinear interpolation is used to perform upsampling by 2 times to restore the resolution of the feature map.

[0065] Step 4. Update network parameters and check for convergence;

[0066] Through the above steps, an iterative solution is performed to obtain the image semantic segmentation result. After obtaining the segmentation result, the loss value of this iteration is calculated using the loss function, and the momentum gradient descent method is used for optimization, and the network parameters are updated. When the maximum number of iterations is reached or the iteration converges, the iteration stops and the final segmentation result is output; otherwise, steps 1, 2, and 3 are repeated.

[0067] Preferably, the number of images per batch during algorithm training is set to 16. The algorithm uses a polynomial decay learning rate update strategy for learning rate updates, with an initial learning rate of 4e-3 and a polynomial exponent of 0.9. The optimization method of the algorithm is momentum gradient descent, where the momentum coefficient is 0.9. The model training iterates for 100 epochs. After each epoch, if the segmentation effect of the currently trained model is better than the saved model, the current model parameters are saved and the iteration proceeds to the next epoch; otherwise, the iteration proceeds directly to the next epoch.

[0068] This invention validates the disclosed method on the PASCAL VOC 2012 dataset. The original dataset contained 1464 images for training, 1449 images for validation, and 1456 images for testing. Later, images from the SBD dataset were added, expanding the training set to 10582 images. The entire dataset is labeled with four main categories: humans, animals, indoor furniture and household items, and vehicles. Animals include six categories: birds, cats, cows, dogs, horses, and sheep; indoor furniture and household items include six categories: bottles, chairs, dining tables, potted plants, sofas, and televisions; and vehicles include seven categories: airplanes, bicycles, boats, buses, cars, motorcycles, and trains. Including the background, there are a total of 21 categories. All original images in the dataset are in RGB mode, and the labels are 8-bit color images.

[0069] This invention uses the mean Intersection over Union (mIoU) and pixel accuracy (PA) as quantitative evaluation metrics. Higher mIoU and PA values ​​indicate a closer match between the network's predicted segmentation result and the labeled image, signifying better segmentation performance. Both DMNet and PANet utilize multi-scale semantic information in their encoders. Table 2 quantitatively compares the mIoU and PA of the method disclosed in this invention with those of DMNet and PANet on the PASCAL VOC 2012 dataset. The table shows that, compared to DMNet, the method disclosed in this invention improves mIoU by 0.87% and PA by 0.30%; compared to PANet, it improves mIoU by 0.64% and PA by 0.27%.

[0070] Table 2 Comparison of the method disclosed in this invention with other methods in terms of mIoU and PA.

[0071]

[0072]

[0073] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined in this invention may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An image semantic segmentation method based on dynamic convolutional attention, characterized in that, Includes the following steps: Step 1. Feature extraction; The backbone network is a ResNet101 residual network with 101 convolutional layers. This network has five sets of residual structures: the first set is a single convolutional layer; the second to fifth sets consist of 3, 4, 23, and 3 residual structures, respectively. Each residual structure consists of three convolutional layers with kernel sizes of 1×1, 3×3, and 1×1, respectively. In the fourth and fifth sets of residual structures, the stride is set to 1, and the convolutional layers are replaced with dilated convolutional layers with dilation rates of 2 and 4, respectively, to expand the receptive field. Step 2. Extract multi-scale information using the Multi-Scale Dynamic Convolutional Attention Module (MDAM); MDAM consists of four dynamic convolutional attention modules (DAMs). High-level features are aggregated with multi-scale semantic information through four parallel DAMs, and then connected with the original high-level features along the channels. The input is then fed into the decoder through a 1×1 convolutional layer. Step 2.1 Generate dynamic convolutional kernels; The high-level features extracted by the ResNet101 feature extraction network are used to obtain a set of standard convolutional kernels through adaptive average pooling. A 1×1 convolutional layer is then used to linearly combine these standard kernels along the channels to generate dynamic convolutional kernels of size k×k. To enhance the spatial contextual information interaction of the kernel parameters, a spatial attention mechanism is used to readjust the parameters of the generated initial dynamic convolutional kernels, as shown in the following formula: F k =σ(f 3×3 ([AvgPool(f k );MaxPool(f k )])) In the formula, f k This represents a dynamic convolution kernel of size k×k, f 3×3 This represents a convolution operation with a kernel size of 3×3, where σ represents the activation operation using the sigmoid activation function, and F... k This represents the calculated kernel parameter weights of the dynamic convolution kernel; multiplying the calculated kernel parameter weights by the parameters at the corresponding positions of the initial convolution kernel yields the final dynamic convolution kernel; Step 2.2 Extract multi-scale information; Step 2.1 generates four sets of dynamic convolutional kernels of different sizes, where k∈[1,3,5,7], and the extracted high-level features are dimensionality-reduced through a 1×1 convolutional layer. Using the generated dynamic convolutional kernels, multi-scale semantic information from the high-level features is aggregated through depthwise separable convolution. The extracted multi-scale features and the original high-level features are concatenated along the channels, reduced in dimensionality by a 1×1 convolutional layer, and then input into the decoder. Step 3. Use the Feature Guidance Module (FGM) to guide low-level features in channel selection and feature fusion; The output of step 2 is used as a high-level feature. The global semantic information of the high-level feature is aggregated through a global pooling layer. In order to enhance the information interaction between adjacent channels, four sets of one-dimensional convolutional layers of different sizes are used to extract channel information at different scales. Then, dimensionality reduction is performed through a 1×1 convolutional layer to obtain the channel weights of the low-level feature. Each channel of the low-level feature is multiplied by the obtained channel weights. The rich semantic information in the high-level feature guides the low-level feature to select channels, enhance the channels that are beneficial to the segmentation performance, and suppress other channels. The upsampled high-level feature and the low-level feature after channel weight adjustment are then fused. Step 4. Update network parameters and check for convergence; Through the above steps, an iterative solution is performed on the semantic segmentation result of the image. After obtaining the segmentation result, the loss value of this iteration is calculated using the loss function, and the momentum gradient descent method is used for optimization, and the network parameters are updated. When the maximum number of iterations is reached or the iteration converges, the iteration stops and the final segmentation result is output; otherwise, steps 1, 2 and 3 are repeated.

2. The image semantic segmentation method based on dynamic convolutional attention according to claim 1, characterized in that, The number of images per batch is set to 16.

3. The image semantic segmentation method based on dynamic convolutional attention according to claim 1, characterized in that, A total of 100 epochs were performed.

4. The image semantic segmentation method based on dynamic convolutional attention according to claim 1, characterized in that, A polynomial decay learning rate update strategy is used to update the learning rate, with an initial learning rate of 4e-3 and a polynomial exponent of 0.

9.

5. The image semantic segmentation method based on dynamic convolutional attention according to claim 1, characterized in that, The optimization method is momentum gradient descent, with a momentum coefficient of 0.9.

Images