An image semantic segmentation method, device and medium
Patent Information
- Application Number
- CN202411135518.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-19
- Publication Date
- 2026-10-27
- Estimated Expiration
- 2044-08-19
AI Technical Summary
然而，深度学习模型通常需要大量的标注数据进行训练，而在实际的工业生产中，获取大量的标注数据是一项非常困难的任务
[0059]本申请通过括骨干网络MobileNetv2和高级特征融合模块以及DenseSPASPP模块有效地解决小目标物体的分割难题和物体边缘的识别问题，增强模型对环境的适应性；构建AFF模块、白化变换模块和优化的损失函数，对DeepLabv3+进行了优化，进一步的提高了在复杂场景下小目标物体的分割效果和模型的域适应性，解决了数据集中正负样本不平衡的问题，从而大大提高了在复杂工业场景下图像分割的准确性和鲁棒性。
Smart Images
Figure CN119206206B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the fields of computer vision and image processing technology, specifically to an image semantic segmentation method, apparatus, and medium. Background Technology
[0002] In intelligent instrument analysis, the segmentation of instruments and their components is a crucial task, significantly impacting automated analysis and inspection. However, due to the complexity of industrial scenarios, instruments often include small objects, making the segmentation of these objects challenging, particularly in terms of segmentation, edge recognition, and adaptation to diverse environments.
[0003] Traditional image processing methods, such as thresholding and edge detection, while achieving good results in simple scenarios, often fail to meet the performance requirements of complex industrial environments. Furthermore, these methods typically require extensive manual parameter tuning and cannot adapt to environmental changes. In recent years, deep learning technology has made significant progress in image processing. Convolutional Neural Networks (CNNs), in particular, have demonstrated superior performance in tasks such as image classification, object detection, and semantic segmentation. However, deep learning models typically require large amounts of labeled data for training, which is a very difficult task in actual industrial production. Moreover, the training process for deep learning models usually requires significant computational resources and time, posing a major challenge for practical industrial applications.
[0004] Therefore, designing a high-precision segmentation device capable of operating in complex industrial scenarios while overcoming limitations in data availability and computational resources is a crucial research topic. This invention aims to address these issues by proposing an image semantic segmentation method. Summary of the Invention
[0005] This invention aims to overcome the limitations of existing technologies by proposing an image semantic segmentation method, apparatus, and medium. This method effectively solves the challenges of segmenting small objects and recognizing object edges, enhances the model's adaptability to the environment, and improves the accuracy and robustness of object segmentation.
[0006] To achieve the above objectives, this application employs the following technical solution:
[0007] In a first aspect, this application provides an image semantic segmentation method, which includes the following steps;
[0008] Obtain the image to be segmented;
[0009] The image features are extracted using the MobileNetv2 backbone network of the pre-trained semantic segmentation model to obtain the semantic information of the image and obtain a set of feature maps.
[0010] The semantic information of the feature map group is fused using the feature fusion module of the semantic segmentation model to obtain a first feature map;
[0011] The feature map group is pooled and dilated convolutionally processed by the DenseSPASPP module of the semantic segmentation model, and the pooling result and the dilated convolution result of the last layer of the feature map group are concatenated to obtain the second feature map.
[0012] The second feature map and the first feature map are concatenated to obtain the semantic segmentation image of the image.
[0013] A further embodiment of this application states that the training method for the semantic segmentation model includes the following steps;
[0014] The training process of the semantic segmentation model includes:
[0015] Obtain the image dataset;
[0016] The images in the dataset are labeled by meticulously outlining the boundaries of objects using the LabelMe tool and the polygon plotting method.
[0017] The pixel proportions of the objects in the dataset are divided according to an 8:1:1 ratio;
[0018] The dataset is input into a pre-built semantic segmentation model for iterative training until the model performance reaches a stable state.
[0019] A further aspect of this application involves fusing the features of the feature map group using the feature fusion module of the semantic segmentation model, including:
[0020] Feature map F1 in the feature map group is convolved to generate feature map F1', and feature map F1' is processed by a coordinate attention mechanism to generate a height feature map. and width feature map The calculation formula is as follows:
[0021]
[0022] Where c is the channel index, xc indicates that pooling operation is performed on channel c; W represents the number of summation loops, h represents the height of feature map F1', and w represents the width of feature map F1'.
[0023] height feature map and width feature map The feature map F0 is generated by concatenating and convolving the data.
[0024]
[0025] r represents the channel change rate of the convolution; H represents the height of the height feature map; and W represents the width of the width feature map.
[0026] The feature map F0 is split into two feature maps F0. h and F w The weight g is obtained by processing it using the following formula. h and g w ;
[0027] g h =σ(F h (f h ))
[0028] g w =σ(F w f w ))
[0029] Among them, F h Represents height feature map, F w σ represents the result of the convolution operation on the width feature map;
[0030] S-shaped bar pooling is performed on feature map F2 in the feature map group. The processed feature map F2 and feature map F0 are added together to obtain an enhanced feature map. The enhanced feature map is then processed by the ReLU activation function to obtain the first feature map.
[0031] Among them, feature map F1 and feature map F2 are two feature maps of different scales obtained by the backbone network MobileNetv2 from the image feature extraction.
[0032] A further aspect of this application involves using the DenseSPASPP module of the semantic segmentation model to perform pooling and dilated convolution processing on the images in the feature map group, including:
[0033] The feature map Y1 is obtained by performing S-shaped bar pooling on the images in the feature map group, as shown in the following formula.
[0034] Y1 = sp(X)
[0035] Where sp represents the bar pooling operation; X represents the image sample;
[0036] The feature map Y2 is obtained by sequential dilated convolution of the images in the feature map group, as shown in the following formula.
[0037]
[0038] Among them, ac l In this context, l represents the void ratio of the l-th layer, DC represents the calculation of dilated convolution, k represents the kernel size, [...] represents the concatenation operation, and y i Let i represent the image features of the i-th layer, where i ranges from [0 to l-1] and l represents the number of layers in the network.
[0039] The feature map Y1 and the feature map Yn processed by the last dilated convolution layer are concatenated to obtain the second feature map;
[0040] The second feature map is concatenated with the first feature map to obtain the image segmentation result.
[0041] In a further embodiment of this application, the feature map group is further processed by a whitening transformation module;
[0042] The whitening transformation module processes the following:
[0043] Select the feature map X from the last layer of the feature map group and perform instance normalization;
[0044] The feature map X after instance normalization is subjected to whitening transformation to obtain the feature map X covariance with the identity matrix I, so that the shape of the image processed by the feature fusion module and the DenseSPASPP module is consistent.
[0045] A further proposed approach is to use the loss function L of the whitening transformation process. iw The formula is as follows:
[0046]
[0047] Where I is an upper triangular matrix consisting entirely of 1s, MSE is the mean squared error loss function, and F... iw It is the product of the feature map of the last layer of the backbone network and the transpose of the feature map.
[0048] A further aspect of this application involves iteratively training the pre-built semantic segmentation model using the dataset during training until the model's performance reaches a stable state. This includes evaluating the performance of the semantic segmentation model using Dice Loss and Focal Loss loss functions, as shown in the following formulas:
[0049] Focal CE Loss=(1-e -ce ) γ ×ce
[0050] Where ce represents the calculated result of cross-entropy loss, γ is the adjustment factor, and e represents a variable of prediction probability;
[0051] L dice =1-dice
[0052]
[0053] Here, X and Y represent two different image samples.
[0054] In a second aspect, this application provides a computer device including a memory for storing computer programs / instructions;
[0055] A processor is used to execute the computer program / instructions to implement the steps of the above-described image semantic segmentation method.
[0056] Thirdly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described image semantic segmentation method.
[0057] Fourthly, this application provides a computer program product, including a computer program / instructions, characterized in that the computer program / instructions, when executed by a processor, implement the steps of the above-described image semantic segmentation method.
[0058] The beneficial effects of this application are as follows:
[0059] This application effectively addresses the challenges of segmenting small objects and identifying object edges by incorporating the MobileNetv2 backbone network, an advanced feature fusion module, and the DenseSPASPP module, thereby enhancing the model's adaptability to the environment. Furthermore, it constructs an AFF module, a whitening transformation module, and an optimized loss function, further improving the segmentation performance of small objects in complex scenes and the model's domain adaptability. This also resolves the problem of imbalanced positive and negative samples in the dataset, significantly enhancing the accuracy and robustness of image segmentation in complex industrial scenarios. Attached Figure Description
[0060] Figure 1 This is a schematic diagram of the AFF module in the embodiments of this application;
[0061] Figure 2 This is a schematic diagram of the structure of the DenseSPASPP module in the embodiments of this application;
[0062] Figure 3 This is a schematic diagram of the structure of the whitening block module in the embodiments of this application;
[0063] Figure 4 This is a schematic diagram of the semantic segmentation model in the embodiments of this application. Detailed Implementation
[0064] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations thereof. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.
[0065] The term "and / or" simply describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship.
[0066] Example 1
[0067] like Figures 1 to 4 As shown, this embodiment discloses an image semantic segmentation method, which includes the following steps;
[0068] Obtain the image to be segmented;
[0069] The image features are extracted using the MobileNetv2 backbone network of a pre-trained semantic segmentation model to obtain a set of feature maps.
[0070] The features of the feature map group are fused using the feature fusion module of the semantic segmentation model to obtain a first feature map;
[0071] The feature map group is pooled and dilated convolutionally processed by the DenseSPASPP module of the semantic segmentation model, and the pooling result and the dilated convolution result of the last layer of the feature map group are concatenated to obtain the second feature map.
[0072] The second feature map and the first feature map are concatenated to obtain the segmentation result of the image.
[0073] In some embodiments, such as Figure 4 As shown, an image semantic segmentation method is implemented as follows:
[0074] (1) Improve and optimize the DeepLabv3+ baseline model to construct a semantic segmentation model;
[0075] (2) Obtain images and construct a dataset;
[0076] (3) Train the semantic segmentation model described above;
[0077] (4) Validate the semantic segmentation model.
[0078] Step (1) Improve and optimize the DeepLabv3+ baseline model to construct a semantic segmentation model, including the following steps:
[0079] S1. An Advanced Feature Fusion (AFF) module was designed to process high-dimensional feature data, reduce the feature extraction loss of small objects, and improve the segmentation effect of small objects.
[0080] S2. The DenseSPASPP module, which integrates S-shaped strip pooling (SP), replaces the adaptive spatial pyramid pooling (ASPP) module in DeepLabv3+, thereby expanding the receptive field of the decoder.
[0081] S3. A whitening transformation module was introduced to ensure the consistency and accuracy of image processing;
[0082] S4. A loss function combining Log-cosh Dice Loss and Focal Loss is adopted to optimize the model's handling of imbalanced positive and negative samples.
[0083] Step S1 involves designing an Advanced Feature Fusion (AFF) module to process high-dimensional feature data, reduce feature extraction loss for small objects, and improve segmentation performance for small objects.
[0084] Its structural diagram is as follows Figure 1 As shown. The AFF module begins with two feature maps F1 and F2 of different scales extracted from the backbone network. F1 is a feature map downsampled by a factor of four, while F2 is a feature map downsampled by a factor of eight. F1 is transformed using a 1x1 convolutional kernel to generate F1', which is then added to the subsequently processed F2. The purpose of this step is to adjust the number of channels in F1 to match the feature dimensions of F2. A coordinate attention mechanism is applied to the transformed F1'. First, F1' is average-pooled in the height (H) and width (W) directions using pooling kernels of size [1,H] and [1,W], generating two feature maps, represented as follows: and
[0085]
[0086] Where c is the channel index, x c This represents a pooling operation performed on channel c, where W represents the number of summation loops, h represents the height of feature map F1', and w represents the width of feature map F1'.
[0087] The height and width feature maps obtained above are concatenated and a new feature map F0 is generated through convolution operation.
[0088]
[0089] Where r represents the channel change rate of the convolution;
[0090] The F1 algorithm is split into two feature maps F. h and F w These correspond to height and width information, respectively. Then, a 1x1 convolution kernel is used to convolve them to unify the channel dimensions, and a non-linearity is introduced through the sigmoid function to obtain the weights g. h and g w .
[0091] g h =σ(F h (f h )),g w =σ(F w (f w ))
[0092] Among them, F h Represents height feature map, F w σ represents the result of the convolution operation on the width feature map;
[0093] The obtained weight g h and g w The feature map is expanded and multiplied with the original feature map to enhance its spatial location information.
[0094] F2 is upsampled by a factor of two using linear interpolation to match the size of F1. Then, a convolution with a dilation rate of 2 is performed on the upsampled feature map to expand the receptive field. The SP (Sparingly Pulsed) bar pooling module is used to process the upsampled F2, capturing long-range features and enhancing global semantic information. The SP module expands the feature map using 1D convolution and uses the Sigmoid function to increase non-linear learning capability. The F1' processed by the coordinate attention mechanism and the F2 processed by the SP module are added together to obtain the enhanced feature map. Finally, the ReLU activation function is used to perform a non-linear transformation on the fused feature map to obtain the final output F3 of the AFF module.
[0095] Step S2 employs the DenseSPASPP module, which integrates S-shaped strip pooling (SP), to replace the adaptive spatial pyramid pooling (ASPP) module in DeepLabv3+, thereby expanding the receptive field of the decoder.
[0096] Its structural diagram is as follows Figure 2As shown, ASPP utilizes dilated convolutions with different dilation rates to expand the model's receptive field. To overcome the shortcomings of ASPP, the DenseSPASPP structure organizes dilated convolutions in a cascaded manner, with the dilation rate arranged from smallest to largest, thereby achieving denser sampling and a wider receptive field. The top layer of DenseSPASPP uses SP bar pooling, a special type of pooling operation that uses long bar-shaped pooling kernels to capture long-range dependencies in the image. The output of SP pooling is represented as:
[0097] Y1 = sp(X)
[0098] Here, sp represents the strip pooling operation. Dilated convolutions in DenseSPASPP are performed in a concatenated manner, with the output of each dilated convolution layer serving as the input to the next layer.
[0099]
[0100] Among them, ac l In this context, l represents the void ratio of the l-th layer, DC represents the calculation of dilated convolution, k represents the kernel size, [...] represents the concatenation operation, and y i Let i represent the image features of the i-th layer, where i ranges from [0 to l-1] and l represents the number of layers in the network.
[0101] The final output of DenseSPASPP is the concatenation of the SP strip pooling result Y1 and the result of the last layer of cascaded dilated convolution:
[0102] Y = [Y1, Yn]
[0103] Here, Yn represents the final feature map after all cascaded dilated convolutional layers. In DenseSPASPP, feature maps with different dilation rates are combined through a concatenation operation, which preserves feature information at different scales while increasing the expressive power of the features.
[0104] Step S3 introduces a whitening transformation module to ensure the consistency and accuracy of image processing:
[0105] Its structural diagram is as follows Figure 3As shown. A feature map X from the last layer of the backbone network is selected, with dimensions C×H×W, where C is the number of channels, and H and W are the height and width, respectively. Instance normalization is performed on feature map X so that the pixel mean of each channel is close to 0 and the standard deviation is close to 1. The normalized feature map X is then transformed to the shape [C, H, W]. The mean vector μ and covariance matrix Σ of the transformed feature map are calculated. The orthogonal matrix Q and diagonal matrix Λ are obtained through eigenvalue decomposition of Σ. These matrices are then used to perform a whitening transformation on feature map X, making the covariance matrix of the transformed feature map the identity matrix I. To achieve the whitening transformation, a loss function L is constructed. iw This loss function uses the mean squared error (MSE) to minimize the difference between the covariance matrix of X and the identity matrix I. The formula for calculating the loss function is:
[0106]
[0107] Where I is an upper triangular matrix consisting entirely of 1s, MSE is the mean squared error loss function, and F... iw It is the product of the feature map of the last layer of the backbone network and the transpose of the feature map.
[0108] Step S4: A loss function combining Log-cosh Dice Loss and Focal Loss is used to optimize the model's handling of imbalanced positive and negative samples.
[0109] Focal Loss dynamically adjusts the weights of easy and difficult samples by introducing a regulating factor γ, making the model focus more on samples that are difficult to classify, such as small targets in substation images with complex backgrounds. Its calculation formula is as follows:
[0110] Focal CE Loss=(1-e -ce ) γ ×ce
[0111] Here, ce represents the calculated cross-entropy loss, γ is a moderating factor that adjusts the weight of the sample loss, and e represents a variable representing the predicted probability. This design helps to reduce the weight of easily classified samples while increasing the weight of difficult-to-classify samples.
[0112] On the other hand, Log-cosh Dice Loss addresses the training instability issue that may arise with Dice Loss when dealing with small targets. Dice Loss is a region-dependent loss function that considers the similarity between predicted and true values, helping to resolve the imbalance between positive and negative samples. The calculation method for Dice Loss is as follows:
[0113] L dice =1-dice
[0114]
[0115] X and Y represent two different image samples. By combining Focal Loss and Log-cosh Dice Loss, the problems of hard-to-classify samples and imbalanced positive and negative samples can be considered simultaneously during training, improving the model's accuracy in recognizing small targets and enhancing the stability of model training. This fusion of loss functions, on the one hand, enhances the model's attention to small targets in complex backgrounds through Focal Loss, and on the other hand, reduces the impact of small target prediction errors on the training process through Log-cosh Dice Loss.
[0116] Step (2) involves obtaining the image dataset, which includes the following steps:
[0117] S1. Obtaining the dataset;
[0118] S2, Image annotation;
[0119] S3, Dataset Partitioning and Data Augmentation.
[0120] Step S1, Obtaining the dataset:
[0121] Image data was acquired from publicly available datasets in the field of power instrumentation. Further enhancements to the data's realism and coverage were achieved through photography at different time periods, including rainy days, daytime, and evening, and under diverse lighting conditions. Key categories such as instrument pointers, scales, oil stains, and damaged covers were successfully captured. Furthermore, to ensure the dataset meets various image processing needs, resolution normalization was performed on the images, thus guaranteeing its broad applicability and high quality.
[0122] Step S2, Image annotation:
[0123] During the annotation process, LabelMe was selected as the annotation tool, and the polygon plotting method was used to carefully delineate the boundaries of objects in order to reduce errors and ensure the high accuracy requirements of the dataset for instrument readings.
[0124] Step S3, the dataset partitioning and data augmentation:
[0125] The dataset was created to accurately calculate the pixel ratio of objects in images, divided in an 8:1:1 ratio. To improve data diversity and model generalization ability, a series of enhancement techniques were employed during training, including random flipping, ZCA whitening, random occlusion, and random adjustment of brightness and saturation. These methods effectively simulated potential robot camera jitter, different weather conditions, and day-night light variations encountered in a power environment, thereby enhancing the model's adaptability to real-world application scenarios.
[0126] The semantic segmentation model is trained in step (3):
[0127] The training method for the semantic segmentation model is explained in detail below.
[0128] The experimental environment and test data used in this case are as follows:
[0129] Graphics card: NVIDIA GeForce RTX 3080 Ti
[0130] CPU: Intel(R)Core(TM)i7-10750H
[0131] Python: 3.7.0
[0132] Neuralnet Framework: Pytorch 1.11.0
[0133] Operating system: Windows 10 Professional.
[0134] The specific process is as follows:
[0135] During model training, the stochastic gradient descent (SGD) algorithm was used as the optimizer. The initial learning rate was set to 2 × 10⁻⁶. -4 Furthermore, during training, the learning rate was adjusted according to a cosine annealing strategy, with a decay rate set to 0.01. Additionally, the batch size for each iteration was set to 16, and the entire training process lasted for 600 epochs until the model performance reached a stable state.
[0136] The validation of the semantic segmentation model in step (4) includes:
[0137] The mean intersection-over-union ratio (mIOU), a commonly used evaluation metric in semantic segmentation, was adopted. To better facilitate deployment in substation scenarios, the model's parameter count and frame detection speed were tested. The mIOU calculation method is as follows, where c represents the number of semantic segmentation task categories.
[0138]
[0139] Example 2
[0140] This embodiment also discloses a computer device, which includes...
[0141] Memory, used to store computer programs / instructions;
[0142] A processor is configured to execute the computer program / instructions to implement the steps of the image semantic segmentation method in Embodiment 1.
[0143] Example 3
[0144] This embodiment also discloses a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the image semantic segmentation method in Embodiment 1.
[0145] Example 4
[0146] A computer program product includes a computer program / instructions, characterized in that, when the computer program / instructions are executed by a processor, they implement the steps of the image semantic segmentation method in Embodiment 1.
[0147] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0148] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0149] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0150] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0151] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.
Claims
1. An image semantic segmentation method, characterized in that, include: Obtain the image to be segmented; The image features are extracted using the MobileNetv2 backbone network of the pre-trained semantic segmentation model to obtain the semantic information of the image and obtain a set of feature maps. The semantic information of the feature map group is fused using the feature fusion module of the semantic segmentation model to obtain a first feature map; The feature fusion module of the semantic segmentation model is used to fuse the features of the feature map group, including: The feature maps in the feature map group Feature maps are generated through convolution. F 1', the feature map F 1' Generates a height feature map after processing using a coordinate attention mechanism. and width feature map The calculation formula is as follows: ； ； Where c is the channel index. x c This represents performing pooling operations on channel c; W This indicates the number of iterations in the summation loop. h Representation of feature map F 1' height w Representation of feature map F 1' width; height feature map and width feature map Concatenate and convolve to generate feature maps ; ； r This represents the rate of change of the channels in the convolution; H Indicates the height of the height feature map. W The width of the feature map is represented; the feature map is... Split into two feature maps and The weights are obtained by processing them using the following formula. and ; ； ； in, F h Represents height feature map, F w This represents the result of the width feature map convolution operation. This represents the Sigmoid operation; For the feature maps in the feature map group S-shaped bar pooling is performed, and the resulting feature map and feature map The sums are used to obtain an enhanced feature map, which is then processed by the ReLU activation function to obtain the first feature map. Among them, feature map Feature map The two feature maps of different scales are obtained by the backbone network MobileNetv2 from the image feature extraction process. The feature map group is pooled and dilated convolutionally processed by the DenseSPASPP module of the semantic segmentation model, and the pooling result and the dilated convolution result of the last layer of the feature map group are concatenated to obtain the second feature map. The DenseSPASPP structure organizes dilated convolutions in a cascade manner, with the dilation rate arranged from small to large. The top layer of DenseSPASPP uses SP bar pooling. The pooling process results are ,in This represents a strip pooling operation; In DenseSPASPP, dilated convolutions are performed in a concatenated manner, with the output of each dilated convolution layer serving as the input to the next layer. The result of dilated convolution is ,in, ac l In l This represents the void ratio of the l-th layer. DC This indicates the calculation of dilated convolution, where k represents the kernel size, [...] represents the concatenation operation. y i Indicates the first i Layer image features, i The value range is [0~ l- 1], l Indicates the number of layers in the network; The second feature map and the first feature map are concatenated to obtain the semantic segmentation image of the image.
2. The image semantic segmentation method according to claim 1, characterized in that, The training process of the semantic segmentation model includes: Obtain the image dataset; The images were meticulously outlined using the LabelMe tool and the polygon plotting method. The images in the dataset were then labeled and the pixel proportions of the objects in the images were divided. The dataset is input into a pre-built semantic segmentation model for iterative training until the model performance reaches a stable state.
3. The image semantic segmentation method according to claim 1, characterized in that, The DenseSPASPP module of the semantic segmentation model is used to perform pooling and dilated convolution processing on the images in the feature map group, including: The feature maps are obtained by performing S-shaped bar pooling on the images in the feature map group. Y 1. The formula is as follows: ； in, This represents a strip pooling operation; X Represents image samples; The feature maps are obtained by sequential dilated convolution of the images in the feature map set. Y 2. The formula is as follows: ； in, ac l In l This represents the void ratio of the l-th layer. DC This indicates the calculation of dilated convolution, where k represents the kernel size, [...] represents the concatenation operation. y i Indicates the first i Layer image features, where i ranges from [0 to 1]. l- 1], l Indicates the number of layers in the network; The feature map Y Feature maps after processing with dilated convolution layer 1 and the last layer. Y n concatenations yield the second feature map.
4. The image semantic segmentation method according to claim 1, characterized in that, After extracting image features from the image using the MobileNetv2 backbone network of a pre-trained semantic segmentation model to obtain a set of feature maps, the process further includes: Select the feature map X from the last layer of the feature map group and perform instance normalization; The feature map X after instance normalization is subjected to whitening transformation to obtain the feature map X covariance with the identity matrix I, so that the shape of the image processed by the feature fusion module and the DenseSPASPP module is consistent.
5. The image semantic segmentation method according to claim 4, characterized in that, The loss function of the whitening transformation process The formula is as follows: ； in, It is an upper triangular matrix consisting entirely of 1s, and MSE is the mean squared error loss function. F iw It is the product of the feature map of the last layer of the backbone network and the transpose of the feature map.
6. The image semantic segmentation method according to claim 2, characterized in that, During training, the dataset is input into the pre-built semantic segmentation model for iterative training until the model performance reaches a stable state. This includes evaluating the performance of the semantic segmentation model using Dice Loss and Focal Loss loss functions, as shown in the following formulas: ； Where ce represents the calculated result of the cross-entropy loss. As a regulating factor, e A variable representing the probability of prediction; ； ； Here, X and Y represent two different image samples.
7. A computer device, characterized in that, include Memory, used to store computer programs / instructions; A processor for executing the computer program / instructions to implement the steps of the semantic segmentation method according to any one of claims 1 to 6.
8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the semantic segmentation method according to any one of claims 1 to 6.
9. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it implements the steps of the semantic segmentation method according to any one of claims 1-6.