A real-time semantic segmentation method based on asymmetric spatial feature convolution
By adopting asymmetric spatial feature convolution technology in real-time semantic segmentation, the feature representation and extraction capabilities are enhanced, and the problem of low image accuracy and efficiency in the existing technology is solved, and higher segmentation accuracy and efficiency, inference speed and average are achieved. The improvement of the exchange and comparison.
Patent Information
- Application Number
- CN202211390123.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-11-08
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2042-11-08
AI Technical Summary
The existing real-time semantic segmentation methods have differences in image accuracy and segmentation efficiency, and are limited by the local receptive field mechanism, and lack a full understanding of the target features of different scales in scene images, resulting in a low accuracy rate.
Using a real-time semantic segmentation method based on asymmetric spatial feature convolution, a network model containing 26 convolution blocks is constructed, each block consisting of jump connections, convolution layers, activation functions and batch normalization layers, using asymmetric Convolution, hollow convolution, group convolution and channel shuffling enhance the ability to represent and extract features, reduce feature information loss, and combine low-level high-level features through bilinear interpolation and splicing operations to improve segmentation accuracy and efficiency.
It achieves higher image segmentation accuracy and efficiency, making the segmented images more in line with the visual effect of the human eye, and significantly improves the inference speed and average intersecting ratio, avoiding artifacts, simplifying the network structure, and improving universality.
Smart Images

Figure CN115564959B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of real-time semantic segmentation of images in computer vision, and is a method for performing semantic segmentation using a convolutional neural network. Background Art
[0002] Semantic segmentation is to obtain the category information of each pixel in the image based on the texture, color, scene and other information of the image itself. In the semantic segmentation task, each pixel in the image needs to be accurately classified into a specific category. With the rise of deep learning, semantic segmentation can provide the category information, location information, posture information and environmental information of the object in the image, and plays a pivotal role in autonomous driving, augmented reality, video surveillance and infrared images. However, the existing semantic segmentation methods mainly focus on improving performance. In order to improve the accuracy of segmentation, these algorithms usually use a more complex backbone network for feature encoders and a intensive computing network structure for decoders. Therefore, the network model is large in scale and the scene segmentation efficiency is low. However, for practical applications, the algorithm needs to maintain a high segmentation accuracy and the real-time requirements must also be met. Although some real-time semantic segmentation algorithms are fast, they are limited by the local receptive field mechanism and lack a full understanding of the target features of different scales in the scene image, resulting in low accuracy.
[0003] The Chinese patent publication number is "CN112215848A", and the name is "A real-time semantic segmentation method based on spatial information guidance". The method first preprocesses the images in the data set; inputs the processed images into the backbone network; uses the backbone network to encode the input image to obtain input features; uses the input feature mapping to obtain the first high-dimensional features and global context features; uses the first high-dimensional features and global context features as inputs to the context propagation module guided by spatial details; uses the different paths in the context propagation module guided by spatial details to alternate and reconstruct spatial information; and obtains the segmentation prediction result. The segmented image obtained by this method does not meet the visual effect of the human eye, and the scene segmentation efficiency is low and the segmentation accuracy is poor. Summary of the invention
[0004] In order to solve the problems of poor image accuracy and low segmentation efficiency obtained by existing real-time semantic segmentation methods, the present invention provides a real-time semantic segmentation method based on asymmetric spatial feature convolution. The image obtained by semantic segmentation has a better segmentation effect and is more in line with human visual observation. At the same time, the method proposed by the present invention has a simple implementation process, higher image segmentation efficiency, and stronger versatility.
[0005] The solution to the technical problem of the present invention is:
[0006] A real-time semantic segmentation method based on asymmetric spatial feature convolution includes the following steps:
[0007] Step 1, building a network model: the entire network includes 26 convolution blocks from convolution block 1 to convolution block 26, each of which consists of a skip connection, a convolution layer, an activation function and a batch normalization layer;
[0008] Step 2, prepare the data set: first train the entire convolutional network with the semantic segmentation data set;
[0009] Step 3, training the network model: input the data set prepared in step 2 into the network model built in step 1 for training;
[0010] Step 4, select the minimized loss function and the optimal evaluation index: By minimizing the loss function of the network output image and the label, until the number of training times reaches the set threshold or the value of the loss function reaches the set range, the model parameters are considered to have been pre-trained and saved; at the same time, the optimal evaluation index is selected to measure the accuracy of the algorithm and evaluate the performance of the system;
[0011] Step 5, fine-tuning the model: Use the infrared image dataset to train and fine-tune the model to obtain stable and usable model parameters, further improve the model's image semantic segmentation ability, and ultimately make the model more effective in image segmentation;
[0012] Step 6, save the model: solidify the final model parameters. When the infrared image semantic segmentation operation is required, directly input the image into the network to obtain the final segmented image.
[0013] Among the twenty-six convolution blocks, convolution block one, convolution block twenty, convolution block twenty-three and convolution block twenty-six use bilinear interpolation; convolution block two, convolution block three, convolution block four, convolution block five, convolution block eight, convolution block nine, convolution block ten, and convolution block eleven, the sizes of convolution kernels are unified to n×1 and 1×n; channel shuffling and channel separation are used in convolution block fourteen, convolution block fifteen, convolution block sixteen and convolution block seventeen, and the sizes of convolution kernels are unified to n×1 and 1×n; the sizes of convolution kernels in convolution block six, convolution block seven, convolution block twelve, convolution block thirteen, convolution block eighteen, convolution block nineteen, convolution block twenty-one, convolution block twenty-two, convolution block twenty-four and convolution block twenty-five are unified to n×n; finally, the size of the feature map is consistent with the size of the input image.
[0014] In the step 1, convolution block 1 performs a downsampling operation on the input image to reduce the size of the feature map, convolution block 2, convolution block 3, convolution block 4 and convolution block 5 perform feature extraction on the image, convolution block 6 and convolution block 7 perform downsampling and channel conversion on the image, convolution block 8, convolution block 9, convolution block 10 and convolution block 11 perform feature extraction on the image, convolution block 12 and convolution block 13 perform downsampling and channel conversion on the image, convolution block 14, convolution block 15, convolution block 16 and convolution block 17 perform feature extraction on the image, convolution block 18 adopts an attention mechanism to make the network pay more attention to the features of the image, convolution block 19 performs channel conversion on the image, convolution block 20, convolution block 21 and convolution block 22 use upsampling, splicing and channel conversion on the image, convolution block 23, convolution block 24 and convolution block 25 use upsampling, splicing and channel conversion on the image, convolution block 26 performs upsampling operation on the input image to restore the size of the feature map, and obtains the final semantic segmentation image;
[0015] In the step 3, CityScape and Camvid are used as semantic segmentation datasets in the pre-training process; supervised training is performed on the images in the dataset; image enhancement is performed on the images in the dataset, random diffraction transformation is performed on the same image, and it is cropped to the size of the input image as the input of the entire network, and the annotated images in the dataset are used as labels.
[0016] In step 4, the cross entropy loss function is selected as the loss function during the training process; the selection of the loss function affects the quality of the model, can truly reflect the difference between the predicted value and the true value, and can correctly feedback the quality of the model.
[0017] In step 4, the inference speed and average intersection-over-union ratio are selected as appropriate evaluation indicators during the training process, which can effectively evaluate the accuracy and efficiency of the algorithm and measure the role of the segmentation network.
[0018] In step 5, the FLIR dataset is used in the process of fine-tuning the model parameters.
[0019] The beneficial effects of the present invention are as follows:
[0020] 1. In convolution blocks 2, 3, 4 and 5, asymmetric convolution, dilated convolution and channel shuffling are used to extract features from feature maps, which can enhance the feature representation of feature maps and reduce the loss of feature information of feature maps.
[0021] 2. In convolution blocks eight, nine, ten and eleven, asymmetric convolution, dilated convolution, group convolution and channel shuffling are used to extract features from feature maps, which can make the image segmentation output by the network more accurate and will not cause artifacts in the segmented image.
[0022] 3. Asymmetric convolution, hole convolution, group convolution, two-stream structure, channel separation and channel shuffling are used in convolution block 14, convolution block 15, convolution block 16 and convolution block 17 to enhance the network's ability to extract image features; adding skip connections helps to reduce network parameters, making the network shallower and the number of network parameters smaller, ultimately making the entire network structure simple and improving segmentation efficiency and accuracy.
[0023] 4. The use of bilinear interpolation and splicing operations in convolution blocks 20, 21 and 22 can combine the low-level features and high-level features of the image, making the network more capable of extracting features from two different images. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] Figure 1 This is a flow chart of a real-time semantic segmentation method based on asymmetric spatial feature convolution in the present invention.
[0025] Figure 2 This is a network structure diagram of a real-time semantic segmentation method based on asymmetric spatial feature convolution in the present invention.
[0026] Figure 3 This is the specific composition of each convolution block in convolution block 2, convolution block 3, convolution block 4 and convolution block 5 described in the present invention.
[0027] Figure 4 This is the specific composition of each convolution block in convolution block eight, convolution block nine, convolution block ten and convolution block eleven described in the present invention.
[0028] Figure 5 This is the specific composition of each of the convolution blocks 14, 15, 16 and 17 described in the present invention.
[0029] Figure 6 This is the specific composition of each convolution block in convolution block 20, convolution block 21 and convolution block 22 described in the present invention. DETAILED DESCRIPTION
[0030] The present invention is further described in detail below with reference to the accompanying drawings.
[0031] like Figure 1 As shown, a flowchart of a real-time semantic segmentation method based on asymmetric spatial feature convolution is provided, and the method specifically comprises the following steps:
[0032] Step 1, build a network model. The entire network includes 26 convolution blocks from convolution block 1 to convolution block 26, each of which consists of a skip connection, a convolution layer, an activation function and a batch normalization layer; convolution block 1 downsamples the input image to reduce the size of the feature map, convolution block 2, convolution block 3, convolution block 4 and convolution block 5 extract features from the image, convolution block 6, convolution block 7 downsamples and converts channels from the image, convolution block 8, convolution block 9, convolution block 10 and convolution block 11 extract features from the image, convolution block 12, convolution block 13 downsamples and converts channels from the image, convolution block 14, convolution block 15, convolution block 16 and convolution block 17 extract features from the image, convolution block 18 uses an attention mechanism to make the network pay more attention to the features of the image, convolution block 19 converts channels from the image, convolution block 20, convolution block 21 and convolution block 22 upsample, concatenate and convert channels from the image, convolution block 23 and convolution block Convolution block 24 and convolution block 25 use upsampling, splicing and channel conversion on the image, convolution block 26 upsamples the input image to restore the size of the feature map and obtain the final semantic segmentation image; convolution block 1, convolution block 20, convolution block 23 and convolution block 26 use bilinear interpolation; convolution block 2, convolution block 3, convolution block 4, convolution block 5, convolution block 8, convolution block 9, convolution block 10, convolution block 11, the size of the convolution kernel is unified to n×1 and 1×n; convolution block 14, convolution block 15, convolution block 16 and convolution block 17 use channel shuffling and channel separation, and the size of the convolution kernel is unified to n×1 and 1×n; the size of the convolution kernel in convolution block 6, convolution block 7, convolution block 12, convolution block 13, convolution block 18, convolution block 19, convolution block 21, convolution block 22, convolution block 24 and convolution block 25 is unified to n×n; finally, the size of the feature map is consistent with the size of the input image.
[0033] Step 2, prepare the dataset. The entire convolutional network is pre-trained with the semantic segmentation dataset. In the pre-training process, CityScapes and Camvid are used as the visible light dataset. Supervised training is performed on the images in the dataset; image enhancement is performed on the images in the dataset, random diffraction transformation is performed on the same image, and it is cropped to the size of the input image as the input of the entire network, and the annotated images in the dataset are used as labels.
[0034] Step 3: Train the network model. Input the data set prepared in step 2 into the network model built in step 1 for training.
[0035] Step 4, select the minimized loss function value and the optimal evaluation index: By minimizing the loss function of the network output image and label, until the number of training times reaches the set threshold or the value of the loss function reaches the set range, the model parameters can be considered to have been pre-trained and the model parameters can be saved. At the same time, the optimal evaluation index is selected to measure the accuracy of the algorithm and evaluate the performance of the system. The choice of loss function affects the quality of the model, can truly reflect the difference between the predicted value and the true value, and can correctly feedback the quality of the model; the optimal evaluation index is selected as inference speed and average intersection-over-union ratio, which can effectively evaluate the accuracy and efficiency of the algorithm and measure the role of the segmentation network.
[0036] Step 5: Fine-tune the model. Use infrared images to train and fine-tune the model to obtain stable and usable model parameters. The image segmentation accuracy of the network output is higher. Use the FLIR dataset in the process of fine-tuning the model parameters.
[0037] Step 6: Save the model and solidify the finalized model parameters.
[0038] If you are performing semantic segmentation on an infrared image, you can directly input the image into the network to get the final segmented image.
[0039] Example:
[0040] like Figure 1 As shown, a real-time semantic segmentation method based on asymmetric spatial feature convolution is provided, and the method specifically comprises the following steps:
[0041] Step 1: Build a network model.
[0042] like Figure 2 As shown in the figure, the entire network includes twenty-six convolution blocks, each of which consists of a skip connection, a convolution layer, an activation function, and a batch normalization layer; convolution block 1 consists of five layers of convolution, a linear activation function, and a batch normalization layer, and the convolution kernel size is 1×1 and 3×3, and the step size is 1 and 2; convolution block 2, convolution block 3, convolution block 4, and convolution block 5 consist of two layers of bilinear interpolation, three layers of convolution, a linear activation function, a batch normalization layer, and channel shuffling, and the convolution kernel size is 3×3, 3×1, and 1×3, and the step size is 1. The specific composition of each convolution block is as follows Figure 3 As shown; convolution block six and convolution block seven consist of a layer of bilinear interpolation, a layer of convolution, a linear activation function and a batch normalization layer, and the convolution kernel size is 1×1; convolution block eight, convolution block nine, convolution block ten and convolution block eleven consist of two layers of bilinear interpolation, four layers of convolution, a linear activation function, a batch normalization layer and channel shuffling, and the convolution kernel size is 3×1 and 1×3, and the step size is 1. The specific composition of each convolution block is as follows Figure 4As shown; convolution block 12 and convolution block 13 consist of a layer of bilinear interpolation, a layer of convolution, a linear activation function and a batch normalization layer, and the convolution kernel size is 1×1; convolution block 14, convolution block 15, convolution block 16 and convolution block 17 consist of two layers of bilinear interpolation, eight layers of convolution, a linear activation function, a batch normalization layer, channel separation and channel shuffling, the convolution kernel size is 3×1 and 1×3, the step size is 1, and the specific composition of each convolution block is as follows Figure 5 As shown in the figure; convolution block 18 adopts the attention mechanism; convolution block 19 consists of a layer of convolution, a linear activation function and a batch normalization layer, and the convolution kernel size is 1×1; convolution block 20, convolution block 21, and convolution block 22 consist of a layer of bilinear interpolation, two layers of convolution, a linear activation function and a batch normalization layer, and the convolution kernel size is 1×1. The specific composition of each convolution block is as follows Figure 6 As shown; convolution block 23, convolution block 24 and convolution block 25 consist of a layer of bilinear interpolation, two layers of convolution, linear activation function and batch normalization layer, and the convolution kernel size is 1×1; convolution block 26 consists of a layer of bilinear interpolation. The linear activation function, batch normalization layer and bilinear interpolation are shown as follows:
[0043]
[0044]
[0045] f(u+i,v+j)=(1-i)(1-j)f(u,v)+i(1-j)f(u+1,v)+(1-i)jf(u,v+1)+ijf(u+1,v+1)
[0046] Among them, f (k) and x (k) are the original input data and the normalized output data, μ (k) and σ (k) are the mean and standard deviation of the input data, β (k) and γ (k) are the translation parameter and scaling parameter of the science department, k represents the kth dimension of the data, and ε is a constant to prevent the denominator from being 0; where f(u,v) is the pixel value of the feature map at the (u,v) position, and the pixel coordinates of the bilinear interpolation result are (u+i,v+j), where u and v are the integer part of the floating-point coordinates, and i and j are the decimal part of the floating-point coordinates. The bilinear interpolation result pixel is represented as f(u+i,v+j), which is calculated by weighting the pixel values of the four neighboring positions f(u,v), f(u,v+1), f(u+1,v), and f(u+1,v+1) in proportion.
[0047] Step 2, prepare the dataset. The image semantic segmentation datasets use CityScapes and Camvid. The CityScapes dataset contains 2975 training images, 500 validation images, and 1525 test images. The image size is 1024×2048, and there are 19 categories in total. The Camvid dataset contains 367 training images, 101 validation images, and 233 test images. The image size is 720×960, and there are 11 categories in total.
[0048] Step 3, training the network model. Perform image enhancement on the images in the dataset, perform random diffraction transformation on the same image, and crop it to the size of the input image as the input of the entire network, and use the annotated images in the dataset as labels. The random size and position can be achieved through software algorithms. The purpose of using annotated images in the dataset as labels is to allow the network to learn better feature extraction capabilities and ultimately achieve better segmentation results.
[0049] Step 4, select the minimized loss function value and the optimal evaluation index. The network output and label calculate the loss function, and achieve a better fusion effect by minimizing the loss function. The loss function selection selects the cross entropy loss function. The calculation formula of the cross entropy loss function is as follows:
[0050]
[0051] Among them, C represents the cost, x represents the sample, y represents the actual value, a represents the output value, and n represents the total number of samples.
[0052] The average intersection-and-union ratio is the ratio of the intersection and union of two sets, which are the true value and the predicted value. The formula for calculating the inference speed and the average intersection-and-union ratio is as follows:
[0053]
[0054] Among them, k represents the number of pixel classes, P ii Indicates the correct number of pixels, P ij and P ji Indicates the number of false positives and false negatives for pixels.
[0055] The number of training times is set to 350, and the number of images input to the network each time is about 8-16. The upper limit of the number of images input to the network each time is mainly determined by the performance of the computer graphics processor. Generally, the number of images input to the network each time is in the range of 8-16, which can make the network training more stable and the training results better. The learning rate of the training process is set to 0.0001, which can ensure fast fitting of the network. The network parameter optimizer selects the Adam optimizer. Its advantages are simple implementation, efficient calculation, low memory requirements, and parameter updates are not affected by the scaling transformation of the gradient, making the parameters relatively stable. The loss function value threshold is set to about 0.0005. When the loss function value is less than 0.0005 or the number of iterations is greater than the preset value, it can be considered that the training of the entire network has been basically completed.
[0056] Step 5: Fine-tune the model. Fine-tune the parameters of the entire network model using the image dataset, where the dataset is the FLIR dataset.
[0057] Step 6, save the model. After the network training is completed, all parameters in the network need to be saved. Then, the infrared image to be segmented can be input into the network to obtain the segmented image. The network has no requirements on the input image size, and any size is acceptable.
[0058] Among them, the implementation of convolution, activation function, splicing operation, batch normalization and residual network are algorithms well known to those skilled in the art, and the specific processes and methods can be found in corresponding textbooks or technical literature.
[0059] The present invention constructs a real-time semantic segmentation method based on asymmetric spatial feature convolution, which can directly generate segmented images from images without going through other intermediate steps, thus avoiding the manual design of relevant segmentation rules. Under the same conditions, the feasibility and superiority of this method are further verified by calculating the relevant indicators of the image obtained by the existing method. The comparison of relevant indicators between the prior art and the method proposed in this invention is shown in Table 1:
[0060] Table 1 Comparison of relevant indicators of the prior art and the method proposed in the present invention
[0061]
[0062] It can be seen from the table that the method proposed in the present invention has higher reasoning speed and average intersection-over-union ratio than the existing method. These indicators also further illustrate that the method proposed in the present invention has faster reasoning speed and higher segmentation image accuracy.
Claims
1. A real-time semantic segmentation method based on asymmetric spatial feature convolution, characterized in that: The method comprises the following steps: Step 1, building a network model: the entire network includes 26 convolution blocks from convolution block 1 to convolution block 26, each of which consists of a skip connection, a convolution layer, an activation function and a batch normalization layer; Step 2, prepare the data set: first train the entire convolutional network with the semantic segmentation data set; Step 3, training the network model: input the data set prepared in step 2 into the network model built in step 1 for training; Step 4, select the minimized loss function and the optimal evaluation index: By minimizing the loss function of the network output image and the label, until the number of training times reaches the set threshold or the value of the loss function reaches the set range, the model parameters are considered to have been pre-trained and saved; at the same time, the optimal evaluation index is selected to measure the accuracy of the algorithm and evaluate the performance of the system; Step 5, fine-tuning the model: Use the infrared image dataset to train and fine-tune the model to obtain stable and usable model parameters, further improve the model's image semantic segmentation ability, and ultimately make the model more effective in image segmentation; Step 6, save the model: solidify the final model parameters, and when the infrared image semantic segmentation operation is required, directly input the image into the network to obtain the final segmented image; The twenty-six convolution blocks described in step 1, each of which consists of a skip connection, a convolution layer, an activation function, and a batch normalization layer; convolution block 1 consists of five layers of convolution, a linear activation function, and a batch normalization layer, and the convolution kernel size is 1×1 and 3×3, and the step size is 1 and 2; convolution block 2, convolution block 3, convolution block 4, and convolution block 5 consist of two layers of bilinear interpolation, three layers of convolution, a linear activation function, a batch normalization layer, and channel shuffling, and the convolution kernel size is 3×3, 3× 1 and 1×3, with a step size of 1; convolution block 6 and convolution block 7 consist of a layer of bilinear interpolation, a layer of convolution, a linear activation function and a batch normalization layer, and the convolution kernel size is 1×1; convolution block 8, convolution block 9, convolution block 10 and convolution block 11 consist of two layers of bilinear interpolation, four layers of convolution, linear activation function, batch normalization layer and channel shuffle, and the convolution kernel size is 3×1 and 1×3, with a step size of 1; convolution block 12 and convolution block 13 consist of a layer of bilinear interpolation, a layer of convolution , linear activation function and batch normalization layer, the convolution kernel size is 1×1; convolution block 14, convolution block 15, convolution block 16 and convolution block 17 are composed of two layers of bilinear interpolation, eight layers of convolution, linear activation function, batch normalization layer, channel separation and channel shuffling, the convolution kernel size is 3×1 and 1×3, and the step size is 1; convolution block 18 adopts the attention mechanism; convolution block 19 is composed of one layer of convolution, linear activation function and batch normalization layer, and the convolution kernel size is 1×1; Convolution block 20, convolution block 21, and convolution block 22 consist of a layer of bilinear interpolation, two layers of convolution, a linear activation function, and a batch normalization layer, and the convolution kernel size is 1×1; convolution block 23, convolution block 24, and convolution block 25 consist of a layer of bilinear interpolation, two layers of convolution, a linear activation function, and a batch normalization layer, and the convolution kernel size is 1×1; convolution block 26 consists of a layer of bilinear interpolation; the linear activation function, batch normalization layer, and bilinear interpolation are as follows: f(u+i,v+j)=(1-i)(1-j)f(u,v)+i(1-j)f(u+1,v)+(1-i)jf(u,v+1)+ijf(u+1,v+1) Among them, f (k) and x (k) are the original input data and the normalized output data, μ (k) and σ (k) are the mean and standard deviation of the input data, β (k) and γ (k) are the translation parameter and scaling parameter of the scientific department respectively, k represents the kth dimension of the data, and ε is a constant to prevent the denominator from being 0; f(u,v) is the pixel value of the feature map at the (u,v) position, and the pixel coordinates of the bilinear interpolation result are (u+i,v+j), where u and v are the integer part of the floating-point coordinates, and i and j are the decimal part of the floating-point coordinates; the bilinear interpolation result pixel is represented as f(u+i,v+j), which is calculated by weighting the pixel values of the four adjacent positions f(u,v), f(u,v+1), f(u+1,v), and f(u+1,v+1) in proportion.
2. According to claim 1, a real-time semantic segmentation method based on asymmetric spatial feature convolution is characterized in that: Among the twenty-six convolution blocks, convolution block one, convolution block twenty, convolution block twenty-three and convolution block twenty-six use bilinear interpolation; convolution block two, convolution block three, convolution block four, convolution block five, convolution block eight, convolution block nine, convolution block ten, and convolution block eleven, the sizes of convolution kernels are unified to n×1 and 1×n; channel shuffling and channel separation are used in convolution block fourteen, convolution block fifteen, convolution block sixteen and convolution block seventeen, and the sizes of convolution kernels are unified to n×1 and 1×n; the sizes of convolution kernels in convolution block six, convolution block seven, convolution block twelve, convolution block thirteen, convolution block eighteen, convolution block nineteen, convolution block twenty-one, convolution block twenty-two, convolution block twenty-four and convolution block twenty-five are unified to n×n; finally, the size of the feature map is consistent with the size of the input image.
3. The real-time semantic segmentation method based on asymmetric spatial feature convolution according to claim 1, characterized in that: In the step 1, the convolution block 1 performs a downsampling operation on the input image to reduce the size of the feature map, the convolution block 2, the convolution block 3, the convolution block 4 and the convolution block 5 perform feature extraction on the image, the convolution block 6 and the convolution block 7 perform downsampling and channel conversion on the image, the convolution block 8, the convolution block 9, the convolution block 10 and the convolution block 11 perform feature extraction on the image, the convolution block 12 and the convolution block 13 perform downsampling and channel conversion on the image, the convolution block 14, the convolution block 15, the convolution block 16 and the convolution block 17 perform feature extraction on the image, the convolution block 18 adopts the attention mechanism to make the network pay more attention to the features of the image, the convolution block 19 performs channel conversion on the image, the convolution block 20, the convolution block 21 and the convolution block 22 use upsampling, splicing operation and channel conversion on the image, the convolution block 23, the convolution block 24 and the convolution block 25 use upsampling, splicing operation and channel conversion on the image, and the convolution block 26 performs upsampling operation on the input image to restore the size of the feature map to obtain the final semantic segmentation image.
4. The real-time semantic segmentation method based on asymmetric spatial feature convolution according to claim 1, characterized in that: In the step 3, CityScape and Camvid are used as semantic segmentation datasets in the pre-training process; supervised training is performed on the images in the dataset; image enhancement is performed on the images in the dataset, random diffraction transformation is performed on the same image, and it is cropped to the size of the input image as the input of the entire network, and the annotated images in the dataset are used as labels.
5. The real-time semantic segmentation method based on asymmetric spatial feature convolution according to claim 1, characterized in that: In step 4, the cross entropy loss function is selected as the loss function during the training process; the selection of the loss function affects the quality of the model, can truly reflect the difference between the predicted value and the true value, and can correctly feedback the quality of the model.
6. The real-time semantic segmentation method based on asymmetric spatial feature convolution according to claim 1, characterized in that: In step 4, the inference speed and average intersection-over-union ratio are selected as appropriate evaluation indicators during the training process, which can effectively evaluate the accuracy and efficiency of the algorithm and measure the role of the segmentation network.
7. The real-time semantic segmentation method based on asymmetric spatial feature convolution according to claim 1, characterized in that: In step 5, the FLIR dataset is used in the process of fine-tuning the model parameters.
