A deep learning-based image segmentation method
By building a semantic segmentation model composed of backbone network, XY network and fully convolutional decoding network, the problem of difficult face image segmentation during vehicle driving is solved, and higher generalization ability and accuracy are achieved, which improves the driver's The accuracy of fatigue recognition solves the problem of high misjudgment rate in the prior art.
Patent Information
- Application Number
- CN202210705586.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-06-21
- Publication Date
- 2024-11-22
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The face images collected during vehicle driving have difficulty in image segmentation, especially in the case of light changes and complex environment, which leads to misjudgment of driver fatigue recognition.
Using deep learning-based image segmentation method, a semantic segmentation model consisting of backbone network, XY network and fully convolutional decoding network is constructed. Through the combination of transfer learning and multiple segmentation algorithms, a new context information extraction module XY network is designed to enhance the The model's ability to identify and locate portraits improves generalization ability and accuracy.
In tests on multiple datasets, the XYNet model performed excellently in robustness, MIoU score, inference speed, model size, and inference floating-point calculation number, compared with other similar segmentation network models, accuracy and deployment rate Both have significant improvements, providing stronger balance and visualization.
Smart Images

Figure CN115082678B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image segmentation, and in particular to an image segmentation method based on deep learning. Background Art
[0002] With the rapid growth of my country's economy, the overall consumption level of the people has gradually increased, resulting in the continuous growth of automobile production. The pressure on road traffic has become unbearable, and traffic accidents have become increasingly frequent. According to statistics, more than 60% of traffic accidents are related to driver fatigue driving. When the driver is driving fatigued, the perception and judgment of the surrounding environment and the ability to control the vehicle will be reduced to varying degrees, which will lead to traffic accidents. The fatigue state of the driver is affected by many situations. Most of the reasons are that the driver drives the car for a long time, which can easily cause the driver's fatigue, thereby affecting the driver's ability to control the vehicle and causing traffic accidents.
[0003] Although by installing a visual detection device on the vehicle to collect the driver's facial image, and by segmenting the driver's facial image to obtain the driver's eyes and mouth (such as closing eyes or yawning, etc.), it is possible to identify whether the driver is driving fatigued, so as to avoid traffic accidents caused by driver fatigue driving. However, due to the complex and changeable environment in which the vehicle is in during driving, the changes in light in these environments will cause the collected facial images to appear in different states (such as overexposure caused by strong sunlight, dark images caused by dark light, glare caused by the lights of oncoming vehicles, and image ghosting caused by vehicle shaking, etc.), which brings great challenges to the subsequent image segmentation. Once the facial image cannot be effectively segmented, it will lead to misjudgment of the driver's fatigue recognition. Summary of the invention
[0004] The present invention aims to solve the problem that face images collected during vehicle driving are difficult to segment, and provides an image segmentation method based on deep learning.
[0005] To solve the above problems, the present invention is achieved through the following technical solutions:
[0006] A deep learning-based image segmentation method comprises the following steps:
[0007] Step 1, construct a semantic segmentation model; the semantic segmentation model consists of a backbone network, an XY network and a fully convolutional decoding network; the input of the backbone network forms the input of the semantic segmentation model, the output of the backbone network is connected to the input of the XY network, the output of the XY network is connected to the input of the fully convolutional decoding network, and the output of the fully convolutional decoding network is the output of the semantic segmentation model;
[0008] Step 2: Use the segmented sample image set to train the semantic segmentation model constructed in step 1 to obtain a trained semantic segmentation model;
[0009] Step 3: Send the image to be segmented into the trained semantic segmentation model obtained in step 2, and the trained semantic segmentation model outputs the segmented image.
[0010] The backbone network is composed of 2 convolutional layers, 4 bottleneck layers, 9 transition layers, 9 branch layers, 3 fusion layers, 4 bilinear interpolation layers and 1 connection layer; the input of the first convolutional layer forms the input of the backbone network; the output of the first convolutional layer is connected to the input of the second convolutional layer, the output of the second convolutional layer is connected to the input of the first bottleneck layer, the output of the first bottleneck layer is connected to the input of the second bottleneck layer, the output of the second bottleneck layer is connected to the input of the third bottleneck layer, the output of the third bottleneck layer is connected to the input of the fourth bottleneck layer. The output of the 4th bottleneck layer is connected to the input of the 1st transition layer and the 2nd transition layer respectively; the output of the 1st transition layer is connected to the input of the 1st branch layer, and the output of the 2nd transition layer is connected to the input of the 2nd branch layer; the outputs of the 1st branch layer and the 2nd branch layer are connected to the input of the 1st fusion layer at the same time; the output of the 1st fusion layer is connected to the input of the 3rd transition layer, the 4th transition layer and the 5th transition layer respectively; the output of the 3rd transition layer is connected to the input of the 3rd branch layer, and the output of the 4th transition layer is connected to the input of the 4th branch layer. The output of the 5th transition layer is connected to the input of the 5th branch layer; the outputs of the 3rd branch layer, the 4th branch layer and the 5th branch layer are connected to the input of the 2nd fusion layer at the same time; the output of the 2nd fusion layer is connected to the input of the 6th transition layer, the 7th transition layer, the 8th transition layer and the 9th transition layer respectively; the output of the 6th transition layer is connected to the input of the 6th branch layer, the output of the 7th transition layer is connected to the input of the 7th branch layer, the output of the 8th transition layer is connected to the input of the 8th branch layer, and the output of the 9th transition layer is connected to the input of the 9th branch layer. The outputs of the 6th branch layer, the 7th branch layer, the 8th branch layer and the 9th branch layer are simultaneously connected to the input of the 3rd fusion layer; the output of the 3rd fusion layer is simultaneously connected to the outputs of the 1st bilinear interpolation layer, the 2nd bilinear interpolation layer, the 3rd bilinear interpolation layer and the 4th bilinear interpolation layer; the outputs of the 1st bilinear interpolation layer, the 2nd bilinear interpolation layer, the 3rd bilinear interpolation layer and the 4th bilinear interpolation layer are simultaneously connected to the input of the connection layer; the output of the connection layer forms the output of the backbone network.
[0011] The above XY network consists of 15 convolutional layers, 4 upsampling layers, 5 channel attention layers, 5 merging layers and 1 connection layer; among them: the inputs of the 1st convolutional layer, the 2nd convolutional layer, the 5th convolutional layer, the 8th convolutional layer, the 11th convolutional layer and the 14th convolutional layer together form the input of the XY network; the output of the 1st convolutional layer is connected to one of the inputs of the 1st merging layer, the 2nd merging layer, the 3rd merging layer and the 4th merging layer; the output of the 2nd convolutional layer is connected to the input of the 3rd convolutional layer, and the output of the 3rd convolutional layer is connected to the input of the 1st upsampling layer. The output of the first upsampling layer is connected to the other input of the first merging layer; the output of the first merging layer is connected to the input of the first channel attention layer, and the output of the first channel attention layer is connected to the input of the fourth convolution layer; the output of the fifth convolution layer is connected to the input of the sixth convolution layer, and the output of the sixth convolution layer is connected to the input of the second upsampling layer, and the output of the second upsampling layer is connected to the other input of the second merging layer; the output of the second merging layer is connected to the input of the second channel attention layer, and the output of the second channel attention layer is connected to the input of the seventh convolution layer; the output of the eighth convolution layer is connected to the input of the sixth convolution layer, and the output of the sixth convolution layer is connected to the input of the second upsampling layer, and the output of the second upsampling layer is connected to the other input of the second merging layer; the output of the second merging layer is connected to the input of the second channel attention layer, and the output of the second channel attention layer is connected to the input of the seventh convolution layer; The output of the 9th convolutional layer is connected to the input of the 3rd upsampling layer, and the output of the 3rd upsampling layer is connected to another input of the 3rd merging layer; the output of the 3rd merging layer is connected to the input of the 3rd channel attention layer, and the output of the 3rd channel attention layer is connected to the input of the 10th convolutional layer; the output of the 11th convolutional layer is connected to the input of the 12th convolutional layer, and the output of the 12th convolutional layer is connected to the input of the 4th upsampling layer, and the output of the 4th upsampling layer is connected to another input of the 4th merging layer; the output of the 4th merging layer is connected to the 4th channel attention layer. The input of the attention layer and the output of the 4th channel attention layer are connected to the input of the 13th convolution layer; the outputs of the 1st, 4th, 7th, 10th and 13th convolution layers are connected to the input of the connection layer, and the output of the connection layer is connected to the input of the 15th convolution layer; the output of the 15th convolution layer is connected to one of the inputs of the 5th merging layer; the output of the 14th convolution layer is connected to the other input of the 5th merging layer; the output of the 5th merging layer is connected to the input of the 5th channel attention layer, and the output of the 5th channel attention layer forms the output of the XY network.
[0012] The above-mentioned fully convolutional decoding network consists of 2 convolutional layers and 1 bilinear interpolation layer; wherein: the input of the first convolutional layer forms the input of the fully convolutional decoding network, the output of the first convolutional layer is connected to the input of the second convolutional layer, the output of the second convolutional layer is connected to the input of the bilinear interpolation layer, and the output of the bilinear interpolation layer forms the output of the fully convolutional decoding network.
[0013] Compared with the prior art, the semantic segmentation model of the present invention adopts XYNet, which is a combination of HRNet and XY network. Compared with the original basic network, it has stronger generalization ability and higher accuracy. On the basis of the fusion of the two basic networks, the DAMMP module is added to improve the original network, and the context information in the original network is fully utilized again, so as to improve the performance effect without significantly increasing the parameters. By optimizing the model internally, the internal modules of the model are improved and innovated while ensuring that the performance is not reduced, which greatly enhances the future feasibility. Under the premise that the model is very lightweight, the accuracy and deployment rate of the semantic segmentation model of the present invention far exceed other segmentation network models of the same type. Using the semantic segmentation model of the present invention to test and compare horizontally on multiple data sets, it can be obtained that our XYNet has achieved good results in robustness, evaluation index MIoU (Mean Intersection over Union), reasoning speed, model size, reasoning floating-point calculation number and image visualization results, and is more balanced than the existing network. BRIEF DESCRIPTION OF THE DRAWINGS
[0014] Figure 1 A schematic diagram of a semantic segmentation model.
[0015] Figure 2 This is the schematic diagram of the backbone network phase one.
[0016] Figure 3 This is the schematic diagram of the backbone network phase 2.
[0017] Figure 4 This is the schematic diagram of the backbone network phase three.
[0018] Figure 5 Schematic diagram of stage 4 of the backbone network.
[0019] Figure 6 This is the schematic diagram of the XY network. DETAILED DESCRIPTION
[0020] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific examples.
[0021] A deep learning-based image segmentation method comprises the following steps:
[0022] Step 1: Build a semantic segmentation model.
[0023] See also Figure 1The semantic segmentation model consists of a backbone network, an XY network, and a fully convolutional decoding network. The input of the backbone network forms the input of the semantic segmentation model, the output of the backbone network is connected to the input of the XY network, the output of the XY network is connected to the input of the fully convolutional decoding network, and the output of the fully convolutional decoding network is the output of the semantic segmentation model. This semantic segmentation model consists of a fully convolutional network FCN composed of a backbone network and a fully convolutional decoding network, and an XY network. The image segmentation model XYNet combines the advantages of multiple segmentation algorithms through transfer learning, model cascading, etc., and designs a new context information extraction module XY network. The network can fully extract the context information of the image, enhance the model's ability to recognize and locate the main portrait of the picture, and the overall model achieves an excellent balance between speed and accuracy.
[0024] The backbone network consists of 2 convolutional layers, 4 bottleneck layers, 9 transition layers, 9 branch layers, 3 fusion layers, 4 bilinear interpolation layers and 1 connection layer.
[0025] The backbone network consists of four stages, and the multi-scale feature maps generated in each stage are configured as shown in Table 1.
[0026] Table 1 Backbone network configuration table
[0027] Multi-scale feature maps Number of branches Number of modules per branch Number of modules Phase 1 1 / 4 1 4 0 Phase 2 1 / 4,1 / 8 2 4,4 1 Phase 3 1 / 4,1 / 8,1 / 16 3 4,4,4 4 Phase 4 1 / 4,1 / 8,1 / 16,1 / 32 4 4,4,4,4 3
[0028] Stage 1: The input of the first convolutional layer forms the input of the backbone network. The output of the first convolutional layer is connected to the input of the second convolutional layer, the output of the second convolutional layer is connected to the input of the first bottleneck layer, the output of the first bottleneck layer is connected to the input of the second bottleneck layer, the output of the second bottleneck layer is connected to the input of the third bottleneck layer, and the output of the third bottleneck layer is connected to the input of the fourth bottleneck layer. The output of the fourth bottleneck layer is connected to the input of the first transition layer and the second transition layer respectively. The output of the first transition layer is connected to the input of the first branch layer, and the output of the second transition layer is connected to the input of the second branch layer. Figure 2 As shown. The bottleneck layer can fix the number of channels in the feature map to a certain value, and then enlarge it, making the number of channels like a bottleneck, thin on the top and wide on the bottom. The two 1X1 convolutions are used to reduce and increase the feature dimension respectively. The main purpose is to reduce the number of parameters, thereby reducing the amount of calculation, and after the dimension reduction, the data training and feature extraction can be more effective and intuitive. The bottleneck layer (Bottleneck) is connected with 4 residuals, which is a classic module in the residual network. In the bottleneck layer, the 256-dimensional data is reduced to 64 dimensions, then convolved, and then increased to 256 dimensions, which effectively reduces the amount of calculation. The transition layer uses two convolutions to reduce the features of the 256 channels into two feature maps, thereby forming two branches to enter the second stage.
[0029] Phase 2: The outputs of the first branch layer and the second branch layer are connected to the input of the first fusion layer at the same time. The output of the first fusion layer is connected to the input of the third transition layer, the fourth transition layer, and the fifth transition layer respectively. The output of the third transition layer is connected to the input of the third branch layer, the output of the fourth transition layer is connected to the input of the fourth branch layer, and the output of the fifth transition layer is connected to the input of the fifth branch layer. Figure 3 As shown in the figure. There is only one high-resolution module (HighResolutionModule) in this stage. The high-resolution module (HighResolutionModule) contains branches and fusion layers. This stage contains two branch modules, which is equivalent to two branches. Each branch consists of 4 residual blocks (BasicBlock). Before entering the third stage, the transition layer changes 2 branches into 3 branches. Feature maps with the same number of channels are not processed. The feature map of the third branch is downsampled from the second branch.
[0030] Stage 3: The outputs of the 3rd branch layer, the 4th branch layer, and the 5th branch layer are simultaneously connected to the input of the 2nd fusion layer. The output of the 2nd fusion layer is connected to the inputs of the 6th transition layer, the 7th transition layer, the 8th transition layer, and the 9th transition layer. The output of the 6th transition layer is connected to the input of the 6th branch layer, the output of the 7th transition layer is connected to the input of the 7th branch layer, the output of the 8th transition layer is connected to the input of the 8th branch layer, and the output of the 9th transition layer is connected to the input of the 9th branch layer. Figure 4 As shown in Figure 2, this stage consists of two high-resolution modules. The high-resolution module in this part has one more branch than that in stage 2. Similarly, the transition layer changes three branches into four branches, and the fourth branch is downsampled from the third branch.
[0031] Stage 4: The outputs of the 6th branch layer, the 7th branch layer, the 8th branch layer, and the 9th branch layer are simultaneously connected to the input of the 3rd fusion layer. The output of the 3rd fusion layer is simultaneously connected to the outputs of the 1st bilinear interpolation layer, the 2nd bilinear interpolation layer, the 3rd bilinear interpolation layer, and the 4th bilinear interpolation layer. The outputs of the 1st bilinear interpolation layer, the 2nd bilinear interpolation layer, the 3rd bilinear interpolation layer, and the 4th bilinear interpolation layer are simultaneously connected to the input of the connection layer. The output of the connection layer forms the output of the backbone network. Figure 5 As shown in the figure, this stage consists of three high-resolution modules, and the number of branches is one more than the third stage. It can be seen that the deeper you go, the lower the resolution of the feature map. There are four levels of resolution here, and the high and low resolutions are still integrated with each other.
[0032] The XY network consists of 15 convolutional layers, 4 upsampling layers, 5 channel attention layers, 5 merging layers and 1 connection layer. The inputs of the 1st convolutional layer, the 2nd convolutional layer, the 5th convolutional layer, the 8th convolutional layer, the 11th convolutional layer and the 14th convolutional layer together form the input of the XY network. The output of the 1st convolutional layer is connected to one of the inputs of the 1st merging layer, the 2nd merging layer, the 3rd merging layer and the 4th merging layer. The output of the 2nd convolutional layer is connected to the input of the 3rd convolutional layer, the output of the 3rd convolutional layer is connected to the input of the 1st upsampling layer, and the output of the 1st upsampling layer is connected to another input of the 1st merging layer. The output of the 1st merging layer is connected to the input of the 1st channel attention layer, and the output of the 1st channel attention layer is connected to the input of the 4th convolutional layer. The output of the 5th convolutional layer is connected to the input of the 6th convolutional layer, and the output of the 6th convolutional layer is connected to the input of the 2nd upsampling layer, and the output of the 2nd upsampling layer is connected to another input of the 2nd merging layer. The output of the 2nd merging layer is connected to the input of the 2nd channel attention layer, and the output of the 2nd channel attention layer is connected to the input of the 7th convolutional layer. The output of the 8th convolutional layer is connected to the input of the 9th convolutional layer, and the output of the 9th convolutional layer is connected to the input of the 3rd upsampling layer, and the output of the 3rd upsampling layer is connected to another input of the 3rd merging layer. The output of the 3rd merging layer is connected to the input of the 3rd channel attention layer, and the output of the 3rd channel attention layer is connected to the input of the 10th convolutional layer. The output of the 11th convolutional layer is connected to the input of the 12th convolutional layer, and the output of the 12th convolutional layer is connected to the input of the 4th upsampling layer, and the output of the 4th upsampling layer is connected to another input of the 4th merging layer. The output of the 4th merging layer is connected to the input of the 4th channel attention layer, and the output of the 4th channel attention layer is connected to the input of the 13th convolutional layer. The outputs of the 1st, 4th, 7th, 10th and 13th convolutional layers are connected to the input of the connection layer, and the output of the connection layer is connected to the input of the 15th convolutional layer. The output of the 15th convolutional layer is connected to one of the inputs of the 5th merging layer. The output of the 14th convolutional layer is connected to another input of the 5th merging layer. The output of the 5th merging layer is connected to the input of the 5th channel attention layer, and the output of the 5th channel attention layer forms the output of the XY network. Figure 6 shown.
[0033] Considering that most of the existing feature extraction networks first extract high-level semantic information from high-resolution downsampling to low-resolution, there is a disadvantage in doing so, that is, the detailed information of the image will be gradually lost during the downsampling process, thereby highlighting the important high-level information in the image, and HRNetV1 can maintain the feature map in a high-resolution state throughout the feature extraction process, effectively reducing the loss of detailed information, so the XY network of the present invention is based on HRNetV1. HRNetV1 takes the high-resolution subnetwork as the first stage, gradually increases the subnetwork from high resolution to low resolution, forms more stages, and connects the multi-resolution subnetworks in parallel, and performs multiple multi-scale fusions at the same time, so that each high-resolution to low-resolution representation repeatedly receives information from other parallel representations, and obtains rich high-resolution representations, so that the predicted feature map is more accurate. However, due to the large number of modules in the original HRNetv1, the model parameters and the amount of calculation are relatively large, so the present invention performs lightweight processing on it, reduces the width of some modules and network models, and greatly improves the computational efficiency of the backbone network. Through comparative experiments, it is found that although the width of the module and network model is reduced, the accuracy has not decreased significantly.
[0034] The fully convolutional decoding network consists of 2 convolutional layers and 1 bilinear interpolation layer. The input of the first convolutional layer forms the input of the fully convolutional decoding network, the output of the first convolutional layer is connected to the input of the second convolutional layer, the output of the second convolutional layer is connected to the input of the bilinear interpolation layer, and the output of the bilinear interpolation layer forms the output of the fully convolutional decoding network. The global context information extracted by the XY network is finally input into a lightweight fully convolutional decoding module (FCNHead), which consists of a 1x1 convolution + regularization + activation function basic module and a 1x1 convolution layer. After the two-part convolution, the feature map is upsampled to the same size as the original image through the bilinear interpolation algorithm to achieve end-to-end training.
[0035] Step 2: Use the segmented sample image set to train the semantic segmentation model constructed in step 1 to obtain a trained semantic segmentation model.
[0036] Step 3: Send the image to be segmented into the trained semantic segmentation model obtained in step 2, and the trained semantic segmentation model outputs the segmented image.
[0037] The semantic segmentation method is used to separate the person from the background to obtain the current pure portrait picture, and then the obtained portrait is cropped, with the focus on retaining the person's head picture. Face detection is then performed on the retained head picture, mainly to identify the closure of the eyes and mouth of the face.
[0038] It should be noted that although the embodiments of the present invention described above are illustrative, they are not intended to limit the present invention, and therefore the present invention is not limited to the above specific embodiments. Without departing from the principles of the present invention, any other embodiments obtained by those skilled in the art under the guidance of the present invention are deemed to be within the protection of the present invention.
Claims
1. A deep learning-based image segmentation method, characterized in that: The steps include: Step 1, construct a semantic segmentation model; the semantic segmentation model consists of a backbone network, an XY network and a fully convolutional decoding network; the input of the backbone network forms the input of the semantic segmentation model, the output of the backbone network is connected to the input of the XY network, the output of the XY network is connected to the input of the fully convolutional decoding network, and the output of the fully convolutional decoding network is the output of the semantic segmentation model; The backbone network is composed of 2 convolutional layers, 4 bottleneck layers, 9 transition layers, 9 branch layers, 3 fusion layers, 4 bilinear interpolation layers and 1 connection layer; the input of the first convolutional layer forms the input of the backbone network; the output of the first convolutional layer is connected to the input of the second convolutional layer, the output of the second convolutional layer is connected to the input of the first bottleneck layer, the output of the first bottleneck layer is connected to the input of the second bottleneck layer, the output of the second bottleneck layer is connected to the input of the third bottleneck layer, the output of the third bottleneck layer is connected to the input of the fourth bottleneck layer. The output of the 4th bottleneck layer is connected to the input of the 1st transition layer and the 2nd transition layer respectively; the output of the 1st transition layer is connected to the input of the 1st branch layer, and the output of the 2nd transition layer is connected to the input of the 2nd branch layer; the outputs of the 1st branch layer and the 2nd branch layer are connected to the input of the 1st fusion layer at the same time; the output of the 1st fusion layer is connected to the input of the 3rd transition layer, the 4th transition layer and the 5th transition layer respectively; the output of the 3rd transition layer is connected to the input of the 3rd branch layer, and the output of the 4th transition layer is connected to the input of the 4th branch layer. The output of the 5th transition layer is connected to the input of the 5th branch layer; the outputs of the 3rd branch layer, the 4th branch layer and the 5th branch layer are connected to the input of the 2nd fusion layer at the same time; the output of the 2nd fusion layer is connected to the input of the 6th transition layer, the 7th transition layer, the 8th transition layer and the 9th transition layer respectively; the output of the 6th transition layer is connected to the input of the 6th branch layer, the output of the 7th transition layer is connected to the input of the 7th branch layer, the output of the 8th transition layer is connected to the input of the 8th branch layer, and the output of the 9th transition layer is connected to the input of the 9th branch layer. The outputs of the 6th branch layer, the 7th branch layer, the 8th branch layer and the 9th branch layer are simultaneously connected to the input of the 3rd fusion layer; the output of the 3rd fusion layer is simultaneously connected to the outputs of the 1st bilinear interpolation layer, the 2nd bilinear interpolation layer, the 3rd bilinear interpolation layer and the 4th bilinear interpolation layer; the outputs of the 1st bilinear interpolation layer, the 2nd bilinear interpolation layer, the 3rd bilinear interpolation layer and the 4th bilinear interpolation layer are simultaneously connected to the input of the connection layer; the output of the connection layer forms the output of the backbone network; The above XY network consists of 15 convolutional layers, 4 upsampling layers, 5 channel attention layers, 5 merging layers and 1 connection layer; among them: the inputs of the 1st convolutional layer, the 2nd convolutional layer, the 5th convolutional layer, the 8th convolutional layer, the 11th convolutional layer and the 14th convolutional layer together form the input of the XY network; the output of the 1st convolutional layer is connected to one of the inputs of the 1st merging layer, the 2nd merging layer, the 3rd merging layer and the 4th merging layer; the output of the 2nd convolutional layer is connected to the input of the 3rd convolutional layer, and the output of the 3rd convolutional layer is connected to the input of the 1st upsampling layer. The output of the first upsampling layer is connected to the other input of the first merging layer; the output of the first merging layer is connected to the input of the first channel attention layer, and the output of the first channel attention layer is connected to the input of the fourth convolution layer; the output of the fifth convolution layer is connected to the input of the sixth convolution layer, and the output of the sixth convolution layer is connected to the input of the second upsampling layer, and the output of the second upsampling layer is connected to the other input of the second merging layer; the output of the second merging layer is connected to the input of the second channel attention layer, and the output of the second channel attention layer is connected to the input of the seventh convolution layer; the output of the eighth convolution layer is connected to the input of the sixth convolution layer, and the output of the sixth convolution layer is connected to the input of the second upsampling layer, and the output of the second upsampling layer is connected to the other input of the second merging layer; the output of the second merging layer is connected to the input of the second channel attention layer, and the output of the second channel attention layer is connected to the input of the seventh convolution layer; The output of the 9th convolutional layer is connected to the input of the 3rd upsampling layer, and the output of the 3rd upsampling layer is connected to another input of the 3rd merging layer; the output of the 3rd merging layer is connected to the input of the 3rd channel attention layer, and the output of the 3rd channel attention layer is connected to the input of the 10th convolutional layer; the output of the 11th convolutional layer is connected to the input of the 12th convolutional layer, and the output of the 12th convolutional layer is connected to the input of the 4th upsampling layer, and the output of the 4th upsampling layer is connected to another input of the 4th merging layer; the output of the 4th merging layer is connected to the 4th channel attention layer. The input of the attention layer and the output of the 4th channel attention layer are connected to the input of the 13th convolution layer; the outputs of the 1st, 4th, 7th, 10th and 13th convolution layers are connected to the input of the connection layer, and the output of the connection layer is connected to the input of the 15th convolution layer; the output of the 15th convolution layer is connected to one of the inputs of the 5th merging layer; the output of the 14th convolution layer is connected to the other input of the 5th merging layer; the output of the 5th merging layer is connected to the input of the 5th channel attention layer, and the output of the 5th channel attention layer forms the output of the XY network; Step 2: Use the segmented sample image set to train the semantic segmentation model constructed in step 1 to obtain a trained semantic segmentation model; Step 3: Send the image to be segmented into the trained semantic segmentation model obtained in step 2, and the trained semantic segmentation model outputs the segmented image.
2. The image segmentation method based on deep learning according to claim 1, characterized in that: The fully convolutional decoding network consists of 2 convolutional layers and 1 bilinear interpolation layer; among them: the input of the first convolutional layer forms the input of the fully convolutional decoding network, the output of the first convolutional layer is connected to the input of the second convolutional layer, the output of the second convolutional layer is connected to the input of the bilinear interpolation layer, and the output of the bilinear interpolation layer forms the output of the fully convolutional decoding network.
