Semantic segmentation method for unmanned aerial vehicle aerial image facing urban scene

Abstract

The invention relates to the technical field of semantic segmentation in computer vision, and discloses an urban scene-oriented semantic segmentation method for unmanned aerial vehicle aerial images, which comprises the following steps of: collecting a large amount of unmanned aerial vehicle aerial image data to form a semantic segmentation label set; taking the semantic segmentation tag set as the input of a feature extraction module, and training the feature extraction module; taking the semantic segmentation tag set as the input of a Laplacian pyramid module, and training the Laplacian pyramid module; taking the feature map output by the feature extraction module and the feature component output by the Laplacian pyramid module as the input of a feature fusion module, and training the feature fusion module; inputting the semantic segmentation tag set into a detail tag processing module to obtain a semantic segmentation edge detail tag graph; and calculating the loss of the semantic segmentation model. According to the method, the semantic segmentation problem of the aerial image of the unmanned aerial vehicle in the urban scene is solved, so that the safety and practicability of the unmanned aerial vehicle in the urban environment are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of semantic segmentation in computer vision, and in particular to a semantic segmentation method for unmanned aerial vehicle aerial images of urban scenes. Background Art

[0002] With the rapid development of drone technology, drone aerial images are increasingly used in urban scenes. These images can be used not only in urban planning, environmental monitoring, and resource management, but also in intelligent transportation, emergency rescue, and security monitoring. However, it is still a challenging problem to achieve automatic analysis and understanding of these images, especially semantic segmentation to identify the semantic information of different objects and regions in the image. Traditional image processing methods and computer vision techniques have some limitations when processing drone aerial images of urban scenes. The variety of objects in urban environments, the different sizes, the complex lighting conditions, and the cluttered and changing backgrounds make it difficult for traditional methods to obtain accurate semantic segmentation results. In addition, semantic segmentation in urban scenes requires accurately segmenting the boundaries of different categories while maintaining a high inference speed. Summary of the invention

[0003] The purpose of the present invention is to solve the problem of semantic segmentation of drone aerial images in urban scenes, so as to improve the safety and practicality of drones in urban environments. By applying semantic segmentation technology in advanced computer vision, the present invention is committed to achieving accurate recognition and segmentation of different objects and areas in urban scenes. This technology is not only helpful in application fields such as urban planning and environmental monitoring, but also can improve the operating efficiency and situational awareness of drones in urban environments. To this end, a semantic segmentation method for drone aerial images in urban scenes is provided.

[0004] In order to achieve the above-mentioned object of the invention, the embodiment of the present invention provides the following technical solutions:

[0005] A semantic segmentation method for unmanned aerial vehicle aerial images of urban scenes is implemented based on a semantic segmentation model, wherein the semantic segmentation model includes a feature extraction module, an ASPP module, a Laplace pyramid module, a feature fusion module, a detail segmentation head, and a detail label processing module; the method includes the following steps:

[0006] Step 1: Collect a large amount of drone aerial image data, annotate each drone aerial image data pixel by pixel with semantic labels, and form a semantic segmentation label set;

[0007] Step 2, using the semantic segmentation label set as input of a feature extraction module to train the feature extraction module;

[0008] Step 3, using the semantic segmentation label set as the input of the Laplacian pyramid module to train the Laplacian pyramid module;

[0009] Step 4, taking the feature map output by the feature extraction module and the feature component output by the Laplacian pyramid module as the input of the feature fusion module, and training the feature fusion module;

[0010] Step 5, inputting the semantic segmentation label set into a detail label processing module to obtain a semantic segmentation edge detail label map;

[0011] Step 6, performing loss calculation on the feature map output by the feature extraction module, the feature component output by the Laplacian pyramid module, and the semantic segmentation edge detail label map output by the detail label processing module to complete the training of the semantic segmentation model;

[0012] Step 7: Input the real-time drone aerial image data into the trained semantic segmentation model to obtain its semantic category.

[0013] Compared with the prior art, the present invention has the following beneficial effects: The present invention achieves more accurate segmentation of drone aerial images in urban scenes while maintaining high processing speed through advanced technologies such as deep learning and convolutional neural networks. In addition, the present invention solves the problem of uneven frequency of objects of different categories in urban environments, and ensures accurate segmentation of each category through a combined loss function. This technology not only enhances the situational awareness and decision-making capabilities of drones in urban scenes, but also provides an efficient image segmentation solution specifically for urban environments, which is suitable for multiple fields such as urban planning, environmental monitoring, and intelligent transportation. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0015] Figure 1 It is a structural diagram of a semantic segmentation model according to an embodiment of the present invention;

[0016] Figure 2 This is a schematic diagram of data processing of a Laplacian pyramid module according to an embodiment of the present invention;

[0017] Figure 3 is a schematic diagram of the structure of any feature fusion unit in an embodiment of the present invention;

[0018] Figure 4This is a schematic diagram of the structure of a detail label processing module according to an embodiment of the present invention;

[0019] Figure 5 Schematic diagram of the structure of the detail segmentation head according to an embodiment of the present invention. DETAILED DESCRIPTION

[0020] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents the selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present invention.

[0021] It should be noted that similar numbers and letters represent similar items in the following figures, so once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. At the same time, in the description of the present invention, the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance, or implying any such actual relationship or order between these entities or operations. In addition, the terms "connected", "connected", etc. can be directly connected between elements, or indirectly connected via other elements.

[0022] Embodiment 1:

[0023] The present invention is achieved through the following technical solutions: Figure 1 As shown, a semantic segmentation method for drone aerial images of urban scenes is implemented based on a semantic segmentation model, wherein the semantic segmentation model includes a feature extraction module, an ASPP module, a Laplace pyramid module, a feature fusion module, a detail segmentation head, and a detail label processing module; the method includes the following steps:

[0024] Step 1: Collect a large amount of drone aerial image data, annotate each drone aerial image data pixel by pixel with semantic labels, and form a semantic segmentation label set.

[0025] In order to train the semantic segmentation model, a large amount of drone aerial image data containing urban scenes is collected in advance, and semantic labels are annotated pixel by pixel for each drone aerial image data. In order to enable the drone to demonstrate good generalization ability when performing target recognition tasks, for these drone aerial image data, the categories of semantic labels that need to be annotated are determined, including: 'Background', 'Car', 'Building', 'Tree', 'Road', 'Obstacle', 'River', 'Plant', 'Human', etc. The labeme image annotation tool is used for annotation, and the annotated data is made into the VOC dataset format to form a semantic segmentation label set.

[0026] Step 2: Using the semantic segmentation label set as input of a feature extraction module, and training the feature extraction module so that the feature extraction module can accurately extract image semantic information.

[0027] The feature extraction module uses a ResNeXt-101 network to extract high-level semantic features from the input image. These semantic features have different dimensions and help to better represent different objects and regions in the image.

[0028] See also Figure 1 , the feature extraction module includes a first-stage feature extraction unit, a second-stage feature extraction unit, a third-stage feature extraction unit, and a fourth-stage feature extraction unit. First, the image size in the semantic segmentation label set is initialized to 224*224*3; the semantic segmentation label set is input into the first-stage feature extraction unit, and a feature map F1 with a size of 112*112*64 is output; the feature map F1 is input into the second-stage feature extraction unit, and the output size is halved again, and the output size is a feature map F2 with a size of 56*56*128; the feature map F2 is input into the third-stage feature extraction unit, and the output size is halved again, and the output size is a feature map F3 with a size of 28*28*256; the feature map F3 is input into the fourth-stage feature extraction unit, and the output size is halved again, and the output size is a feature map F4 with a size of 14*14*512.

[0029] The network structures of the feature extraction units in the four stages of the feature extraction module are the same. The feature extraction units in each stage reduce or increase the dimension through a 1×1 convolution layer, introduce group convolution through a 3×3 convolution layer to increase the interaction between channels, and maintain information transmission through residual connections, thereby achieving high-performance feature extraction in the model.

[0030] The ResNeXt-101 network is a deep convolutional neural network (CNN) architecture. It has the following characteristics and advantages as a backbone network in semantic segmentation tasks:

[0031] High-performance feature extraction: The ResNeXt-101 network is a deep neural network with 101 layers, so it can effectively learn and extract high-level features in images, which is very important for semantic segmentation tasks because these tasks require the network to understand the semantic relationship between different regions in the image;

[0032] Multi-channel information learning: The ResNeXt-101 network adopts the concept of "cardinality" by combining multiple independent sub-networks together to increase the network's information capacity. This helps the network better learn different types of features and multiple information, which helps improve the performance of semantic segmentation;

[0033] Strong transfer learning ability: The ResNeXt-101 network has been trained on large-scale image classification tasks, so it has strong transfer learning ability. This means that you can use the trained ResNeXt-101 model as the backbone of the semantic segmentation task, and then fine-tune it on a relatively small dataset to obtain good segmentation performance;

[0034] Scalability: The ResNeXt-101 network is a scalable architecture that can increase network capacity by increasing the number of cards or channels. This makes it suitable for semantic segmentation tasks of different scales and can be adjusted according to the complexity of the task;

[0035] Excellent performance: The ResNeXt-101 network performs well in image classification tasks, so using it as a backbone in semantic segmentation tasks usually leads to very good performance, especially on challenging datasets.

[0036] Step 3: Using the semantic segmentation label set as the input of the Laplacian pyramid module, and training the Laplacian pyramid module, so that the feature components output by the Laplacian pyramid module can accurately describe the edge features of the image.

[0037] See also Figure 1 The Laplace pyramid module includes a first-scale Laplace pyramid unit, a second-scale Laplace pyramid unit, a third-scale Laplace pyramid unit, and a fourth-scale Laplace pyramid unit.

[0038] See also Figure 2First, the image size in the semantic segmentation label set is initialized to 224*224, and the image in the semantic segmentation label set is defined as the original image G0; the original image G0 is input into the first scale Laplacian pyramid unit for Gaussian blur and downsampling processing to obtain a feature map G1 with a size of 112*112, and then the feature map G1 is upsampled to restore it to the size of the original image G0, recorded as feature map T0, and then the original image G0 and the feature map T0 are differentially processed to output the Laplacian pyramid feature component L0;

[0039] The feature map G1 is input into the second-scale Laplacian pyramid unit for Gaussian blur and downsampling to obtain a feature map G2 with a size of 56*56. The feature map G2 is then upsampled to restore it to the size of the feature map G1, recorded as feature map T1, and then the feature map G1 and the feature map T1 are differentially processed to output the Laplacian pyramid feature component L1;

[0040] The feature map G2 is input into the third-scale Laplacian pyramid unit for Gaussian blur and downsampling to obtain a feature map G3 with a size of 28*28. The feature map G3 is then upsampled to restore it to the size of the feature map G2, recorded as feature map T2, and then the feature map G2 and the feature map T2 are differentially processed to output the Laplacian pyramid feature component L2;

[0041] The feature map G3 is input into the fourth-scale Laplacian pyramid unit for Gaussian blur and downsampling to obtain a feature map G4 with a size of 14*14. The feature map G4 is then upsampled to restore it to the size of the feature map G3, recorded as feature map T3, and then the feature map G3 and the feature map T3 are differentially processed to output the Laplacian pyramid feature component L3.

[0042] The multi-scale Laplacian pyramid can extract feature information at multiple scales from an image and is very useful for semantic segmentation tasks because the target object may have different appearances at different scales. By analyzing images at different scales, the model can better capture the details and global context of the object, thereby improving the accuracy of segmentation.

[0043] Step 4: Take the feature map output by the feature extraction module and the feature components output by the Laplacian pyramid module as the input of the feature fusion module, and train the feature fusion module so that the segmentation result map output by the feature fusion module can accurately express the image semantic information.

[0044] Please continue to see Figure 1 The feature fusion module includes a first feature fusion unit, a second feature fusion unit, a third feature fusion unit, and a fourth feature fusion unit.

[0045] The feature map F4 is input into the ASPP module (atrous convolution pyramid pooling module) to improve the receptive field of the model when extracting context features, and the feature map F4 is processed to generate a semantic feature map F4_1. The semantic feature map F4_1 and the Laplacian pyramid feature component L0 are input into the first feature fusion unit for feature fusion to generate a first-level feature map FL1; the feature map F3, the Laplacian pyramid feature component L1 and the first-level feature map FL1 are input into the second feature fusion unit for feature fusion to generate a second-level feature map FL2; the feature map F2, the Laplacian pyramid feature component L2 and the second-level feature map FL2 are input into the third feature fusion unit for feature fusion to generate a third-level feature map FL3; the feature map F1, the Laplacian pyramid feature component L3 and the third-level feature map FL3 are input into the fourth feature fusion unit for feature fusion to generate a fourth-level feature map FL4; the fourth-level feature map FL4 is used as the segmentation result map.

[0046] The network structures of the four feature fusion units in the feature fusion module are the same, see Figure 3 , taking the first feature fusion unit as an example, the first feature fusion unit includes an upsampling convolution layer upconv, a stacked convolution layer conv block, and a concatenation layer concat; the semantic feature map F4_1 is input into the upsampling convolution layer upconv as a backbone feature component, the feature map output by the upsampling convolution layer upconv is processed by the stacked convolution layer conv block, the Laplacian pyramid feature component F4 is input into the concatenation layer concat as a Laplacian pyramid component, the feature map output by the stacked convolution layer convblock is concatenated with the Laplacian pyramid feature component F4 in the concatenation layer concat, thereby outputting a first-level feature map FL1.

[0047] Similarly, for the second feature fusion unit, the feature map F3 is input into the upsampling convolution layer upconv as the backbone feature component, the primary feature map FL1 is input into the upsampling convolution layer upconv as the upper layer fusion feature, the feature map output by the upsampling convolution layer upconv is processed by the stacked convolution layer conv block, the Laplacian pyramid feature component F3 is input into the concatenation layer concat as the Laplacian pyramid component, the feature map output by the stacked convolution layer conv block is concatenated with the Laplacian pyramid feature component F3 in the concatenation layer concat, thereby outputting the secondary feature map FL2.

[0048] The third and fourth feature fusion units are similar, so they will not be described here. Feature layer-by-layer fusion allows feature information from different convolutional layers to communicate and integrate with each other, which helps improve the model's semantic understanding of pixels, because the underlying features contain more detailed information, while the high-level features contain more semantic information. By fusing this information, the model can better understand the contextual information of the image.

[0049] Step 5: Input the semantic segmentation label set into the detail label processing module to obtain a semantic segmentation edge detail label map.

[0050] like Figure 4 As shown in the figure, the semantic segmentation label set is input into the detail label processing module, and Laplace convolution operations with different strides (Stride=1, Stride=2, Stride=4) are performed to obtain its multi-scale edge detail features. Then, the multi-scale edge detail features are restored to the same size as the semantic segmentation label set through upsampling operations with different magnifications (2 times, 4 times), and then reduced in dimension through a 1×1 convolution operation to generate a semantic segmentation edge detail label map.

[0051] Step 6: Perform loss calculation on the feature map output by the feature extraction module, the feature component output by the Laplacian pyramid module, and the semantic segmentation edge detail label map output by the detail label processing module to complete the training of the semantic segmentation model.

[0052] The feature map F2 output by the second stage feature extraction unit in the feature extraction module is input into the detail segmentation head for processing to generate the predicted edge detail map. Figure 5 As shown, the detail segmentation head includes a 3×3 convolution layer, a batch normalization layer (BN layer), a ReLU activation function, and a 1×1 convolution layer.

[0053] Since the number of detail pixels is far less than that of non-detail pixels, detail prediction is a class imbalance problem, and the weighted cross entropy loss function usually leads to rough results. Therefore, this scheme adopts the binary cross entropy loss function and the Dice loss function to jointly optimize detail learning. The detail label loss is calculated for the edge detail map predicted by the detail segmentation head and the semantic segmentation edge detail label map generated by the detail label processing module:

[0054] L detail =L dice (p d ,g d )+L bce (p d ,g d );

[0055] Among them, L detail represents the detail label loss function; p dRepresents the predicted edge detail map, i.e., the predicted label; g d Represents the semantic segmentation edge detail label map, that is, the real label; L dice (p d ,g d ) represents the Dice loss function; L bce (p d ,g d ) represents the binary cross entropy loss function.

[0056]

[0057] Wherein, H and W represent the height and width of the predicted edge detail image, respectively; i represents the i-th pixel in the edge detail image; ∈ represents the Laplace smoothing term, which is used to avoid zero division, and ∈=1.

[0058] In order to achieve real-time results in semantic segmentation tasks, a detail label loss function is designed that is only used in the model training phase. This design can effectively improve the model's edge detail extraction capability without increasing the number of model parameters. When it is mounted on a drone platform, it can achieve high-precision real-time reasoning results.

[0059] Next, the cross entropy loss is calculated for the segmentation result map output by the fourth feature fusion unit and the semantic segmentation edge detail label map generated by the detail label processing module:

[0060]

[0061] Among them, L ce represents the cross entropy loss function; y jk Indicates that the j-th pixel belongs to the true label of the k-th category, which is obtained from the semantic segmentation edge detail label map; p jk represents the probability that the j-th pixel predicted by the semantic segmentation model belongs to the k-th category; N represents the number of pixels in the segmentation result image, j = 1, 2, ..., N; C represents the number of categories, k = 1, 2, ..., C.

[0062] Then the total loss function LOSS of the semantic segmentation model is:

[0063] LOSS=L detail +L ce .

[0064] Step 7: Input the real-time drone aerial image data into the trained semantic segmentation model to obtain its semantic category.

[0065] In summary, the semantic segmentation model uses the ResNext101 network as the backbone of the model, which can fully extract image features, use the multi-scale Laplacian pyramid to extract multi-scale feature information, and use detail loss optimization to reduce the number of model parameters while improving the model's edge detail extraction ability. The model was deployed to TX2 for testing, and the semantic segmentation MIoU reached 85% and the fps reached 64, indicating that the model has efficient segmentation performance and real-time performance in the drone aerial photography dataset established in the present invention.

[0066] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A semantic segmentation method for drone aerial images of urban scenes, characterized by: It is implemented based on a semantic segmentation model, which includes a feature extraction module, an ASPP module, a Laplace pyramid module, a feature fusion module, a detail segmentation head, and a detail label processing module; The method comprises the following steps: Step 1: Collect a large amount of drone aerial image data, annotate each drone aerial image data pixel by pixel with semantic labels, and form a semantic segmentation label set; Step 2, using the semantic segmentation label set as input of a feature extraction module to train the feature extraction module; Step 3, using the semantic segmentation label set as the input of the Laplacian pyramid module to train the Laplacian pyramid module; Step 4, taking the feature map output by the feature extraction module and the feature component output by the Laplacian pyramid module as the input of the feature fusion module, and training the feature fusion module; Step 5, inputting the semantic segmentation label set into a detail label processing module to obtain a semantic segmentation edge detail label map; Step 6, performing loss calculation on the feature map output by the feature extraction module, the feature component output by the Laplacian pyramid module, and the semantic segmentation edge detail label map output by the detail label processing module to complete the training of the semantic segmentation model; Step 7: Input the real-time drone aerial image data into the trained semantic segmentation model to obtain its semantic category.

2. The semantic segmentation method of drone aerial images for urban scenes according to claim 1 is characterized by: In step 2, the feature extraction module includes a first-stage feature extraction unit, a second-stage feature extraction unit, a third-stage feature extraction unit, and a fourth-stage feature extraction unit; First, the image size of the semantic segmentation label set is initialized to 224*224*3; the semantic segmentation label set is input into the first stage feature extraction unit, and a feature map F1 with a size of 112*112*64 is output; The feature map F1 is input into the second stage feature extraction unit, and the output size is halved again, and the output size is a feature map F2 of 56*56*128; The feature map F2 is input into the third stage feature extraction unit, and the output size is halved again, and the output size is a feature map F3 of 28*28*256; The feature map F3 is input into the fourth stage feature extraction unit, and the output size is halved again, and the output size is a feature map F4 of 14*14*512.

3. The semantic segmentation method of drone aerial images for urban scenes according to claim 2 is characterized by: In step 3, the Laplace pyramid module includes a first-scale Laplace pyramid unit, a second-scale Laplace pyramid unit, a third-scale Laplace pyramid unit, and a fourth-scale Laplace pyramid unit; First, the image size in the semantic segmentation label set is initialized to 224*224, and the image in the semantic segmentation label set is defined as the original image G0; the original image G0 is input into the first scale Laplacian pyramid unit for Gaussian blur and downsampling processing to obtain a feature map G1 with a size of 112*112, and then the feature map G1 is upsampled to restore it to the size of the original image G0, recorded as feature map T0, and then the original image G0 and the feature map T0 are differentially processed to output the Laplacian pyramid feature component L0; The feature map G1 is input into the second-scale Laplacian pyramid unit for Gaussian blur and downsampling to obtain a feature map G2 with a size of 56*56. The feature map G2 is then upsampled to restore it to the size of the feature map G1, recorded as feature map T1, and then the feature map G1 and the feature map T1 are differentially processed to output the Laplacian pyramid feature component L1; The feature map G2 is input into the third-scale Laplacian pyramid unit for Gaussian blur and downsampling to obtain a feature map G3 with a size of 28*28. The feature map G3 is then upsampled to restore it to the size of the feature map G2, recorded as feature map T2, and then the feature map G2 and the feature map T2 are differentially processed to output the Laplacian pyramid feature component L2; The feature map G3 is input into the fourth-scale Laplacian pyramid unit for Gaussian blur and downsampling to obtain a feature map G4 with a size of 14*14. The feature map G4 is then upsampled to restore it to the size of the feature map G3, recorded as feature map T3, and then the feature map G3 and the feature map T3 are differentially processed to output the Laplacian pyramid feature component L3.

4. The semantic segmentation method of drone aerial images for urban scenes according to claim 3 is characterized by: In step 4, the feature fusion module includes a first feature fusion unit, a second feature fusion unit, a third feature fusion unit, and a fourth feature fusion unit; The feature map F4 is input into the ASPP module, and the feature map F4 is processed to generate a semantic feature map F4_1; The semantic feature map F4_1 and the Laplacian pyramid feature component L0 are input into the first feature fusion unit for feature fusion to generate a first-level feature map FL1; The feature map F3, the Laplacian pyramid feature component L1 and the primary feature map FL1 are input into the second feature fusion unit for feature fusion to generate a secondary feature map FL2; The feature map F2, the Laplacian pyramid feature component L2 and the secondary feature map FL2 are input into the third feature fusion unit for feature fusion to generate a third-level feature map FL3; The feature map F1, the Laplacian pyramid feature component L3 and the third-level feature map FL3 are input into the fourth feature fusion unit for feature fusion to generate a fourth-level feature map FL4; the fourth-level feature map FL4 is used as the segmentation result map.

5. The semantic segmentation method of drone aerial images for urban scenes according to claim 1, characterized in that: The step 5 specifically comprises the following steps: The semantic segmentation label set is input into the detail label processing module, and Laplace convolution operations of different step lengths are performed to obtain its multi-scale edge detail features. The multi-scale edge detail features are then restored to the same size as the semantic segmentation label set through upsampling operations of different magnifications, and then reduced in dimension through a 1×1 convolution operation to generate a semantic segmentation edge detail label map.

6. The semantic segmentation method of drone aerial images for urban scenes according to claim 4, characterized in that: The step 6 specifically comprises the following steps: The detail label loss is calculated for the edge detail map predicted by the detail segmentation head and the semantic segmentation edge detail label map generated by the detail label processing module: L detail =L dice (p d ,g d )+L bce (p d ,g d ); Among them, L detail represents the detail label loss function; p d Represents the predicted edge detail map, i.e., the predicted label; g d Represents the semantic segmentation edge detail label map, that is, the real label; L dice (p d ,g d ) represents the Dice loss function; L bce (p d ,g d ) represents the binary cross entropy loss function; Where H and W represent the height and width of the predicted edge detail map, respectively; i represents the i-th pixel in the edge detail map; ∈ represents the Laplace smoothing term; The cross entropy loss is calculated for the segmentation result map output by the fourth feature fusion unit and the semantic segmentation edge detail label map generated by the detail label processing module: Among them, L ce represents the cross entropy loss function; y jk Indicates that the j-th pixel belongs to the true label of the k-th category, which is obtained from the semantic segmentation edge detail label map; p jk represents the probability that the j-th pixel predicted by the semantic segmentation model belongs to the k-th category; N represents the number of pixels in the segmentation result image, j = 1, 2, ..., N; C represents the number of categories, k = 1, 2, ..., C; The total loss function of the semantic segmentation model is: LOSS=L detail +L ce ; Among them, LOSS is the total loss function.

Images