Urban scene real-time semantic segmentation method based on bilateral fusion network
Through the bilateral fusion network of the bilateral fusion network, combining spatial branches and semantic branches, the context information aggregator and semantic detail fusion module are used to solve the information fusion problem in the existing technology, and the accuracy and speed balance of real-time semantic segmentation of urban market scenes is achieved. .
Patent Information
- Application Number
- CN202410941236.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-15
- Publication Date
- 2024-11-08
AI Technical Summary
The existing semantic segmentation model is difficult to effectively integrate spatial details and high-level semantic features in the information fusion stage, which makes it difficult to balance the accuracy and reasoning speed when dealing with urban market scenarios.
A bilateral fusion network is adopted to extract the spatial detail features and high-level semantic information of the image through spatial branches and semantic branches, and a context information aggregator and semantic detail fusion module are introduced to realize the organic fusion of low-level spatial details and high-level semantic information.
The segmentation accuracy and calculation efficiency of the model are improved, the accuracy and inference speed are balanced in real-time semantic segmentation are ensured, and the organic combination of image space details and semantic information is realized.
Smart Images

Figure CN118918326A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of image semantic segmentation, and in particular relates to a real-time semantic segmentation method for urban scenes based on a bilateral fusion network. Background Art
[0002] In today's computer vision field, semantic segmentation is an important task that aims to assign specific semantic labels to each pixel in an image, providing key support for intelligent image understanding and scene analysis. Since FCN, the pioneering work of deep learning methods in the field of semantic segmentation, has achieved outstanding performance, the existing segmentation models have been dominated by the fully convolutional neural network it represents.
[0003] FCN is an encoder-decoder architecture, which divides the network structure into encoder and decoder by downsampling and upsampling step by step. The encoder is mainly responsible for capturing the multi-scale features of the image, and the decoder produces pixel-level feature representation. With the rapid development of deep learning technology, various semantic segmentation models have emerged in an endless stream. In addition to the encoder-decoder architecture, dual-branch architecture, multi-branch architecture and other architectures have also emerged. These architectures have their own characteristics in balancing computational performance and segmentation accuracy. On the one hand, the traditional encoder-decoder architecture performs well in processing high-level semantic information, but may lose sensitivity to spatial details. On the other hand, some models using dual-branch architectures focus on capturing spatial details and high-level semantics at the same time, and have achieved certain success. However, these methods often face challenges in the information fusion stage, and need to effectively integrate the feature information extracted by the two branches.
[0004] The invention with patent publication number CN115995002B discloses a network construction method and a real-time semantic segmentation method for urban scenes. The encoder module is constructed using a backbone network, a strip pooling semantic enhancement module SPSEM and a gated fusion module GFM; the decoder module is constructed using 6 context guidance modules CGM; the output end of the encoder module is connected to the input end of the decoder module, the output end of the decoder module is connected to the input end of the splicing operation layer, the output end of the splicing operation layer is connected to the input end of the convolution layer, and the output end of the convolution layer is connected to the input end of the upsampling layer to form an initial semantic segmentation model; the initial semantic segmentation model is trained using sample data to obtain a target semantic segmentation model. The invention uses the target semantic segmentation model to realize the semantic segmentation of common objects in urban scenes, making full use of multi-scale global and local information and more remote context information in a large receptive field, effectively improving the accuracy of semantic segmentation. However, the invention adopts an encoder-decoder architecture and does not involve the organic fusion problem between low-level spatial details and high-level semantics. Summary of the invention
[0005] Technical problem solved: The present invention discloses a real-time semantic segmentation method for urban scenes based on a bilateral fusion network. The spatial detail features and high-level semantic information of an image are obtained through spatial branches and semantic branches respectively. At the same time, a context information aggregator is introduced to capture the semantics of a larger receptive field, and a semantic detail fusion module is used to more effectively fuse low-level spatial detail features and high-level semantic information. The present invention realizes the organic combination of image spatial details and semantic information, while improving the model segmentation accuracy, maintaining the real-time performance of the network, ensuring the balance between accuracy and reasoning speed in real-time semantic segmentation, and improving the model segmentation effect.
[0006] Technical solution:
[0007] A method for real-time semantic segmentation of urban scenes based on a bilateral fusion network, the method comprising the following steps:
[0008] S1: Obtain the urban scene dataset and perform data preprocessing, and divide the dataset into training set, validation set, and test set in proportion;
[0009] S2: Build a bilateral fusion network, which includes a detail branch, a semantic branch and a semantic detail fusion module; wherein the detail branch uses the first three convolution blocks of ResNet18 without skip connections as the backbone network to extract the spatial detail feature map Y3 of the input image; the semantic branch uses the part after removing the fully connected layer at the tail of ResNet18 as the backbone network to extract the high-level semantic feature map X5 of the input image and transmit it to the context information aggregator, which integrates context information of different scales through a multi-pooling fusion mechanism to capture the semantics of a larger receptive field and outputs the feature map C; the semantic detail fusion module fuses the spatial detail feature map Y3 output by the backbone network of the detail branch and the feature map C output by the context information aggregator through mutual guidance of spatial details and high-level semantics, and outputs the final semantic segmentation result;
[0010] S3: Use the training set and validation set obtained in step S1 to train the bilateral fusion network to obtain the optimal network model;
[0011] S4: Input the test set obtained in step S1 into the optimal network model obtained in step S3 to obtain the semantic segmentation result of the urban scene image.
[0012] Step S1 further comprises:
[0013] S1.1: Download or create your own urban scene dataset from an open source dataset website;
[0014] S1.2: Preprocess the images and labels in the urban scene dataset, including random jitter, scale adjustment, horizontal flipping, and cropping;
[0015] S1.3: Divide the images and their labels in the preprocessed urban scene dataset into training set, validation set and test set in proportion.
[0016] Further, in step S2, the context information aggregator includes a 3×3 average pooling layer, a global average pooling layer, a strip pooling layer in width and height, a splicing unit, a first 1×1 convolutional layer, a first fusion unit and a first 3×3 convolutional layer;
[0017] After the high-level semantic feature map X5 is imported into the context information aggregator, the 3×3 average pooling layer, the global average pooling layer and the strip pooling layer perform 3×3 average pooling, global average pooling and strip pooling in width and height directions in parallel. The three feature maps after pooling are spliced by channels by the splicing unit and then imported into the first 1×1 convolution layer to extract features; the first fusion unit adds the input high-level semantic feature map X5 and the convolutional features output by the first 1×1 convolution layer through a jump connection for fusion, and the fusion result is imported into the first 3×3 convolution layer to generate an output feature map C.
[0018] Furthermore, the semantic detail fusion module includes a second 3×3 convolution layer, an upsampling layer, a first 3×3 depth-separable convolution layer, a third 3×3 convolution layer, an average pooling layer, a second 3×3 depth-separable convolution layer, a second fusion layer, and a second 1×1 convolution layer;
[0019] The feature map C is respectively input into the second 3×3 convolutional layer and the first 3×3 depth-separable convolutional layer; the spatial detail feature map Y3 is respectively input into the third 3×3 convolutional layer and the second 3×3 depth-separable convolutional layer;
[0020] After the feature map C passes through the second 3×3 convolution layer and the upsampling layer in sequence, it guides the spatial detail feature map Y3 after passing through the second 3×3 depth-separable convolution layer, and outputs the feature map YC; after the spatial detail feature map Y3 passes through the third 3×3 convolution layer and the average pooling layer in sequence, it guides the high-level semantic feature map C after passing through the first 3×3 depth-separable convolution layer, and outputs the feature map CY; the second fusion layer fuses the feature maps YC and CY through an addition operation, and uses the second 1×1 convolution layer to refine the information, and outputs the final semantic segmentation result.
[0021] Step S3 further comprises:
[0022] S3.1: Set the initial training parameters of the bilateral fusion network;
[0023] S3.2: The cross entropy loss function is used to calculate the loss during training:
[0024] L=-[ylogy′+(1-y)log(1-y′)]
[0025] In the formula, y is the true value and y′ is the predicted value; when y=1, L=-log′, the closer y′ is to 1, the smaller the loss value is, and the closer y′ is to 0, the larger the loss value is; when y=0, L=-log(1-y′), the closer y′ is to 1, the larger the loss value is, and the closer y′ is to 0, the smaller the loss value is;
[0026] The gradient is calculated according to the loss function, the random gradient descent optimizer is used to update the network model parameters, and the poly strategy is used to decay the learning rate lr:
[0027]
[0028] Among them, base_lr is the initial learning rate, cur_iter is the current number of training rounds, and total_iter is the total number of training rounds set;
[0029] S3.3: Use the average intersection over union (mIoU) and frame rate (FPS) to evaluate the model. The higher the mIoU value, the more accurate the segmentation result obtained by the model. The larger the FPS value, the faster the segmentation speed of the model.
[0030]
[0031] In the formula, N represents the number of categories; i represents the i-th category; TP represents the true positive example, that is, the model prediction and the true label are both positive examples, TP i Indicates the number of true positive examples in the i-th category; FP is a false positive example, that is, the model predicts that the category is a positive example, but the category is a negative example in the true label, FP i represents the number of false positive examples in the i-th category; FN is a false negative example, that is, the model predicts that the category is a negative example, but the category is a positive example in the true label, FN i represents the number of false negative examples of the i-th category; figure_num represents the total number of predicted images; time represents the total prediction time;
[0032] S3.4: Repeat the training process from step S3.2 to step S3.3, and use the validation set to evaluate the bilateral fusion network after each round of training until the set total number of training rounds is completed and the optimal network model is saved.
[0033] Step S4 further comprises:
[0034] S4.1: Import the optimal network model saved in step S3, and read in the images and labels of the test set in step S1;
[0035] S4.2: Calculate the average intersection over union (mIoU) and frame rate (FPS), and save the test results.
[0036] Beneficial effects:
[0037] First, the real-time semantic segmentation method of urban scenes based on bilateral fusion network of the present invention effectively captures image microstructure and edge details by introducing detail branches, and retains more high-resolution spatial detail features;
[0038] Second, the real-time semantic segmentation method of urban scenes based on bilateral fusion network of the present invention uses semantic branches to perform deeper feature extraction and capture high-level semantic information. It also flexibly integrates context information of different scales by introducing a multi-pooling fusion mechanism of context information aggregator.
[0039] Thirdly, the real-time semantic segmentation method of urban scenes based on bilateral fusion network of the present invention efficiently fuses the features of the two branches through the semantic detail fusion module, and realizes the organic combination of image spatial details and semantic information through the mutual guidance of spatial details and high-level semantics;
[0040] Fourth, the real-time semantic segmentation method of urban scenes based on the bilateral fusion network of the present invention not only effectively improves the computational efficiency of the model, but also retains the organic combination of spatial and semantic features, thereby achieving a balance between accuracy and reasoning speed in real-time semantic segmentation. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] Figure 1 This is a flow chart of a method for real-time semantic segmentation of urban scenes based on a bilateral fusion network according to an embodiment of the present invention.
[0042] Figure 2 is a schematic diagram of a bilateral fusion network structure according to an embodiment of the present invention;
[0043] Figure 3 is a schematic diagram of the structure of a context information aggregator (CIA) according to an embodiment of the present invention;
[0044] Figure 4 It is a structural diagram of a semantic detail fusion module (SDFM) according to an embodiment of the present invention. DETAILED DESCRIPTION
[0045] The following examples will enable those skilled in the art to more fully understand the present invention, but are not intended to limit the present invention in any way.
[0046] The embodiment of the present invention discloses a method for real-time semantic segmentation of urban scenes based on a bilateral fusion network, and the method for real-time semantic segmentation of urban scenes comprises the following steps:
[0047] S1: Obtain the urban scene dataset and perform data preprocessing, and divide the dataset into training set, validation set, and test set in proportion;
[0048] S2: Build a bilateral fusion network, which includes a detail branch, a semantic branch and a semantic detail fusion module; wherein the detail branch uses the first three convolution blocks of ResNet18 without skip connections as the backbone network to extract the spatial detail feature map Y3 of the input image; the semantic branch uses the part after removing the fully connected layer at the tail of ResNet18 as the backbone network to extract the high-level semantic feature map X5 of the input image and transmit it to the context information aggregator, which integrates context information of different scales through a multi-pooling fusion mechanism to capture the semantics of a larger receptive field and outputs the feature map C; the semantic detail fusion module fuses the spatial detail feature map Y3 output by the backbone network of the detail branch and the feature map C output by the context information aggregator through mutual guidance of spatial details and high-level semantics, and outputs the final semantic segmentation result;
[0049] S3: Use the training set and validation set obtained in step S1 to train the bilateral fusion network to obtain the optimal network model;
[0050] S4: Input the test set obtained in step S1 into the optimal network model obtained in step S3 to obtain the semantic segmentation result of the urban scene image.
[0051] Examples
[0052] like Figure 1 As shown, this embodiment provides a method for real-time semantic segmentation of urban scenes based on a bilateral fusion network, and the specific steps are as follows:
[0053] Step S1: Obtain the urban scene dataset and perform data preprocessing, and divide the dataset into a training set, a validation set, and a test set in proportion.
[0054] In this example, step S1 mainly includes the following steps:
[0055] Step S1.1: Download the Cityscapes dataset from the Cityscapes official website https: / / www.cityscapes-dataset.com / ; the Cityscapes dataset consists of pictures of different urban road scenes, including 5,000 finely labeled pictures and 20,000 roughly labeled pictures with a total of 30 segmentation categories. This embodiment only uses 5,000 finely labeled pictures and 19 segmentation categories. The 5,000 finely labeled pictures include 2,975 training set pictures, 500 validation set pictures, and 1,525 test set pictures;
[0056] Step S1.2: Preprocess the dataset, including random color jitter (i.e., randomly adjusting the brightness, contrast, and saturation of the image), random resizing between 0.5 and 2.0, random horizontal flipping with a probability of 0.5, and random cropping at an image resolution of 1024×512;
[0057] Step S1.3: Put the preprocessed images and their labels into the corresponding training set, validation set and test set.
[0058] Step S2: Build a bilateral fusion network.
[0059] The bilateral fusion network structure is as follows Figure 2 As shown, it includes detail branch, semantic branch and semantic detail fusion module SDFM;
[0060] In the detail branch, the first three convolutional blocks of ResNet18 without skip connections are used as the backbone network of the detail branch, and the original output of the third convolutional block is extracted from the input image. Figure 1 / 8 size spatial detail feature map Y3; in the semantic branch, the part after removing the fully connected layer at the tail of ResNet18 is used as the backbone network of the semantic branch, and the original output of the fifth convolutional block is extracted from the input image. Figure 1 / 32 size high-level semantic feature graph X5; in the semantic branch, the high-level semantic feature graph X5 output by the backbone network of the semantic branch is input into the context information aggregator CIA, and the feature graph C is output; wherein the structure of the context information aggregator is as follows Figure 3 As shown in the figure, after the high-level semantic feature map X5 is input into the context information aggregator, 3×3 average pooling, global average pooling, and strip pooling in both width and height directions are performed in parallel. Then, the three feature maps after pooling are spliced by channel and 1×1 convolution is used. Then, the input feature map X5 and the convolution feature are added and fused through a jump connection, and then a 3×3 convolution is used to generate the output feature map C. Finally, the spatial detail feature map Y3 output by the backbone network of the detail branch and the feature map C output by the context information aggregator are input into the semantic detail fusion module to output the final semantic segmentation result. The structure of the semantic detail fusion module is shown in the figure. Figure 4 As shown in the figure, the feature map C guides the spatial detail feature map Y3 after the 3×3 depth-separable convolution through 3×3 convolution and upsampling operations, and outputs the feature map YC; the spatial detail feature map Y3 guides the high-level semantic feature map C after the 3×3 depth-separable convolution through 3×3 convolution and average pooling operations, and outputs the feature map CY; finally, the feature maps YC and CY are fused through the addition operation, and the information is refined using 1×1 convolution to output the final semantic segmentation result.
[0061] Step S3: Use the training set and validation set obtained in step S1 to train the bilateral fusion network to obtain the optimal network model.
[0062] In this example, step S3 mainly includes the following steps:
[0063] Step S3.1: Set the initial parameters of model training as follows;
[0064] Initial learning rate: 0.01
[0065] Optimizer: SGD optimizer;
[0066] Learning rate decay: poly strategy;
[0067] Weight decay: 0.0004
[0068] Momentum: 0.9
[0069] Batch size: 16
[0070] Number of training epochs: 300.
[0071] Step S3.2: During the training process, the cross entropy loss function is used to calculate the loss. The calculation formula is as follows:
[0072] L=-[ylogy′+(1-y)log(1-y′)]
[0073] In the formula, y is the true value and y' is the predicted value. When y = 1, L = -log'. The closer y' is to 1, the smaller the loss value is, and the closer y' is to 0, the larger the loss value is. When y = 0, L = -log(1-y'). The closer y' is to 1, the larger the loss value is, and the closer y' is to 0, the smaller the loss value is.
[0074] The gradient is calculated according to the loss function, the SGD izer is used to update the network model parameters, and the poly strategy is used to decay the learning rate. The decay formula is as follows:
[0075]
[0076] Among them, base_lr is the initial learning rate, cur_iter is the current number of training rounds, and total_iter is the total number of training rounds set;
[0077] Step S3.3: Use mean intersection over union (mIoU) and frame rate (FPS) to evaluate the model. The higher the mIoU value, the more accurate the segmentation result obtained by the model. The larger the FPS value, the faster the segmentation speed of the model. The calculation formula is as follows:
[0078]
[0079] In the formula, N represents the number of categories; i represents the i-th category; TP (True Positive) represents a true positive example, that is, the model prediction and the true label are both positive examples, TP i Indicates the number of true positive examples in the i-th category; FP (False Positive) is a false positive example, that is, the model predicts that the category is a positive example, but the category is a negative example in the real label, FP i Indicates the number of false positive examples in the i-th category; FN (False Negative) is a false negative example, that is, the model predicts that the category is a negative example, but the category is a positive example in the true label. i represents the number of false negative examples of the i-th category; figure_num represents the total number of predicted images; time represents the total prediction time in seconds.
[0080] Step S3.4: Repeat the training process from step S3.2 to step S3.3, and use the validation set to evaluate the network model after each round of training until the set total number of training rounds of 300 rounds are completed, and save the optimal network model according to the optimal mIoU value.
[0081] Step S4: Input the test set obtained in step S1 into the optimal network model obtained in step S3 to obtain the semantic segmentation result of the urban scene image.
[0082] In this example, step S4 mainly includes the following steps:
[0083] Step S4.1: Import the optimal network model saved in step S3, and read in the images and labels of the test set in step S1;
[0084] Step S4.2: Calculate mIoU and FPS and save the test results.
[0085] The above are only preferred embodiments of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should be regarded as the protection scope of the present invention.
Claims
1. A real-time semantic segmentation method for urban scenes based on bilateral fusion network, characterized in that: The urban scene real-time semantic segmentation method comprises the following steps: S1: Obtain the urban scene dataset and perform data preprocessing, and divide the dataset into training set, validation set, and test set in proportion; S2: Build a bilateral fusion network, which includes a detail branch, a semantic branch and a semantic detail fusion module; wherein the detail branch uses the first three convolution blocks of ResNet18 without skip connections as the backbone network to extract the spatial detail feature map Y3 of the input image; the semantic branch uses the part after removing the fully connected layer at the tail of ResNet18 as the backbone network to extract the high-level semantic feature map X5 of the input image and transmit it to the context information aggregator, which integrates context information of different scales through a multi-pooling fusion mechanism to capture the semantics of a larger receptive field and outputs the feature map C; the semantic detail fusion module fuses the spatial detail feature map Y3 output by the backbone network of the detail branch and the feature map C output by the context information aggregator through mutual guidance of spatial details and high-level semantics, and outputs the final semantic segmentation result; S3: Use the training set and validation set obtained in step S1 to train the bilateral fusion network to obtain the optimal network model; S4: Input the test set obtained in step S1 into the optimal network model obtained in step S3 to obtain the semantic segmentation result of the urban scene image.
2. The urban scene real-time semantic segmentation method based on bilateral fusion network according to claim 1 is characterized in that: Step S1 further comprises: S1.1: Download or create your own urban scene dataset from an open source dataset website; S1.2: Preprocess the images and labels in the urban scene dataset, including random jitter, scale adjustment, horizontal flipping, and cropping; S1.3: Divide the images and their labels in the preprocessed urban scene dataset into training set, validation set and test set in proportion.
3. The urban scene real-time semantic segmentation method based on bilateral fusion network according to claim 1 is characterized in that: In step S2, the context information aggregator includes a 3×3 average pooling layer, a global average pooling layer, strip pooling layers in width and height, a splicing unit, a first 1×1 convolutional layer, a first fusion unit and a first 3×3 convolutional layer; After the high-level semantic feature map X5 is imported into the context information aggregator, 3×3 average pooling, global average pooling, and strip pooling in width and height are performed in parallel by a 3×3 average pooling layer, a global average pooling layer, and a strip pooling layer, and the three feature maps after pooling are spliced by channels by a splicing unit and then imported into the first 1×1 convolutional layer to extract features; The first fusion unit adds the input high-level semantic feature map X5 and the post-convolution feature output by the first 1×1 convolutional layer through a skip connection for fusion, and the fusion result is introduced into the first 3×3 convolutional layer to generate the output feature map C.
4. The urban scene real-time semantic segmentation method based on bilateral fusion network according to claim 1 is characterized in that: The semantic detail fusion module includes a second 3×3 convolution layer, an upsampling layer, a first 3×3 depth-separable convolution layer, a third 3×3 convolution layer, an average pooling layer, a second 3×3 depth-separable convolution layer, a second fusion layer, and a second 1×1 convolution layer; The feature map C is respectively input into the second 3×3 convolutional layer and the first 3×3 depth-separable convolutional layer; the spatial detail feature map Y3 is respectively input into the third 3×3 convolutional layer and the second 3×3 depth-separable convolutional layer; After the feature map C passes through the second 3×3 convolution layer and the upsampling layer in sequence, it guides the spatial detail feature map Y3 after passing through the second 3×3 depth-separable convolution layer, and outputs the feature map YC; after the spatial detail feature map Y3 passes through the third 3×3 convolution layer and the average pooling layer in sequence, it guides the high-level semantic feature map C after passing through the first 3×3 depth-separable convolution layer, and outputs the feature map CY; The second fusion layer fuses the feature maps YC and CY through an addition operation, and uses the second 1×1 convolutional layer to refine the information and output the final semantic segmentation result.
5. The urban scene real-time semantic segmentation method based on bilateral fusion network according to claim 1 is characterized in that: Step S3 further comprises: S3.1: Set the initial training parameters of the bilateral fusion network; S3.2: The cross entropy loss function is used to calculate the loss during training: L=-[ylogy′+(1-y)log(1-y′)] In the formula, y is the true value and y′ is the predicted value; when y=1, L=-log′, the closer y′ is to 1, the smaller the loss value is, and the closer y′ is to 0, the larger the loss value is; when y=0, L=-log(1-y′), the closer y′ is to 1, the larger the loss value is, and the closer y′ is to 0, the smaller the loss value is: The gradient is calculated according to the loss function, the random gradient descent optimizer is used to update the network model parameters, and the poly strategy is used to decay the learning rate lr: Among them, base_lr is the initial learning rate, cur_iter is the current number of training rounds, and total_iter is the total number of training rounds set; S3.3: Use the average intersection over union (mIoU) and frame rate (FPS) to evaluate the model. The higher the mIoU value, the more accurate the segmentation result obtained by the model. The larger the FPS value, the faster the segmentation speed of the model. In the formula, N represents the number of categories; i represents the i-th category; TP represents true positive examples, that is, both the model prediction and the true label are positive examples, TPi represents the number of true positive examples of the i-th category; FP represents false positive examples, that is, the model predicts that the category is a positive example, but the true label shows that the category is a negative example, FPi represents the number of false positive examples of the i-th category; FN represents false negative examples, that is, the model predicts that the category is a negative example, but the true label shows that the category is a positive example, FN i represents the number of false negative examples of the i-th category; figure_num represents the total number of predicted images; time represents the total prediction time; S3.4: Repeat the training process from step S3.2 to step S3.3, and use the validation set to evaluate the bilateral fusion network after each round of training until the set total number of training rounds is completed and the optimal network model is saved.
6. The urban scene real-time semantic segmentation method based on bilateral fusion network according to claim 1 is characterized in that: Step S4 further comprises: S4.1: Import the optimal network model saved in step S3, and read in the images and labels of the test set in step S1; S4.2: Calculate the average intersection over union (mIoU) and frame rate (FPS), and save the test results.
Citation Information
Patent Citations
Indoor scene image real-time semantic segmentation method based on multi-scale refinement
CN116229461A
Cited By
City scene real-time semantic segmentation method based on hybrid parallel network
CN121074403A
Lightweight multi-branch semantic segmentation method and system for complex road scene
CN121811418A
Lightweight multi-branch semantic segmentation method and system for complex road scenes
CN121811418B
