RGB-D SOD method based on local feature extraction and semantic segmentation

By using local feature extraction and semantic segmentation methods in RGB-D significance detection, an encoder-decoder structure network is constructed, and the problem of insufficient fusion of color maps and depth maps and semantic segmentation accuracy in the prior art is solved, and the problem of insufficient fusion of color maps and depth maps and semantic segmentation accuracy is realized. Higher significance object extraction accuracy and stronger versatility, suitable for robot vision tasks.

CN118918342AActive Publication Date: 2024-11-08COLLEGE OF SCI & TECH NINGBO UNIV
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202411027640.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-30
Publication Date
2024-11-08
Estimated Expiration
2044-07-30

AI Technical Summary

Technical Problem

The existing RGB-D significance detection methods have insufficient accuracy in the fusion of color maps and depth maps and image semantic segmentation, which cannot meet the needs of robot applications.

Method used

Using a method based on local feature extraction and semantic segmentation, the network model of encoder-decoder structure is constructed, and multimodal feature extraction and fusion is used to use PVTv2 and ConvNeXt for multimodal feature extraction and fusion, and combining CSA, MFE, SSE and LGE modules to improve significance. Feature accuracy and segmentation effect.

Benefits of technology

The accuracy of the salient objects is significantly improved, which can be better applied in robotics technology, providing higher salient target extraction accuracy and stronger versatility.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118918342A_ABST
    Figure CN118918342A_ABST
Patent Text Reader

Abstract

An RGB-D SOD method based on local feature extraction and semantic segmentation belongs to the technical field of image saliency detection, aims to solve the problem of low precision of the existing saliency extraction method, and comprises the following steps: step 1, constructing a network model: the whole network is based on framework design of an encoder-decoder structure model; step 2, preparing a data set: selecting common RGB-D pictures to construct the data set; 3, training a network model: inputting the data set prepared in the step 2 into the network model constructed in the step 1 for training; step 4, selecting a minimum loss function value and an optimal evaluation index; 5, storing the model: solidifying the finally determined model parameters; and if the RGB-D image saliency detection operation is carried out, directly inputting the color image and the depth image into the network to obtain a final saliency image. The method is higher in precision, can be well applied to the robot technology, and is higher in universality.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to an RGB-D SOD (Salient Object Detection) method based on local feature extraction and semantic segmentation, and belongs to the technical field of image saliency detection. Background Art

[0002] Salient object detection (SOD) aims to simulate biological vision and capture the most important parts of an image. It is widely used in many downstream tasks, such as object detection, image classification, semantic segmentation, co-saliency detection, object tracking, etc. SOD models are usually used to complete the following two tasks: 1. Detect the most salient object. 2. Segment the accurate area of ​​the object. In order to improve the effectiveness of the above two tasks, intrinsic and extrinsic cues are usually used in saliency detection. Depth map is an extrinsic cue that can improve the effectiveness of saliency models. The emergence of depth cameras allows users to more easily access the depth map of the current scene. Appropriate feature extractors and fusion schemes can be selected to make depth information play a better role in saliency extraction. Saliency models can use edge information, multi-scale aggregation, completeness and context to extract salient objects. These methods have improved the effectiveness of saliency modeling to varying degrees. However, the existing models use external cues in a rough way, especially the fusion scheme of color and depth maps and the method for image semantic segmentation, and the results are still unsatisfactory.

[0003] BBS-Net is an internationally recognized excellent saliency detection model. See the journal "Cham: Springer International Publishing, 2020: 275-292", Deng-Ping Fan et al. published a paper titled "BBS-Net: RGB-D Salient Object Detection with a Bifurcated Backbone Strategy Network". This method first uses channel / spatial attention to fuse the color image and depth image of the image in the dataset; then, the deep enhancement module (DEM) is used to mine the information part of the depth clues from the channel and spatial views; then, the model parameters are adjusted through the loss function to obtain the optimal model; finally, the model is saved and the performance of the model is evaluated. Although this method runs faster, the accuracy of the salient target obtained by this method is low and cannot meet the current robot application scenarios. Summary of the invention

[0004] In order to solve the problem of low precision of existing saliency extraction methods, the present invention provides an RGB-D SOD method based on local feature extraction and semantic segmentation. By applying the local feature extraction method, the effect is improved in the multimodal multiscale fusion of color images and depth images, and the effect is improved in the enhancement of saliency features by applying image semantic segmentation, so that the final saliency object has higher precision and is more in line with the visual perception method of organisms. At the same time, the method proposed by the present invention is more accurate than the original method, can be well applied to robotics technology, and has stronger versatility.

[0005] The solution to the technical problem of the present invention is:

[0006] The RGB-D SOD method based on local feature extraction and semantic segmentation includes the following steps:

[0007] Step 1, build the network model: the whole network is based on the framework design of the encoder-decoder structure model; the image preprocessing module of the encoder part loads the pre-trained weight backbone network of the dual PVTv2 (Pyramid Vision Transformer v2), and extracts basic features from the color image and depth image of the input multimodal image respectively. The entire fusion is performed on four scales (96x96, 48x48, 24x24, 12x12); among them, a ConvNeXt (Convolutional Next Xtreme) module is added to the depth image for local feature processing, while the color image is not processed by the ConvNeXt module; then, the depth image features processed by PVTv2+ConvNeXt are passed to the CSA (Channel and SpatialAttention) module, and the result is added to the color image features processed by PVTv2; finally, the addition result is passed to the ConvNeXt module for a local feature extraction, and then a convolution is performed through the convolution module to complete the entire RGB-D fusion;

[0008] Step 2, prepare the data set: select commonly used RGB-D images to build a data set;

[0009] Step 3, training the network model: input the data set prepared in step 2 into the network model constructed in step 1 for training;

[0010] Step 4: Select the minimized loss function value and the optimal evaluation index: By minimizing the loss function of the network output image and label, until the number of training times reaches the set threshold and the loss value tends to converge, the model parameters can be considered to have been pre-trained and saved; at the same time, the optimal evaluation index is selected to measure the accuracy of the algorithm and evaluate the performance of the system;

[0011] Step 5, save the model: solidify the final model parameters; if performing RGB-D image saliency detection operation, directly input the color image and depth image into the network to obtain the final saliency map.

[0012] The decoder part includes: MFE (Multi-feature Extraction) module, SSE (Semantic Segmentation Enhancement) module and LGE (Local / Global Agreement and Edge Detection) module;

[0013] The MFE module uses convolution kernels of three different types and sizes, namely, ordinary convolution, cross convolution and dynamic convolution;

[0014] The SSE module enhances the feature output of RGB-D saliency through semantic segmentation based on DeepLabv3+ with spatial pyramid pooling and Encoder-Decoder architecture optimized by dilated convolution; wherein, the up-down sampling UDS (Up-Down-Sampling) module is used to integrate the connection between the upper and lower scales; and BN is added before ReLU; then, DeepLabv3+ modules with three different dilation rates of 3, 6, and 9 are used and added, and the saliency features are enhanced by using image segmentation;

[0015] The LGE module completes saliency detection through the interaction of four scales under the guidance of local / global consistency check and edge detection; a capsule network is used to process the local and global consistency relationship of salient objects; in addition, a Double Fusion module guided by edge detection is introduced to improve the effect of the LGE module; then, the posture vector output by the capsule network is connected with the edge information processed by Double Fusion as the final output posture vector; in the Double Fusion module, in addition to convolution and Pixel Shuffle operations, a Smish activation function is designed, which effectively improves the optimization effect during training when there is no BN through convolution, sigmoid, ln and tanh operations; finally, the output posture of the capsule network and the output of Double Fusion are connected as the output of the LGE module; the output of the LGE module is the final output of the entire network.

[0016] The step 2 is to collect the same RGB-D saliency dataset according to the comparison network BBS-Net, using NJU2K, NLPR, STERE, LFSD, SSD and SIP; conduct supervised training on the images in the dataset; perform image enhancement on the images in the dataset, normalize, randomly crop and randomly flip the same set of images as the input of the entire network, and use the annotated images in the dataset as labels.

[0017] In the step 3, during the pre-training process, the RGB-D saliency dataset uses NJU2K, NLPR, STERE, LFSD, SSD and SIP; supervised training is performed on the images in the dataset; image enhancement is performed on the images in the dataset, and the same group of images including color images, depth images and label images are normalized, randomly cropped, randomly flipped, and adjusted to a uniform size of 384×384 as input to the entire network, and the labeled images in the dataset are used as saliency labels.

[0018] The optimal evaluation indicator selection in step 4 is: S α Focus on capturing the structural similarity of images; E ξ It integrates local pixel matching and image-level statistics, and obtains the maximum value by changing the threshold; F β Use precision and recall to evaluate performance, or change the threshold to obtain the maximum value; finally, MAE calculates the average pixel-by-pixel difference between the predicted image and the true value; the above four indicators can evaluate the accuracy and quality of the algorithm and measure the role of the segmentation network.

[0019] The beneficial effects of the present invention are as follows:

[0020] 1. Use the image preprocessing module as the encoder. With the support of the PVTv2 backbone network, the channel and spatial attention (CSA) module and the local feature extraction module are used to process RGB and Depth in different ways to achieve better cross-modal fusion, and the absolute position encoding (APE) is added to embed more information about the relationship between image patches.

[0021] 2. In the decoder part, the semantic segmentation enhancement (SSE) module is introduced. The SSE module combines spatial pyramid pooling and encoder-decoder structure to integrate semantic information of different scales and enhance complex semantic segmentation tasks. In addition, the SSE module uses natural convolution to optimize the boundary recognition ability of spatial pyramid pooling.

[0022] 3. Comparisons with BBS-Net are made on 6 challenging datasets, and saliency maps generated by these datasets are given. Experimental results show that the proposed method has higher accuracy than other state-of-the-art techniques. The proposed method shows excellent performance in multi-scale, multi-modal fusion RGB-D saliency detection tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is a flow chart of the RGB-D SOD method based on local feature extraction and semantic segmentation of the present invention.

[0024] Figure 2 This is the network structure diagram of the RGB-D SOD method based on local feature extraction and semantic segmentation of the present invention.

[0025] Figure 3 This is a network structure diagram of the multiple feature extraction (MFE) module described in the present invention.

[0026] Figure 4 This is a network structure diagram of the semantic segmentation enhancement (SSE) module based on hole convolution optimization and spatial pyramid pooling described in the present invention.

[0027] Figure 5 This is a network structure diagram of the local / global consistency check and edge detection (LGE) module described in the present invention. DETAILED DESCRIPTION

[0028] The present invention is further described in detail below with reference to the accompanying drawings.

[0029] like Figure 1 As shown, the RGB-D SOD method based on local feature extraction and semantic segmentation includes the following steps:

[0030] Step 1: Build a network model. Figure 2 As shown in the figure, the entire network is designed based on the framework of the encoder-decoder structure model. The image preprocessing module of the encoder part loads the pre-trained weight backbone network of the dual PVTv2, and performs basic feature extraction on the color image and depth image of the input multimodal image respectively. The entire fusion is performed at four scales. Among them, adding a ConvNeXt module to the depth map for local feature processing will improve the fusion accuracy of RGB-D. For the color image, it is not processed by the ConvNeXt module. Then, the depth map features processed by PVTv2+ConvNeXt are passed to the CSA module, and the results are added to the color map features processed by PVTv2. Finally, the addition result is passed to the ConvNeXt module for a local feature extraction, and finally a convolution is performed through the convolution module to complete the entire RGB-D fusion.

[0031] The decoder part includes: MFE module, SSE module and LGE module.

[0032] The decoder part includes the following in order: 1. First, Figure 3 As shown in Figure 1, the MFE module uses three different types and sizes of convolution kernels to extract diverse features to enrich feature information at more levels. The three different convolutions are: ordinary convolution, cross convolution, and dynamic convolution. 2. Then, as Figure 4 As shown in the figure, the SSE module is based on DeepLabv3+ with spatial pyramid pooling and Encoder-Decoder architecture optimized by dilated convolution, and uses semantic segmentation to enhance the feature output of RGB-D saliency. Among them, the up-down sampling (UDS) module is used to integrate the connection between the upper and lower scales. In addition, BN is added before ReLU to alleviate the optimization difficulty caused by channel transformation. Then, DeepLabv3+ modules with three different dilation rates of 3, 6, and 9 are used and added together, and the saliency features are enhanced by using image segmentation. 3. Finally, as shown in the figure, Figure 5 As shown in the figure, LGE completes saliency detection through the interaction of four scales under the guidance of local / global consistency check and edge detection. Among them, a capsule network is used to process the local and global consistency relationship of salient objects. In addition, a Double Fusion module guided by edge detection is introduced to improve the effect of the LGE module. Then, the posture vector output by the capsule network is connected with the edge information processed by Double Fusion as the final output posture vector. In the Double Fusion module, in addition to the convolution and Pixel Shuffle operations, a Smish activation function is also designed. Through convolution, sigmoid, ln and tanh operations, the optimization effect during training is effectively improved when there is no BN. Finally, the output posture of the capsule network and the output of Double Fusion are connected as the output of the LGE module. The output of the LGE module is the final output of the entire network.

[0033] Step 2, prepare the dataset. Collect the same RGB-D saliency dataset based on the comparison network BBS-Net, using NJU2K, NLPR, STERE, LFSD, SSD and SIP. Perform supervised training on the images in the dataset; perform image enhancement on the images in the dataset, normalize, randomly crop, and randomly flip the same set of images as the input of the entire network, and use the annotated images in the dataset as labels.

[0034] Step 3: Train the network model. Input the data set prepared in step 2 into the network model built in step 1 for training.

[0035] Step 4, select the minimized loss function value and the optimal evaluation index: By minimizing the loss function of the network output image and label, until the number of training times reaches the set threshold and the loss value tends to converge, the model parameters can be considered to have been pre-trained and the model parameters can be saved. At the same time, the optimal evaluation index is selected to measure the accuracy of the algorithm and evaluate the performance of the system. The choice of loss function affects the quality of the model, can truly reflect the difference between the predicted value and the true value, and can correctly feedback the quality of the model. The optimal evaluation index selection: S α Focus on capturing the structural similarity of images; E ξ It represents the integration of local pixel matching and image-level statistics. We can obtain the maximum value by changing the threshold. β Use precision and recall to evaluate performance, and we can also change the threshold to get the maximum value; finally, MAE calculates the average pixel-by-pixel difference between the predicted image and the true value. The above four indicators can evaluate the accuracy and quality of the algorithm and measure the role of the segmentation network.

[0036] Step 5: Save the model and solidify the finalized model parameters.

[0037] If RGB-D image saliency detection is performed, the final salient object can be obtained by directly inputting the color image and depth image into the network.

[0038] Example:

[0039] In the step 1, in the decoder part: 1. In order to dynamically adjust the weight of each pixel when extracting local features by convolution in the MFE module, the present invention proposes a dynamic convolution method. Dynamic convolution enhances the perception of local features by the convolution operation by introducing the attention mechanism, and realizes a more flexible determination of the importance of different pixels in the image during the convolution process. Dynamic convolution makes the basic feature extraction more robust, and experiments have shown that it is helpful for enriching the aggregated features in saliency detection. 2. In order to mine the information between multiple scales in the SSE module, the up-down-sampling (UDS) module is used to integrate the connection between the upper and lower scales. For the current level, the next level is upsampled and the previous level is downsampled, so that the three adjacent levels can reach the same image size. If there is no previous / next level, the current level is used as the result of up / down sampling. Then the features of the three levels are connected and output. A series of operations are used to enhance the output features of the SSE module. Through a convolution, the number of channels increased by the output layer of the UDS module is restored, and then the optimization difficulty caused by channel transformation is alleviated by adding BN before ReLU. Next, we use DeepLabv3+ modules with three different void rates of 3, 6, and 9 and add them together, and enhance the significant features by using image segmentation. After that, all operations of the module are completed through convolution, BN, and convolution. After the SSE enhancement module, the edges of the overall significant objects become clearer and more certain. 3. The Double Fusion module is used in the LGE module to extract image edge information in order to enhance the posture of the capsule network output. In the Double Fusion module, in addition to convolution and Pixel Shuffle operations, a Smish activation function is also designed. Through +1, sigmoid, ln and tanh operations, it effectively improves the optimization effect during training when there is no BN. 4. Finally, the output of the network is 5 items, in is one of the outputs of the SSE module, f lge _ pose is the output of the LGE module. The loss function is calculated for each of these five items, and then the five loss values ​​are added together to get a total loss value.

[0040] In step 2, in order to be able to make a completely equal comparison with BBS-Net, the RGB-D image saliency monitoring model is evaluated on six challenging datasets: NJU2K, NLPR, STERE, LFSD, SSD and SIP. The present invention selects 1485 images from NJU2K as the training dataset and 700 images from NLPR as the training dataset, and the remaining images in these two datasets are used as the test dataset during the training process. All NJU2K, NLPR, stee, LFSD, SSD and SIP are evaluated as test datasets. NJU2k contains 1,985 images collected from the Internet, movies and Fuji W3 cameras, and the remaining 500 images are used for testing here in addition to the training set. NLPR consists of 1000 RGB images and depth maps, and the remaining 300 images are used for testing here in addition to the training set. STERE collects 1000 stereo images, using 100 indoor and outdoor scenes collected by Lytro light field cameras. SSD uses 80 samples constructed from three stereoscopic movies. SIP contains 1000 high-resolution photorealistic photos in different poses.

[0041] In step 3, the images of the data set are enhanced, random diffraction transformation is performed on the same set of images, and the same set of images (color images, depth images, label images) are normalized, randomly cropped, randomly flipped, and adjusted to a uniform size of 384×384 as the input of the entire network. The normalization, random cropping and random flipping can be achieved through the data set processing software algorithm. Then, the labeled images in the data set are used as labels to allow the network to learn better feature extraction capabilities and ultimately achieve better segmentation effects.

[0042] The output of the network in step 4 and the loss function of the label are used to calculate the loss function, and a better fusion effect is achieved by minimizing the loss function. The binary cross entropy (BCE) and intersection over union (IoU) loss functions are used as supervision strategies. The final loss function is L lfss , where L bce is the BCE loss function, L iou is the IoU loss function. The formula of the final loss function is:

[0043] L lfss =L bce +L iou

[0044] The BCE loss function is: H and W give the height and width of the image, respectively, giving the ground truth and position prediction values. Its calculation formula is:

[0045]

[0046] The calculation formula of IoU loss function is:

[0047]

[0048] The entire network operation obtains 5 features as the final output of the model. The final prediction under the constraint of the loss function is used as the supervision strategy. The sum of these 5 losses is the final loss.

[0049] The present invention runs all experimental codes on a PC with an Intel i7-12700K 3.6GHz CPU, a Nvidia GeForce RTX 3090 (24G video memory) GPU, and a 32G 3000MHz RAM. Python 3.7.2 for Windows 11 and torch 1.13.0 + cu117 are used as the deep learning platform. During training / testing, the PVTv2 backbone scales the input image to 384 × 384. We normalize, randomly crop, and randomly flip the image. The SGD optimizer parameters are initialized to lr = 0.05, momentum = 0.9, weight_decay = 1e-4, nesterov = True, and the batch size is set to batch_size = 4, epoch = 200. Finally, APEX 0.1 is used to accelerate the training.

[0050] After the network training is completed in step 5, all parameters in the network need to be saved, and then the color image and depth image of the salient target to be extracted are input into the network to obtain the salient map. The network has no requirements on the input image size, and any size is acceptable.

[0051] Among them, convolution, Transformer (PVTv2), activation function, matrix concatenation, matrix multiplication, batch normalization (BN), residual network, ConvNeXt, DeepLabv3+ and other implementations are algorithms well known to those skilled in the art, and the specific processes and methods can be found in the corresponding textbooks or technical literature.

[0052] The present invention constructs an RGB-D salient object detection network based on local feature extraction and semantic segmentation, which can directly generate salient maps from images, achieve more accurate salient object extraction in robot vision, and provide good preliminary processing for downstream tasks. Under the same conditions, the feasibility and superiority of this method are further verified by calculating the relevant indicators of the image obtained by the existing method. The comparison of the relevant indicators of the existing technology and the method proposed in this invention is shown in Table 1:

[0053] Table 1 Comparison of 4 indicators between the prior art and the method proposed in the present invention on 6 data sets (“↑” represents the higher the indicator, the better, “↓” represents the lower the indicator, the better, and bold represents the best accuracy)

[0054]

[0055] It can be seen from the table that the method proposed in this invention has a higher accuracy rate than the existing method in four basic indicators (S α 、E ξ 、F β , MAE), which comprehensively surpasses the original methods. These indicators also further illustrate that the method proposed in the present invention has better saliency extraction accuracy.

Claims

1. The RGB-D SOD method based on local feature extraction and semantic segmentation is characterized by: The method comprises the following steps: Step 1, build the network model: the whole network is based on the framework design of the Encoder-Decoder structure model; the image preprocessing module of the encoder part loads the pre-trained weight backbone network of the dual PVTv2, and performs basic feature extraction on the color image and depth image of the input multimodal image respectively, and the entire fusion is performed at four scales; among them, a ConvNeXt module is added to the depth image for local feature processing, while the color image is not processed by the ConvNeXt module; then, the depth image features processed by PVTv2+ConvNeXt are passed to the CSA module, and the results are added to the color image features processed by PVTv2; finally, the addition result is passed to the ConvNeXt module for a local feature extraction, and then a convolution is performed through the convolution module to complete the entire RGB-D fusion; Step 2, prepare the data set: select commonly used RGB-D images to build a data set; Step 3, training the network model: input the data set prepared in step 2 into the network model constructed in step 1 for training; Step 4: Select the minimized loss function value and the optimal evaluation index: By minimizing the loss function of the network output image and label, until the number of training times reaches the set threshold and the loss value tends to converge, the model parameters can be considered to have been pre-trained and saved; at the same time, the optimal evaluation index is selected to measure the accuracy of the algorithm and evaluate the performance of the system; Step 5, save the model: solidify the final model parameters; if performing RGB-D image saliency detection operation, directly input the color image and depth image into the network to obtain the final saliency map.

2. The RGB-D SOD method based on local feature extraction and semantic segmentation according to claim 1, characterized in that: The decoder part includes: an MFE module, an SSE module and an LGE module; The MFE module uses convolution kernels of three different types and sizes, namely, ordinary convolution, cross convolution and dynamic convolution; The SSE module enhances the feature output of RGB-D saliency through semantic segmentation based on DeepLabv3+ with spatial pyramid pooling and Encoder-Decoder architecture optimized by dilated convolution. The up- and down-sampling UDS modules are used to integrate the connection between the upper and lower scales. BN is added before ReLU. Then, DeepLabv3+ modules with three different dilation rates of 3, 6, and 9 are used and added together to enhance the saliency features by using image segmentation. The LGE module completes saliency detection through the interaction of four scales under the guidance of local / global consistency check and edge detection; a capsule network is used to process the local and global consistency relationship of salient objects; in addition, a Double Fusion module guided by edge detection is introduced to improve the effect of the LGE module; then, the posture vector output by the capsule network is connected with the edge information processed by Double Fusion as the final output posture vector; in the Double Fusion module, in addition to convolution and Pixel Shuffle operations, a Smish activation function is designed, which effectively improves the optimization effect during training when there is no BN through convolution, sigmoid, ln and tanh operations; finally, the output posture of the capsule network and the output of Double Fusion are connected as the output of the LGE module; the output of the LGE module is the final output of the entire network.

3. The RGB-D SOD method based on local feature extraction and semantic segmentation according to claim 1, characterized in that: The step 2 is to collect the same RGB-D saliency dataset according to the comparison network BBS-Net, using NJU2K, NLPR, STERE, LFSD, SSD and SIP; conduct supervised training on the images in the dataset; perform image enhancement on the images in the dataset, normalize, randomly crop and randomly flip the same set of images as the input of the entire network, and use the annotated images in the dataset as labels.

4. The RGB-D SOD method based on local feature extraction and semantic segmentation according to claim 1, characterized in that: In the step 3, during the pre-training process, the RGB-D saliency dataset uses NJU2K, NLPR, STERE, LFSD, SSD and SIP; supervised training is performed on the images in the dataset; image enhancement is performed on the images in the dataset, and the same group of images including color images, depth images and label images are normalized, randomly cropped, randomly flipped, and adjusted to a uniform size of 384×384 as input to the entire network, and the labeled images in the dataset are used as saliency labels.

5. The RGB-D SOD method based on local feature extraction and semantic segmentation according to claim 1, characterized in that: The optimal evaluation indicator selection in step 4 is: S α Focus on capturing the structural similarity of images; E ξ It integrates local pixel matching and image-level statistics, and obtains the maximum value by changing the threshold; F β Use precision and recall to evaluate performance, or change the threshold to obtain the maximum value; finally, MAE calculates the average pixel-by-pixel difference between the predicted image and the true value; the above four indicators can evaluate the accuracy and quality of the algorithm and measure the role of the segmentation network.

Citation Information

Patent Citations

  • Infrared image semantic segmentation method based on U-Net

    CN114092481A

  • RGB-D salient object detection method

    GB202403824D0

  • Salient object detection method and system for weak supervision-based spatio-temporal cascade neural network

    WO2019136591A1