Image segmentation method and device, terminal equipment and storage medium

Abstract

An image segmentation method and device, terminal equipment and storage medium of the present application, comprising: acquiring an RGB image and a depth image, extracting features of the RGB image to obtain an RGB feature map, extracting features of the depth image to obtain a depth image feature map, fusing the RGB feature map and the depth image feature map to obtain a fused feature map, performing multiple down-sampling on the fused feature map, fusing the convolutional features obtained after each down-sampling to obtain a multi-scale fused feature map, and obtaining a segmentation result after segmenting the multi-scale fused feature map. The present application fuses the features of the RGB image and the depth image to obtain a fused feature map, and extracts features of the fused feature map and the multi-scale fused feature map step by step, thereby ensuring higher segmentation accuracy.

Description

Technical Field

[0001] This application relates to the field of image segmentation technology, and in particular to an image segmentation method, apparatus, terminal device and storage medium. Background Technology

[0002] Existing image segmentation methods typically use semantic segmentation algorithms with RGB images as input. For example, semantic segmentation algorithms such as FCN and UNet can achieve high-precision semantic segmentation through end-to-end training. However, semantic segmentation based on RGB images faces segmentation errors caused by changes in lighting and shadows in practical applications, resulting in low segmentation accuracy for these methods. Summary of the Invention

[0003] The purpose of this application is to provide an image segmentation method, apparatus, terminal device, and storage medium to improve the segmentation accuracy of images.

[0004] To achieve the above objectives, this application provides an image segmentation method, comprising the following steps:

[0005] Acquire RGB and depth images;

[0006] Extract the features from the RGB image to obtain an RGB feature map;

[0007] Extract the features from the depth image to obtain a depth image feature map;

[0008] The RGB feature map is fused with the depth image feature map to obtain a fused feature map;

[0009] The fused feature map is downsampled multiple times, and the convolutional features obtained after each downsampling are fused to obtain a multi-scale fused feature map.

[0010] After segmenting the multi-scale fused feature map, the segmentation result is obtained.

[0011] This application also provides an image segmentation apparatus, comprising:

[0012] The acquisition module is used to acquire RGB images and depth images;

[0013] The first extraction module is used to extract features from the RGB image to obtain an RGB feature map;

[0014] The second extraction module is used to extract features from the depth image to obtain a depth image feature map.

[0015] The first fusion module is used to fuse the RGB feature map with the depth image feature map to obtain a fused feature map;

[0016] The second fusion module is used to downsample the fused feature map multiple times, and fuse the convolutional features obtained after each downsampling to obtain a multi-scale fused feature map.

[0017] The segmentation module is used to segment the multi-scale fused feature map to obtain the segmentation result.

[0018] This application also provides a terminal device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the image segmentation method described in any of the above claims.

[0019] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the image segmentation method described in any of the preceding claims.

[0020] The image segmentation method, apparatus, terminal device, and storage medium of this application acquire RGB images and depth images, extract features from the RGB images to obtain RGB feature maps, extract features from the depth images to obtain depth image feature maps, fuse the RGB feature maps and depth image feature maps to obtain fused feature maps, perform multiple downsampling on the fused feature maps, fuse the convolutional features obtained after each downsampling to obtain multi-scale fused feature maps, and segment the multi-scale fused feature maps to obtain segmentation results. By fusing the features of the RGB images and the depth images to obtain the fused feature maps, and extracting features from the fused feature maps and multi-scale fused feature maps step by step, higher segmentation accuracy is ensured. Attached Figure Description

[0021] Figure 1 A schematic diagram illustrating the steps of an image segmentation method;

[0022] Figure 2 A schematic block diagram of an image segmentation device;

[0023] Figure 3 This is a schematic block diagram of the terminal device.

[0024] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0026] Reference Figure 1This is a schematic diagram illustrating the steps of an image segmentation method in one embodiment of this application. The method includes:

[0027] S11. Obtain the RGB image and depth image;

[0028] S12. Extract the features of the RGB image to obtain an RGB feature map;

[0029] S13. Extract the features of the depth image to obtain a depth image feature map;

[0030] S14. The RGB feature map is fused with the depth image feature map to obtain a fused feature map;

[0031] S15. The fused feature map is downsampled multiple times, and the convolutional features obtained after each downsampling are fused to obtain a multi-scale fused feature map.

[0032] S16. After segmenting the multi-scale fused feature map, the segmentation result is obtained.

[0033] Currently, the following methods are generally used for image segmentation:

[0034] Semantic segmentation methods based on RGB images: Semantic segmentation algorithms that use RGB images as input are currently the most widely used methods. High-precision semantic segmentation can usually be achieved by using semantic segmentation algorithms such as FCN and UNet through end-to-end training. However, semantic segmentation based on RGB images faces segmentation errors caused by changes in lighting and shadows in practical applications. In addition, the diversity of real-world scenes tests the generalization ability of the algorithm.

[0035] Semantic segmentation methods based on depth images: Depth images are robust to changes in light and have a unique advantage over RGB images because they contain information about the depth of objects. However, depth images are rarely used in semantic segmentation due to the lack of color and texture, high noise levels, and the presence of holes.

[0036] Taking cleaning robots as an example, with the development of robot-related technologies, more and more types of robots are entering the market and beginning to serve human society in various scenarios. One type of cleaning robot is used to clean public restrooms, and a key task is cleaning urinals. To enable cleaning robots to clean urinals, precise path planning for the urinals is first required. The prerequisite for this is to perform precise and accurate image segmentation of the inner and outer walls and edges of the urinals. In other words, how to segment the inner and outer walls of the urinals with high precision, high efficiency, and high edge detail is an important task in this field.

[0037] Therefore, this application proposes an end-to-end image segmentation method based on RGB-D images as input, which addresses the requirements of high-precision, high-efficiency, and high-resolution edge segmentation of the inner and outer walls of urinals. This method offers better edge segmentation and higher inference efficiency. The RGB-D image is actually two images: a standard RGB three-channel color image and a depth image. The depth image is similar to a grayscale image, except that each pixel value represents the actual distance between the sensor and the object. Furthermore, the RGB image and the depth image are typically registered, resulting in a one-to-one correspondence between pixels.

[0038] Specifically, as described in steps S11-S14 above, this embodiment can complete image segmentation based on an end-to-end RGB-D segmentation model. It is designed based on the STDC-Seg lightweight segmentation algorithm and incorporates scene characteristics and RGB-D features. First, it fully utilizes its lightweight network structure to extract features from the RGB image and depth image respectively, obtaining RGB feature maps and depth image feature maps, ensuring a balance between accuracy and efficiency. Second, it fuses the RGB feature maps and depth image feature maps to obtain a fused feature map.

[0039] As described in steps S15-S16 above, after the RGB-D features are fused, downsampling can be performed by continuously stacking multiple feature extraction modules, such as three RGB-D feature extraction modules (i.e., RGB-D feature extraction module 1, RGB-D feature extraction module 2, and RGB-D feature extraction module 3), to further fuse the RGB feature map and the depth image feature map step by step, thereby extracting features that are different and complementary between the two, that is, extracting a multi-scale fusion feature map that is more robust to the scene, providing a foundation for high-precision segmentation.

[0040] The three RGB-D feature extraction modules can have similar structures. For example, each RGB-D feature extraction module consists of two RGB-D blocks and a set of convolutional layers with stacked kernels of 3*3 and a stride of 2, a Batch Normalization (BN) layer, and a ReLU activation layer. Furthermore, the feature map output by each RGB-D feature extraction module is downsampled by a factor of 2 compared to its input feature map. Combined with the original feature map, this achieves a factor of 4 downsampling of the original image. In this stage, the output of RGB-D feature extraction module 1 achieves an 8-fold downsampling relative to the original image, while RGB-D feature extraction modules 2 and 3 achieve 16-fold and 32-fold downsampling, respectively. Generally, in deep networks, a larger downsampling factor in the feature map implies stronger semantic information. Thus, the output features of these three modules represent semantic information at three different scale levels.

[0041] In one embodiment, the RGB-D feature extraction module can be replaced as needed, for example, using a MobileNet module, a ResNet module, a DenseNet module, or an Inception module. To ensure a certain level of inference efficiency, this application may use an STDC (Short-Term Dense Concatenate network) structure as the RGB-D block, which consists of three sets of convolutional layers with a relatively small number of channels, and fuses the three sets of convolutional features through a channel concatenation layer.

[0042] Since the output feature scales of RGB-D feature extraction modules 1, 2, and 3 are different, the feature maps are first upsampled to make the width and height of the three output feature maps consistent. Specifically, RGB-D feature extraction module 2 needs to be upsampled by a factor of 2, and RGB-D feature extraction module 3 needs to be upsampled by a factor of 4. In this application, upsampling can be achieved using bilinear interpolation or deconvolution. Then, a channel concatenation layer is used to concatenate the three feature maps. Finally, a channel attention mechanism is used to obtain a better multi-scale fused feature map. After segmenting the multi-scale fused feature map, the segmentation result is obtained.

[0043] In one embodiment, the step of acquiring the RGB image and the depth image includes the following steps:

[0044] Obtain the original RGB image and the original depth image;

[0045] The original RGB image and the original depth image are subjected to resolution registration and cropping to obtain an RGB image and a depth image; wherein the RGB image and the depth image have the same resolution and the same scene range.

[0046] In this embodiment, the original RGB image and the original depth image can be obtained by capturing them with an RGB-D camera. The original RGB image and the original depth image are pre-registered and cropped to ensure that the obtained RGB image and depth image have the same resolution and the same scene range before being used as the RGB-D input of this application.

[0047] In one embodiment, the step of extracting features from the RGB image to obtain an RGB feature map includes the following steps:

[0048] The RGB feature extraction module is invoked to extract features from the RGB image to obtain an RGB feature map; wherein, the RGB feature extraction module includes a batch normalization layer, a ReLU activation layer and a preset number of convolutional layers, the preset number of convolutional layers being stacked consecutively.

[0049] In this embodiment, after the RGB image and the depth image are input, two networks with the same structure are first used to extract the features of the RGB image and the depth image respectively, so as to extract the features of the RGB image and the depth image independently using two different networks and obtain features with differences.

[0050] The RGB feature extraction module consists of two sets of consecutively stacked convolutional layers, batch normalization (BN) layers, and ReLU activation layers. Both convolutional layers have a kernel size of 3*3 and a stride of 2. In addition to initially extracting features from the RGB or depth images, the module also downsamples the output feature map to a scale four times that of the original image.

[0051] In one embodiment, the step of fusing the RGB feature map with the depth image feature map to obtain a fused feature map includes the following steps:

[0052] The RGB feature map and the depth image feature map are concatenated according to a preset channel dimension to obtain a fused feature map.

[0053] In this embodiment, a channel stitching layer can be used to stitch the RGB feature map and the depth image feature map according to a preset channel dimension, so as to achieve preliminary fusion of the features of the RGB feature map and the depth image feature map.

[0054] After fusion is completed, the RGB feature map and the depth image feature map can be further fused step by step by stacking three RGB-D feature extraction modules (i.e., RGB-D feature extraction module 1, RGB-D feature extraction module 2 and RGB-D feature extraction module 3) to extract features that are different and complementary to each other. That is, more robust RGB-D features to the scene are extracted, which provides a foundation for high-precision segmentation.

[0055] In one embodiment, after segmenting the multi-scale fused feature map to obtain the segmentation result, the method further includes the following steps:

[0056] Edge detection is performed on the depth image and the RGB image respectively to obtain the first edge feature of the depth image and the second edge feature of the RGB image;

[0057] The first edge feature and the second edge feature are fused to obtain the fused edge feature;

[0058] The fused edge features are filtered using preset annotation information to obtain reference edge features;

[0059] Edge features are extracted from the multi-scale fused feature map, and edge loss is calculated based on the edge features and reference edge features.

[0060] The segmentation result is adjusted based on the edge loss.

[0061] This embodiment integrates edge information from RGB and depth images. This is because, for urinals, the colors and textures of their inner and outer walls are similar, and in some scenarios, it is impossible to extract the edge features of the boundary between the inner and outer walls using only RGB images. Or, due to lighting or other reasons, the outline edges of the urinal cannot be extracted. However, since there is a significant height difference between the inner and outer walls, and the depth image is robust to lighting, extracting edge information from the depth image can effectively compensate for the loss of such edge information.

[0062] Therefore, this embodiment extracts edge features, enabling the features output by the RGB-D feature extraction module 1 to have more robust edge information, thus achieving finer edge segmentation for the final inner and outer walls of the urinal. Since the extraction of edge features and the adjustment of the segmentation results only participate in training and not in segmentation, no additional computation is required, thereby improving image segmentation efficiency.

[0063] Specifically, this embodiment performs edge detection on the RGB image and the depth image respectively. The Laplacian algorithm can be used for edge detection. After processing, the first edge feature of the RGB image and the second edge feature of the depth image are output respectively. Then, the corresponding pixel values ​​of the first edge feature of the RGB image and the second edge feature of the depth image are ORed to merge the first edge feature and the second edge feature to obtain the fused edge feature. Then, the fused edge feature is filtered using the annotation information, retaining only the edge features within the annotated image area, and setting the remaining edge values ​​to 0. Finally, a reference edge feature is generated, which is used as the reference edge feature for calculating the edge loss.

[0064] Edge features are extracted from the multi-scale fused feature map. The edge loss is calculated based on the edge features and reference edge features. The edge loss is used to guide the model to enhance its ability to extract image edge features. For example, it aims to enhance the ability to segment the edge details of urinals and the inner and outer walls of urinals. The edge loss in this application can be composed of binary cross-entropy loss and dice loss.

[0065] In one embodiment, the step of performing edge detection on the depth image and the RGB image respectively to obtain a first edge feature of the depth image and a second edge feature of the RGB image includes the following steps:

[0066] Edge detection is performed on the depth image and the RGB image using the Laplacian algorithm to obtain the first edge feature of the depth image and the second edge feature of the RGB image.

[0067] This embodiment can use the Laplacian algorithm for edge detection, and after processing, output the first edge features of the RGB image and the second edge features of the depth image respectively.

[0068] In one embodiment, the step of fusing the first edge feature and the second edge feature to obtain the fused edge feature includes the following steps:

[0069] The pixel values ​​corresponding to the first edge feature of the RGB image and the second edge feature of the depth image are ORed to obtain the fused edge feature.

[0070] In this embodiment, the corresponding pixel values ​​of the first edge feature of the RGB image and the second edge feature of the depth image are ORed to merge the first edge feature and the second edge feature, thereby obtaining the fused edge feature and achieving fine fusion.

[0071] The image segmentation method of this application acquires RGB and depth images, extracts features from the RGB image to obtain an RGB feature map, extracts features from the depth image to obtain a depth image feature map, fuses the RGB and depth image feature maps to obtain a fused feature map, performs multiple downsampling on the fused feature map, fuses the convolutional features obtained after each downsampling to obtain a multi-scale fused feature map, and segments the multi-scale fused feature map to obtain the segmentation result. By fusing the features of the RGB image and the depth image to obtain the fused feature map, and extracting features from the fused feature map and the multi-scale fused feature map step by step, higher segmentation accuracy is ensured.

[0072] Reference Figure 2 This is a structural block diagram of an image segmentation device disclosed in this application. The device includes:

[0073] The acquisition module 100 is used to acquire RGB images and depth images;

[0074] The first extraction module 200 is used to extract features from the RGB image to obtain an RGB feature map;

[0075] The second extraction module 300 is used to extract features from the depth image to obtain a depth image feature map.

[0076] The first fusion module 400 is used to fuse the RGB feature map with the depth image feature map to obtain a fused feature map;

[0077] The second fusion module 500 is used to downsample the fused feature map multiple times and fuse the convolutional features obtained after each downsampling to obtain a multi-scale fused feature map.

[0078] The segmentation module 600 is used to segment the multi-scale fused feature map to obtain the segmentation result.

[0079] In one embodiment, the acquisition module 100 is specifically configured as follows:

[0080] Obtain the original RGB image and the original depth image;

[0081] The original RGB image and the original depth image are subjected to resolution registration and cropping to obtain an RGB image and a depth image; wherein the RGB image and the depth image have the same resolution and the same scene range.

[0082] In one embodiment, the first extraction module 200 is specifically configured as follows:

[0083] The RGB feature extraction module is invoked to extract features from the RGB image to obtain an RGB feature map; wherein, the RGB feature extraction module includes a batch normalization layer, a ReLU activation layer and a preset number of convolutional layers, the preset number of convolutional layers being stacked consecutively.

[0084] In one embodiment, the first fusion module 400 is specifically configured as follows:

[0085] The RGB feature map and the depth image feature map are concatenated according to a preset channel dimension to obtain a fused feature map.

[0086] In one embodiment, the image segmentation apparatus further includes an adjustment module, which is specifically configured to:

[0087] Edge detection is performed on the depth image and the RGB image respectively to obtain the first edge feature of the depth image and the second edge feature of the RGB image;

[0088] The first edge feature and the second edge feature are fused to obtain the fused edge feature;

[0089] The fused edge features are filtered using preset annotation information to obtain reference edge features;

[0090] Edge features are extracted from the multi-scale fused feature map, and edge loss is calculated based on the edge features and reference edge features.

[0091] The segmentation result is adjusted based on the edge loss.

[0092] In one embodiment, the adjustment module is further configured to:

[0093] Edge detection is performed on the depth image and the RGB image using the Laplacian algorithm to obtain the first edge feature of the depth image and the second edge feature of the RGB image.

[0094] In one embodiment, the adjustment module is further configured to: perform an OR operation on the pixel values ​​corresponding to the first edge feature of the RGB image and the second edge feature of the depth image to obtain fused edge features.

[0095] As described above, it is understood that each component of the image segmentation apparatus proposed in this application can realize the function of any of the above image segmentation methods, and the specific structure will not be described in detail.

[0096] Reference Figure 3 This application also provides a terminal device, which can be a cleaning robot, and its internal structure can be as follows: Figure 3 As shown. The terminal device may include components such as a processor, memory, network interface, and database connected via a system bus. Those skilled in the art will understand that... Figure 3 The components shown are not intended to limit all terminal devices; they may include more or fewer components, or combinations thereof. For example, in most cases, a cleaning robot may not require a network interface and database. The processor in this terminal device provides computational and control capabilities. The terminal device's memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. This memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The terminal device's database stores data such as depth images. The terminal device's network interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements an image segmentation method. The image segmentation method includes: acquiring an RGB image and a depth image; extracting features from the RGB image to obtain an RGB feature map; extracting features from the depth image to obtain a depth image feature map; fusing the RGB feature map and the depth image feature map to obtain a fused feature map; performing multiple downsampling operations on the fused feature map, fusing the convolutional features obtained after each downsampling to obtain a multi-scale fused feature map; and segmenting the multi-scale fused feature map to obtain a segmentation result.

[0097] One embodiment of this application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements an image segmentation method. It is understood that the computer-readable storage medium in this embodiment can be a volatile readable storage medium or a non-volatile readable storage medium. The image segmentation method includes: acquiring an RGB image and a depth image; extracting features from the RGB image to obtain an RGB feature map; extracting features from the depth image to obtain a depth image feature map; fusing the RGB feature map and the depth image feature map to obtain a fused feature map; performing multiple downsampling operations on the fused feature map, fusing the convolutional features obtained after each downsampling to obtain a multi-scale fused feature map; and segmenting the multi-scale fused feature map to obtain a segmentation result.

[0098] In summary, the image segmentation method, apparatus, terminal device, and storage medium of this application acquire RGB images and depth images, extract features from the RGB images to obtain RGB feature maps, extract features from the depth images to obtain depth image feature maps, fuse the RGB feature maps and depth image feature maps to obtain fused feature maps, perform multiple downsampling on the fused feature maps, fuse the convolutional features obtained after each downsampling to obtain multi-scale fused feature maps, and segment the multi-scale fused feature maps to obtain segmentation results. By fusing the features of the RGB images and the depth images to obtain the fused feature maps, and extracting features from the fused feature maps and multi-scale fused feature maps step by step, higher segmentation accuracy is ensured.

[0099] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media provided in this application and in the embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual-speed SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0100] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, apparatus, article, or method. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0101] The above description is only a preferred embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural changes made based on the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. An image segmentation method, wherein, Includes the following steps: Acquire RGB and depth images; Extract the features from the RGB image to obtain an RGB feature map; Extract the features from the depth image to obtain a depth image feature map; The RGB feature map is fused with the depth image feature map to obtain a fused feature map; The fused feature map is downsampled multiple times, and the convolutional features obtained after each downsampling are fused to obtain a multi-scale fused feature map. After segmenting the multi-scale fused feature map, the segmentation result is obtained; Edge detection is performed on the depth image and the RGB image respectively to obtain the first edge feature of the depth image and the second edge feature of the RGB image; The first edge feature and the second edge feature are fused to obtain the fused edge feature; The fused edge features are filtered using preset annotation information to obtain reference edge features; Edge features are extracted from the multi-scale fused feature map, and edge loss is calculated based on the edge features and reference edge features. The segmentation result is adjusted based on the edge loss; In this process, the Laplacian algorithm is used for edge detection. After processing, the first edge feature of the RGB image and the second edge feature of the depth image are output respectively. Then, the corresponding pixel values ​​of the first edge feature of the RGB image and the second edge feature of the depth image are ORed to merge the first edge feature and the second edge feature, resulting in a fused edge feature. Then, the fused edge feature is filtered using the annotation information, retaining only the edge features within the annotated image area, and setting the remaining edge values ​​to 0. Finally, a reference edge feature is generated, which is used as the reference edge feature for calculating the edge loss.

2. The image segmentation method according to claim 1, wherein, The steps for acquiring RGB and depth images include the following: Obtain the original RGB image and the original depth image; The original RGB image and the original depth image are subjected to resolution registration and cropping to obtain an RGB image and a depth image; wherein the RGB image and the depth image have the same resolution and the same scene range.

3. The image segmentation method according to claim 1, wherein, The step of extracting features from the RGB image to obtain an RGB feature map includes the following steps: The RGB feature extraction module is invoked to extract features from the RGB image to obtain an RGB feature map; wherein, the RGB feature extraction module includes a batch normalization layer, a ReLU activation layer and a preset number of convolutional layers, the preset number of convolutional layers being stacked consecutively.

4. The image segmentation method according to claim 1, wherein, The step of fusing the RGB feature map with the depth image feature map to obtain the fused feature map includes the following steps: The RGB feature map and the depth image feature map are concatenated according to a preset channel dimension to obtain a fused feature map.

5. An image segmentation apparatus for implementing the method of any one of claims 1-4, wherein, include: The acquisition module is used to acquire RGB images and depth images; The first extraction module is used to extract features from the RGB image to obtain an RGB feature map; The second extraction module is used to extract features from the depth image to obtain a depth image feature map. The first fusion module is used to fuse the RGB feature map with the depth image feature map to obtain a fused feature map; The second fusion module is used to downsample the fused feature map multiple times, and fuse the convolutional features obtained after each downsampling to obtain a multi-scale fused feature map. The segmentation module is used to segment the multi-scale fused feature map to obtain the segmentation result.

6. A terminal device, comprising a memory and a processor, wherein the memory stores a computer program, wherein... When the processor executes the computer program, it implements the steps of the image segmentation method according to any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program stored thereon, wherein, When the computer program is executed by a processor, it implements the steps of the image segmentation method according to any one of claims 1 to 4.

Images