Image semantic segmentation method and device

Abstract

The present disclosure provides an image semantic segmentation method and device. It includes: inputting the image to be processed into an image recognition model; the image recognition model includes: a short-time dense connection network layer and a decoding network layer, the short-time dense connection network layer includes: a convolution module, multiple short-time dense connection layers and an output module, and the short-time dense connection layer includes multiple short-time dense connection modules; calling the convolution module to process the image to be processed to obtain a first feature map corresponding to the image to be processed; calling the short-time dense connection layer to process the first feature map to obtain a second feature map; calling the output module to process the second feature map to obtain a third feature map; calling the decoding network layer to upsample the third feature map and map it to the segmentation category to obtain a fourth feature map with the number of channels being the number of segmentation categories; determining the semantic segmentation result corresponding to the image to be processed based on the fourth feature map. The present disclosure can reduce network structural redundancy and improve image semantic segmentation performance and efficiency.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of image processing technology, and in particular to a method and device for image semantic segmentation. Background Art

[0002] Semantic segmentation is a fundamental area in the field of computer vision. With the advancement of deep learning methods in recent years, semantic segmentation has also made significant progress and has been applied in an increasing number of scenarios, such as autonomous driving, human-computer interaction, medical analysis, and augmented reality.

[0003] In order to balance the real-time and effectiveness requirements of semantic segmentation, a current mainstream approach to the design of real-time semantic segmentation is to use an existing lightweight backbone network for encoding and independently design a lightweight decoding network to improve segmentation efficiency. ResNet-18, Xception-39, etc. are commonly used backbone networks, but these backbone networks lack customized design for segmentation tasks. Their own structures may produce structural redundancy for segmentation tasks. Directly using them as backbone networks will lead to a decrease in semantic segmentation performance and efficiency. Summary of the Invention

[0004] The embodiments of the present disclosure provide a method and apparatus for image semantic segmentation, which enable the network to capture the multi-scale receptive field information required for the segmentation task and remove structural redundancy, so that the network operates efficiently and meets the requirements of real-time semantic segmentation.

[0005] According to a first aspect of an embodiment of the present disclosure, a method for image semantic segmentation is provided, comprising:

[0006] Inputting the image to be processed into an image recognition model; the image recognition model includes: a short-term dense connection network layer and a decoding network layer, the short-term dense connection network layer includes: a convolution module, multiple short-term dense connection layers connected in series, and an output module, each short-term dense connection layer includes multiple short-term dense connection modules;

[0007] Calling the convolution module to process the image to be processed to obtain a first feature map corresponding to the image to be processed;

[0008] Calling the short-term dense connection layer to process the first feature map to obtain a second feature map corresponding to the first feature map;

[0009] Calling the output module to process the second feature map to obtain a third feature map corresponding to the second feature map;

[0010] Calling the decoding network layer to upsample the third feature map and map it to the segmentation category to obtain a fourth feature map with the same number of channels as the number of segmentation categories;

[0011] Determine a semantic segmentation result corresponding to the image to be processed according to the fourth feature map.

[0012] Optionally, the convolution module includes: a first convolution unit and a second convolution unit,

[0013] The calling the convolution module to process the image to be processed to obtain a first feature map corresponding to the image to be processed includes:

[0014] Calling the first convolution unit to downsample the image to be processed by twice using a convolution kernel of a set size to obtain an intermediate feature map having a first number of channels;

[0015] Calling the second convolution unit to perform two-fold downsampling on the intermediate feature map using a convolution kernel of a set size to obtain a first feature map having a second number of channels;

[0016] The number of the second channels is twice the number of the first channels.

[0017] Optionally, the multiple short-term dense connection modules in each short-term dense connection layer include: a first short-term dense connection module and at least one second short-term dense connection module,

[0018] The calling the short-term dense connection layer to process the first feature map to obtain a second feature map corresponding to the first feature map includes:

[0019] Calling a first short-term dense connection module in a first short-term dense connection layer among the multiple short-term dense connection layers to downsample the first feature map by a factor of two to obtain a first initial feature map having a third number of channels; the third number of channels is twice the second number of channels;

[0020] Calling at least one second short-term dense connection module in the first short-term dense connection layer to perform short-term dense connection processing on the first initial feature map to obtain a first processed feature map with a third number of channels;

[0021] Calling an nth short-time dense connection layer other than the first short-time dense connection layer among the multiple short-time dense connection layers, performing two-fold downsampling and short-time dense connection processing on the feature map output by the n-1th short-time dense connection layer, to obtain an nth processed feature map;

[0022] Obtaining a processed feature map output by a last short-term dense connection layer among the multiple short-term dense connection layers, and using the processed feature map as the second feature map;

[0023] Wherein, n is a positive integer greater than or equal to 2, and the number of channels of the n-th processed feature map is twice the number of channels of the n-1th processed feature map output by the n-1th short-time dense connection layer.

[0024] Optionally, the output module includes: a convolutional layer, an average pooling layer and a full convolutional network layer,

[0025] The calling the output module to process the second feature map to obtain a third feature map corresponding to the second feature map includes:

[0026] Calling the convolutional layer to downsample the second feature map to obtain a sampled feature map;

[0027] Calling the average pooling layer to perform pooling processing on the sampled feature map to obtain a pooled feature map;

[0028] Call the fully convolutional network layer to perform convolution processing on the pooled feature map to obtain the third feature map.

[0029] Optionally, each short-term dense connection module includes multiple operation blocks, and the number of channels from the second operation block to the penultimate operation block in the operation blocks of each short-term dense connection module forms a geometric progression, and the number of channels of the last operation block is the same as that of the penultimate operation block.

[0030] According to a second aspect of an embodiment of the present disclosure, there is provided an image semantic segmentation apparatus, comprising:

[0031] An image input module to be processed is used to input the image to be processed into an image recognition model; the image recognition model includes: a short-term dense connection network layer and a decoding network layer, the short-term dense connection network layer includes: a convolution module, multiple short-term dense connection layers connected in series, and an output module, each short-term dense connection layer includes multiple short-term dense connection modules;

[0032] A first feature map acquisition module is used to call the convolution module to process the image to be processed to obtain a first feature map corresponding to the image to be processed;

[0033] A second feature map acquisition module, configured to call the short-term dense connection layer to process the first feature map to obtain a second feature map corresponding to the first feature map;

[0034] a third feature map acquisition module, configured to call the output module to process the second feature map to obtain a third feature map corresponding to the second feature map;

[0035] a fourth feature map acquisition module, configured to call the decoding network layer to upsample the third feature map and map it to the segmentation categories to obtain a fourth feature map having the same number of channels as the number of segmentation categories;

[0036] The semantic segmentation result determination module is used to determine the semantic segmentation result corresponding to the image to be processed based on the fourth feature map.

[0037] Optionally, the convolution module includes: a first convolution unit and a second convolution unit,

[0038] The first feature map acquisition module includes:

[0039] an intermediate feature map acquisition submodule, configured to call the first convolution unit to perform two-fold downsampling on the image to be processed using a convolution kernel of a set size, to obtain an intermediate feature map having a first number of channels;

[0040] a first feature map acquisition submodule, configured to call the second convolution unit to perform two-fold downsampling on the intermediate feature map using a convolution kernel of a set size, to obtain a first feature map having a second number of channels;

[0041] The number of the second channels is twice the number of the first channels.

[0042] Optionally, the multiple short-term dense connection modules in each short-term dense connection layer include: a first short-term dense connection module and at least one second short-term dense connection module,

[0043] The second feature map acquisition module includes:

[0044] a first initial feature map acquisition submodule, configured to call a first short-time dense connection module in a first short-time dense connection layer among the multiple short-time dense connection layers to downsample the first feature map by a factor of two to obtain a first initial feature map having a third number of channels; the third number of channels being twice the second number of channels;

[0045] A first processing feature map acquisition submodule is configured to call at least one second short-time dense connection module in the first short-time dense connection layer to perform short-time dense connection processing on the first initial feature map to obtain a first processed feature map having a third number of channels;

[0046] an nth processed feature map acquisition submodule, configured to call an nth short-time dense connection layer other than the first short-time dense connection layer among the multiple short-time dense connection layers, perform two-fold downsampling and short-time dense connection processing on the feature map output by the n-1th short-time dense connection layer, and obtain an nth processed feature map;

[0047] A second feature map acquisition submodule is configured to acquire a processed feature map output by a last short-term dense connection layer among the multiple short-term dense connection layers, and use the processed feature map as the second feature map;

[0048] Wherein, n is a positive integer greater than or equal to 2, and the number of channels of the n-th processed feature map is twice the number of channels of the n-1th processed feature map output by the n-1th short-time dense connection layer.

[0049] Optionally, the output module includes: a convolutional layer, an average pooling layer and a full convolutional network layer,

[0050] The third feature map acquisition module includes:

[0051] A sampling feature map acquisition submodule, configured to call the convolutional layer to downsample the second feature map to obtain a sampling feature map;

[0052] A pooling feature map acquisition submodule is used to call the average pooling layer to perform pooling processing on the sampled feature map to obtain a pooling feature map;

[0053] The third feature map acquisition submodule is used to call the full convolutional network layer to perform convolution processing on the pooled feature map to obtain the third feature map.

[0054] Optionally, each short-term dense connection module includes multiple operation blocks, and the number of channels from the second operation block to the penultimate operation block in the operation blocks of each short-term dense connection module forms a geometric progression, and the number of channels of the last operation block is the same as that of the penultimate operation block.

[0055] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, including:

[0056] A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements any one of the above-described image semantic segmentation methods when executing the program.

[0057] According to a fourth aspect of an embodiment of the present disclosure, a readable storage medium is provided. When instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform any of the above-mentioned image semantic segmentation methods.

[0058] The embodiments of the present disclosure provide an image semantic segmentation method and apparatus, which inputs a to-be-processed image into an image recognition model, the image recognition model comprising a short-term dense connection network layer and a decoding network layer, the short-term dense connection network layer comprising a convolution module, a plurality of short-term dense connection layers connected in series, and an output module, each short-term dense connection layer comprising a plurality of short-term dense connection modules, calls the convolution module to process the to-be-processed image to obtain a first feature map corresponding to the to-be-processed image, calls the short-term dense connection layer to process the first feature map to obtain a second feature map corresponding to the first feature map, calls the output module to process the second feature map to obtain a third feature map corresponding to the second feature map, calls the decoding network layer to upsample the third feature map and maps it to the segmentation category to obtain a fourth feature map having the same number of channels as the number of segmentation categories, and determines the semantic segmentation result corresponding to the to-be-processed image based on the fourth feature map. The embodiments of the present disclosure use short-term dense connection modules connected in series for image processing, and the feature dimension gradually decreases with the depth of the network module, so that the network can encode the receptive field information required for segmentation while reducing the structural redundancy of the network, thereby improving the performance and efficiency of image semantic segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the description of the embodiments of the present disclosure. Obviously, the drawings described below are only some embodiments of the embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0060] Figure 1 A flowchart of a method for image semantic segmentation according to an embodiment of the present disclosure is provided;

[0061] Figure 2 A flowchart of another method for image semantic segmentation according to an embodiment of the present disclosure;

[0062] Figure 3 A schematic diagram of model configuration parameters provided in an embodiment of the present disclosure;

[0063] Figure 4 A schematic diagram of the structure of an image semantic segmentation device provided in an embodiment of the present disclosure;

[0064] Figure 5 A schematic diagram of the structure of another image semantic segmentation device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0065] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, but not all of them. Based on the embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the embodiments of the present disclosure.

[0066] Example 1

[0067] Reference Figure 1 , shows a flowchart of the steps of an image semantic segmentation method provided by an embodiment of the present disclosure, such as Figure 1 As shown, the image semantic segmentation method may specifically include the following steps:

[0068] Step 101: Input the image to be processed into an image recognition model; the image recognition model includes: a short-time dense connection network layer and a decoding network layer, the short-time dense connection network layer includes: a convolution module, multiple short-time dense connection layers connected in series and an output module, and each short-time dense connection layer includes multiple short-time dense connection modules.

[0069] The embodiments of the present disclosure can be applied to scenarios where semantic segmentation of images is performed.

[0070] The image to be processed refers to an image that needs to be subjected to image semantic segmentation. In this example, the image to be processed can be a dish image, or a map image, etc. Specifically, it can be determined according to business needs, and this embodiment does not impose any restrictions on this.

[0071] An image recognition model is a model used to perform semantic segmentation on an image, that is, a model that identifies the pixel category of each pixel in the image.

[0072] The model architecture of the image recognition model is: a short-term dense connection network layer (STDC net) and a decoding network layer. The short-term dense connection network layer may include: a convolution module, multiple short-term dense connection layers connected in series, and an output module. Each short-term dense connection layer contains multiple short-term dense connection modules.

[0073] A short-term dense connection layer refers to a network layer formed by multiple short-term dense connection modules connected in series. The structures of the multiple short-term dense connection modules are the same. Each short-term dense connection module can contain multiple operation blocks (for example, each short-term dense connection module can contain Block1, Block2, Block3, ... Blockn). The number of channels from the second operation block to the second-to-last operation block in the operation blocks of each short-term dense connection module is in geometric progression, and the number of channels of the last operation block is the same as the number of channels of the second-to-last operation block. Specifically, from the second operation block (i.e., the short-term dense connection module) to the second-to-last operation block, the number of channels of the kernel is half of the number of channels of the previous feature block, and the number of channels of the kernel of the last operation block is consistent with the number of channels of the second-to-last kernel. In this embodiment, in the short-term dense connection module, the kernel size of the first operation block is 1x1, and its main function is to reduce the dimension of the input feature to half of the output dimension. Except for the first operation block, the kernels of the remaining operation blocks are all 3x3.

[0074] After the image to be processed is acquired, the image to be processed may be input into an image recognition model, and then step 102 is executed.

[0075] Step 102: calling the convolution module to process the image to be processed to obtain a first feature map corresponding to the image to be processed.

[0076] The first feature map refers to the image obtained after the image to be processed is processed by the convolution module.

[0077] After the image to be processed is input into the image recognition model, the convolution module can be called to process the image to be processed to obtain a first feature map corresponding to the image to be processed. Specifically, the convolution module can include two convolution units. First, the first convolution unit can be used to downsample the image to be processed to obtain an intermediate image, and then the second convolution unit can be used to downsample the intermediate image to obtain the first feature map. Specifically, this will be described in detail in the following embodiment 2, and the embodiments of the present disclosure will not be repeated here.

[0078] After calling the convolution module to process the image to be processed to obtain a first feature map corresponding to the image to be processed, step 103 is executed.

[0079] Step 103: Call the short-term dense connection layer to process the first feature map to obtain a second feature map corresponding to the first feature map.

[0080] The second feature map refers to the image obtained after the first feature map is processed by the short-term dense connection module.

[0081] After calling the convolution module to process the image to be processed to obtain the first feature map corresponding to the image to be processed, the short-time dense connection layer can be called to process the first feature image to obtain a second feature map corresponding to the first feature map. Specifically, the first short-time dense connection module in the first short-time dense connection layer among multiple short-time dense connection modules can be called to downsample and perform short-time dense connection processing on the first feature map to obtain a first processed image of the first feature map object. Then, the second short-time dense connection layer is used to downsample and perform short-time dense connection processing on the first processed image to obtain a second processed image,..., and so on, until the last short-time dense connection layer downsamples and performs short-time dense connection processing on the processed image output by the second-to-last short-time dense connection layer, and then the second feature map can be obtained.

[0082] It can be understood that in the description of steps 102 to 103 above, the number of channels of the feature map output by each network structure layer is twice the number of channels of the feature map output by the previous network structure layer, and the feature channels of the short-term dense connection module are reduced layer by layer, thereby effectively removing feature redundancy and improving network efficiency.

[0083] After calling the short-term dense connection layer to process the first feature map to obtain a second feature map corresponding to the first feature map, step 104 is executed.

[0084] Step 104: calling the output module to process the second feature map to obtain a third feature map corresponding to the second feature map.

[0085] The third feature map refers to the feature map output after the output module is called to process the second feature map.

[0086] After calling the short-time dense connection layer to process the first feature map to obtain a second feature map corresponding to the first feature map, the output module can be called to process the second feature map to obtain a third feature map corresponding to the second feature map. Specifically, this process will be described in detail in the following embodiment 2 and will not be repeated here in this embodiment.

[0087] After calling the output module to process the second feature map to obtain a third feature map corresponding to the second feature map, step 105 is executed.

[0088] Step 105: Call the decoding network layer to upsample the third feature map and map it to the segmentation category to obtain a fourth feature map with the same number of channels as the number of segmentation categories.

[0089] The fourth feature map refers to a feature map whose number of channels is the number of segmentation categories obtained after upsampling the third feature map through the decoding network layer and mapping it to the segmentation category.

[0090] After calling the output module to process the second feature map to obtain a third feature map corresponding to the second feature map, the decoding network layer can be called to upsample the third feature map and map it to the segmentation category to obtain a fourth feature map with the same number of channels as the number of segmentation categories.

[0091] After obtaining the fourth feature map having the same number of channels as the number of segmentation categories, step 106 is executed.

[0092] Step 106: Determine a semantic segmentation result corresponding to the image to be processed according to the fourth feature map.

[0093] After obtaining the fourth feature map whose number of channels is the number of segmentation categories, the semantic segmentation result corresponding to the image to be processed can be determined based on the fourth feature map. That is, the pixel category of each pixel in the image to be processed can be determined through the fourth feature map. Combined with the pixel categories, it can be clear which area in the image to be processed belongs to which category.

[0094] The embodiments of the present disclosure perform image processing by adopting short-term dense connection modules connected in series. The feature dimension gradually decreases with the depth of the network module, so that the network can encode the receptive field information required for segmentation while reducing the structural redundancy of the network.

[0095] The image semantic segmentation method provided by the embodiment of the present disclosure is to input the image to be processed into an image recognition model, wherein the image recognition model includes a short-term dense connection network layer and a decoding network layer, the short-term dense connection network layer includes a convolution module, multiple short-term dense connection layers connected in series, and an output module, each short-term dense connection layer includes multiple short-term dense connection modules, the convolution module is called to process the image to be processed to obtain a first feature map corresponding to the image to be processed, the short-term dense connection layer is called to process the first feature map to obtain a second feature map corresponding to the first feature map, the output module is called to process the second feature map to obtain a third feature map corresponding to the second feature map, the decoding network layer is called to upsample the third feature map and map it to the segmentation category to obtain a fourth feature map with the same number of channels as the number of segmentation categories, and the semantic segmentation result corresponding to the image to be processed is determined based on the fourth feature map. The embodiment of the present disclosure uses the short-term dense connection modules connected in series to process the image, and the feature dimension gradually decreases with the depth of the network module, so that the network can encode the receptive field information required for segmentation while reducing the structural redundancy of the network, thereby improving the performance and efficiency of image semantic segmentation.

[0096] Example 2

[0097] Reference Figure 2 , shows a flowchart of the steps of another image semantic segmentation method provided by an embodiment of the present disclosure, such as Figure 2As shown, the image semantic segmentation method may specifically include the following steps:

[0098] Step 201: Input the image to be processed into an image recognition model; the image recognition model includes: a short-time dense connection network layer and a decoding network layer, the short-time dense connection network layer includes: a convolution module, multiple short-time dense connection layers connected in series and an output module, and each short-time dense connection layer includes multiple short-time dense connection modules.

[0099] The embodiments of the present disclosure can be applied to scenarios where semantic segmentation of images is performed.

[0100] The image to be processed refers to an image that needs to be subjected to image semantic segmentation. In this example, the image to be processed can be a dish image, or a map image, etc. Specifically, it can be determined according to business needs, and this embodiment does not impose any restrictions on this.

[0101] An image recognition model is a model used to perform semantic segmentation on an image, that is, a model that identifies the pixel category of each pixel in the image.

[0102] The model architecture of the image recognition model is: a short-term dense connection network layer (STDC net) and a decoding network layer. The short-term dense connection network layer may include: a convolution module, multiple short-term dense connection layers connected in series, and an output module. Each short-term dense connection layer contains multiple short-term dense connection modules.

[0103] A short-term dense connection layer refers to a network layer formed by multiple short-term dense connection modules connected in series. The structures of the multiple short-term dense connection modules are the same. Each short-term dense connection module contains multiple operation blocks. The number of channels from the second operation block to the penultimate operation block in the operation block of each short-term dense connection module is in geometric progression, and the number of channels of the last operation block is the same as the number of channels of the penultimate operation block. Specifically, from the second operation block (i.e., the short-term dense connection module) to the penultimate operation block, the number of channels of the kernel is half of the number of channels of the previous feature block, and the number of channels of the kernel of the last operation block is consistent with the number of channels of the penultimate kernel. In this embodiment, in the short-term dense connection module, the kernel size of the first operation block is 1x1, and its main function is to reduce the dimension of the input feature to half of the output dimension. Except for the first operation block, the kernels of the remaining operation blocks are all 3x3.

[0104] After the image to be processed is acquired, the image to be processed may be input into an image recognition model, and then step 202 is executed.

[0105] Step 202: Call the first convolution unit to downsample the image to be processed by twice using a convolution kernel of a set size to obtain an intermediate feature map with a first number of channels.

[0106] Next, combine Figure 3 The technical solutions of the embodiments of the present disclosure are described in detail as follows.

[0107] In this embodiment, the first convolution unit may be ConvX1, which is used to process the image to be processed ( Figure 3 After the image (with an output size of 224×224) is input into the image recognition model, ConvX1 can be called to perform two times downsampling to obtain an intermediate feature map with the same number of channels as the first channel, as shown in the following example: Figure 3 As shown in the figure, after two times downsampling, an intermediate feature map with 32 channels is obtained. Specifically, ConvX1 can be called to process the image to be processed with a 3×3 convolution kernel with a step size of 2, so that ConvX1 can output an intermediate feature map with 32 channels.

[0108] After calling the first convolution unit to perform two-fold downsampling on the image to be processed using a convolution kernel of a set size to obtain an intermediate feature map having the first number of channels, step 203 is executed.

[0109] Step 203: Call the second convolution unit to downsample the intermediate feature map by a factor of two using a convolution kernel of a set size to obtain a first feature map having a second number of channels.

[0110] The second convolution unit is Figure 3 As shown in ConvX2, after obtaining the intermediate feature map, ConvX2 can be called to downsample the intermediate feature map by twice through a convolution kernel of a set size to obtain a first feature map with a second number of channels. The processing method of this process is similar to that in the above step 202, and the embodiments of the present disclosure will not be repeated here.

[0111] In this embodiment, the number of the second channels is twice the number of the first channels. The increase in the number of feature map channels can improve the receptive field of image coding.

[0112] After calling the second convolution unit to perform two-fold downsampling on the intermediate feature map using a convolution kernel of a set size to obtain a first feature map having a second number of channels, step 204 is executed.

[0113] Step 204: Call the first short-time dense connection module in a short-time dense connection layer of the multiple short-time dense connection layers to downsample the first feature map by two times to obtain a first initial feature map having a third number of channels; the third number of channels is twice the second number of channels.

[0114] In this embodiment, the multiple short-time dense connection modules in each short-time dense connection layer include: a first short-time dense connection module and at least one second short-time dense connection module, wherein the first short-time dense connection module can be used to downsample the feature map output by the previous module by two times, and the at least one second short-time dense connection module can be used to perform short-time dense connection processing on the feature map output by the first short-time dense connection module.

[0115] The first initial feature map refers to a feature map obtained by downsampling the first feature map using the first short-term dense connection module in the first short-term dense connection layer in multiple short-term dense connection layers.

[0116] The third number of channels refers to the number of channels of the obtained first initial feature map.

[0117] After calling the second convolution unit to downsample the intermediate feature map by twice using the convolution kernel of a set size to obtain a first feature map with the second number of channels, the first short-term dense connection module in the first short-term dense connection layer can be called to downsample the first feature map by twice to obtain a first initial feature map with a third number of channels, where the third number of channels is twice the second number of channels.

[0118] After calling the first short-term dense connection module in the first short-term dense connection layer in the multiple short-term dense connection layers to downsample the first feature map by a factor of two to obtain a first initial feature map with a third number of channels, step 205 is executed.

[0119] Step 205: Call at least one second short-time dense connection module in the first short-time dense connection layer to perform short-time dense connection processing on the first initial feature map to obtain a first processed feature map with a third number of channels.

[0120] After obtaining the first initial feature map, at least one second short-time dense connection module in the first short-time dense connection layer can be called to perform short-time dense connection processing on the first initial feature map to obtain a first processed feature map, wherein the number of channels of the first processed feature map is the same as that of the first initial feature map, which is the third number of channels.

[0121] After obtaining the first processing feature map, step 206 is executed.

[0122] Step 206: Call the nth short-time dense connection layer among the multiple short-time dense connection layers except the first short-time dense connection layer, perform two-fold downsampling and short-time dense connection processing on the feature map output by the n-1th short-time dense connection layer, and obtain the nth processed feature map.

[0123] In this embodiment, for multiple short-time dense connection layers, the nth short-time dense connection layer except the first short-time dense connection layer among the multiple short-time dense connection layers can be called, and the feature map output by the n-1th short-time dense connection layer is downsampled twice and short-time densely connected to obtain the nth processed feature map, where n is a positive integer greater than or equal to 2, and the number of channels of the nth processed feature map is twice the number of channels of the n-1th processed feature map output by the n-1th short-time dense connection layer.

[0124] Specifically, after obtaining the first processing feature map, the first short-time dense connection module in the second short-time dense connection layer is used to downsample the first processing feature map by two times to obtain a second initial feature map, and then at least one second short-time dense connection module in the second short-time dense connection layer is used to perform short-time dense connection processing on the second initial feature map to obtain a second processing feature map, and then, the first short-time dense connection module in the third short-time dense connection layer is used to downsample the second processing feature map by two times to obtain three initial feature maps, and then at least one second short-time dense connection module in the third short-time dense connection layer is used to perform short-time dense connection processing on the third initial feature map to obtain a third processing feature map, and so on, until the last short-time dense connection layer performs two times downsampling and short-time dense connection processing on the processing feature map output by the penultimate short-time dense connection layer.

[0125] Step 207: Obtain a processed feature map output by the last short-term dense connection layer among the multiple short-term dense connection layers, and use the processed feature map as the second feature map.

[0126] After all the short-term dense connection layers are subjected to corresponding image processing, a processed feature map output by the last short-term dense connection layer among the multiple short-term dense connection layers can be obtained, and the processed feature map is used as the second feature map.

[0127] After the second feature map is obtained, step 208 is executed.

[0128] Step 208: Call the convolutional layer to downsample the second feature map to obtain a sampled feature map.

[0129] In this embodiment, the output module may include a convolutional layer, an average pooling layer, and a full convolutional network layer, such as Figure 3 As shown in the figure, the convolution layer is ConvX6, the average pooling layer is GlobalPool, and the full convolution network layer is FC (i.e. FC1 and FC2).

[0130] After obtaining the second feature map, the convolution layer can be called to downsample the second feature map to obtain a sampled feature map, the number of channels of the sampled feature map is the same as the number of channels of the second feature map, such as Figure 3 shown.

[0131] After calling the convolutional layer to downsample the second feature map to obtain a sampled feature map, step 209 is executed.

[0132] Step 209: Call the average pooling layer to perform pooling processing on the sampled feature map to obtain a pooled feature map.

[0133] After obtaining the sampled feature map, the average pooling layer can be called to perform pooling processing on the sampled feature map to obtain a pooled feature map.

[0134] The process of pooling is a commonly used technical means in the art and will not be described in detail in this embodiment.

[0135] After calling the average pooling layer to perform pooling processing on the sampled feature map to obtain the pooled feature map, step 210 is executed.

[0136] Step 210: Call the fully convolutional network layer to perform convolution processing on the pooled feature map to obtain the third feature map.

[0137] After calling the average pooling layer to perform pooling processing on the sampled feature map to obtain the pooled feature map, the full convolutional network layer can be called to perform convolution processing on the pooled feature map to obtain the third feature map.

[0138] After obtaining the third feature map, step 211 is executed.

[0139] Step 211: calling the decoding network layer to upsample the third feature map and map it to the segmentation category to obtain a fourth feature map with the same number of channels as the number of segmentation categories.

[0140] The fourth feature map refers to a feature map whose number of channels is the number of segmentation categories obtained after upsampling the third feature map through the decoding network layer and mapping it to the segmentation category.

[0141] After calling the output module to process the second feature map to obtain a third feature map corresponding to the second feature map, the decoding network layer can be called to upsample the third feature map and map it to the segmentation category to obtain a fourth feature map with the same number of channels as the number of segmentation categories.

[0142] After obtaining the fourth feature map having the same number of channels as the number of segmentation categories, step 212 is executed.

[0143] Step 212: Determine a semantic segmentation result corresponding to the image to be processed based on the fourth feature map.

[0144] After obtaining the fourth feature map whose number of channels is the number of segmentation categories, the semantic segmentation result corresponding to the image to be processed can be determined based on the fourth feature map. That is, the pixel category of each pixel in the image to be processed can be determined through the fourth feature map. Combined with the pixel categories, it can be clear which area in the image to be processed belongs to which category.

[0145] The embodiments of the present disclosure perform image processing by adopting short-term dense connection modules connected in series. The feature dimension gradually decreases with the depth of the network module, so that the network can encode the receptive field information required for segmentation while reducing the structural redundancy of the network.

[0146] The image semantic segmentation method provided by the embodiment of the present disclosure is to input the image to be processed into an image recognition model, wherein the image recognition model includes a short-term dense connection network layer and a decoding network layer, the short-term dense connection network layer includes a convolution module, multiple short-term dense connection layers connected in series, and an output module, each short-term dense connection layer includes multiple short-term dense connection modules, the convolution module is called to process the image to be processed to obtain a first feature map corresponding to the image to be processed, the short-term dense connection layer is called to process the first feature map to obtain a second feature map corresponding to the first feature map, the output module is called to process the second feature map to obtain a third feature map corresponding to the second feature map, the decoding network layer is called to upsample the third feature map and map it to the segmentation category to obtain a fourth feature map with the same number of channels as the number of segmentation categories, and the semantic segmentation result corresponding to the image to be processed is determined based on the fourth feature map. The embodiment of the present disclosure uses the short-term dense connection modules connected in series to process the image, and the feature dimension gradually decreases with the depth of the network module, so that the network can encode the receptive field information required for segmentation while reducing the structural redundancy of the network, thereby improving the performance and efficiency of image semantic segmentation.

[0147] Example 3

[0148] Reference Figure 4 , which shows a structural diagram of an image semantic segmentation device provided by an embodiment of the present disclosure, such as Figure 4 As shown, the image semantic segmentation device 400 may specifically include the following modules:

[0149] The image input module 410 is used to input the image to be processed into the image recognition model; the image recognition model includes: a short-term dense connection network layer and a decoding network layer, the short-term dense connection network layer includes: a convolution module, multiple short-term dense connection layers connected in series, and an output module, each short-term dense connection layer includes multiple short-term dense connection modules;

[0150] A first feature map acquisition module 420 is configured to call the convolution module to process the image to be processed to obtain a first feature map corresponding to the image to be processed;

[0151] A second feature map acquisition module 430 is configured to call the short-term dense connection layer to process the first feature map to obtain a second feature map corresponding to the first feature map;

[0152] A third feature map acquisition module 440 is configured to call the output module to process the second feature map to obtain a third feature map corresponding to the second feature map;

[0153] A fourth feature map acquisition module 450 is configured to call the decoding network layer to upsample the third feature map and map it to the segmentation categories to obtain a fourth feature map having the same number of channels as the number of segmentation categories;

[0154] The semantic segmentation result determination module 460 is used to determine the semantic segmentation result corresponding to the image to be processed based on the fourth feature map.

[0155] Optionally, each short-term dense connection module includes multiple operation blocks, and the number of channels from the second operation block to the penultimate operation block in the operation blocks of each short-term dense connection module forms a geometric progression, and the number of channels of the last operation block is the same as that of the penultimate operation block.

[0156] The image semantic segmentation device provided by the embodiment of the present disclosure inputs the image to be processed into an image recognition model, wherein the image recognition model includes a short-term dense connection network layer and a decoding network layer, the short-term dense connection network layer includes a convolution module, multiple short-term dense connection layers connected in series, and an output module, each short-term dense connection layer includes multiple short-term dense connection modules, the convolution module is called to process the image to be processed to obtain a first feature map corresponding to the image to be processed, the short-term dense connection layer is called to process the first feature map to obtain a second feature map corresponding to the first feature map, the output module is called to process the second feature map to obtain a third feature map corresponding to the second feature map, the decoding network layer is called to upsample the third feature map and map it to the segmentation category to obtain a fourth feature map with the same number of channels as the number of segmentation categories, and the semantic segmentation result corresponding to the image to be processed is determined based on the fourth feature map. The embodiment of the present disclosure uses the short-term dense connection modules connected in series for image processing, and the feature dimension gradually decreases with the depth of the network module, so that the network can encode the receptive field information required for segmentation while reducing the structural redundancy of the network, thereby improving the performance and efficiency of image semantic segmentation.

[0157] Example 4

[0158] Reference Figure 5, which shows a structural diagram of another image semantic segmentation device provided by an embodiment of the present disclosure, such as Figure 5 As shown, the image semantic segmentation device 500 may specifically include the following modules:

[0159] The image input module 510 is used to input the image to be processed into the image recognition model; the image recognition model includes: a short-term dense connection network layer and a decoding network layer, the short-term dense connection network layer includes: a convolution module, multiple short-term dense connection layers connected in series, and an output module, each short-term dense connection layer includes multiple short-term dense connection modules;

[0160] A first feature map acquisition module 520 is configured to call the convolution module to process the image to be processed to obtain a first feature map corresponding to the image to be processed;

[0161] A second feature map acquisition module 530 is configured to call the short-term dense connection layer to process the first feature map to obtain a second feature map corresponding to the first feature map;

[0162] A third feature map acquisition module 540 is configured to call the output module to process the second feature map to obtain a third feature map corresponding to the second feature map;

[0163] A fourth feature map acquisition module 550 is configured to call the decoding network layer to upsample the third feature map and map it to the segmentation categories to obtain a fourth feature map having the same number of channels as the number of segmentation categories;

[0164] The semantic segmentation result determination module 560 is used to determine the semantic segmentation result corresponding to the image to be processed based on the fourth feature map.

[0165] Optionally, the convolution module includes: a first convolution unit and a second convolution unit,

[0166] The first feature map acquisition module 520 includes:

[0167] an intermediate feature map acquisition submodule 521, configured to call the first convolution unit to perform two-fold downsampling on the image to be processed using a convolution kernel of a set size, to obtain an intermediate feature map having a first number of channels;

[0168] A first feature map acquisition submodule 522 is configured to call the second convolution unit to perform two-fold downsampling on the intermediate feature map using a convolution kernel of a set size to obtain a first feature map having a second number of channels;

[0169] The number of the second channels is twice the number of the first channels.

[0170] Optionally, the multiple short-term dense connection modules in each short-term dense connection layer include: a first short-term dense connection module and at least one second short-term dense connection module,

[0171] The second feature map acquisition module 530 includes:

[0172] A first initial feature map acquisition submodule 531 is configured to call a first short-term dense connection module in a first short-term dense connection layer among the multiple short-term dense connection layers to downsample the first feature map by a factor of two to obtain a first initial feature map having a third number of channels; the third number of channels is twice the second number of channels;

[0173] A first processed feature map acquisition submodule 532 is configured to call at least one second short-time dense connection module in the first short-time dense connection layer to perform short-time dense connection processing on the first initial feature map to obtain a first processed feature map having a third number of channels;

[0174] The nth processed feature map acquisition submodule 533 is configured to call the nth short-time dense connection layer except the first short-time dense connection layer among the multiple short-time dense connection layers, perform two-fold downsampling and short-time dense connection processing on the feature map output by the n-1th short-time dense connection layer, and obtain the nth processed feature map;

[0175] A second feature map acquisition submodule 534 is configured to acquire a processed feature map output by a last short-term dense connection layer among the multiple short-term dense connection layers, and use the processed feature map as the second feature map;

[0176] Wherein, n is a positive integer greater than or equal to 2, and the number of channels of the n-th processed feature map is twice the number of channels of the n-1th processed feature map output by the n-1th short-time dense connection layer.

[0177] Optionally, the output module includes: a convolutional layer, an average pooling layer and a full convolutional network layer,

[0178] The third feature map acquisition module 540 includes:

[0179] A sampling feature map acquisition submodule 541 is configured to call the convolutional layer to downsample the second feature map to obtain a sampling feature map;

[0180] The pooling feature map acquisition submodule 542 is used to call the average pooling layer to perform pooling processing on the sampled feature map to obtain a pooling feature map;

[0181] The third feature map acquisition submodule 543 is used to call the full convolutional network layer to perform convolution processing on the pooled feature map to obtain the third feature map.

[0182] The image semantic segmentation device provided by the embodiment of the present disclosure inputs the image to be processed into an image recognition model, wherein the image recognition model includes a short-term dense connection network layer and a decoding network layer, the short-term dense connection network layer includes a convolution module, multiple short-term dense connection layers connected in series, and an output module, each short-term dense connection layer includes multiple short-term dense connection modules, the convolution module is called to process the image to be processed to obtain a first feature map corresponding to the image to be processed, the short-term dense connection layer is called to process the first feature map to obtain a second feature map corresponding to the first feature map, the output module is called to process the second feature map to obtain a third feature map corresponding to the second feature map, the decoding network layer is called to upsample the third feature map and map it to the segmentation category to obtain a fourth feature map with the same number of channels as the number of segmentation categories, and the semantic segmentation result corresponding to the image to be processed is determined based on the fourth feature map. The embodiment of the present disclosure uses the short-term dense connection modules connected in series for image processing, and the feature dimension gradually decreases with the depth of the network module, so that the network can encode the receptive field information required for segmentation while reducing the structural redundancy of the network, thereby improving the performance and efficiency of image semantic segmentation.

[0183] An embodiment of the present disclosure further provides an electronic device, comprising: a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the image semantic segmentation method of the aforementioned embodiment when executing the program.

[0184] An embodiment of the present disclosure further provides a readable storage medium, which, when instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to execute the image semantic segmentation method of the aforementioned embodiment.

[0185] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0186] The algorithms and displays provided herein are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems may also be used together with the teachings herein. Based on the above description, it is apparent that the structure required for constructing such systems is suitable. In addition, the embodiments of the present disclosure are not directed to any specific programming language. It should be understood that various programming languages ​​may be utilized to implement the contents of the embodiments of the present disclosure described herein, and the above description of specific languages ​​is intended to disclose the best mode of implementation of the embodiments of the present disclosure.

[0187] In the description provided herein, numerous specific details are described. However, it is understood that embodiments of the present disclosure may be practiced without these specific details. In some instances, well-known methods, structures, and techniques are not shown in detail so as not to obscure the understanding of this description.

[0188] Similarly, it should be understood that in order to streamline the present disclosure and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present disclosure, various features of the embodiments of the present disclosure are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed approach should not be interpreted as reflecting an intention that the claimed embodiments of the present disclosure require more features than are expressly recited in each claim. Rather, as reflected in the claims below, inventive aspects lie in less than all of the features of the individual embodiments disclosed above. Accordingly, the claims that follow the detailed description are hereby expressly incorporated into the detailed description, with each claim standing on its own as a separate embodiment of the embodiments of the present disclosure.

[0189] Those skilled in the art will appreciate that the modules in the devices of the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and further may be divided into a plurality of submodules or subunits or subcomponents. All features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device so disclosed may be combined in any combination, except that at least some of such features and / or processes or units are mutually exclusive. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0190] The various component embodiments of the embodiments of the present disclosure may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art will appreciate that a microprocessor or digital signal processor (DSP) may be used in practice to implement some or all of the functions of some or all of the components of the device for generating dynamic images according to the embodiments of the present disclosure. The embodiments of the present disclosure may also be implemented as a device or apparatus program for executing part or all of the methods described herein. Such a program implementing the embodiments of the present disclosure may be stored on a computer-readable medium, or may be in the form of one or more signals. Such a signal may be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0191] It should be noted that the above embodiments illustrate rather than limit the embodiments of the present disclosure, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbols placed between brackets should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The embodiments of the present disclosure may be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third etc. does not indicate any order. These words may be interpreted as names.

[0192] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0193] The above description is merely a preferred embodiment of the embodiments of the present disclosure and is not intended to limit the embodiments of the present disclosure. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the embodiments of the present disclosure shall be included in the protection scope of the embodiments of the present disclosure.

[0194] The above description is merely a specific implementation of the embodiments of the present disclosure, but the scope of protection of the embodiments of the present disclosure is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the embodiments of the present disclosure should be included in the scope of protection of the embodiments of the present disclosure. Therefore, the scope of protection of the embodiments of the present disclosure should be based on the scope of protection of the claims.

Claims

1. A method for image semantic segmentation, characterized in that: include: Input the image to be processed into the image recognition model; The image recognition model includes: a short-term dense connection network layer and a decoding network layer, wherein the short-term dense connection network layer includes: a convolution module, a plurality of short-term dense connection layers connected in series, and an output module, and each short-term dense connection layer includes a plurality of short-term dense connection modules; Calling the convolution module to process the image to be processed to obtain a first feature map corresponding to the image to be processed; Calling the short-term dense connection layer to process the first feature map to obtain a second feature map corresponding to the first feature map; Calling the output module to process the second feature map to obtain a third feature map corresponding to the second feature map; Calling the decoding network layer to upsample the third feature map and map it to the segmentation category to obtain a fourth feature map with the same number of channels as the number of segmentation categories; Determining a semantic segmentation result corresponding to the image to be processed according to the fourth feature map; The short-term dense connection layer refers to a network layer formed by multiple short-term dense connection modules connected in series. The structures of the multiple short-term dense connection modules are the same. Each short-term dense connection module contains multiple operation blocks. From the second operation block to the second-to-last operation block, the number of channels of the kernel is half of the number of channels of the previous feature block, and the number of channels of the kernel of the last operation block is consistent with the number of channels of the second-to-last kernel; and the number of channels of the feature map output by each network structure layer is twice the number of channels of the feature map output by the previous network structure layer.

2. The method according to claim 1, characterized in that The convolution module includes: a first convolution unit and a second convolution unit, and calling the convolution module to process the image to be processed to obtain a first feature map corresponding to the image to be processed includes: Calling the first convolution unit to downsample the image to be processed by twice using a convolution kernel of a set size to obtain an intermediate feature map having a first number of channels; Calling the second convolution unit to perform two-fold downsampling on the intermediate feature map using a convolution kernel of a set size to obtain a first feature map having a second number of channels; The number of the second channels is twice the number of the first channels.

3. The method according to claim 1, characterized in that The multiple short-term dense connection modules in each short-term dense connection layer include: a first short-term dense connection module and at least one second short-term dense connection module, and calling the short-term dense connection layer to process the first feature map to obtain a second feature map corresponding to the first feature map includes: Calling a first short-term dense connection module in a first short-term dense connection layer among the multiple short-term dense connection layers to downsample the first feature map by a factor of two to obtain a first initial feature map having a third number of channels; the third number of channels is twice the second number of channels; Calling at least one second short-term dense connection module in the first short-term dense connection layer to perform short-term dense connection processing on the first initial feature map to obtain a first processed feature map with a third number of channels; Calling an nth short-time dense connection layer other than the first short-time dense connection layer among the multiple short-time dense connection layers, performing two-fold downsampling and short-time dense connection processing on the feature map output by the n-1th short-time dense connection layer, to obtain an nth processed feature map; Obtaining a processed feature map output by a last short-term dense connection layer among the multiple short-term dense connection layers, and using the processed feature map as the second feature map; Wherein, n is a positive integer greater than or equal to 2, and the number of channels of the n-th processed feature map is twice the number of channels of the n-1th processed feature map output by the n-1th short-time dense connection layer.

4. The method according to claim 1, wherein The output module includes: a convolution layer, an average pooling layer, and a full convolutional network layer. The calling of the output module to process the second feature map to obtain a third feature map corresponding to the second feature map includes: Calling the convolutional layer to downsample the second feature map to obtain a sampled feature map; Calling the average pooling layer to perform pooling processing on the sampled feature map to obtain a pooled feature map; Call the fully convolutional network layer to perform convolution processing on the pooled feature map to obtain the third feature map.

5. The method according to claim 1, wherein Each short-term dense connection module includes multiple operation blocks, and the number of channels from the second operation block to the penultimate operation block in the operation blocks of each short-term dense connection module forms a geometric progression, and the number of channels of the last operation block is the same as that of the penultimate operation block.

6. An image semantic segmentation device, characterized in that: include: An image input module to be processed, used to input the image to be processed into the image recognition model; The image recognition model includes: a short-term dense connection network layer and a decoding network layer, wherein the short-term dense connection network layer includes: a convolution module, a plurality of short-term dense connection layers connected in series, and an output module, and each short-term dense connection layer includes a plurality of short-term dense connection modules; A first feature map acquisition module is used to call the convolution module to process the image to be processed to obtain a first feature map corresponding to the image to be processed; A second feature map acquisition module, configured to call the short-term dense connection layer to process the first feature map to obtain a second feature map corresponding to the first feature map; a third feature map acquisition module, configured to call the output module to process the second feature map to obtain a third feature map corresponding to the second feature map; a fourth feature map acquisition module, configured to call the decoding network layer to upsample the third feature map and map it to the segmentation categories to obtain a fourth feature map having the same number of channels as the number of segmentation categories; a semantic segmentation result determination module, configured to determine a semantic segmentation result corresponding to the image to be processed based on the fourth feature map; The short-term dense connection layer refers to a network layer formed by multiple short-term dense connection modules connected in series. The structures of the multiple short-term dense connection modules are the same. Each short-term dense connection module contains multiple operation blocks. From the second operation block to the second-to-last operation block, the number of channels of the kernel is half of the number of channels of the previous feature block, and the number of channels of the kernel of the last operation block is consistent with the number of channels of the second-to-last kernel; and the number of channels of the feature map output by each network structure layer is twice the number of channels of the feature map output by the previous network structure layer.

7. The device according to claim 6, characterized in that The convolution module includes: a first convolution unit and a second convolution unit, and the first feature map acquisition module includes: an intermediate feature map acquisition submodule, configured to call the first convolution unit to perform two-fold downsampling on the image to be processed using a convolution kernel of a set size, to obtain an intermediate feature map having a first number of channels; a first feature map acquisition submodule, configured to call the second convolution unit to perform two-fold downsampling on the intermediate feature map using a convolution kernel of a set size, to obtain a first feature map having a second number of channels; The number of the second channels is twice the number of the first channels.

8. The device according to claim 6, characterized in that The multiple short-term dense connection modules in each short-term dense connection layer include: a first short-term dense connection module and at least one second short-term dense connection module, and the second feature map acquisition module includes: a first initial feature map acquisition submodule, configured to call a first short-time dense connection module in a first short-time dense connection layer among the multiple short-time dense connection layers to downsample the first feature map by a factor of two to obtain a first initial feature map having a third number of channels; the third number of channels being twice the second number of channels; A first processing feature map acquisition submodule is configured to call at least one second short-time dense connection module in the first short-time dense connection layer to perform short-time dense connection processing on the first initial feature map to obtain a first processed feature map having a third number of channels; an nth processed feature map acquisition submodule, configured to call an nth short-time dense connection layer other than the first short-time dense connection layer among the multiple short-time dense connection layers, perform two-fold downsampling and short-time dense connection processing on the feature map output by the n-1th short-time dense connection layer, and obtain an nth processed feature map; A second feature map acquisition submodule is configured to acquire a processed feature map output by a last short-term dense connection layer among the multiple short-term dense connection layers, and use the processed feature map as the second feature map; Wherein, n is a positive integer greater than or equal to 2, and the number of channels of the n-th processed feature map is twice the number of channels of the n-1th processed feature map output by the n-1th short-time dense connection layer.

9. The device according to claim 6, characterized in that The output module includes: a convolutional layer, an average pooling layer and a full convolutional network layer, and the third feature map acquisition module includes: A sampling feature map acquisition submodule, configured to call the convolutional layer to downsample the second feature map to obtain a sampling feature map; A pooling feature map acquisition submodule is used to call the average pooling layer to perform pooling processing on the sampled feature map to obtain a pooling feature map; The third feature map acquisition submodule is used to call the full convolutional network layer to perform convolution processing on the pooled feature map to obtain the third feature map.

10. The device according to claim 6, characterized in that Each short-term dense connection module includes multiple operation blocks, and the number of channels from the second operation block to the penultimate operation block in the operation blocks of each short-term dense connection module forms a geometric progression, and the number of channels of the last operation block is the same as that of the penultimate operation block.

11. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the image semantic segmentation method according to any one of claims 1 to 5 when executing the program.

12. A readable storage medium, characterized in that: When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the image semantic segmentation method according to any one of claims 1 to 5.

Images