An image segmentation method, device, terminal equipment and storage medium

Abstract

The application provides an image segmentation method and device, terminal equipment and storage medium, and relates to the technical field of image processing. The image segmentation method comprises the following steps: acquiring an image to be processed; inputting the image to be processed into a trained image segmentation model for processing, and outputting a segmented image; the image segmentation model is an encoder-decoder structure, and a group integration module is arranged between the corresponding encoder and decoder, the group integration module is used for extracting m pieces of compensation information of different scales from the input features of the encoder, and inputting the m pieces of compensation information of different scales into the decoder, wherein m is greater than or equal to 2. The image segmentation method, device, terminal equipment and storage medium provided by the application can solve the problem of inaccurate image segmentation result of the existing image segmentation method to a certain extent.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and particularly relates to an image segmentation method and device, a terminal device and a storage medium. BACKGROUND

[0002] Image segmentation technology is a technology of dividing an image into a plurality of specific regions, thereby extracting a specific type of target. The image segmentation technology has been widely applied in the fields of remote sensing, medicine and the like. Taking the medical field as an example, the region where an organ or a lesion in a medical image is located can be extracted through the image segmentation technology, so as to assist medical experts to make corresponding medical diagnosis.

[0003] With the rapid development of deep learning technology, the image segmentation method based on deep learning has become an important image segmentation technology. In the prior art, a U-Net network model is generally used to segment an image. However, the network model only uses a simple skip connection to fuse low-level feature information and high-level semantic information, so that the high-level semantic information is diluted, thereby resulting in an inaccurate image segmentation result. SUMMARY

[0004] The embodiments of the present application provide an image segmentation method, device, terminal device and storage medium, which can solve the problem of inaccurate image segmentation result of the existing image segmentation method.

[0005] In a first aspect, the embodiments of the present application provide an image segmentation method, which comprises: acquiring a to-be-processed image; inputting the to-be-processed image into a trained image segmentation model for processing, and outputting a segmented image; the image segmentation model is an encoder-decoder structure, and a group integration module is arranged between the mutually corresponding encoder and decoder, and the group integration module is used for extracting m different scales of compensation information from the first input feature of the encoder, and inputting the m different scales of compensation information into the decoder, m≥2.

[0006] In one possible implementation manner, the group integration module comprises a first convolutional layer and m-1 semantic compensation modules; the first convolutional layer is used for extracting compensation information of a first scale from the first input feature of the encoder; the m-1 semantic compensation modules are used for extracting m-1 compensation information of different scales from the compensation information of the first scale, wherein the output of an i-th semantic compensation module in the m-1 semantic compensation modules is the input of an i+1-th semantic compensation module, 1≤i≤m-1.

[0007] In one possible implementation manner, the semantic compensation module comprises a second convolutional layer, a plurality of third convolutional layers and a dilated convolutional layer connected in sequence.

[0008] In one possible implementation manner, the decoder comprises m group convolutional modules;

[0009] inputting m pieces of compensation information of different scales into the decoder, including:

[0010] dividing the second input features of the decoder into m groups;

[0011] decoding and processing, by each of the m group convolution modules, a group of the second input features and compensation information of one scale to obtain m groups of decoded features;

[0012] obtaining output features of the decoder according to the m groups of decoded features.

[0013] In a possible implementation, the group convolution module includes an up-sampling module and a first convolution module.

[0014] decoding and processing, by each of the m group convolution modules, a group of the second input features and compensation information of one scale, including:

[0015] up-sampling, by the up-sampling module in each group convolution module, a group of the second input features, and concatenating the output of the up-sampling module with compensation information of one scale to serve as the input of the first convolution module in the group convolution module.

[0016] In a possible implementation, the encoder includes a second convolution module, a down-sampling module and a compression and excitation module connected in sequence, and the second convolution module includes a batch normalization layer, an excitation layer and a convolution layer connected in sequence.

[0017] In a possible implementation, the image to be processed is a medical image.

[0018] In a second aspect, an embodiment of the present application provides an image segmentation device, including: an acquisition unit configured to acquire an image to be processed; and a segmentation unit configured to input the image to be processed into a trained image segmentation model for processing, and output a segmented image, wherein the image segmentation model is an encoder-decoder structure, and a group integration module is arranged between mutually corresponding encoders and decoders, the group integration module is configured to extract m pieces of compensation information of different scales from input features of the encoder, and input the m pieces of compensation information of different scales into the decoder, and m is greater than or equal to 2.

[0019] In a third aspect, an embodiment of the present application provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the method of any one of the above first aspect when executing the computer program.

[0020] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method of any one of the above first aspect.

[0021] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when running on a terminal device, causes the terminal device to perform the method of any one of the first aspect.

[0022] According to the image segmentation method and device, the terminal device and the storage medium provided in the present application, the group integration module is arranged between the mutually corresponding encoder and decoder, the compensation information of multiple different scales is extracted from the first input feature of the encoder by the group integration module, and the compensation information of multiple different scales is input into the corresponding decoder for processing. The group integration module can increase the receptive field of the network layer, reduce the difference between the low-layer edge detail information in the encoder and the high-level semantic feature information in the corresponding decoder through the compensation information of multiple different scales, avoid the high-level semantic feature information from being diluted due to the direct fusion of the low-layer edge detail information and the high-level semantic feature information, and thus improve the segmentation accuracy of the model on the image to be processed. BRIEF DESCRIPTION OF DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0024] Figure 1 is a structural schematic diagram of an image segmentation model provided by an embodiment of the present application;

[0025] Figure 2 is another structural schematic diagram of an image segmentation model provided by an embodiment of the present application;

[0026] Figure 3 is a structural schematic diagram of an encoder provided by an embodiment of the present application;

[0027] Figure 4 is a structural schematic diagram of a compression and excitation module provided by an embodiment of the present application;

[0028] Figure 5 is a structural schematic diagram of a group integration module provided by an embodiment of the present application;

[0029] Figure 6 is a structural schematic diagram of a semantic compensation module provided by an embodiment of the present application;

[0030] Figure 7 is a structural schematic diagram of a group convolution module in a decoder provided by an embodiment of the present application;

[0031] Figure 8 is a structural schematic diagram of an image segmentation device provided by an embodiment of the present application;

[0032] Figure 9 is a structural schematic diagram of a terminal device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0033] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below in a clear and complete manner with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0034] It should be understood that, when used in the specification and the appended claims, the term “comprising” indicates the presence of the described features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. And the terms used in the specification of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification and the appended claims of the present application, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise. The terms “first,” “second,” “third,” and the like are used only to distinguish descriptions, and cannot be understood as indicating or implying relative importance.

[0035] In view of the problem that, when a U-Net network model is used to process an image segmentation task, only simple skip connections are used to fuse low-level edge detail information in an encoder into high-level semantic information in a decoder, so that the high-level semantic information is diluted, resulting in low accuracy of the image segmentation result, the present application provides an image segmentation method. After obtaining a to-be-processed image, the to-be-processed image is input into an image segmentation model based on an encoder-decoder structure provided by the present application for processing, to obtain a segmentation image of the to-be-processed image.

[0036] In the image segmentation model based on the encoder-decoder structure provided in the application, a group integration module is arranged between the mutually corresponding encoder and decoder, compensation information of equal multiple different scales is extracted from the first input feature of the encoder by the group integration module and input into the corresponding decoder, the receptive field of the network layer is increased, the difference between the low-level edge detail information in the encoder and the high-level semantic feature information in the corresponding decoder is reduced through the compensation information of multiple different scales, the problem that the high-level semantic feature information is diluted due to the direct splicing of the low-level edge detail information and the high-level semantic feature information in the existing U-Net network model is solved, and thus the segmentation accuracy of the model for the image to be processed is improved.

[0037] For example, referring to Figure 1 The structure diagram of the image segmentation model provided in an embodiment of the application is shown in FIG. 1. The image segmentation model includes two convolutional layers, three encoders and three decoders corresponding to the three encoders one by one, and specifically includes: convolutional layer 1, encoder 1, encoder 2, encoder 3, decoder 3, decoder 2, decoder 1 and convolutional layer 2 connected in sequence.

[0038] As shown in Figure 2 Convolutional layer 1 includes 32 convolutional kernels, and the input image to be processed is processed by convolutional layer 1 to increase the channel number of the feature map, and a feature map with a channel number of 32 and a size of 176x144 is obtained. Convolutional layer 2 includes one convolutional kernel, and the feature map with a channel number of 32 and a size of 176x144 output by decoder 1 is processed by convolutional layer 2 to obtain a segmentation image with a size of 176x144.

[0039] As shown in Figure 3 Each encoder includes a second convolutional module, a down-sampling module and a Squeeze and Excitation (SE) module connected in sequence. The second convolutional module is used to increase the channel number of the feature map, the down-sampling module is used to reduce the scale of the feature map, and the Squeeze and Excitation module is used to perform re-scaling processing on the feature map of each channel.

[0040] The second convolutional module includes a Batch Normalization (BN) layer, an activation layer and a convolutional layer connected in sequence. For example, the activation layer can be a ReLU (Rectified Linear Unit) activation layer.

[0041] As shown in Figure 4 The Squeeze and Excitation module includes convolution operation F tr (·, θ), Squeeze operation F sq (·) and Excitation operation F ex(·, w) and a weighting operation F scal (·). Specifically, the input feature map with c_1 channels is converted into a feature map with c_2 channels through a convolution operation. The feature map with c_2 channels is compressed through a Squeeze operation, and the feature map of each channel is converted into a real number, so that the feature map with c_2 channels is converted into a feature vector with c_2 dimensions. The feature map of each channel is evaluated through an Excitation operation to obtain a weight w corresponding to each channel. Finally, the weight w and the feature map with c_2 channels are weighted through a weighting operation, so that the original feature is re-scaled in the channel dimension.

[0042] In the embodiments of the present application, a group integration module is arranged between the corresponding encoder and decoder, and the group integration module is used to extract m compensation information of different scales from the input feature of the encoder and input the m compensation information of different scales into the decoder, m≥2. That is, as shown in Figure 1 and Figure 2 , the group integration module 1 is used to extract m compensation information of different scales from the input feature of the encoder 1 and input the m compensation information of different scales into the decoder 1, the group integration module 2 is used to extract m compensation information of different scales from the input feature of the encoder 2 and input the m compensation information of different scales into the decoder 2, and the group integration module 3 is used to extract m compensation information of different scales from the input feature of the encoder 3 and input the m compensation information of different scales into the decoder 3.

[0043] As shown in Figure 5 , the group integration module (GE block) includes a first convolution layer and m-1 semantic compensation modules (SCM). The first convolution layer is used to extract compensation information of a first scale from the input feature of the encoder, and the first convolution layer is a convolution kernel with a size of 1×1. The m-1 semantic compensation modules are used to extract m-1 compensation information of different scales from the compensation information of the first scale, wherein the output of the i-th semantic compensation module in the m-1 semantic compensation modules is the input of the i+1-th semantic compensation module, 1≤i≤m-1.

[0044] As shown in Figure 6 , each semantic compensation module includes a second convolution layer, at least two third convolution layers and an Atrous convolution layer connected in sequence. The size of the convolution kernel of the second convolution layer is 1×1, and the size of the convolution kernel of each third convolution layer is 3×3. The Atrous convolution layer can increase the receptive field in order to eliminate the semantic gap between the low-level edge information of the encoder and the high-level semantic information of the decoder.

[0045] For example, as shown in Figure 5As shown, the m-1 semantic compensation modules in the group ensemble module include SCM_1 to SCM_m-1. The first convolutional layer in the group ensemble module is used to extract compensation information y1 at the first scale from the input features of the encoder. SCM_1 is used to process the compensation information y1 at the first scale to obtain y2. SCM_3 is used to process the compensation information y2 at the second scale to obtain the compensation information y3 at the third scale, and so on, until SCM_m-1 processes the compensation information y1 at the (m-1)th scale. m-1 The process is performed to obtain the compensation information y at the m-th scale. m .

[0046] like Figure 7 As shown, each decoder includes m groups of convolutional modules. Each group of convolutional modules includes an upsampling module and a first convolutional module. The second input features input to the decoder are divided into m groups, with each of the m groups of second input features corresponding one-to-one with one of the m group of convolutional modules. Each of the m group of convolutional modules decodes one group of second input features and one scale of compensation information to obtain m groups of decoded features. The output features of the decoder can be obtained from the m groups of decoded features. The second input features of the decoder are the output of the network structure located in the layer above the decoder. The network structure in the layer above the decoder can be the last decoder in the image segmentation model, or it can be another decoder located in the layer above the decoder.

[0047] Specifically, for each group convolutional module, an upsampling module within the group convolutional module upsamples a set of second input features, and the output of the upsampling module is then compared with compensation information y at a certain scale. i After concatenation, the outputs of the m group convolutional modules are used as the input to the first convolutional module in the group convolutional module, where 1 ≤ i ≤ m. For each decoder, the outputs of the m group convolutional modules are concatenated and used as the output of that decoder.

[0048] The upsampling module includes a convolutional layer with a 1×1 kernel and a deconvolutional layer. The first convolutional module includes a batch normalization (BN) layer, an activation layer, and a convolutional layer connected in sequence. For example, the activation layer can be a ReLU (rectified linear unit).

[0049] In one example, taking the encoder 3, the group integration module 3 and the decoder 3 as an example, assuming that m = 4, the first input feature of the encoder 3 is a feature map with a channel number of 128 and a size of 44x36, and the decoder includes 4 group convolution modules, i.e., group convolution module 1 to group convolution module 4. The first input feature of the encoder 3 is input into the group integration module 3 for processing to obtain 4 different scale compensation information, i.e., y1, y2, y3 and y4. In addition, the second convolution block, the downsampling module and the compression and excitation module in the encoder 3 sequentially process the first input feature, and the output feature obtained is the second input feature of the decoder 3, and the second input feature includes a feature map with a channel number of 256 and a size of 22x18. The second input feature is divided into 4 groups on the channel, and each group of the second input feature includes a feature map with a channel number of 128 / 4 = 32. Each group of the second input feature and the compensation information of one scale are decoded by each group convolution module in the 4 group convolution modules to obtain 4 groups of decoding features, and the 4 groups of decoding features are spliced to obtain the output of the decoder 3.

[0050] For example, for the first group of the second input feature, the upsampling module in the group convolution module 1 is used to decode the features of the latent space from the first group of the second input feature, and the output of the upsampling module is spliced with the compensation information y1 of the first scale on the channel and input into the first convolution module in the group convolution module 1, and the output is a feature map with a channel number of 128 / 4 = 32 and a size of 44x36 after sequentially passing through the batch normalization layer, the excitation layer and the convolution layer in the first convolution module. Similarly, for the second group of the second input feature, the upsampling module in the group convolution module 2 is used to process the second group of the second input feature, and the output of the upsampling module is spliced with the compensation information y2 of the second scale on the channel and input into the first convolution module in the group convolution module 2, and the output is a feature map with a channel number of 128 / 4 = 32 and a size of 44x36 after sequentially passing through the batch normalization layer, the excitation layer and the convolution layer in the first convolution module. Similarly, the 4 group convolution modules in the decoder 3 all output a feature map with a channel number of 128 / 4 = 32 and a size of 44x36. The output features of the 4 group convolution modules in the decoder 3 are spliced on the channel to obtain a feature map with a channel number of 128 and a size of 44x36 as the second input feature of the decoder 2.

[0051] It should be noted that the decoder in the present application uses group convolution instead of the convolution block in the traditional decoder. The m group convolution modules in each decoder are subnetworks with the same structure, and each subnetwork is equivalent to a classifier. The use of group convolution not only effectively reduces the number of parameters, but also improves the segmentation accuracy of the model according to the prediction diversity of multiple classifiers.

[0052] The image segmentation model provided in the application can be applied to the field of medical image segmentation, realizes extraction of regions such as lesions and organs, for example, brain tissue segmentation, brain tumor segmentation, lung nodule segmentation and the like. It can also be applied to any segmentation or extraction task of a target region in a to-be-processed image.

[0053] Exemplarily, taking the brain tumor segmentation task as an example, the training process and effect of the image segmentation model provided in the application are exemplarily described.

[0054] First, an initial image segmentation model is constructed. For the brain tumor segmentation task, a corresponding training set is collected, the training set including a plurality of image sample pairs, each image sample pair including a brain tumor image sample and a segmentation image sample corresponding to the brain tumor image sample. The training set can be used to iteratively train the initial image segmentation model to minimize the loss function, thereby obtaining a trained image segmentation model.

[0055] It can be understood that the image segmentation method using the trained image segmentation model described above and the method of training the initial image segmentation model described above can be executed by the same terminal device or by different terminal devices. The terminal device can be, but is not limited to, various smartphones, portable notebooks, tablet computers, smart wearable devices, computers, robots and the like.

[0056] In order to verify the feasibility of the image segmentation model provided in the embodiments of the application, the image segmentation model provided in the application is applied to brain tumor segmentation. First, the number of group convolution modules in the decoder in the image segmentation model GEU-Net provided in the application is verified by experiment to affect the brain tumor segmentation result, and the implementation result is compared with the segmentation result of the traditional small U-Net network based on a plurality of different indexes.

[0057] Table 1

[0058]

[0059] As shown in Table 1, the evaluation indexes include Dice similarity coefficient, Sensitivity, Specificity, Hausdorff 95 and Parameter. Among them, the Dice similarity coefficient is used to measure the similarity between the network segmentation result and the standard segmentation result. The Sensitivity is used to measure the recognition ability of the model to positive examples. The Specificity is used to measure the recognition ability of the model to negative examples. The Hausdorff 95 is used to measure the shape similarity. The greater the values of the Dice similarity coefficient, the Sensitivity and the Specificity, the better the performance of the model, and the smaller the Parameter value, the smaller the parameter amount in the model.

[0060] As can be seen from Table 1, the number of group convolution modules in the decoder has a greater influence on the brain tumor segmentation result. When the number of group convolution modules is 4, the performance of the GEU-Net model provided in the present application is optimal in the index of Dice similarity coefficient. In actual application, the number of group convolution modules can be designed according to different segmentation tasks.

[0061] In addition, by reducing the different amounts of parameters in the traditional U-Net model, a small U-Net model and a medium U-Net model are obtained. Based on the multiple evaluation indexes in the above, the segmentation results of the image segmentation model of the present application are compared with the brain tumor segmentation results of the traditional U-Net model, the small U-Net model, the medium U-Net model, the DeepLabV3+ model, the PSP-Net (Pyramid Scene Parsing Network) model and the Attention U-Net model. The experimental results are shown in Table 2. As can be seen from Table 2, the GEU-Net model provided in the present application has a strong advantage in the indexes of Dice similarity coefficient, Sensitivity and parameter amount compared with other models.

[0062] Table 2

[0063]

[0064] In summary, the image segmentation model provided in the embodiments of the present application sets a group integration module between the corresponding encoder and decoder, extracts m different scale compensation information from the first input feature of the encoder through the group integration module and inputs the m different scale compensation information into the corresponding decoder, increases the receptive field of the network layer, and reduces the difference between the low-level edge detail information in the encoder and the high-level semantic feature information in the corresponding decoder through the m different scale compensation information. Secondly, the multiple group convolution modules are used in the decoder to replace the traditional single convolution block, so that the decoder is similar to the integrated multiple classifiers, which not only can greatly reduce the number of parameters, but also can integrate the outputs of the multiple classifiers to improve the segmentation accuracy of the model. The image segmentation model provided in the embodiments of the present application solves the problem that the high-level semantic feature information is diluted due to the direct splicing of the low-level edge detail information and the high-level semantic feature information in the existing U-Net network model, thereby improving the segmentation accuracy of the model on the to-be-processed image and to a certain extent solving the problem of low segmentation accuracy of the traditional U-Net model.

[0065] An image segmentation method is provided according to the embodiments of the present application, Figure 8 An embodiment structure diagram of an image segmentation device provided in the embodiments of the present application is shown. Referring to Figure 8 The image segmentation device can include:

[0066] The acquisition unit 801 acquires a to-be-processed image.

[0067] The segmentation unit 802 is configured to input the to-be-processed image into the trained image segmentation model for processing, and output a segmented image. The image segmentation model is an encoder-decoder structure, and a group integration module is arranged between the corresponding encoder and decoder. The group integration module is configured to extract m different scale compensation information from the input feature of the encoder, and input the m different scale compensation information into the decoder, where m is greater than or equal to 2.

[0068] Optionally, the group integration module includes a first convolution layer and m-1 semantic compensation modules. The first convolution layer is configured to extract compensation information of a first scale from the first input feature of the encoder. The m-1 semantic compensation modules are configured to extract m-1 compensation information of different scales from the compensation information of the first scale. The output of an i-th semantic compensation module in the m-1 semantic compensation modules is the input of an i+1-th semantic compensation module, where 1≤i≤m-1.

[0069] Optionally, the semantic compensation module includes a second convolution layer, a plurality of third convolution layers and a dilated convolution layer connected in sequence.

[0070] Optionally, the decoder includes m group convolution modules.

[0071] The m pieces of compensation information of different scales are input into the decoder, including:

[0072] The second input features of the decoder are divided into m groups;

[0073] The m pieces of compensation information of different scales are input into the decoder, including:

[0074] Optionally, the group convolution module comprises an up-sampling module and a first convolution module.

[0075] The m pieces of compensation information of different scales are input into the decoder, including:

[0076] The m pieces of compensation information of different scales are input into the decoder, including:

[0077] Optionally, the encoder comprises a second convolution module, a down-sampling module and a compression and excitation module connected in sequence, and the second convolution module comprises a batch normalization layer, an excitation layer and a convolution layer connected in sequence.

[0078] Optionally, the image to be processed is a medical image.

[0079] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional units and modules is exemplified, and in actual application, the above functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit or module in the embodiment can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of software functional unit. In addition, the specific names of each functional unit or module are only for easy distinction, and do not limit the protection scope of the present application. The specific working process of the units and modules in the system can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0080] Based on the same inventive concept, the embodiments of the present application also provide a terminal device. As Figure 9As shown, the terminal device 900 of this embodiment includes a processor 901, a memory 902, and a computer program 904 stored in the memory 902 and executable on the processor 901. The computer program 904 can be executed by the processor 901 to generate instructions 903, and the processor 901 can implement the steps in each of the above image color optimization method embodiments according to the instructions 903. Alternatively, the processor 901 implements the functions of each module / unit in each of the above device embodiments when executing the computer program 904, for example Figure 8 The functions of the units 801 and 802 are shown.

[0081] For example, the computer program 904 can be divided into one or more modules / units, one or more modules / units are stored in the memory 902 and executed by the processor 901 to complete the present application. One or more modules / units can be a series of computer program instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program 904 in the terminal device 900.

[0082] Those skilled in the art can understand that Figure 9 The terminal device 900 is only an example and does not constitute a limitation on the terminal device 900, which can include more or fewer components than shown, or combine certain components, or different components, for example, the terminal device 900 can also include an input / output device, a network access device, a bus, etc.

[0083] The processor 901 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0084] The memory 902 can be an internal storage unit of the terminal device 900, for example, a hard disk or a memory of the terminal device 900. The memory 902 can also be an external storage device of the terminal device 900, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, and the like equipped on the terminal device 900. Further, the memory 902 can also include both the internal storage unit and the external storage device of the terminal device 900. The memory 902 is used to store computer programs and other programs and data required by the terminal device 900. The memory 902 can also be used to temporarily store data that has been output or will be output.

[0085] The terminal device provided by the embodiment can execute the method embodiments described above, and the implementation principle and technical effects are similar, which will not be described here.

[0086] The embodiment of the present application also provides a computer readable storage medium, which stores a computer program. The computer program is executed by a processor to implement the method described in the above method embodiments.

[0087] The embodiment of the present application also provides a computer program product, which, when running on a terminal device, causes the terminal device to implement the method described in the above method embodiments.

[0088] The integrated units described above, if realized in the form of software function units and sold or used as independent products, can be stored in a computer readable storage medium. Based on this understanding, the present application can implement all or part of the processes in the above embodiments by a computer program to instruct related hardware to complete, and the computer program can be stored in a computer readable storage medium. The computer program is executed by a processor to implement the steps of each method embodiment described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms, etc. The computer readable storage medium at least includes any entity or device capable of carrying the computer program code to the photographing device / terminal device, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium. For example, U disk, mobile hard disk, magnetic disk or optical disk, etc.

[0089] Reference within this application to "one embodiment" or "some embodiments" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the application. The appearances of the phrase "in one embodiment" or "in some embodiments" in various places in the specification are not necessarily all referring to the same embodiment, nor are they necessarily referring to some, but not all, embodiments. The terms "including," "comprising," "having" and variations thereof as used in this application are intended to be equivalent to the term "consisting of" except where otherwise explicitly disclaimed. The term "consisting of" is intended to mean the inclusion of the recited item or group of items and the exclusion of items that are not recited except for impurities and other such non-specified elements accepted in the art as necessary to the completion of the stated applications.

[0090] In the description of the application, it is to be understood that the terms "first", "second", "third", "fourth", "fifth", "sixth", etc., merely identify features belonging to distinct categories and do not imply a relative position or order of such features, unless specifically stated as such. Thus, a feature specified as "first" can imply that there is at least a "second" feature of the same category. The terms "comprise", "comprising", "include", "including", and "contain", "containing" where used herein are not intended to exclude other additives, components, integers or steps.

[0091] Furthermore, in this application, unless expressly specified and limited otherwise, the terms "connected", "connected", etc., are used broadly and encompass both direct and indirect connections, as well as mechanical and / or electrical connections, and can also imply that two elements are in communication with each other, unless otherwise expressly specified and limited. The specific meaning of the above terms in this application can be understood by those skilled in the art according to the specific circumstances.

[0092] The above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or make equivalent replacements to some or all of the technical features; and such modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. An image segmentation method, characterized in that, The method includes: Obtain the image to be processed; The image to be processed is input into a trained image segmentation model for processing, and a segmented image is output. The image segmentation model is an encoder-decoder structure, which includes multiple encoders, multiple decoders corresponding to the multiple encoders, and multiple group integration modules corresponding to the multiple encoders. Each encoder and its corresponding decoder are connected by a group integration module. Each group integration module is used to extract m compensation information of different scales from the first input features of the encoder and input the m compensation information of different scales into the decoder, where m≥3. Each of the group integration modules includes a first convolutional layer and m-1 semantic compensation modules. The first convolutional layer is used to extract compensation information at a first scale from the first input features of the encoder. The m-1 semantic compensation modules are used to extract m-1 compensation information at different scales from the compensation information at the first scale. The output of the i-th semantic compensation module among the m-1 semantic compensation modules is the input of the (i+1)-th semantic compensation module. The output of each semantic compensation module is a compensation information at a different scale from the first scale, where 1 ≤ i ≤ m-1.

2. The image segmentation method according to claim 1, characterized in that, The semantic compensation module includes a second convolutional layer, multiple third convolutional layers, and a dilated convolutional layer connected in sequence.

3. The image segmentation method according to claim 1, characterized in that, The decoder includes m groups of convolutional modules; The step of inputting the compensation information of the m different scales into the decoder includes: The second input features of the decoder are divided into m groups; Each of the m group convolutional modules decodes a set of the second input features and the compensation information at one scale to obtain m sets of decoded features. The output features of the decoder are obtained based on the m sets of decoding features.

4. The image segmentation method according to claim 3, characterized in that, The group convolution module includes an upsampling module and a first convolution module; The step of decoding a set of second input features and compensation information at one scale using each of the m group convolutional modules includes: The upsampling module in each of the group convolutional modules performs upsampling processing on a group of the second input features. The output of the upsampling module is then concatenated with the compensation information at one scale and used as the input of the first convolutional module in the group convolutional module.

5. The image segmentation method according to any one of claims 1 to 4, characterized in that, The encoder includes a second convolution module, a downsampling module, and a compression and excitation module connected in sequence. The second convolution module includes a batch normalization layer, an excitation layer, and a convolution layer connected in sequence.

6. The image segmentation method according to claim 1, characterized in that, The image to be processed is a medical image.

7. An image segmentation apparatus, characterized in that, include: The acquisition unit acquires the image to be processed; A segmentation unit is used to input the image to be processed into a trained image segmentation model for processing and output a segmented image. The image segmentation model is an encoder-decoder structure, which includes multiple encoders, multiple decoders corresponding one-to-one with the multiple encoders, and multiple group integration modules corresponding one-to-one with the multiple encoders. Each encoder and its corresponding decoder are provided with a group integration module. The group integration module is used to extract m compensation information of different scales from the input features of the encoder and input the m compensation information of different scales into the decoder, where m ≥ 3. Each group integration module includes a first convolutional layer and m-1 semantic compensation modules. The first convolutional layer is used to extract compensation information of a first scale from the first input features of the encoder. The m-1 semantic compensation modules are used to extract m-1 compensation information of different scales from the compensation information of the first scale. The output of the i-th semantic compensation module in the m-1 semantic compensation modules is the input of the (i+1)-th semantic compensation module, where 1 ≤ i ≤ m-1. The output of each semantic compensation module is a compensation information of a different scale from the first scale.

8. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 6.

Images