Image segmentation model generation and image segmentation method, device and electronic equipment

Abstract

The application provides generation of an image segmentation model and an image segmentation method, device and electronic equipment, and belongs to the technical fields of medical image processing and computer vision. The method comprises the following steps: constructing an initial model, the structure of the initial model comprises in sequence: an image representation module for converting a medical image into a first feature vector, a text representation module for converting a medical text into a second feature vector, a feature fine alignment module for aligning based on the first feature vector and the second feature vector to obtain a text enhanced feature, and a mask segmentation module for obtaining a prediction result based on the text enhanced feature and calculating a loss function value of the prediction result; and training the initial model based on obtained medical image data and medical text data to obtain a trained image segmentation model. The application effectively solves the technical problem of poor image segmentation performance in the prior art.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of medical image processing and computer vision, in particular to an image segmentation model generation method and an image segmentation method, device and electronic equipment. BACKGROUND

[0002] Medical image segmentation is an important task in medical image analysis. Text-guided medical image segmentation aims to segment medical images by using medical images and corresponding medical texts, thereby assisting doctors in diagnosis and treatment.

[0003] Compared with single-modal image segmentation, text-guided medical image segmentation can make full use of medical texts to make up for the quality defects of medical images without additional costs. Medical texts are naturally complementary to medical images, and medical texts can provide the number and spatial information of feature regions, thereby more accurately segmenting the feature regions. This is particularly important for clinical diagnosis and medical research, such as polyp segmentation and ultrasound image segmentation.

[0004] Due to the huge difference between different modalities, the main problem of text-guided medical image segmentation is how to learn a joint embedding representation and accurately establish the consistency between images and texts. Many works propose some deep learning segmentation methods, but most of them fail to effectively capture the discriminative regions of images, and roughly aligning images and texts affects the use of text information, ultimately affecting the segmentation performance. Therefore, how to effectively align medical images with corresponding medical texts is a very important task in text-guided image segmentation. SUMMARY

[0005] Therefore, it is necessary to provide an image segmentation model generation method and an image segmentation method, device and electronic equipment to solve the technical problem of poor image segmentation performance in the prior art.

[0006] To solve the above technical problems, in one aspect, the present application provides an image segmentation model generation method, comprising:

[0007] An initial model is constructed, and the structure of the initial model comprises in sequence: an image representation module for converting a medical image into a first feature vector, a text representation module for converting a medical text into a second feature vector, a feature fine alignment module for aligning based on the first feature vector and the second feature vector to obtain a text enhanced feature, and a mask segmentation module for obtaining a prediction result based on the text enhanced feature and calculating a loss function value of the prediction result;

[0008] The initial model is trained based on the obtained medical image data and medical text data to obtain a trained image segmentation model.

[0009] In a possible implementation, the image representation module comprises a plurality of down-sampling convolution modules.

[0010] The down-sampling convolution module comprises a max-pooling layer, a convolution layer, a normalization module, and an activation layer.

[0011] The text representation module comprises a linear layer, an activation layer, and a BERT model pre-trained based on the medical text data and a medical vocabulary, wherein the medical vocabulary comprises medical words and corresponding index values.

[0012] The feature fine alignment module comprises a scale attention module and a spatial attention module, both of which comprise an average pooling layer and a Sigmoid function.

[0013] The mask segmentation module comprises an up-sampling convolution module and a preset loss function.

[0014] In a possible implementation, the training of the initial model based on the obtained medical image data and medical text data comprises:

[0015] The medical image data is down-sampled and convoluted based on the image representation module to obtain a first feature vector comprising a plurality of scales.

[0016] The medical text data is converted into a second feature vector based on the text representation module.

[0017] The first feature vector and the second feature vector are aligned on a scale level and a spatial level in sequence based on the scale attention module and the spatial attention module to obtain the text-enhanced feature.

[0018] A prediction is made based on the mask segmentation module and the text-enhanced feature to obtain a prediction result, and a loss function value of the prediction result is calculated.

[0019] The model parameters are adjusted based on the loss function value until the loss function value is less than a preset threshold or a preset maximum number of iterations is reached.

[0020] In a possible implementation, before the medical text data is converted into a second feature vector based on the text representation module, the method further comprises:

[0021] The medical text data is subjected to word segmentation processing to obtain a sub-word sequence.

[0022] A preset mark is added before and after the sub-word sequence.

[0023] convert each token in the subword sequence into an ID value based on the medical vocabulary, to obtain a Token text sequence, wherein if the token appears in the medical vocabulary, the token is converted into a corresponding index value, otherwise the token is converted into the mark;

[0024] generate a position encoding and an attention mask for the Token text sequence.

[0025] In a possible implementation, the converting the medical text data into a second feature vector based on the text representation module comprises:

[0026] input the ID value, the position encoding and the attention mask into the BERT model for forward propagation, to obtain a hidden state of each Token text sequence, and arrange the hidden state to obtain an output feature;

[0027] input the output feature into the linear layer and the activation layer in sequence, to obtain the second feature vector.

[0028] In a possible implementation, the aligning the first feature vector and the second feature vector on a scale level and a spatial level based on the feature fine alignment module to obtain the text enhanced feature comprises:

[0029] calculate a first similarity between the first feature vector and the second feature vector based on the average pooling layer;

[0030] generate a first attention weight based on the Sigmoid function and the first similarity, and generate a third feature vector enhanced on the scale level based on the first attention weight and the first feature vector;

[0031] calculate a second similarity between the third feature vector and the second feature vector based on the average pooling layer;

[0032] generate a second attention weight based on the Sigmoid function and the second similarity, and generate the text enhanced feature enhanced on the scale level and the spatial level based on the second attention weight and the first feature vector.

[0033] In a possible implementation, the adjusting the model parameter based on the loss function value comprises:

[0034] perform back propagation based on the loss function value, and optimize a connection weight inside an initial model based on a preset optimizer and the gradient.

[0035] The application further provides an image segmentation method, comprising:

[0036] The generation method of the image segmentation model based on any one of the above method items obtains an image segmentation model.

[0037] inputting a medical image to be segmented and a medical text description corresponding to the medical image into the image segmentation model;

[0038] obtaining a segmented mask medical image.

[0039] The application further provides a generation device of an image segmentation model, comprising:

[0040] a model construction module, configured to construct an initial model, a structure of the initial model comprising in sequence: an image representation module for converting a medical image into a first feature vector, a text representation module for converting a medical text into a second feature vector, a feature fine alignment module for aligning based on the first feature vector and the second feature vector to obtain a text enhanced feature, and a mask segmentation module for obtaining a prediction result based on the text enhanced feature and calculating a loss function value of the prediction result;

[0041] a training optimization module, configured to train the initial model based on obtained medical image data and medical text data to obtain a training complete image segmentation model.

[0042] The application further provides an electronic device, comprising:

[0043] a memory, configured to store a program;

[0044] a processor, coupled with the memory, configured to execute the program stored in the memory to implement steps in the generation method of the image segmentation model or the image segmentation method according to any one of the above method items.

[0045] The application has the beneficial effects that: the application provides a generation method of an image segmentation model, which first learns important feature information in a medical image through a new text information fusion module, and captures the detailed corresponding relationship between a text description and the medical image through a feature fine alignment module, so that the semantic representations of different modalities can be better aligned, the medical image capturing features are enhanced by introducing text information, the lost detailed information in spatial position and feature size is supplemented, and the precision of image segmentation is effectively enhanced. On this basis, a loss function is introduced to maintain the similarity between the segmentation result and the real label, retain the detailed features of the medical image and the semantic content of the text information, and eliminate the differences between different modalities. Finally, the efficiency and accuracy of image segmentation are improved, and the technical problem of poor image segmentation performance of the prior art is effectively solved. BRIEF DESCRIPTION OF DRAWINGS

[0046] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 A flowchart illustrating an embodiment of the image segmentation model generation method provided by the present invention;

[0048] Figure 2 For the present invention Figure 1 A flowchart of an embodiment of S102;

[0049] Figure 3 For the present invention Figure 2 A flowchart illustrating the other steps prior to S202 in one embodiment;

[0050] Figure 4 For the present invention Figure 2 A flowchart illustrating an embodiment of S202;

[0051] Figure 5 For the present invention Figure 2 A flowchart illustrating an embodiment of S203;

[0052] Figure 6 This is a flowchart illustrating an embodiment of the image segmentation method provided by the present invention;

[0053] Figure 7 A schematic diagram of the structure of an embodiment of the image segmentation model generation device provided by the present invention;

[0054] Figure 8 A schematic diagram of the structure of an embodiment of the electronic device provided by the present invention. Detailed Implementation

[0055] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0056] In the description of the embodiments of the present invention, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0057] The terms "first", "second", and the like in the description and in the claims, if any, are used for distinguishing between similar elements and not necessarily for describing a sequential or chronological order. It is to be understood that the terms so used are interchangeable under appropriate circumstances and are not to be construed as limited to the order of application set forth herein.

[0058] Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase "in an embodiment" in various places in the specification are not necessarily all referring to the same embodiment, nor are they necessarily mutually exclusive of one another. It is expressly understood that any of the embodiments described herein can be combined with any of the other embodiments unless specifically noted otherwise.

[0059] The application provides an image segmentation model generation method and device, and an image segmentation method and device, and an electronic device.

[0060] Figure 1 An embodiment flowchart of the image segmentation model generation method provided by the application is shown in FIG. 1. Figure 1 The image segmentation model generation method includes the following steps.

[0061] Step S101, constructing an initial model.

[0062] Specifically, the structure of the initial model includes, in sequence, an image representation module for converting a medical image into a first feature vector, a text representation module for converting a medical text into a second feature vector, a feature fine alignment module for aligning based on the first feature vector and the second feature vector to obtain a text enhanced feature, and a mask segmentation module for obtaining a prediction result based on the text enhanced feature and calculating a loss function value of the prediction result.

[0063] Step S102, training the initial model based on the obtained medical image data and medical text data to obtain a trained image segmentation model.

[0064] In a possible implementation, the image representation module includes a plurality of down-sampling convolution modules.

[0065] It should be noted that the down-sampling convolution module is used to sample the input medical image, and the image features with the same center and different scales obtained by sampling are used as the multi-scale feature representation of the image.

[0066] F v = [ F v 0 , F v1 , F v 2 , F v 3 ];

[0067] Preferably, the down-sampling convolution module is implemented by a max-pooling layer, a convolution layer, a normalization module and an activation layer.

[0068] The text representation module comprises a linear layer, an activation layer and a BERT model pre-trained based on medical text data and a medical vocabulary, wherein the medical vocabulary comprises medical words and corresponding index values.

[0069] The feature fine alignment module comprises a scale attention module and a spatial attention module, both of which are composed of an average pooling layer and a Sigmoid function.

[0070] The mask segmentation module comprises an up-sampling convolution module and a preset loss function.

[0071] As Figure 2 In a possible implementation, step S102 comprises:

[0072] Step S201, performing down-sampling convolution on the medical image data based on the image representation module to obtain a first feature vector comprising a plurality of scales;

[0073] Step S202, converting the medical text data into a second feature vector based on the text representation module;

[0074] Step S203, sequentially aligning the first feature vector and the second feature vector on a scale level and a spatial level based on the scale attention module and the spatial attention module to obtain a text-enhanced feature;

[0075] Step S204, performing prediction based on the mask segmentation module and the text-enhanced feature to obtain a prediction result and calculating a loss function value of the prediction result;

[0076] Specifically, the total loss function value of the calculation model comprises three parts: L a Focal loss function L f a Dice loss function L d and a similarity loss function L c

[0077] (1) The Focal loss function L l ​This approach aims to address the class imbalance problem by assigning higher weights to samples that are difficult to classify, thereby improving model performance. Its calculation formula is as follows:

[0078] (1)

[0079] In equation (1), p i Indicates the first i The predicted probability that a sample belongs to the correct category. The balancing factor is used to control the weights of positive and negative samples. The adjustment factor is used to adjust the weights of easy-to-classify and hard-to-classify samples. N This represents the total number of samples.

[0080] (2) Dice loss function L d The formula is as follows:

[0081] (2)

[0082] In equation (2), p i This is the predicted value of the i-th pixel in the segmentation result predicted by the model. In binary classification problems, it is usually 0 or 1, but in practical applications, it can be a probability value between 0 and 1, representing the probability that the pixel belongs to the target class. g i This represents the true value of the i-th pixel in the actual label. It is usually 0 or 1, indicating whether the pixel belongs to the target class.

[0083] Similarity loss function L c The formula is as follows:

[0084] (3)

[0085] Furthermore, the model's total loss function L The calculation formula is:

[0086] L = L f + L d + L c (4)

[0087] In equation (4), Represents the weight parameters. Represents the loss function for controlling reduced modal differences. L c The hyperparameter value of the percentage.

[0088] Step S205, adjust the model parameters based on the loss function value until the loss function value is less than a preset threshold or reaches a preset maximum number of iterations.

[0089] As Figure 3 In a possible implementation, before step S202, further comprising:

[0090] Step S301, performing word segmentation processing on the medical text data to obtain a subword sequence;

[0091] Step S302, adding a preset mark before and after the subword sequence;

[0092] Exemplarily, the input medical text is split into words or subwords, which can use the WordPiece word segmentation method. Special marks [CLS] and [SEP] are added before and after the segmented text, the [CLS] mark is used to represent the features of the entire sentence, and the [SEP] mark is used to separate the sentences or mark the end of the sentence. Let the input medical text be T, and the result of the word segmentation is { t 1, t 2, t 3, …, t n}, and the result after adding special marks is { | CLS | , t 1, t 2, t 3, …, t n , | SEP |}.

[0093] Step S303, converting each subword of the subword sequence into an ID value based on a medical vocabulary table to obtain a Token text sequence;

[0094] Wherein, if the subword appears in the medical vocabulary table, it is converted into the corresponding index value, otherwise it is converted into a mark;

[0095] Specifically, the word segmentation text is converted into an index (token ID) in the vocabulary table by using the vocabulary table pre-trained by the BERT model.

[0096] Further, each token is converted into a corresponding ID by using the pre-training vocabulary table of the BERT model. The result of converting it into Token IDs is { ID CLS , ID 1, ID 2, ID 3, …, ID n , ID SEP}.

[0097] Step S304, position encoding and attention mask are generated for the Token text sequence.

[0098] It should be noted that the position information of each token is generated, mainly to indicate the position of each token in the sentence. The position encoding is P={ P CLS , P 1, P 2, P 3, …, P n , P SEP}. A mask is created to indicate which tokens are actual inputs (1) and which are padding parts (0). The attention mask is M={ M CLS , M 1, M 2, M 3, …, M n , M SEP}.

[0099] As Figure 4 In one possible implementation, step S202 includes:

[0100] Step S401, input the ID value, position encoding and attention mask into the BERT model for forward propagation to obtain the hidden state of each Token text sequence, and arrange to obtain the output feature;

[0101] Specifically, the Token IDs, position encoding and attention mask are input into the BERT model to obtain the hidden state H={ H CLS , H 1, H 2, H 3, …, H n , H SEP}.

[0102] Step S402, input the output feature into the linear layer and the activation layer in turn to obtain the second feature vector.

[0103] Specifically, the output feature of the BERT model is extracted, the hidden state of the last layer is obtained, and the feature vector corresponding to the [CLS] mark is extracted. The hidden state contains the feature vector of each token. After processing, the second feature vector F t。

[0104] As Figure 5 In a possible implementation, step S203 includes:

[0105] Step S501, calculate the first similarity between the first feature vector and the second feature vector after the average pooling layer processing;

[0106] Specifically, the calculation formula is as follows:

[0107] (5)

[0108] In formula (5), denotes the first feature vector after processing F va and the second feature vector F ta .

[0109] Step S502, generate the first attention weight based on the Sigmoid function and the first similarity, and generate the third feature vector enhanced in the scale layer based on the first attention weight and the first feature vector;

[0110] Specifically, the calculation formula is as follows:

[0111] F v' = F va × θ ( S ( F va, F ta )) (6)

[0112] In formula (6), θ denotes the Sigmoid function operation, F v’ denotes the third feature vector.

[0113] Step S503, calculate the second similarity between the third feature vector and the second feature vector based on the average pooling layer;

[0114] Specifically, the calculation formula is as follows:

[0115] (7)

[0116] In formula (7), denotes the second feature vector F t and the third feature vector F v’ .

[0117] Step S504, generating a second attention weight based on a Sigmoid function and the second similarity, and generating a text enhancement feature enhanced in a scale level and a spatial level based on the second attention weight and the first feature vector.

[0118] Specifically, the calculation formula is as follows:

[0119] F vt F v θ S F v', F t

[0120] In formula (8), F vt denotes the text enhancement feature, F v denotes the first feature vector.

[0121] In one possible implementation, step S205 includes:

[0122] Based on the loss function value, the gradient is back propagated, and the connection weight inside the initial model is optimized based on the preset optimizer and the gradient.

[0123] As Figure 6 The application also provides an image segmentation method, comprising:

[0124] Step S601, obtaining an image segmentation model based on the image segmentation model generation method in any one of the above method items;

[0125] Step S602, inputting a medical image to be segmented and a medical text description corresponding to the medical image into the image segmentation model;

[0126] Step S603, obtaining a segmented mask medical image.

[0127] As Figure 7 The application also provides an image segmentation model generation device 70, comprising:

[0128] The model construction module 710 is configured to construct an initial model, and the structure of the initial model comprises, in sequence, an image representation module for converting a medical image into a first feature vector, a text representation module for converting a medical text into a second feature vector, a feature fine alignment module for aligning the first feature vector and the second feature vector to obtain a text enhancement feature, and a mask segmentation module for obtaining a prediction result based on the text enhancement feature and calculating a loss function value of the prediction result.​​​​​

[0129] The training optimization module 720 is configured to train the initial model based on the obtained medical image data and medical text data to obtain a trained image segmentation model.

[0130] As Figure 8 The present application also provides an electronic device 80, comprising:

[0131] The memory 810 is configured to store a program.

[0132] The processor 820 is coupled to the memory 810 and is configured to execute the program stored in the memory 810 to implement the steps of the image segmentation model generation method or the image segmentation method according to any one of the above method items.

[0133] Those skilled in the art can understand that all or part of the processes of the above-mentioned embodiments can be completed by a computer program instructing relevant hardware (such as a processor, a controller, etc.) to complete. The computer program can be stored in a computer readable storage medium. The computer readable storage medium includes a magnetic disk, an optical disk, a read-only memory, a random access memory, etc.

[0134] The above describes in detail the image segmentation model generation method, the image segmentation method, the device and the electronic device provided by the present application. The principles and implementation manners of the present application are described by using specific examples. The above description of the embodiments is only used to help understand the method of the present application and its core idea. Meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation manner and application range can be changed. In summary, the content of the present application should not be understood as a limitation of the present application.

Claims

1. A method for generating an image segmentation model, characterized in that, The method comprises the following steps: An initial model is constructed, and a structure of the initial model comprises in sequence: an image representation module for converting a medical image into a first feature vector, a text representation module for converting a medical text into a second feature vector, a feature fine alignment module for aligning the first feature vector and the second feature vector to obtain a text enhanced feature, and a mask segmentation module for obtaining a prediction result based on the text enhanced feature and calculating a loss function value of the prediction result, the image representation module comprises a plurality of down-sampling convolution modules, wherein the down-sampling convolution module comprises a max-pooling layer, a convolution layer, a normalization module and an activation layer, the text representation module comprises a linear layer, an activation layer and a BERT model pre-trained based on the medical text data and a medical vocabulary, wherein the medical vocabulary comprises medical words and corresponding index values, the feature fine alignment module comprises a scale attention module and a spatial attention module, each of which comprises an average pooling layer and a Sigmoid function, and the mask segmentation module comprises an up-sampling convolution module and a preset loss function; The initial model is trained based on the obtained medical image data and medical text data to obtain a trained image segmentation model; The feature fine alignment module aligns the first feature vector and the second feature vector on a scale level and a spatial level to obtain the text enhanced feature, which comprises the following steps: A first similarity between the first feature vector and the second feature vector is calculated based on the average pooling layer; A first attention weight is generated based on the Sigmoid function and the first similarity, and a third feature vector enhanced on the scale level is generated based on the first attention weight and the first feature vector; A second similarity between the third feature vector and the second feature vector is calculated based on the average pooling layer; A second attention weight is generated based on the Sigmoid function and the second similarity, and a text enhanced feature enhanced on the scale level and the spatial level is generated based on the second attention weight and the first feature vector.

2. The method of claim 1, wherein, The training of the initial model based on the obtained medical image data and medical text data comprises the following steps: The medical image data is down-sampled and convoluted based on the image representation module to obtain a first feature vector comprising a plurality of scales; The medical text data is converted into a second feature vector based on the text representation module; The first feature vector and the second feature vector are aligned on the scale level and the spatial level in sequence based on the scale attention module and the spatial attention module to obtain the text enhanced feature; A prediction result is obtained based on the mask segmentation module and the text enhanced feature, and a loss function value of the prediction result is calculated; The model parameters are adjusted based on the loss function value until the loss function value is less than a preset threshold or a preset maximum number of iterations is reached.

3. The method of claim 2, wherein, Before the medical text data is converted into a second feature vector based on the text representation module, the method further comprises the following steps: The medical text data is subjected to word segmentation processing to obtain a sub-word sequence; adding preset marks before and after the sub-word sequence; converting each sub-word of the sub-word sequence into an ID value based on the medical vocabulary table to obtain a Token text sequence, wherein if the sub-word appears in the medical vocabulary table, the sub-word is converted into a corresponding index value, otherwise, the sub-word is converted into the mark; generating position encoding and attention mask for the Token text sequence.

4. The method of claim 3, wherein, The method for converting the medical text data into a second feature vector based on the text representation module comprises: inputting the ID value, the position encoding and the attention mask into the BERT model for forward propagation to obtain a hidden state of each Token text sequence, and arranging the hidden state to obtain an output feature; inputting the output feature into the linear layer and the activation layer in sequence to obtain the second feature vector.

5. The method of claim 1, wherein, The method for adjusting the model parameters based on the loss function value comprises: performing back propagation based on the loss function value, and optimizing the connection weight inside the initial model based on a preset optimizer and gradient.

6. An image segmentation method characterized by, The method comprises: obtaining an image segmentation model based on the method for generating the image segmentation model according to any one of claims 1-5; inputting a medical image to be segmented and a medical text description corresponding to the medical image into the image segmentation model; obtaining a segmented mask medical image.

7. An apparatus for generating an image segmentation model, the apparatus comprising: a processor configured to: receive a plurality of images; and generate a model for segmenting the plurality of images. The method comprises: a model construction module configured to construct an initial model, the structure of the initial model comprising, in sequence, an image representation module configured to convert a medical image into a first feature vector, a text representation module configured to convert a medical text into a second feature vector, a feature fine alignment module configured to align the first feature vector and the second feature vector based on the first feature vector and the second feature vector to obtain a text enhanced feature, and a mask segmentation module configured to obtain a prediction result based on the text enhanced feature and calculate a loss function value of the prediction result, the image representation module comprising a plurality of down-sampling convolution modules, wherein each down-sampling convolution module comprises a max-pooling layer, a convolution layer, a normalization module and an activation layer, the text representation module comprising a linear layer, an activation layer and a BERT model pre-trained based on the medical text data and a medical vocabulary table, wherein the medical vocabulary table comprises medical words and corresponding index values, the feature fine alignment module comprising a scale attention module and a spatial attention module, each of which comprises an average-pooling layer and a Sigmoid function, and the mask segmentation module comprising an up-sampling convolution module and a preset loss function; a training optimization module configured to train the initial model based on obtained medical image data and medical text data to obtain a trained image segmentation model; wherein the feature fine alignment module aligns the first feature vector and the second feature vector on a scale level and a spatial level to obtain the text enhanced feature, comprising: calculating a first similarity of the first feature vector and the second feature vector based on the average-pooling layer; generating a first attention weight based on the Sigmoid function and the first similarity, and generating a third feature vector enhanced on the scale level based on the first attention weight and the first feature vector; calculate a second similarity between the third feature vector and a second feature vector based on the average pooling layer; generate a second attention weight based on the Sigmoid function and the second similarity, and generate a text enhancement feature enhanced in a scale level and a spatial level based on the second attention weight and the first feature vector.

8. An electronic device, comprising: comprise: a memory for storing a program; a processor coupled to the memory, configured to execute the program stored in the memory, so as to realize the steps of the image segmentation model generation method in any one of claims 1 to 5 or the image segmentation method in claim 6.

Images