Training method of medical image segmentation model, image segmentation method and related device

Abstract

The invention provides a medical image segmentation model training method, an image segmentation method and a related device. The method comprises the following steps: enhancing a training medical image; extracting image features of the enhanced image and text features of the text description, fusing the image features and the text features, calculating to obtain similarity according to the image features and the text features, and generating prompt information according to the similarity; extracting attention features of the fusion features; performing image segmentation on the training medical image according to prompt information and the attention features; adjusting model parameters according to the image segmentation result to obtain a trained medical image segmentation model; according to the method, features of different fields are distinguished and learned by using text information of different fields, the robustness and generalization of the model are improved on the basis of limited training data, the overall parameter quantity and consumption of computing resources are emphasized, and high-accuracy medical image segmentation is realized in few computing resources.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image processing, and in particular to a training method for a medical image segmentation model, an image segmentation method, and related devices. Background Art

[0002] Traditional medical segmentation models usually use a network architecture that combines downsampling and upsampling. This network architecture gradually captures more accurate local feature information through downsampling, and effectively fuses global and local features by combining jump connections. However, this network architecture mainly focuses on local features, and global features in complex images cannot be effectively integrated. The Transformer architecture can fully utilize its advantages in extracting features from images, and its core self-attention mechanism can effectively capture global features and calculate the relationship between any two positions. However, the heavy computational workload and large number of parameters in this architecture require more computing resources and more space to store numerous parameters. Summary of the invention

[0003] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0004] The purpose of the present application is to solve at least one of the technical problems existing in the related art to a certain extent. The embodiments of the present application provide a training method for a medical image segmentation model, an image segmentation method and related devices, which can achieve high-precision image segmentation with less resources.

[0005] An embodiment of the first aspect of the present application is a method for training a medical image segmentation model, comprising:

[0006] Acquire a training medical image and a text description corresponding to the training medical image, and input the training medical image and the text description into an initial medical image segmentation model;

[0007] The training medical image is enhanced to obtain an enhanced image, wherein the quantity difference between images in different medical fields in the enhanced image is less than a preset difference;

[0008] Extracting image features of the enhanced image and extracting text features of the text description, fusing the image features and the text features to obtain fused features, calculating similarities based on the image features and the text features, and generating prompt information based on the similarities;

[0009] Extracting attention features of the fused features;

[0010] Performing image segmentation on the training medical image according to the prompt information and the attention feature to obtain an image segmentation result;

[0011] The parameters of the initial medical image segmentation model are adjusted according to the image segmentation result to obtain a trained medical image segmentation model.

[0012] According to certain embodiments of the first aspect of the present application, the step of enhancing the training medical image to obtain an enhanced image includes:

[0013] Randomly selecting a plurality of target enhancement operations from an enhancement operation group, wherein the enhancement operation group includes a plurality of spatial level enhancement operations and a plurality of pixel level enhancement operations;

[0014] The multiple target enhancement operations are performed on the training medical image in a random order to obtain an enhanced image.

[0015] According to some embodiments of the first aspect of the present application, extracting image features of the enhanced image includes:

[0016] Performing a position embedding operation on the enhanced image to obtain a first image processing feature;

[0017] Learning the dependency relationship between the plurality of first image processing features from different subspace perspectives to obtain a second image processing feature;

[0018] The second image processing feature is extracted through a multi-layer sensor to obtain an image feature.

[0019] According to some embodiments of the first aspect of the present application, extracting text features of the text description includes:

[0020] Performing a position embedding operation on the text description to obtain a first text processing feature;

[0021] Learning the dependency relationship between the plurality of first text processing features from different subspace perspectives to obtain a second text processing feature;

[0022] The second text processing feature is extracted through a multi-layer sensor to obtain a text feature.

[0023] According to certain embodiments of the first aspect of the present application, the calculating the similarity according to the image feature and the text feature, and generating prompt information according to the similarity, includes:

[0024] Calculate the cosine similarity between the image feature and the text feature according to the image feature and the text feature;

[0025] Determine a target cosine similarity from among the plurality of cosine similarities;

[0026] Determine the medical field of the image feature corresponding to the target cosine similarity as the target medical field;

[0027] Prompt information is generated according to the target medical field.

[0028] According to some embodiments of the first aspect of the present application, extracting the attention feature of the fused feature includes:

[0029] Extracting channel attention features and spatial attention features of the fused features;

[0030] The channel attention feature and the spatial attention feature are fused to obtain the attention feature.

[0031] According to certain embodiments of the first aspect of the present application, extracting the channel attention feature and the spatial attention feature of the fused feature includes:

[0032] Processing the fused features through a multi-head attention mechanism to obtain a multi-head attention feature;

[0033] Performing average pooling processing on the multi-head attention feature to obtain a first channel feature, performing linear transformation and activation function processing on the first channel feature to obtain a second channel feature, and performing matrix multiplication of the second channel feature and the multi-head attention feature to obtain a channel attention feature;

[0034] The spatial dimension of the channel attention feature is downsampled and compressed to obtain a first spatial feature, the first spatial feature is processed by an activation function to obtain a second spatial feature, the spatial dimension of the second spatial feature is upsampled and expanded to obtain a third spatial feature, and the third spatial feature is matrix multiplied with the channel attention feature to obtain a spatial attention feature.

[0035] An embodiment of the second aspect of the present application, an image segmentation method, includes:

[0036] Acquire a medical image to be segmented;

[0037] Inputting the medical image to be segmented into a trained medical image segmentation model to obtain a segmentation result;

[0038] The trained medical image segmentation model is obtained by training according to the training method of the medical image segmentation model as described in the embodiment of the first aspect of the present application.

[0039] An embodiment of the third aspect of the present application is an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the training method for the medical image segmentation model as described in the embodiment of the first aspect of the present application and the image segmentation method as described in the embodiment of the second aspect of the present application are implemented.

[0040] An embodiment of the fourth aspect of the present application is a computer storage medium storing computer executable instructions, wherein the computer executable instructions are used to execute the training method of the medical image segmentation model as described in the embodiment of the first aspect of the present application and the image segmentation method as described in the embodiment of the second aspect of the present application.

[0041] The above scheme has at least the following beneficial effects: by enhancing the training medical image; extracting the image features of the enhanced image and the text features of the text description, fusing the image features and the text features, calculating the similarity based on the image features and the text features, and generating prompt information based on the similarity; extracting the attention features of the fused features; performing image segmentation on the training medical image based on the prompt information and the attention features; adjusting the model parameters based on the image segmentation results to obtain a trained medical image segmentation model; using text information from different fields to distinguish and learn features from different fields, thereby solving the problem of improving the robustness and generalization of the model on limited training data, and paying attention to the overall parameter size and the consumption of computing resources, so as to achieve high-accuracy medical image segmentation with less computing resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings are used to provide further understanding of the technical solution of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the technical solution of the present application and do not constitute a limitation on the technical solution of the present application.

[0043] Figure 1 is a step diagram of the image segmentation method provided in an embodiment of the present application;

[0044] Figure 2 is a step diagram of a training method for a medical image segmentation model provided in an embodiment of the present application;

[0045] Figure 3 It is a sub-step diagram of step S300 provided in an embodiment of the present application;

[0046] Figure 4 is a sub-step diagram of extracting image features of an enhanced image provided by an embodiment of the present application;

[0047] Figure 5 It is a sub-step diagram of extracting text features of text description provided by an embodiment of the present application;

[0048] Figure 6 It is a sub-step diagram of calculating similarity based on image features and text features and generating prompt information based on the similarity provided by an embodiment of the present application;

[0049] Figure 7 It is a sub-step diagram of step S500 provided in an embodiment of the present application;

[0050] Figure 8 It is a sub-step diagram of step S510 provided in an embodiment of the present application. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0052] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims or the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0053] The embodiments of the present application are further described below in conjunction with the accompanying drawings.

[0054] An embodiment of the present application provides an image segmentation method.

[0055] Reference Figure 1 , the image segmentation method comprises the following steps:

[0056] Step S110, obtaining a medical image to be segmented;

[0057] Step S120, inputting the medical image to be segmented into a trained medical image segmentation model to obtain a segmentation result.

[0058] The trained medical image segmentation model is obtained by training according to the following medical image segmentation model training method.

[0059] Reference Figure 2 , the training method of the medical image segmentation model includes the following steps:

[0060] Step S200, obtaining a training medical image and a text description corresponding to the training medical image, and inputting the training medical image and the text description into an initial medical image segmentation model;

[0061] Step S300, enhancing the training medical image to obtain an enhanced image;

[0062] Step S400, extracting image features of the enhanced image and text features of the text description, fusing the image features and the text features to obtain fused features, calculating similarities based on the image features and the text features, and generating prompt information based on the similarities;

[0063] Step S500, extracting attention features of fused features;

[0064] Step S600, performing image segmentation on the training medical image according to the prompt information and the attention feature to obtain an image segmentation result;

[0065] Step S700, adjusting the parameters of the initial medical image segmentation model according to the image segmentation result to obtain a trained medical image segmentation model.

[0066] In step S200, medical images are obtained from an image database as training medical images, and the training medical images include images in different medical fields, such as CT images in the field of lung medicine, B-ultrasound images in the field of fetal examination medicine, etc. The medical records of the training medical images are used as text descriptions of the corresponding training medical images. The training medical images and the text descriptions are input into the initial medical image segmentation model.

[0067] Reference Figure 3 For step S300, the training medical image is enhanced to obtain an enhanced image, including the following steps:

[0068] Step S310, randomly selecting multiple target enhancement operations from the enhancement operation group;

[0069] Step S320, performing multiple target enhancement operations on the training medical image in a random order to obtain an enhanced image.

[0070] Exemplarily, the enhancement space of the medical image segmentation model is divided into the spatial enhancement space D spatial and pixel enhancement space D pixel The spatial enhancement space includes multiple spatial enhancement operations, such as rotation, horizontal flip, vertical flip, scaling, translation, and shearing. The pixel enhancement space includes multiple pixel enhancement operations, such as brightness adjustment, contrast enhancement, sharpening, Gaussian blur, and Gaussian noise. Multiple spatial enhancement operations and multiple pixel enhancement operations are considered as an enhancement operation group.

[0071] During the sampling process, three modules are generated for each training medical image. Each module extracts a random value R1 from the Beta distribution. According to the random value R1, multiple target enhancement operations are randomly selected from the enhancement operation group. For example, three target enhancement operations are selected as horizontal flip, vertical flip and sharpening. The random value R2 extracted from the Dirichlet distribution is used to determine the execution order of the target enhancement operations. Multiple target enhancement operations are performed on the training medical image in a random order to obtain an enhanced image. Among them, the intensity and probability of each enhancement operation are controlled by the hyperparameter L, which determines the maximum intensity and the corresponding probability. By adjusting this hyperparameter L, it can be ensured that the enhanced image meets the characteristics and requirements of medical images.

[0072] The number of images in different medical fields in the enhanced image is similar, that is, the numerical difference is less than the preset difference, thereby achieving a balance of medical image samples in different medical fields. By balancing the medical image samples in the same medical field, there will be no deviation when learning images in multiple medical fields, and the adaptability and robustness of the model are enhanced.

[0073] Image enhancement can alleviate the imbalance of medical image samples in different medical fields when there are not enough medical images to support segmentation, and solve the differences between natural images and medical images.

[0074] For step S400, refer to Figure 4 , extracting image features of the enhanced image, including the following steps:

[0075] Step S411, performing a position embedding operation on the enhanced image to obtain a first image processing feature;

[0076] Step S412, learning the dependency relationship between the plurality of first image processing features from different subspace perspectives to obtain second image processing features;

[0077] Step S413: extract the second image processing feature through the multi-layer sensor to obtain the image feature.

[0078] Reference Figure 5 , extracting text features of text description, including the following steps:

[0079] Step S421, performing a position embedding operation on the text description to obtain a first text processing feature;

[0080] Step S422, learning the dependency relationship between the plurality of first text processing features from different subspace perspectives to obtain second text processing features;

[0081] Step S423: extracting the second text processing feature through the multi-layer sensor to obtain text features.

[0082] The image features and text features are fused to obtain fused features.

[0083] Reference Figure 6 , similarity is calculated based on image features and text features, and prompt information is generated based on the similarity, including the following steps:

[0084] Step S431, calculating the cosine similarity between the image feature and the text feature according to the image feature and the text feature;

[0085] Step S432, determining a target cosine similarity from among the multiple cosine similarities;

[0086] Step S433, determining the medical field of the image feature corresponding to the target cosine similarity as the target medical field;

[0087] Step S434: Generate prompt information according to the target medical field.

[0088] For example, the enhanced image x belongs to an unknown medical domain and its dimensions are width w and height h. Consider a set of text descriptions T = {T1, T2, ..., T x}, where T i is the text description of the image from the ith domain, Represents a discrete numerical index. An initial feature extraction is performed on the enhanced image using a 2×2 convolutional layer. The output of the convolutional layer is then converted into a series of vectors, each of which corresponds to the features of a position. A position embedding operation is then performed on each vector in the sequence to add position information to it and obtain the first image processing feature. Among them, the position embedding is a predefined vector used to provide position data to the model. Before entering the Transformer module, layer normalization is applied to the sequence to help the training stability of the model and increase the convergence speed of the model. 12 Transformer modules are used for feature extraction. Each Transformer module uses a multi-head attention mechanism, which enables the model to learn the dependencies between inputs from different subspace perspectives at the same time to obtain the second image processing feature. Multilayer perceptrons help capture more complex features, while layer normalization normalizes these features. The image features extracted from the enhanced image can be expressed as: in, represents the reshape operator, represents a 2x2 convolution operation, Used to obtain the position information of each vector, All features within each sample are normalized. Refers to the multi-head self-attention module in Transformer, and MLP(.) represents a multilayer perceptron with two linear layers and one activation function layer.

[0089] When processing text information, the text description T is first converted into a fixed-dimensional vector [B, N, D] through an embedding layer, i.e., the first text processing feature. Among them, B represents the batch size, N represents the context length, and D represents the dimension. Then the fixed-dimensional vector is positionally embedded. The obtained vector is input into the same 12 Transformer modules for image feature extraction to obtain the second text processing feature. Finally, the second text processing feature is normalized. The text feature of the extracted text description can be expressed by the following formula: in, Map each text token to a vector of dimension D.

[0090] In the image feature extraction process, the image feature dimension A is [1, D], where 1 represents the length of the image feature and D represents the dimension of the feature vector. Similarly, in text feature extraction, the text feature dimension B is [3, D], where 3 represents the number of text descriptions in different fields and D represents the dimension of the feature vector. Subsequently, the image feature vector is matrix multiplied with the transposed text feature vector to calculate the similarity between each text feature vector and all image feature vectors. This process identifies the medical field that is most similar to the input image, namely the target medical field. The medical field label with the highest similarity is provided to the prompt generation module and automatically provides the box prompt required for the segmentation stage. In addition, the text information across different fields can be used to convert the box into a point prompt to realize an automatic multimodal prompt generator. The prompt generation process can be expressed by the following formula: Among them, f cos (.) represents matrix multiplication, that is, calculating the cosine similarity, f max (.) represents the highest similarity value in the matrix.

[0091] Reference Figure 7 , for step S500, extracting the attention feature of the fused feature includes the following steps:

[0092] Step S510, extracting channel attention features and spatial attention features of the fused features;

[0093] Step S520, fusing the channel attention feature and the spatial attention feature to obtain the attention feature.

[0094] Reference Figure 8 , for step S510, extracting the channel attention feature and the spatial attention feature of the fused feature includes the following steps:

[0095] Step S511, processing the fused features through a multi-head attention mechanism to obtain a multi-head attention feature;

[0096] Step S512, average pooling the multi-head attention features to obtain a first channel feature, linearly transforming and performing activation function processing on the first channel feature to obtain a second channel feature, and matrix multiplying the second channel feature with the multi-head attention feature to obtain a channel attention feature;

[0097] Step S513, downsample and compress the spatial dimension of the channel attention feature to obtain the first spatial feature, perform activation function processing on the first spatial feature to obtain the second spatial feature, upsample and expand the spatial dimension of the second spatial feature to obtain the third spatial feature, and perform matrix multiplication of the third spatial feature and the channel attention feature to obtain the spatial attention feature.

[0098] Exemplarily, a lightweight domain-adaptive image encoder is used to improve the efficiency and accuracy of cross-domain image segmentation. On the premise of retaining the Transformer architecture's expertise in global feature extraction, the pre-trained ViT-Tiny is used as the backbone of the encoder. Because ViT-Tiny has a small number of parameters and high feature extraction efficiency. Considering the limitations of ViT-Tiny's small number of parameters and poor segmentation effect when processing complex and changeable medical multi-domain images, a multi-domain adaptive module is added to the model to improve the capabilities of the image encoder. In order to retain the strong generalization parameters obtained by ViT-Tiny after a large number of image training, and considering the limitations of medical images, some parameters of ViT-Tiny are frozen, and the domain adapter is continuously optimized on a limited data set. The fused features are multi-domain adaptive in the Transformer module of the domain-adaptive image encoder. After the fused features are processed by the multi-head attention mechanism, the image is presented as [B,C,H,W], where B represents the batch size, C represents the channel, H represents the height, and W represents the width.

[0099] The image features are adapted to the medical domain images through the domain adapter. The image features pass through the channel attention consisting of two linear layers and an activation function, and then enter the spatial attention consisting of two 2D convolutions and an activation function. Through the feature extraction of the domain adapter, the spatial features and channel features of images in different medical fields are effectively learned. The attention features for extracting fusion features can be expressed as follows: I spatial =I+f spatial (I),I DA =I channel +I spatial .in, represents the channel attention features extracted from image features, Represents the spatial attention features extracted from the image. The residual connection method is used to extract features and obtain spatial attention features. DA It is composed of channel attention features and spatial attention features.

[0100] Specifically, after being processed by the multi-head attention module, the image feature dimension is [1, C, W, H]. It should be noted that average pooling is used to compress the spatial dimension information while retaining the channel dimension information. Subsequently, two linear layers and a ReLU activation function are used to learn channel-level features from the input data. The processed features are then matrix-multiplied with the original input to enhance or weaken certain channels of the original feature map to obtain channel attention features. After channel attention processing, the image feature dimension is still [1, C, W, H]. The image features are processed by spatial attention. Then a 3×3 convolution kernel is used to downsample and compress the spatial dimension of the image features. In addition, a ReLU activation function is used to increase the nonlinearity of the network. Finally, a 4×4 convolution kernel is used to upsample and expand the spatial dimension of the image features. The processed image features are matrix-added with the original input to produce spatial attention features. Features in different fields can be learned more efficiently with fewer parameters.

[0101] The domain-adaptive image encoder focuses on extracting channel and spatial features from images. This encoder significantly improves overall efficiency, accelerates the segmentation of a single image, and improves segmentation accuracy in various medical fields, which is beneficial for performing medical image segmentation operations on resource-constrained devices.

[0102] For step S600, the attention features are classified based on the medical field of the prompt information, which can improve the accuracy of the classification. The training medical image is segmented according to the classification result to obtain the image segmentation result.

[0103] While object detection mechanisms require different types of datasets, the model only requires a segmentation dataset during training and does not require additional object detection annotations. The model uses a mask algorithm during the training phase to calculate the minimum bounding box based on the masks in the training dataset, thereby obtaining the required data without adding additional cost to the automatic hint generator. This approach ensures fully automated training and operation while improving the segmentation accuracy of the model in different medical fields.

[0104] For step S700, a loss function is obtained according to the image segmentation result, and the parameters of the initial medical image segmentation model are adjusted according to the loss function to obtain a trained medical image segmentation model.

[0105] After obtaining the trained medical image segmentation model, the performance of the model can be evaluated by three evaluation indicators, including intersection over union (IoU), mean intersection over union (mIoU), and accuracy (Dice).

[0106] The intersection over union (IoU) is expressed as: The mean intersection over union (mIoU) is expressed as: The accuracy Dice is expressed as:

[0107] In a specific embodiment, the input image size of all stages of the experiment (training, validation, and testing) is standardized to 224×224 pixels. This ensures a fair comparison between the medical image segmentation model and other baseline models. All experiments were performed on a system equipped with a high-performance graphics card, using Python3.9 and PyTorch2.22 under the CUDA11.8 framework. During the experiment, the open source visual image processing library OpenCV was used. Each GPU runs with a batch size of 16 and 8 threads. During the training process of the entire medical image segmentation model, Adam was used as the optimizer and Focal loss was used as the loss function. The same data augmentation technique was applied in the image preprocessing step. In addition, all five data sets were randomly divided into three subsets (80% for training, 10% for validation, and 10% for testing).

[0108] The experimental results are shown in Table 1, where the medical image segmentation model is represented by LAM-SAM.

[0109] Table 1 Comparison of evaluation indicators of medical image segmentation model and baseline model

[0110]

[0111]

[0112] An embodiment of the present application provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the training method of the medical image segmentation model and the medical image segmentation method as described above when executing the computer program.

[0113] The electronic device may be any intelligent terminal including a computer.

[0114] In general, for the hardware structure of the electronic device, the processor can be implemented using a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, etc., to execute related programs to implement the technical solutions provided in the embodiments of the present application.

[0115] The memory can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory can store an operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory, and the processor is called to execute the methods of the embodiments of this application.

[0116] The input / output interface is used to realize information input and output.

[0117] The communication interface is used to realize the communication interaction between this device and other devices. Communication can be achieved through wired methods (such as USB, network cable, etc.) or wireless methods (such as mobile network, WIFI, Bluetooth, etc.).

[0118] The bus transmits information between the various components of the device (such as the processor, memory, input / output interface, and communication interface). The processor, memory, input / output interface, and communication interface communicate with each other within the device through the bus.

[0119] An embodiment of the present application provides a computer storage medium storing computer executable instructions for executing the medical image segmentation model training method and medical image segmentation method as described above.

[0120] It will be appreciated by those skilled in the art that all or some of the steps and systems in the disclosed method above may be implemented as software, firmware, hardware and appropriate combinations thereof. Some physical components or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or a non-transitory medium) and a communication medium (or a temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage or other magnetic storage devices, or any other medium that may be used to store desired information and may be accessed by a computer. In addition, it is well known to those of ordinary skill in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium. In the above description of this specification, the description of reference terms "one embodiment / example", "another embodiment / example" or "certain embodiments / examples" and the like means that the specific features, structures, materials or characteristics described in conjunction with the embodiments or examples are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0121] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0122] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0123] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0124] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0125] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above-mentioned units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. Although the embodiments of the present application have been shown and described, it can be understood by those of ordinary skill in the art that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and purposes of the present application, and the scope of the present application is defined by the claims and their equivalents.

[0126] The above is a specific description of the preferred implementation of the present application, but the present application is not limited to the embodiments. Technical personnel familiar with the field can make various equivalent modifications or substitutions without violating the spirit of the present application. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present application.

Claims

1. A training method for a medical image segmentation model, It is characterized in that include: Acquire a training medical image and a text description corresponding to the training medical image, and input the training medical image and the text description into an initial medical image segmentation model; The training medical image is enhanced to obtain an enhanced image, wherein the quantity difference between images in different medical fields in the enhanced image is less than a preset difference; Extracting image features of the enhanced image and extracting text features of the text description, fusing the image features and the text features to obtain fused features, calculating similarities based on the image features and the text features, and generating prompt information based on the similarities; Extracting attention features of the fused features; Performing image segmentation on the training medical image according to the prompt information and the attention feature to obtain an image segmentation result; The parameters of the initial medical image segmentation model are adjusted according to the image segmentation result to obtain a trained medical image segmentation model.

2. The training method of the medical image segmentation model according to claim 1, It is characterized in that The step of enhancing the training medical image to obtain an enhanced image comprises: Randomly selecting a plurality of target enhancement operations from an enhancement operation group, wherein the enhancement operation group includes a plurality of spatial level enhancement operations and a plurality of pixel level enhancement operations; The multiple target enhancement operations are performed on the training medical image in a random order to obtain an enhanced image.

3. The training method of the medical image segmentation model according to claim 1, It is characterized in that The step of extracting the image features of the enhanced image comprises: Performing a position embedding operation on the enhanced image to obtain a first image processing feature; Learning the dependency relationship between the plurality of first image processing features from different subspace perspectives to obtain a second image processing feature; The second image processing feature is extracted through a multi-layer sensor to obtain an image feature.

4. The training method of the medical image segmentation model according to claim 1, It is characterized in that The extracting text features of the text description includes: Performing a position embedding operation on the text description to obtain a first text processing feature; Learning the dependency relationship between the plurality of first text processing features from different subspace perspectives to obtain a second text processing feature; The second text processing feature is extracted through a multi-layer sensor to obtain a text feature.

5. The training method of the medical image segmentation model according to claim 1, It is characterized in that The calculating the similarity according to the image feature and the text feature, and generating prompt information according to the similarity, comprises: Calculate the cosine similarity between the image feature and the text feature according to the image feature and the text feature; Determine a target cosine similarity from among the plurality of cosine similarities; Determine the medical field of the image feature corresponding to the target cosine similarity as the target medical field; Prompt information is generated according to the target medical field.

6. The method for training a medical image segmentation model according to claim 1, It is characterized in that The extracting the attention feature of the fusion feature includes: Extracting channel attention features and spatial attention features of the fused features; The channel attention feature and the spatial attention feature are fused to obtain the attention feature.

7. The method for training a medical image segmentation model according to claim 6, It is characterized in that The extracting the channel attention feature and the spatial attention feature of the fusion feature includes: Processing the fused features through a multi-head attention mechanism to obtain a multi-head attention feature; Performing average pooling processing on the multi-head attention feature to obtain a first channel feature, performing linear transformation and activation function processing on the first channel feature to obtain a second channel feature, and performing matrix multiplication of the second channel feature and the multi-head attention feature to obtain a channel attention feature; The spatial dimension of the channel attention feature is downsampled and compressed to obtain a first spatial feature, the first spatial feature is processed by an activation function to obtain a second spatial feature, the spatial dimension of the second spatial feature is upsampled and expanded to obtain a third spatial feature, and the third spatial feature is matrix multiplied with the channel attention feature to obtain a spatial attention feature.

8. An image segmentation method, It is characterized in that include: Acquire a medical image to be segmented; Inputting the medical image to be segmented into a trained medical image segmentation model to obtain a segmentation result; The trained medical image segmentation model is obtained by training according to the training method for a medical image segmentation model as described in any one of claims 1 to 7.

9. An electronic device, It is characterized in that include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the training method for the medical image segmentation model according to any one of claims 1 to 7 and the image segmentation method according to claim 8 are implemented.

10. A computer storage medium, It is characterized in that Computer executable instructions are stored, and the computer executable instructions are used to execute the training method of the medical image segmentation model according to any one of claims 1 to 7 and the image segmentation method according to claim 8.

Images