Target region segmentation method, electronic equipment and storage medium
By introducing encoder, decoder and Transformer modules into the area segmentation model, using jump connection and self-attention mechanisms, the problem of inaccurate pancreatic placeholder area segmentation caused by fewer training medical images is solved, and the limited training data is achieved The efficiency and accuracy of target area segmentation under the target area.
Patent Information
- Application Number
- CN202411470075.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-21
- Publication Date
- 2025-02-11
AI Technical Summary
In medical image analysis, fewer training medical images lead to the inability to accurately segment target areas such as pancreatic placeholder areas.
The area segmentation model including encoder, decoder and Transformer module is adopted. Through the jump connection and the self-attention mechanism of the Transformer module, the global context dependence relationship of the target encoding feature is captured, thereby achieving accurate segmentation of the target area.
With fewer training medical images, the target area can be accurately segmented, which improves the efficiency and accuracy of medical image processing.
Smart Images
Figure CN119417844A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present invention relate to the technical field of medical image processing, and in particular to a target area segmentation method, electronic equipment and storage medium. Background Art
[0002] Pancreatic mass segmentation is a key task in medical image analysis, which mainly involves automatically segmenting the pancreatic mass area from brain computed tomography (CT) images or magnetic resonance imaging (MRI) images. Fast and accurate pancreatic mass segmentation can help doctors better assess the severity and scope of bleeding, thereby formulating more personalized treatment plans.
[0003] However, in the medical field, there are relatively few training medical images (i.e., medical images that can be used for model training), and the regional segmentation model obtained based on this training cannot accurately segment target areas such as the pancreatic space-occupying area, which needs to be urgently solved. Summary of the invention
[0004] The embodiments of the present invention provide a target region segmentation method, an electronic device and a storage medium, so as to achieve accurate segmentation of the target region when there are few training medical images.
[0005] According to one aspect of the present invention, a method for segmenting a target region is provided, which may include:
[0006] Acquire a target medical image including a target region to be segmented, and a region segmentation model, wherein the region segmentation model includes an encoder, a decoder jump-connected to the encoder, and a Transformer module connected to the encoder and the decoder respectively;
[0007] The target medical image is input into the region segmentation model, and the target region is segmented from the target medical image by using the region segmentation model through the following three steps:
[0008] Based on the encoder, the target medical image is encoded to obtain the target encoding features;
[0009] Based on the Transformer module, the target encoding features are encoded to obtain global encoding features;
[0010] Based on the decoder, decoding processing related to the global encoding features is performed to obtain the regional segmentation result of the target area, and the regional segmentation result is output.
[0011] According to another aspect of the present invention, a target region segmentation device is provided, which may include:
[0012] A region segmentation model acquisition module, used to acquire a target medical image containing a target region to be segmented, and a region segmentation model, wherein the region segmentation model includes an encoder, a decoder jump-connected to the encoder, and a Transformer module respectively connected to the encoder and the decoder;
[0013] The target medical image input module is used to input the target medical image into the region segmentation model, so as to segment the target region from the target medical image by using the region segmentation model through the following three steps:
[0014] A target coding feature obtaining module is used to encode the target medical image based on the encoder to obtain the target coding feature;
[0015] The global coding feature acquisition module is used to encode the target coding feature based on the Transformer module to obtain the global coding feature;
[0016] The region segmentation result obtaining module is used to perform decoding processing related to the global encoding features based on the decoder, obtain the region segmentation result of the target area, and output the region segmentation result.
[0017] According to another aspect of the present invention, there is provided an electronic device, which may include:
[0018] at least one processor; and
[0019] a memory communicatively connected to at least one processor; wherein,
[0020] The memory stores a computer program executable by at least one processor, and the computer program is executed by at least one processor so that the at least one processor implements the target region segmentation method or the target region segmentation method provided by any embodiment of the present invention when executing the computer program.
[0021] According to another aspect of the present invention, a computer-readable storage medium is provided, on which computer instructions are stored, and the computer instructions are used to enable a processor to implement the target area segmentation method or the target area segmentation method provided by any embodiment of the present invention when executed.
[0022] The technical solution of the embodiment of the present invention is to obtain a target medical image containing a target area to be segmented, and a region segmentation model, wherein the region segmentation model includes an encoder, a decoder jump-connected to the encoder, and a Transformer module respectively connected to the encoder and the decoder. The encoder and the decoder based on the jump connection can realize the training of the model when there are fewer training medical images; the target medical image is input into the region segmentation model, so as to use the region segmentation model to segment the target area from the target medical image through the following three steps: based on the encoder, encoding processing is performed on the target medical image to obtain target encoding features, and the efficiency and accuracy of feature extraction of the target medical image can be improved through encoding processing; based on the Transformer module, encoding processing is performed on the target encoding features to obtain global encoding features, so as to capture the global context dependency of the target encoding features through the self-attention mechanism of the Transformer module, and further improve the accuracy of feature extraction; based on the decoder, decoding processing related to the global encoding features is performed to obtain the region segmentation result of the target area, and the region segmentation result is output to realize accurate segmentation of the target area. The above technical solution, through the jump-connected encoder and decoder, can train a region segmentation model with fewer training medical images. Through the Transformer module, it can capture the global contextual dependencies of the target coding features on the basis of extracting local features of the target medical image to obtain the target coding features, thereby achieving accurate segmentation of the target region with fewer training medical images.
[0023] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0025] Figure 1 is a flow chart of a target area segmentation method provided according to an embodiment of the present invention;
[0026] Figure 2 is a flow chart of another target area segmentation method provided according to an embodiment of the present invention;
[0027] Figure 3is a schematic diagram of a model structure of a specific example in another target region segmentation method provided in an embodiment of the present invention;
[0028] Figure 4 is a structural block diagram of a target region segmentation device provided according to an embodiment of the present invention;
[0029] Figure 5 It is a schematic diagram of the structure of an electronic device for implementing the target area segmentation method according to an embodiment of the present invention. DETAILED DESCRIPTION
[0030] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.
[0031] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. The situations of "target", "original", etc. are similar and will not be repeated here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0032] Before introducing the embodiment of the present invention, the relevant contents described in the background technology are first exemplarily described to better understand the target region segmentation method described in the embodiment of the present invention.
[0033] When segmenting the pancreatic space-occupying area through a neural network model based on deep learning, a large number of training medical images are needed to train the neural network model to improve the accuracy of pancreatic space-occupying area segmentation. However, there are often relatively few training medical images, and the Unet network adopts a U-shaped architecture, which consists of an encoder (contracting path) and a decoder (expanding path), and directly connects the feature map of the encoder to the corresponding layer of the decoder through a jump connection. This design enables the Unet network to fully utilize the low-level features extracted in the encoding stage during the decoding process, which helps to capture more detailed information when there are fewer training medical images. Therefore, a small number of training medical images can be used to train the Unet network, so that good training results can be achieved through the Unet network when there are relatively few training sample images.
[0034] However, since the Unet network is based on convolution operations, it mainly focuses on the extraction of local features and is difficult to capture global contextual information. In particular, it has certain limitations when dealing with long-range dependencies. Therefore, using only the Unet network to segment the pancreatic space-occupying area is still not accurate enough and needs to be solved urgently.
[0035] Figure 1 It is a flow chart of a target region segmentation method provided in an embodiment of the present invention. This embodiment is applicable to the case of segmenting a target region in a target medical image, and is particularly applicable to the case of segmenting a target region in a target medical image when a training medical image is insufficient. The method can be executed by a target region segmentation device provided in an embodiment of the present invention, and the device can be implemented by software and / or hardware, and the device can be integrated in an electronic device, and the electronic device can be various user terminals or servers.
[0036] See also Figure 1 The method of the embodiment of the present invention specifically includes the following steps:
[0037] S110, obtaining a target medical image including a target region to be segmented, and a region segmentation model, wherein the region segmentation model includes an encoder, a decoder jump-connected to the encoder, and a Transformer module respectively connected to the encoder and the decoder.
[0038] Among them, the target medical image can be understood as an image containing the target area to be segmented obtained by photographing or scanning the human body through medical equipment. Optionally, the target medical image is a brain computed tomography image and / or magnetic resonance imaging image, which can clearly reflect the specific situation inside the human body. The target area can be understood as an area in the target medical image that requires surgical operation or observation, such as a lesion area and a bleeding area. Optionally, the target area is a pancreatic space-occupying area. Since pancreatic space-occupying is a life-threatening emergency, it is crucial to quickly and accurately segment the pancreatic space-occupying area from the target medical image for formulating a treatment plan.
[0039] The region segmentation model can be understood as a trained model that can segment the target region in the target medical image. Optionally, the region segmentation model may include an encoder, a decoder jump-connected to the encoder, and a Transformer module connected to the encoder and the decoder respectively. Through the encoder and the decoder jump-connected to the encoder, the feature map of the encoder can be directly connected to the corresponding layer of the decoder through a jump connection when there are fewer training medical images, so that the low-level features extracted in the encoding stage can be fully utilized during the decoding process, thereby improving the segmentation accuracy. Through the Transformer module connected to the encoder and the decoder respectively, the Transformer module can be used at the bottom of the encoder to capture global context information, further improve the accuracy of the encoding features, and thus further improve the segmentation accuracy.
[0040] Optionally, the region segmentation model can be pre-trained through the following steps: obtaining an initial segmentation model, a training medical image, and a region segmentation label corresponding to the training medical image, wherein the model structure of the initial segmentation model is the same as the region segmentation model; inputting the training medical image into the initial segmentation model to obtain a training segmentation region; comparing the training segmentation region with the region segmentation label and calculating the model loss; and adjusting the model parameters in the initial segmentation model based on the model loss to train the region segmentation model.
[0041] First, it is necessary to build an initial segmentation model with the same model structure as the regional segmentation model; then input the acquired training medical image into the initial segmentation model, segment the training medical image, and obtain the training segmentation region; then compare the training segmentation region with the acquired regional segmentation label corresponding to the training medical image, and calculate the model loss. Optionally, the model loss can be calculated by cross entropy loss. Finally, based on the model loss, the model parameters in the initial segmentation model are adjusted. After continuously training and adjusting the initial segmentation model, when the calculated model loss is less than the preset threshold, the regional segmentation model can be trained.
[0042] S120 , input the target medical image into the region segmentation model, and use the region segmentation model to segment the target region from the target medical image through three steps S130 - S150 .
[0043] S130. Based on the encoder, encode the target medical image to obtain target encoding features.
[0044] The target coding feature can be understood as a feature obtained after the target medical image is coded by the encoder. The target medical image can be coded by the encoder, and the coding feature output by the middle coding layer or the last coding layer of the encoder is used as the target coding feature.
[0045] S140. Based on the Transformer module, the target coding feature is encoded to obtain a global coding feature.
[0046] Among them, the global coding feature can be understood as a coding feature that contains the global context information of the target coding feature. After obtaining the target coding feature, since the encoder is based on convolution operations, it mainly focuses on the extraction of local features and it is difficult to capture the global context information. Therefore, the target coding feature can be encoded through the Transformer module to capture the global context information of the target coding feature and obtain the global coding feature.
[0047] S150, based on the decoder, perform decoding processing related to the global coding feature to obtain a region segmentation result of the target area, and output the region segmentation result.
[0048] After obtaining the global coding features, relevant decoding processing can be performed on the global coding features based on the decoder to obtain the regional segmentation result of the target area, and the regional segmentation result is output.
[0049] The technical solution of the embodiment of the present invention is to obtain a target medical image containing a target area to be segmented, and a region segmentation model, wherein the region segmentation model includes an encoder, a decoder jump-connected to the encoder, and a Transformer module respectively connected to the encoder and the decoder. The encoder and the decoder based on the jump connection can realize the training of the model when there are fewer training medical images; the target medical image is input into the region segmentation model, so as to use the region segmentation model to segment the target area from the target medical image through the following three steps: based on the encoder, encoding processing is performed on the target medical image to obtain target encoding features, and the efficiency and accuracy of feature extraction of the target medical image can be improved through encoding processing; based on the Transformer module, encoding processing is performed on the target encoding features to obtain global encoding features, so as to capture the global context dependency of the target encoding features through the self-attention mechanism of the Transformer module, and further improve the accuracy of feature extraction; based on the decoder, decoding processing related to the global encoding features is performed to obtain the region segmentation result of the target area, and the region segmentation result is output to realize accurate segmentation of the target area. The above technical solution, through the jump-connected encoder and decoder, can train a region segmentation model with fewer training medical images. Through the Transformer module, it can capture the global contextual dependencies of the target coding features on the basis of extracting local features of the target medical image to obtain the target coding features, thereby achieving accurate segmentation of the target region with fewer training medical images.
[0050] An optional technical solution is to encode the target medical image based on an encoder to obtain target coding features, including: encoding the target medical image based on an encoder, and using the bottom-level coding features output by the bottom-level coding layer in the encoder as the target coding features.
[0051] Among them, the residual convolutional network in the encoder can be used to continuously downsample and extract features of the target medical image, and finally the bottom-level coding features output by the bottom-level coding layer in the encoder are used as the target coding features.
[0052] The above technical solution can maximize the accuracy of target medical image feature extraction to maximize the accuracy of target region segmentation.
[0053] On this basis, optionally, based on the decoder, decoding processing related to the global coding features is performed to obtain the regional segmentation result of the target area, including: for the intermediate coding layer adjacent to the underlying coding layer in the encoder, the global coding features are spliced with the intermediate coding features output by the intermediate coding layer to obtain the spliced coding features; the spliced coding features are input into the decoder to use the decoder to decode the spliced coding features to obtain the regional segmentation result of the target area.
[0054] In the process of encoding the target medical image through the encoder, each intermediate coding layer in the encoder will output the corresponding intermediate coding features, which are connected to the corresponding intermediate decoding layer of the decoder by means of jump connection. When decoding the global coding features, it is necessary to splice the intermediate coding features corresponding to the intermediate coding layers in the encoder adjacent to the bottom coding layer with the global coding features through the jump connection between the encoder and the decoder to obtain the spliced coding features, and then use the decoder and the intermediate coding features corresponding to each decoding layer in the decoder to decode the spliced coding features and obtain the regional segmentation result of the target area.
[0055] The above technical solution, through jump connection, uses intermediate coding features and decoders to decode the global coding features, which can make full use of the intermediate coding features extracted in the encoding stage, help capture more detail information when there are fewer training medical images, and improve the accuracy of target area segmentation.
[0056] Figure 2 It is a flow chart of another target area segmentation method provided in an embodiment of the present invention. This embodiment is optimized based on the above-mentioned technical solutions. In this embodiment, optionally, based on the Transformer module, the target coding feature is encoded to obtain a global coding feature, including: based on the Transformer module, the target coding feature is expanded to obtain feature blocks with the number of channels of the target coding feature and the target position number corresponding to each feature block; based on the Transformer module, all the obtained feature blocks and all the target position numbers are encoded to obtain a global coding feature. Among them, the explanations of the terms that are the same or corresponding to the above-mentioned embodiments are not repeated here.
[0057] See also Figure 2 The method of this embodiment may specifically include the following steps:
[0058] S210, obtaining a target medical image including a target region to be segmented, and a region segmentation model, wherein the region segmentation model includes an encoder, a decoder jump-connected to the encoder, and a Transformer module respectively connected to the encoder and the decoder.
[0059] S220 , input the target medical image into the region segmentation model, and use the region segmentation model to segment the target region from the target medical image through four steps S230 - S260 .
[0060] S230: Based on the encoder, encode the target medical image to obtain target coding features.
[0061] S240, based on the Transformer module, the target coding feature is expanded to obtain feature blocks whose number is the number of channels of the target coding feature and the target position number corresponding to each feature block.
[0062] Among them, after the target medical image is encoded to obtain the target coding features, the target coding features can be expanded according to the channel dimension, that is, the target coding features are extracted based on each channel respectively to obtain feature blocks with the same number of channels as the target coding features, and the target position number representing the corresponding position of each feature block is determined according to the channel order.
[0063] S250: Based on the Transformer module, all feature blocks and all target position numbers are encoded to obtain global encoding features.
[0064] Among them, the self-attention mechanism of the Transformer module can be used for encoding processing to obtain global encoding features.
[0065] S260: Based on the decoder, perform decoding processing related to the global coding feature to obtain a region segmentation result of the target area, and output the region segmentation result.
[0066] The technical solution of the embodiment of the present invention processes the target coding features through the Transformer module, so as to capture the global context information of the target coding features and improve the accuracy of target region segmentation.
[0067] An optional technical solution, based on a Transformer module, expands the target coding feature to obtain feature blocks whose number is the number of channels of the target coding feature and a target position number corresponding to each feature block, including: based on the Transformer module and the number of channels of the target coding feature, expands the target coding feature to obtain feature blocks whose number is the number of channels; for each feature block, based on the order of the channels corresponding to the feature block in the channels of the number of channels, determines the target position number corresponding to the feature block.
[0068] Among them, the target coding feature can be expanded based on the Transformer module and the number of channels of the target coding feature. Optionally, the target coding feature can be expanded according to the channel dimension based on the Transformer module to obtain feature blocks with the number of channels of the target coding feature. And for each feature block, based on the order of the channels corresponding to the feature block in the channels of the number of channels, the target position number corresponding to the feature block is determined. For example, if the channel corresponding to the feature block is the 9th channel, the position number of the feature block can be numbered as 9, so that the target position number corresponding to each feature block can be obtained.
[0069] In the above technical solution, during the process of expanding the target coding features, the position information of each feature block is represented by the target position number, so as to facilitate the capture of the global context information of the target coding features.
[0070] Another optional technical solution is to encode all the obtained feature blocks and all the target position numbers based on the Transformer module to obtain a global coding feature, including: for each feature block in the feature block of the number of channels, based on the Transformer module, the feature block is expanded to obtain a one-dimensional vector of the number of channels; based on the Transformer module, the target position number and the one-dimensional vector of the number of channels corresponding to the feature block are encoded to obtain a feature vector of the number of channels; the feature vectors of the number of channels corresponding to all the feature blocks are resized to obtain a global coding feature with the same size as the target coding feature.
[0071] Among them, for each feature block in the feature block of the number of channels, based on the Transformer module, the feature block can be expanded to obtain a one-dimensional vector of the number of channels; then, based on the Transformer module, the target position number and the one-dimensional vector of the number of channels corresponding to the feature block are encoded, so that the position information of each one-dimensional vector in the target image can be determined, and then the dependency relationship between each one-dimensional vector can be determined through the self-attention mechanism of the Transformer module, so as to obtain the feature vector of the number of channels; finally, the feature vectors of the number of channels corresponding to all the feature blocks are resized to form a global coding feature with the same size as the target coding feature.
[0072] The above technical solution can effectively extract the global context information in all feature blocks by performing encoding processing based on the Transformer module.
[0073] In order to better understand the above-mentioned technical solutions as a whole, the following is an exemplary description of them in combination with specific examples. For example, in this specific example, the encoder and decoder are located in the U-Net network, and the specific model structure is as follows Figure 3 As shown, the specific steps are as follows:
[0074] S1. Input data: input pancreatic lesion medical image x (i) and the corresponding label y (i)
[0075] S2, the medical image of the pancreas x (i) Perform z-axis slicing, starting from x (i) Randomly take out a number of consecutive slices and the corresponding labels on the z-axis
[0076] S3, data enhancement, sliced image data x (i) Perform operations such as rotation, translation, and noise addition to obtain training medical images; if the operation involves spatial transformation, the corresponding label y (i) Do the same operation to get the region segmentation label
[0077] S4, image encoding, input the training medical image into the encoder of the Unet network of the pre-built initial segmentation model, use the residual convolution network of the encoder to continuously downsample and extract features of the input, and finally output the final target encoding features in the last layer of the encoder network The dimension of this feature is c L × L ×d L , where c L Represents the number of channels, w L and d L Represents the two-dimensional dimensions of the target encoding feature
[0078] S5, Transformer network will Expand by channel dimension and get c L feature patches, and number each patch according to the channel order of the feature, and obtain the target position number of each patch, and then further expand each patch into a one-dimensional vector, so that c L With (w L ×d L ) dimension, and finally c L With (w L ×d L )-dimensional one-dimensional vector and the target position number corresponding to each one-dimensional vector are input into the Transformer network, and the Transformer self-attention is used to calculate the features, and finally c is obtained.L With (w L ×d L )-dimensional feature vectors, and finally these feature vectors are resized to obtain the global encoding features with the same size as the original encoding features.
[0079] S6, image decoding, convert S5 The input is sent to the decoder of Unet, and the decoding operation is performed after being concatenated with the intermediate encoding features corresponding to the size in each decoding layer of the decoder in the channel dimension to obtain the training segmentation area.
[0080] S8, loss calculation, by training segmentation region and region segmentation label y (i) For comparison, the model loss is calculated using the cross entropy loss, and finally the model parameters are updated using the loss.
[0081] S9. Select a regional segmentation model from the trained initial segmentation model based on the validation set.
[0082] S10. Import the region segmentation model.
[0083] S11. Input brain CT and / or MRI images (ie, target medical images) into the region segmentation model, and output the region segmentation results.
[0084] In the above specific example, through the Unet network, a regional segmentation model can be trained with fewer training medical images. Through the Transformer module, the global contextual dependencies of the target coding features can be captured on the basis of extracting local features of the target medical image to obtain the target coding features, thereby achieving accurate segmentation of the target region with fewer training medical images.
[0085] Figure 4 : is a structural block diagram of a target region segmentation device provided in an embodiment of the present invention, and the device is used to execute the target region segmentation method provided in any of the above embodiments. The device and the target region segmentation method of the above embodiments belong to the same inventive concept, and the details not described in detail in the embodiment of the target region segmentation device can refer to the embodiment of the above target region segmentation method. Figure 4 The device may specifically include: a region segmentation model acquisition module 310, a target medical image input module 320, a target coding feature acquisition module 330, a global coding feature acquisition module 340 and a region segmentation result acquisition module 350. Among them,
[0086] A region segmentation model acquisition module 310 is used to acquire a target medical image containing a target region to be segmented, and a region segmentation model, wherein the region segmentation model includes an encoder, a decoder jump-connected to the encoder, and a Transformer module connected to the encoder and the decoder respectively;
[0087] The target medical image input module 320 is used to input the target medical image into the region segmentation model, so as to segment the target region from the target medical image by using the region segmentation model through the following three steps:
[0088] A target coding feature obtaining module 330 is used to perform coding processing on the target medical image based on the encoder to obtain the target coding feature;
[0089] A global coding feature obtaining module 340 is used to encode the target coding feature based on the Transformer module to obtain a global coding feature;
[0090] The region segmentation result obtaining module 350 is used to perform decoding processing related to the global coding feature based on the decoder, obtain the region segmentation result of the target region, and output the region segmentation result.
[0091] An optional global encoding feature obtaining module 340 includes:
[0092] The target position number obtaining submodule is used to expand the target coding feature based on the Transformer module to obtain feature blocks with the number of channels of the target coding feature and the target position number corresponding to each feature block;
[0093] The global coding feature acquisition submodule is used to encode all the feature blocks and all the target position numbers obtained based on the Transformer module to obtain the global coding features.
[0094] On this basis, an optional, target position numbering submodule is obtained, including:
[0095] A feature block obtaining unit is used to expand the target coding feature based on the Transformer module and the number of channels of the target coding feature to obtain feature blocks whose number is the number of channels;
[0096] The target position number determination unit is used to determine, for each feature block, the target position number corresponding to the feature block based on the order of the channels corresponding to the feature block in the number of channels.
[0097] Another optional submodule for obtaining global encoding features includes:
[0098] A one-dimensional vector obtaining unit is used to expand each feature block in the feature block of the number of channels based on the Transformer module to obtain a one-dimensional vector of the number of channels;
[0099] A feature vector obtaining unit is used to encode the one-dimensional vector of the target position number and the number of channels corresponding to the feature block based on the Transformer module to obtain a feature vector of the number of channels;
[0100] The global coding feature obtaining unit is used to transform the size of the feature vectors of the number of channels corresponding to all feature blocks, so as to obtain the global coding feature with the same size as the target coding feature.
[0101] Alternatively, the target coding feature obtaining module 330 includes:
[0102] The target coding feature obtaining submodule is used to encode the target medical image based on the encoder, and use the bottom coding features output by the bottom coding layer at the bottom of the encoder as the target coding features.
[0103] On this basis, optionally, a region segmentation result obtaining module 350 includes:
[0104] A submodule for obtaining concatenated coding features is used to concatenate the global coding features with the intermediate coding features output by the intermediate coding layer for the intermediate coding layer adjacent to the bottom coding layer in the encoder to obtain concatenated coding features;
[0105] The region segmentation result obtaining submodule is used to input the spliced coding features into the decoder so as to use the decoder to decode the spliced coding features and obtain the region segmentation result of the target region.
[0106] Alternatively, the target area segmentation device further includes:
[0107] An initial segmentation model acquisition module, used to acquire an initial segmentation model, a training medical image, and a regional segmentation label corresponding to the training medical image, wherein the model structure of the initial segmentation model is the same as that of the regional segmentation model;
[0108] A training segmentation region obtaining module is used to input the training medical image into the initial segmentation model to obtain the training segmentation region;
[0109] Model loss calculation module, used to compare the training segmentation region with the region segmentation label and calculate the model loss;
[0110] The region segmentation model obtaining module is used to adjust the model parameters in the initial segmentation model based on the model loss to train the region segmentation model.
[0111] Based on any of the above devices, optionally, the target area is a pancreatic space-occupying area, and the target medical image is a brain computed tomography image or a magnetic resonance imaging image.
[0112] The target region segmentation device provided by the embodiment of the present invention obtains a target medical image containing a target region to be segmented and a region segmentation model through a region segmentation model acquisition module, wherein the region segmentation model includes an encoder, a decoder jump-connected to the encoder, and a Transformer module respectively connected to the encoder and the decoder. The encoder and the decoder based on the jump connection can realize the training of the model when there are fewer training medical images; the target medical image is input into the region segmentation model through the target medical image input module, so as to use the region segmentation model to segment the target region from the target medical image through the following three steps: through the target coding feature acquisition module, based on the encoder, the target medical image is encoded to obtain the target coding feature, and the encoding process can improve the efficiency and accuracy of the target medical image feature extraction; through the global coding feature acquisition module, based on the Transformer module, the target coding feature is encoded to obtain the global coding feature, so as to capture the global context dependency of the target coding feature through the self-attention mechanism of the Transformer module, and further improve the accuracy of feature extraction; through the region segmentation result acquisition module, based on the decoder, the global coding feature-related decoding process is performed to obtain the region segmentation result of the target region, and the region segmentation result is output to realize the accurate segmentation of the target region. The above-mentioned device, through the jump-connected encoder and decoder, can train a region segmentation model with fewer training medical images. Through the Transformer module, it can capture the global contextual dependencies of the target coding features on the basis of extracting local features of the target medical image to obtain the target coding features, thereby achieving accurate segmentation of the target region with fewer training medical images.
[0113] The target region segmentation device provided in the embodiment of the present invention can execute the target region segmentation method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.
[0114] It is worth noting that in the embodiment of the above-mentioned target area segmentation device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.
[0115] Figure 5A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.
[0116] like Figure 5 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.
[0117] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0118] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor 11 executes the various methods and processes described above, such as the target area segmentation method.
[0119] In some embodiments, the target region segmentation method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the target region segmentation method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the target region segmentation method in any other appropriate manner (e.g., by means of firmware).
[0120] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0121] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.
[0122] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0123] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).
[0124] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.
[0125] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.
[0126] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.
[0127] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. A target region segmentation method, characterized in that: include: Acquire a target medical image containing a target region to be segmented, and a region segmentation model, wherein the region segmentation model comprises an encoder, a decoder jump-connected to the encoder, and a Transformer module respectively connected to the encoder and the decoder; The target medical image is input into the region segmentation model, so as to segment the target region from the target medical image by using the region segmentation model through the following three steps: Based on the encoder, encoding the target medical image to obtain a target encoding feature; Based on the Transformer module, the target coding feature is encoded to obtain a global coding feature; Based on the decoder, a decoding process related to the global coding feature is performed to obtain a region segmentation result of the target region, and the region segmentation result is output.
2. The method according to claim 1, characterized in that The encoding process of the target encoding feature based on the Transformer module to obtain the global encoding feature includes: Based on the Transformer module, the target coding feature is expanded to obtain feature blocks whose number is the number of channels of the target coding feature and the target position number corresponding to each feature block; Based on the Transformer module, all the feature blocks and all the target position numbers are encoded to obtain global encoding features.
3. The method according to claim 2, characterized in that The target coding feature is expanded based on the Transformer module to obtain feature blocks whose number is the number of channels of the target coding feature and target position numbers corresponding to each feature block, including: Based on the Transformer module and the number of channels of the target coding feature, the target coding feature is expanded to obtain feature blocks whose number is the number of channels; For each of the feature blocks, a target position number corresponding to the feature block is determined based on the order of the channels corresponding to the feature block in the number of channels.
4. The method according to claim 2, characterized in that: Based on the Transformer module, encoding is performed on all the obtained feature blocks and all the target position numbers to obtain global encoding features, including: For each feature block in the feature blocks of the number of channels, based on the Transformer module, the feature block is expanded to obtain a one-dimensional vector of the number of channels; Based on the Transformer module, encoding the target position number corresponding to the feature block and the one-dimensional vector of the number of channels is performed to obtain a feature vector of the number of channels; The feature vectors of the number of channels corresponding to all the feature blocks are resized to obtain a global coding feature with the same size as the target coding feature.
5. The method according to claim 1, characterized in that The encoding process of the target medical image based on the encoder to obtain a target encoding feature includes: Based on the encoder, the target medical image is encoded, and the bottom-level encoding features output by the bottom-level encoding layer in the encoder are used as target encoding features.
6. The method according to claim 5, characterized in that The decoding process related to the global coding feature is performed based on the decoder to obtain the region segmentation result of the target region, including: For an intermediate coding layer adjacent to the bottom coding layer in the encoder, concatenating the global coding feature with an intermediate coding feature output by the intermediate coding layer to obtain a concatenated coding feature; The spliced coding features are input into the decoder, so as to use the decoder to decode the spliced coding features and obtain a region segmentation result of the target region.
7. The method according to claim 1, characterized in that The region segmentation model is pre-trained by the following steps: Acquire an initial segmentation model, a training medical image, and a region segmentation label corresponding to the training medical image, wherein the model structure of the initial segmentation model is the same as that of the region segmentation model; Inputting the training medical image into the initial segmentation model to obtain a training segmentation region; Comparing the training segmentation region with the region segmentation label, and calculating the model loss; The model parameters in the initial segmentation model are adjusted based on the model loss to train the region segmentation model.
8. The method according to any one of claims 1 to 7, characterized in that The target region is a pancreatic space-occupying region, and the target medical image is a brain computed tomography image or a magnetic resonance imaging image.
9. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor performs the target region segmentation method according to any one of claims 1 to 8.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the target region segmentation method according to any one of claims 1 to 8 when executed.