Tumor image segmentation method and system

Abstract

The invention relates to a tumor image segmentation method and system, and the method comprises the steps: constructing a tumor image segmentation model which comprises a coarse segmentation block, a U-shaped Transform backbone network, a segmentation head, an uncertainty head and a boundary head; training the tumor image segmentation model through the training data; a clinical tumor image is input into the trained tumor image segmentation model to obtain a segmentation result, an uncertainty head is used for generating a conditional probability mask to introduce uncertain information brought by a fuzzy boundary, a boundary head is used for generating a boundary mask to introduce foreground information, and a boundary head is used for generating a boundary mask to introduce foreground information; and the segmentation head is used for obtaining a final segmentation result according to the conditional probability mask and the boundary mask. According to the method and the device, the uncertain information and the foreground information are introduced through the uncertain head and the boundary head, so that the segmentation head can be better helped to process boundary blur and uncertainty, the segmentation accuracy is further improved, and the problem of how to improve the tumor image segmentation accuracy is solved.

Description

Technical Field

[0001] The present application relates to the technical field of medical image processing, and in particular to a tumor image segmentation method and system. Background Art

[0002] Tumor image segmentation refers to the identification of tumor areas from magnetic resonance imaging. Magnetic resonance imaging uses multiple modalities in order to capture different tissue characteristics and tumor diffusion. Multimodal data provides rich information, but also introduces redundant information, because some modalities may contain features that are not related to the tissue of interest, thus affecting the segmentation results. At the same time, different levels of tumor invasion into surrounding tissues will lead to differences and uncertainties in boundary shapes, making the boundaries of tumors in the image blurred. This boundary blur will also reduce the performance of segmentation methods. Existing methods are usually based on clear boundary segmentation, ignoring the boundary blur that is prevalent in actual situations, thus affecting the segmentation effect.

[0003] Currently, no effective solution has been proposed for the problem of how to improve the accuracy of tumor image segmentation in related technologies. Summary of the invention

[0004] The embodiments of the present application provide a tumor image segmentation method and system to at least solve the problem of how to improve the accuracy of tumor image segmentation in the related art.

[0005] In a first aspect, an embodiment of the present application provides a tumor image segmentation method, the method comprising:

[0006] Constructing a tumor image segmentation model, wherein the tumor image segmentation includes a coarse segmentation block, a U-shaped Transformer backbone network, a segmentation head, an uncertainty head, and a boundary head;

[0007] Acquiring training data, and training the tumor image segmentation model using the training data to obtain a trained tumor image segmentation model;

[0008] The clinical tumor image is input into the trained tumor image segmentation model to obtain a segmentation result, wherein the uncertainty head is used to generate a conditional probability mask to introduce uncertainty information brought by the fuzzy boundary, the boundary head is used to generate a boundary mask to introduce foreground information, and the segmentation head is used to obtain the final segmentation result based on the conditional probability mask and the boundary mask.

[0009] In some embodiments, the tumor image segmentation model is trained by the training data to obtain a trained tumor image segmentation model, which includes:

[0010] Performing a first-stage training on the tumor image segmentation model using the training data to obtain a tumor image segmentation model after the first-stage training;

[0011] Outputting a boundary mask of the training data through the boundary head of the tumor image segmentation model trained in the first stage;

[0012] Outputting a conditional probability mask of the training data through the uncertainty head of the tumor image segmentation model trained in the first stage;

[0013] Based on the boundary mask, the conditional probability mask and the prior probability mask, an exit mask is calculated;

[0014] Calculating an uncertainty map based on the training data and the exit mask;

[0015] The tumor image segmentation model trained in the first stage is trained in the second stage using the uncertainty graph to obtain a trained tumor image segmentation model.

[0016] In some embodiments, outputting the boundary mask of the training data through the boundary head of the tumor image segmentation model trained in the first stage includes:

[0017] The boundary head of the tumor image segmentation model after the first stage of training is calculated by the boundary mask formula Output the boundary mask u of the training data (1) , where F i represents the sum of the rough segmentation results of the i-th rough segmentation block, T represents the preset threshold, Represents the i-th boundary mask value.

[0018] In some embodiments, based on the boundary mask, the conditional probability mask and the prior probability mask, calculating the exit mask includes:

[0019] Based on the boundary mask, the conditional probability mask and the prior probability mask, the formula Calculate the exit mask, where u (1) represents the boundary mask, u (2) represents the prior probability mask, u (2) Obeying Bernoulli distribution, u (3) represents the conditional probability mask, u (3) It obeys the jointly independent normal distribution.

[0020] In some embodiments, an uncertainty map is calculated based on the training data and the exit mask.

[0021] Based on the training data and the exit mask, the formula The uncertainty map is calculated, where u represents the exit mask and x represents the training data.

[0022] In some embodiments, the tumor image segmentation model is trained in the first stage using the training data to obtain the tumor image segmentation model after the first stage training, including:

[0023] The first stage of training is performed on the U-shaped Transformer backbone network, the uncertainty head and the boundary head of the tumor image segmentation model using the training data, and the network parameters of the U-shaped Transformer backbone network, the uncertainty head and the boundary head are updated to obtain the tumor image segmentation model after the first stage of training.

[0024] In some embodiments, updating the network parameters of the U-shaped Transformer backbone network, the uncertainty head, and the boundary head includes:

[0025] Using a first preset loss function, network parameters of the U-shaped Transformer backbone network, the uncertainty head, and the boundary head are updated, wherein the first preset loss function is a weighted combination of a cross entropy loss function, a soft dice loss function, and a KL loss function.

[0026] In some embodiments, performing a second-stage training on the tumor image segmentation model trained in the first stage by using the uncertainty graph to obtain a trained tumor image segmentation model includes:

[0027] The Transformer backbone network and the segmentation head of the tumor image segmentation model are trained in the second stage through the uncertainty graph, and the network parameters of the U-shaped Transformer backbone network and the segmentation head are updated to obtain a trained tumor image segmentation model.

[0028] In some embodiments, updating the network parameters of the U-shaped Transformer backbone network and the segmentation head includes:

[0029] The network parameters of the U-shaped Transformer backbone network and the segmentation head are updated by using a second preset loss function, wherein the second preset loss function is a weighted combination of a cross entropy loss function and a soft dice loss function.

[0030] In some embodiments, obtaining training data includes:

[0031] Obtain training data from open source datasets;

[0032] Enhanced tumor labeling, tumor core labeling and full tumor labeling are performed on the training data to obtain labeled training data.

[0033] In some embodiments, the clinical tumor image is input into the trained tumor image segmentation model to obtain the segmentation result, including:

[0034] The clinical tumor image is input into the trained tumor image segmentation model to obtain a segmentation result, wherein the segmentation result includes an enhanced tumor segmentation result, a tumor core segmentation result and a whole tumor segmentation result.

[0035] In some embodiments, building a tumor image segmentation model includes:

[0036] Construct a tumor image segmentation model, where the coarse segmentation block is a 3D patch segmentation block and the U-shaped Transformer backbone network includes a U-shaped Transformer encoder and a U-shaped Transformer decoder;

[0037] The Transformer encoder consists of four swin-Transformer blocks, and the first three swin-Transformer blocks are followed by a 3D patch merging block;

[0038] The Transformer decoder consists of three swin-Transformer blocks, each of which is preceded by a 3D patch expansion block;

[0039] The boundary head, segmentation head, and uncertainty head all include convolutional layers and ReLU layers, and the uncertainty head performs a reparameterization operation to generate a conditional probability mask.

[0040] In a second aspect, an embodiment of the present application provides a tumor image segmentation system, the system is used to perform the method described in any one of the first aspects above, the system includes a model construction module, a model training module and a target segmentation module;

[0041] The model building module is used to build a tumor image segmentation model, wherein the tumor image segmentation includes a coarse segmentation block, a U-shaped Transformer backbone network, a segmentation head, an uncertainty head and a boundary head;

[0042] The model training module is used to obtain training data, and train the tumor image segmentation model through the training data to obtain a trained tumor image segmentation model;

[0043] The target segmentation module is used to input the clinical tumor image into the trained tumor image segmentation model to obtain a segmentation result, wherein the uncertainty head is used to generate a conditional probability mask to introduce uncertainty information brought by the fuzzy boundary, the boundary head is used to generate a boundary mask to introduce foreground information, and the segmentation head is used to obtain the final segmentation result based on the conditional probability mask and the boundary mask.

[0044] Compared with the related art, the embodiment of the present application provides a tumor image segmentation method and system, wherein the method constructs a tumor image segmentation model, wherein the tumor image segmentation includes a coarse segmentation block, a U-shaped Transformer backbone network, a segmentation head, an uncertainty head and a boundary head; obtains training data, trains the tumor image segmentation model through the training data, and obtains a trained tumor image segmentation model; inputs the clinical tumor image into the trained tumor image segmentation model to obtain a segmentation result, wherein the uncertainty head is used to generate a conditional probability mask to introduce uncertainty information brought by the fuzzy boundary, the boundary head is used to generate a boundary mask to introduce foreground information, and the segmentation head is used to obtain the final segmentation result based on the conditional probability mask and the boundary mask, thereby realizing the introduction of uncertainty information and foreground information through the uncertainty head and the boundary head, which can better help the segmentation head to deal with boundary fuzziness and uncertainty, thereby improving segmentation accuracy, and solving the problem of how to improve the accuracy of tumor image segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0046] Figure 1 is a flowchart of the steps of the tumor image segmentation method according to an embodiment of the present application;

[0047] Figure 2 is a schematic diagram of the network structure of a tumor image segmentation model according to an embodiment of the present application;

[0048] Figure 3 is a flow chart of a tumor image segmentation method according to an embodiment of the present application;

[0049] Figure 4 is a structural block diagram of a tumor image segmentation system according to an embodiment of the present application;

[0050] Figure 5 It is a schematic diagram of the internal structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments provided in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0052] Obviously, the drawings described below are only some examples or embodiments of the present application. For ordinary technicians in this field, the present application can also be applied to other similar scenarios based on these drawings without creative work. In addition, it can also be understood that although the efforts made in this development process may be complicated and lengthy, for ordinary technicians in this field related to the content disclosed in this application, some changes in design, manufacturing or production based on the technical content disclosed in this application are just conventional technical means, and should not be understood as insufficient content disclosed in this application.

[0053] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those of ordinary skill in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0054] Unless otherwise defined, the technical terms or scientific terms involved in this application should be understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the" and the like involved in this application do not indicate a quantitative limitation, and may represent the singular or plural. The terms "include", "comprise", "have" and any of their variations involved in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships, for example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects before and after are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0055] The present application embodiment provides a tumor image segmentation method. Figure 1 is a flowchart of the steps of the tumor image segmentation method according to an embodiment of the present application, such as Figure 1 As shown, the method comprises the following steps:

[0056] Step S102, constructing a tumor image segmentation model, wherein the tumor image segmentation includes a coarse segmentation block, a U-shaped Transformer backbone network, a segmentation head, an uncertainty head, and a boundary head;

[0057] Specifically, step S102: Figure 2 is a schematic diagram of the network structure of the tumor image segmentation model according to an embodiment of the present application, such as Figure 2As shown in the figure, the coarse segmentation block is a 3D patch segmentation block, the U-shaped Transformer backbone network includes a U-shaped Transformer encoder and a U-shaped Transformer decoder; the Transformer encoder includes four swin-Transformer blocks, and the first three swin-Transformer blocks are followed by a 3D patch merging block; the Transformer decoder includes three swin-Transformer blocks, and each swin-Transformer block is preceded by a 3D patch expansion block; the boundary head, segmentation head and uncertainty head all include convolutional layers and ReLU layers, and the uncertainty head generates a conditional probability mask by performing a re-parameterization operation.

[0058] The U-shaped Transformer in the backbone extracts self-attention features at different scales. The feature map of the encoder is input to the decoder through a jump connection between the U-shaped Transformer encoder and decoder. The three heads connected to the decoder are used for boundary tasks, uncertainty tasks, and segmentation tasks respectively.

[0059] Step S104, obtaining training data, and training the tumor image segmentation model using the training data to obtain a trained tumor image segmentation model;

[0060] Step S104 specifically includes the following steps:

[0061] Step S1041, performing first-stage training on the tumor image segmentation model using training data to obtain a tumor image segmentation model after first-stage training;

[0062] Step S1041, specifically, the training of the tumor image segmentation model is supervised training, and the training data can be obtained from open source data sets, such as BraTS2020, BraTS2021, etc.; the training data can be enhanced to generate more training samples, such as randomly mirroring the training data on three axes with a probability of 50%, randomly adjusting the intensity of each channel of the training data in the range of 0.9-1.1, and randomly shifting the intensity of each pixel of each channel in the range of -0.1-0.1, thereby achieving the effect of data enhancement; the training data is enhanced with tumor annotation (ET), tumor core annotation (TC) and whole tumor annotation (WT) to obtain labeled training data;

[0063] It should be noted that the processing flow of training data or clinical data in the tumor image segmentation model is as follows: multimodal MRI data of size D×H×W×C is transformed into The size is then sent to the U-type Transformer. In this embodiment, C is 96;

[0064] The input feature map passes through the 3D patch merging block and a linear layer in each layer of the encoder to halve the feature map height and width, double the number of feature channels, and finally passes through the standard swin-Transformer block for feature transformation. This is repeated 4 times and the final encoder output size is The feature map of

[0065] The feature map is transformed by the standard swin-Transformer block at each scale of the decoder, and finally passes through a 3D patch expansion block and a linear layer to double the height and width of the feature map and halve the number of feature channels. This is repeated three times and then upsampled to finally output a feature map of size D×H×W×C at the decoder.

[0066] The output features of the decoder are input to the boundary head and uncertainty head respectively; the boundary head outputs the tumor rough segmentation boundary mask u (1) , whose size is D×H×W×1; the uncertainty head outputs the conditional probability mask u (3) , its size is D×H×W×4;

[0067] The first stage of training is performed on the U-shaped Transformer backbone network, uncertainty head, and boundary head of the tumor image segmentation model using the training data;

[0068] The first stage of training the network involves the part of the exit mask (uncertainty head F u (including mean head Variance Head ), Network backbone F B and border header F b ), for supervised training, the training labels are the stacking of binary masks of three sub-regions of the tumor (enhanced tumor (ET), tumor core (TC), and whole tumor (WT)) along the channel direction.

[0069] In the first stage of training, the first preset loss function is used to update the network parameters of the U-shaped Transformer backbone network, the uncertainty head and the boundary head to obtain the tumor image segmentation model after the first stage of training, wherein the first preset loss function is a weighted combination of the cross entropy loss function, the soft dice loss function and the KL loss function. Specifically, the loss function is as follows:

[0070]

[0071] λ3(L CE (c,y)+L KL (N(μ,σ),N(0,I))))

[0072] where χ={(x1,y1),...,(x n ,y n )} are n pairs of multimodal data and corresponding tumor labels, x represents the multimodal data, y represents the corresponding tumor label, λ1, λ2, λ3 are weight values, preferably set to 0.7, 0.3 and 0.2 respectively, c is the result sampled from the reparameterization operation in S1043, μ and σ are and Output. b o F B The o in (x) is a compound operation, and x is first input into the backbone network F B , the output of the backbone network is then input to the boundary head F b N(0,I) is a normal distribution with a mean of 0 and a variance of 1, and N(μ,σ) is a normal distribution with a mean of μ and a variance of σ.

[0073] Step S1042, outputting a boundary mask of the training data through the boundary head of the tumor image segmentation model trained in the first stage;

[0074] Specifically, step S1042: Figure 3 FIG. 1 is a flow chart of a tumor image segmentation method according to an embodiment of the present application. Figure 3 As shown in the figure, the boundary head of the tumor image segmentation model after the first stage of training is calculated by the boundary mask formula Output the boundary mask u of the training data (1) , where F i represents the sum of the rough segmentation results of the i-th rough segmentation block, T represents the preset threshold, Represents the boundary mask value of the i-th patch, all Splice to get u (1) .

[0075] Step S1043, outputting the conditional probability mask of the training data through the uncertainty head of the tumor image segmentation model trained in the first stage;

[0076] Specifically, step S1043 is as follows: Figure 3 As shown, it is assumed that the foreground probability c follows a joint independent normal distribution P(c|μ,σ):c i :Ν(μ i , σ i ),in Represent the mean and variance of the input data respectively, and the uncertainty head (Uncertainty Head) F of the tumor image segmentation model after the first stage of training u Learn the distribution of mean and variance, i.e. in and F u The two branches in Figure 2 As shown, F B is the backbone network, and x is the input data. Perform reparameterization operation Among them, ε: N (0, I) follows a normal distribution with a mean of 0 and a variance of 1, and μ and σ are and The output of K random reparameterization operations calculate Normalized variance That is, we get the conditional probability mask u of the training data (3) , the i-th voxel value Bernoulli distribution where i represents the i-th voxel of the normalized variance V.

[0077] Step S1044, calculating an exit mask based on the boundary mask, the conditional probability mask and the prior probability mask;

[0078] Step S1044 is specifically as follows: Figure 3 As shown, based on the boundary mask, conditional probability mask and prior probability mask, the formula Calculate the drop-out mask, where u (1) represents the boundary mask, u (2) represents the prior probability mask, u (3) Represents a conditional probability mask.

[0079] Furthermore, the prior probability mask u (2) is preset, u (2) The prior probability mask value on the i-th patch follows the Bernoulli distribution Here, p is set based on experience and is set to 0.5 in this embodiment.

[0080] Step S1045, calculating an uncertainty map based on the training data and the exit mask;

[0081] Step S1045 specifically, as Figure 3 As shown, based on the training data and the exit mask, the formula The uncertainty map is calculated, where u represents the exit mask and x represents the training data.

[0082] Step S1046, performing a second stage of training on the tumor image segmentation model trained in the first stage by using the uncertainty graph to obtain a trained tumor image segmentation model.

[0083] Step S1046 specifically, performing the second phase training on the U-shaped Transformer backbone network and the segmentation head of the tumor image segmentation model through the uncertainty graph;

[0084] The second stage of training involves the final segmentation of the network (segmentation head F S and network backbone F B ), for supervised training, the training labels are the stacking of binary masks of three sub-regions of the tumor (enhanced tumor (ET), tumor core (TC), and whole tumor (WT)) along the channel direction.

[0085] In the second stage of training, the second preset loss function is used to update the network parameters of the U-shaped Transformer backbone network, uncertainty head and boundary head to obtain a trained tumor image segmentation model, wherein the second preset loss function is a weighted combination of the cross entropy loss function and the soft dice loss function. Specifically, the loss function is as follows:

[0086]

[0087] Among them, x represents multimodal data, y represents the corresponding tumor label, and F B is the backbone network, F S Segmentation head, u is the exit mask, λ1, λ2 are weight values, preferably set to 0.7 and 0.3 respectively. s o F B The o in (x) is a compound operation, and x is first input into the backbone network F B , and then the output of the backbone network is input to the segmentation head F s χ={(x1,y1),...,(x n ,y n )} are n pairs of multimodal data and corresponding tumor labels. It is voxel-level multiplication, and the voxels corresponding to x and u are multiplied.

[0088] Step S106, input the clinical tumor image into the trained tumor image segmentation model to obtain the segmentation result, where Figure 3 As shown in the figure, the uncertainty head is used to generate a conditional probability mask to introduce the uncertainty information brought by the fuzzy boundary, the boundary head is used to generate a boundary mask to introduce foreground information, and the segmentation head is used to obtain the final segmentation result based on the conditional probability mask and the boundary mask.

[0089] Specifically, step S106 inputs the clinical tumor image into the trained tumor image segmentation model to obtain a segmentation result, wherein the segmentation result includes an enhanced tumor segmentation (ET) result, a tumor core (TC) segmentation result, and a whole tumor (WT) segmentation result.

[0090] Through the above steps in the embodiment of the present application, the introduction of uncertainty information and foreground information through the uncertainty head and the boundary head is achieved, which can better help the segmentation head to deal with boundary fuzziness and uncertainty, thereby improving the segmentation accuracy and solving the problem of how to improve the accuracy of tumor image segmentation.

[0091] It should be noted that the steps shown in the above process or the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0092] In some of the embodiments, the embodiments of the present application provide an experimental comparison between the "tumor image segmentation method" of the present application and the existing tumor image segmentation method.

[0093] Experiment 1: The quantitative and qualitative results of this application are compared with those of six currently advanced multimodal brain tumor segmentation methods (3DUnet, SEVnet, DMFnet, nnUnet, TransBTS and VTUnet) on three sub-regions: enhanced tumor (ET), tumor core (TC) and whole tumor (WT).

[0094] Experiment 2: This application compares the quantitative results of the three sub-regions of enhanced tumor, tumor core and whole tumor under different exit masks.

[0095] Experimental conditions:

[0096] Test data: The test data for Experiment 1 are 125 test data from BraTS (2020) and 417 test data from BraTS (2021), each of which contains four modalities: T1, T1ce, T2, and Flair. The test data for Experiment 2 are the test data from the Brain Tumor Challenge (2021).

[0097] Evaluation index: Dice similarity coefficent and 95% Hausdorff distance were used as evaluation indexes for segmentation results in Experiment 1. Dice similarity coefficent was used as evaluation index for segmentation results in Experiment 2.

[0098] Experimental results and analysis:

[0099] As shown in Tables 1 and 2, Tables 1 and 2 give the DSC and HD indicators of the segmentation results of the three sub-regions of the brain tumor in Experiment 1 on eight methods, where Ours represents this application, and Ours (No Mask) represents that the network model of this application is trained without uncertainty guidance. The segmentation results obtained by training this application without uncertainty guidance still have competitive performance compared with the other six more advanced segmentation methods. Compared with nnUnet, the exit mask method designed by this application increased the DSC of ET, TC, and WT on BraTS (2021) by 0.23%, 0.03%, and 0.19%, respectively. Considerable improvements have also been made in the Hausdorff distance metric.

[0100] Table 1

[0101]

[0102]

[0103] Table 2

[0104]

[0105] As shown in Table 3, Table 3 gives the segmentation result indicators of this application on BraTS (2021) using different exit masks in Experiment 2, where No Mask refers to a deterministic training strategy that does not use an exit mask (referring to using only two of the boundary mask, prior probability mask, and conditional probability mask). Random (p = 0.95) means using a random mask, in which each voxel obeys a Bernoulli distribution with parameter p. It can be seen that the exit mask of this application can help the model significantly improve the segmentation accuracy. Compared with the unmasked method, the exit mask of this application brings an average DSC improvement of 1.05%. The conditional probability mask generated by the uncertainty map plays a major promoting role. In the method that does not use the conditional probability mask, the DSC drops by 0.79%.

[0106] Table 3

[0107]

[0108] The present application embodiment provides a tumor image segmentation system. Figure 4 is a structural block diagram of a tumor image segmentation system according to an embodiment of the present application, such as Figure 4 As shown, the system includes a model building module, a model training module and a target segmentation module;

[0109] A model building module is used to build a tumor image segmentation model, wherein the tumor image segmentation includes a coarse segmentation block, a U-shaped Transformer backbone network, a segmentation head, an uncertainty head, and a boundary head;

[0110] A model training module is used to obtain training data, and train the tumor image segmentation model through the training data to obtain a trained tumor image segmentation model;

[0111] The target segmentation module is used to input clinical tumor images into the trained tumor image segmentation model to obtain the segmentation results, wherein the uncertainty head is used to generate a conditional probability mask to introduce the uncertainty information brought by the fuzzy boundary, the boundary head is used to generate a boundary mask to introduce the foreground information, and the segmentation head is used to obtain the final segmentation result based on the conditional probability mask and the boundary mask.

[0112] Through the model building module, model training module and target segmentation module in the embodiments of the present application, it is possible to introduce uncertainty information and foreground information through the uncertainty head and the boundary head, which can better help the segmentation head to deal with boundary blur and uncertainty, thereby improving segmentation accuracy and solving the problem of how to improve the accuracy of tumor image segmentation.

[0113] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0114] This embodiment further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0115] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0116] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0117] In addition, in combination with the tumor image segmentation method in the above embodiment, the embodiment of the present application can provide a storage medium for implementation. The storage medium stores a computer program; when the computer program is executed by a processor, any tumor image segmentation method in the above embodiment is implemented.

[0118] In one embodiment, a computer program product is provided, including a computer program, characterized in that when the computer program is executed by a processor, any one of the tumor image segmentation methods in the above embodiments is implemented.

[0119] In one embodiment, a computer device is provided, which may be a terminal. The computer device includes a processor, a memory, a network interface, a display screen, and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a tumor image segmentation method is implemented. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covered on the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0120] In one embodiment, Figure 5 is a schematic diagram of the internal structure of an electronic device according to an embodiment of the present application, such as Figure 5 As shown, an electronic device is provided, which may be a server, and its internal structure diagram may be as shown in Figure 5 As shown. The electronic device includes a processor, a network interface, an internal memory and a non-volatile memory connected through an internal bus, wherein the non-volatile memory stores an operating system, a computer program and a database. The processor is used to provide computing and control capabilities, the network interface is used to communicate with an external terminal through a network connection, the internal memory is used to provide an environment for the operation of the operating system and the computer program, the computer program is executed by the processor to implement a tumor image segmentation method, and the database is used to store data.

[0121] Those skilled in the art will understand that Figure 5 The structure shown in the figure is merely a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the electronic device to which the scheme of the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0122] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0123] Those skilled in the art should understand that the technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0124] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A tumor image segmentation method, characterized in that: The method comprises: Constructing a tumor image segmentation model, wherein the tumor image segmentation model includes a coarse segmentation block, a U-shaped Transformer backbone network, a segmentation head, an uncertainty head, and a boundary head; Acquiring training data, and training the tumor image segmentation model using the training data to obtain a trained tumor image segmentation model; The clinical tumor image is input into the trained tumor image segmentation model to obtain a segmentation result, wherein the uncertainty head is used to generate a conditional probability mask to introduce uncertainty information brought by the fuzzy boundary, the boundary head is used to generate a boundary mask to introduce foreground information, and the segmentation head is used to obtain the final segmentation result based on the conditional probability mask and the boundary mask.

2. The method according to claim 1, characterized in that: The tumor image segmentation model is trained by using the training data to obtain a trained tumor image segmentation model, which includes: Performing a first-stage training on the tumor image segmentation model using the training data to obtain a tumor image segmentation model after the first-stage training; Outputting a boundary mask of the training data through the boundary head of the tumor image segmentation model trained in the first stage; Outputting a conditional probability mask of the training data through the uncertainty head of the tumor image segmentation model trained in the first stage; Based on the boundary mask, the conditional probability mask and the prior probability mask, an exit mask is calculated; Calculating an uncertainty map based on the training data and the exit mask; The tumor image segmentation model trained in the first stage is trained in the second stage using the uncertainty graph to obtain a trained tumor image segmentation model.

3. The method according to claim 2, characterized in that Outputting the boundary mask of the training data through the boundary head of the tumor image segmentation model trained in the first stage includes: The boundary head of the tumor image segmentation model after the first stage of training is calculated by the boundary mask formula Output the boundary mask u of the training data (1) , where F i represents the sum of the rough segmentation results of the i-th rough segmentation block, T represents the preset threshold, Represents the i-th boundary mask value.

4. The method according to claim 2, characterized in that: Based on the boundary mask, the conditional probability mask and the prior probability mask, calculating the exit mask includes: Based on the boundary mask, the conditional probability mask and the prior probability mask, the formula Calculate the exit mask, where u (1) represents the boundary mask, u (2) represents the prior probability mask, u (2) Obeying Bernoulli distribution, u (3) represents the conditional probability mask, u (3) It obeys the jointly independent normal distribution.

5. The method according to claim 2, characterized in that: Based on the training data and the exit mask, an uncertainty map is calculated Based on the training data and the exit mask, the formula The uncertainty map is calculated, where u represents the exit mask and x represents the training data.

6. The method according to claim 2, characterized in that The tumor image segmentation model is trained in the first stage by using the training data, and the tumor image segmentation model obtained after the first stage training includes: The U-shaped Transformer backbone network, uncertainty head and boundary head of the tumor image segmentation model are trained in the first stage using the training data, and the network parameters of the Transformer backbone network, the uncertainty head and the boundary head are updated to obtain the tumor image segmentation model after the first stage training.

7. The method according to claim 6, characterized in that Updating the network parameters of the U-shaped Transformer backbone network, the uncertainty head, and the boundary head includes: Using a first preset loss function, network parameters of the U-shaped Transformer backbone network, the uncertainty head, and the boundary head are updated, wherein the first preset loss function is a weighted combination of a cross entropy loss function, a soft dice loss function, and a KL loss function.

8. The method according to claim 2, characterized in that: The tumor image segmentation model trained in the first stage is trained in the second stage by using the uncertainty graph to obtain a trained tumor image segmentation model, which includes: The U-shaped Transformer backbone network and the segmentation head of the tumor image segmentation model are trained in the second stage through the uncertainty graph, and the network parameters of the U-shaped Transformer backbone network and the segmentation head are updated to obtain a trained tumor image segmentation model.

9. The method according to claim 8, characterized in that Updating the network parameters of the U-shaped Transformer backbone network and the segmentation head includes: The network parameters of the U-shaped Transformer backbone network and the segmentation head are updated by using a second preset loss function, wherein the second preset loss function is a weighted combination of a cross entropy loss function and a soft dice loss function.

10. The method according to claim 1, characterized in that Obtaining training data includes: Get training data from open source datasets; Enhanced tumor labeling, tumor core labeling and full tumor labeling are performed on the training data to obtain labeled training data.

11. The method according to claim 8, characterized in that Inputting the clinical tumor image into the trained tumor image segmentation model, obtaining the segmentation results includes: The clinical tumor image is input into the trained tumor image segmentation model to obtain a segmentation result, wherein the segmentation result includes an enhanced tumor segmentation result, a tumor core segmentation result and a whole tumor segmentation result.

12. The method according to claim 1, characterized in that Building a tumor image segmentation model includes: Construct a tumor image segmentation model, where the coarse segmentation block is a 3D patch segmentation block and the U-shaped Transformer backbone network includes a U-shaped Transformer encoder and a U-shaped Transformer decoder; The Transformer encoder includes four swin-Transformer blocks, and the first three swin-Transformer blocks are followed by a 3D patch merging block; The Transformer decoder includes three swin-Transformer blocks, each of which is preceded by a 3D patch expansion block; The boundary head, segmentation head, and uncertainty head all include convolutional layers and ReLU layers, and the uncertainty head performs a reparameterization operation to generate a conditional probability mask.

13. A tumor image segmentation system, characterized in that: The system is used to perform the method according to any one of claims 1 to 12, and the system includes a model building module, a model training module and a target segmentation module; The model building module is used to build a tumor image segmentation model, wherein the tumor image segmentation includes a coarse segmentation block, a Transformer backbone network, a segmentation head, an uncertainty head and a boundary head; The model training module is used to obtain training data, and train the tumor image segmentation model through the training data to obtain a trained tumor image segmentation model; The target segmentation module is used to input the clinical tumor image into the trained tumor image segmentation model to obtain a segmentation result, wherein the uncertainty head is used to generate a conditional probability mask to introduce uncertainty information brought by the fuzzy boundary, the boundary head is used to generate a boundary mask to introduce foreground information, and the segmentation head is used to obtain the final segmentation result based on the conditional probability mask and the boundary mask.

14. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to perform the method according to any one of claims 1 to 12.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 12 is implemented.

16. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 12 is implemented.

Images