Model training method, tumor segmentation method and electronic equipment

Abstract

The embodiment of the invention discloses a model training method, a tumor segmentation method and electronic equipment, and the method comprises the steps: obtaining an original medical image containing a tumor; determining a gradient map of the original medical image, obtaining a seed point based on a local maximum value in the gradient map, and segmenting the original medical image by using the seed point to obtain at least two regions; respectively generating an enhancement coefficient for each region to obtain a regional enhancement image, and performing image enhancement of the original medical image according to the regional enhancement image to obtain an enhanced medical image; and training an initial segmentation model based on the original medical image and the enhanced medical image to obtain a tumor segmentation model. According to the technical scheme of the embodiment of the invention, the problem that the tumor cannot be accurately segmented from the medical image is solved.

Description

Technical Field

[0001] The embodiments of the present invention relate to the technical field of medical image processing, and in particular to a model training method, a tumor segmentation method and an electronic device. Background Art

[0002] Medical image segmentation models based on deep learning are constantly developing and improving. The model is usually based on the independent and identically distributed assumption, that is, it is assumed that the training set images and the test set images are independent and identically distributed. However, in clinical practice, differences in imaging equipment, protocols, and parameters, and even differences in patient radiation exposure and imaging time, can lead to differences in imaging distribution, which is a major obstacle to the application of the model in clinical practice.

[0003] In order to deal with the above obstacles, relevant scholars have conducted research on single-source domain generalization. Single-source domain generalization aims to train a robust model based on only one source domain image, which can be directly applied to unseen target domain images. In order to deal with domain distribution differences and improve the robustness of the model, one implementation solution is to generate new images by enhancing the source domain images, thereby simulating images from different domains and enriching the training set images.

[0004] However, the model trained based on the currently used image enhancement scheme can be applied to tissues or organs with relatively obvious segmentation boundaries, but is not suitable for tumors with unclear segmentation boundaries. In other words, it is impossible to accurately segment tumors from medical images, which needs to be solved urgently. Summary of the invention

[0005] The embodiments of the present invention provide a model training method, a tumor segmentation method and an electronic device, which solve the problem that a tumor cannot be accurately segmented from a medical image.

[0006] According to one aspect of the present invention, a model training method is provided, which may include:

[0007] Acquire original medical images containing tumors;

[0008] Determine a gradient map of the original medical image, obtain a seed point based on a local maximum value in the gradient map, and use the seed point to segment the original medical image to obtain at least two regions;

[0009] Generate an enhancement coefficient for each region to obtain a sub-region enhancement map, and perform image enhancement on the original medical image according to the sub-region enhancement map to obtain an enhanced medical image;

[0010] Based on the original medical image and the enhanced medical image, an initial segmentation model is trained to obtain a tumor segmentation model.

[0011] According to another aspect of the present invention, a tumor segmentation method is provided, which may include:

[0012] Acquire a target medical image containing a tumor and a tumor segmentation model trained according to the model training method described in any embodiment of the present invention;

[0013] The target medical image is input into the tumor segmentation model, and a target segmentation result of the target medical image for the tumor is obtained according to the output result of the tumor segmentation model.

[0014] According to another aspect of the present invention, there is provided an electronic device, which may include:

[0015] at least one processor; and

[0016] a memory communicatively connected to at least one processor; wherein,

[0017] The memory stores a computer program that can be executed by at least one processor, and the computer program is executed by at least one processor so that the at least one processor implements the model training method or tumor segmentation method provided by any embodiment of the present invention when executing.

[0018] The technical solution of the embodiment of the present invention obtains an original medical image containing a tumor; determines the gradient map of the original medical image, and then uses the local maximum value in the gradient map as a seed point to segment the original medical image using the seed point to obtain at least two regions, thereby realizing a regional division process based on anatomical information; further, generates an enhancement coefficient for each region to obtain a sub-region enhancement map, and then performs image enhancement on the original medical image according to the sub-region enhancement map to obtain an enhanced medical image, thereby realizing image enhancement of a single region, that is, image enhancement according to anatomical information; thus, an initial segmentation model can be trained based on the original medical image and the enhanced medical image to obtain a tumor segmentation model. The above technical solution performs a sub-region image enhancement process based on anatomical information during the model training process, which divides the original medical image into different regions according to different tissues and / or organs in the anatomical information, and then performs different image enhancements on each region at the regional level, so that while enhancing the image, the intensity difference between the tumor and objects other than the tumor in the tissue and / or organ where the tumor is located can be retained, so that the model can use the feature of intensity difference to improve its segmentation performance, that is, train a tumor segmentation model suitable for segmenting tumors, and achieve the effect of accurate tumor segmentation.

[0019] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 is a flow chart of a model training method provided according to an embodiment of the present invention;

[0022] Figure 2 is a schematic diagram of an example of regional image enhancement based on anatomical information in a model training method provided according to an embodiment of the present invention;

[0023] Figure 3 is a flowchart of another model training method provided according to an embodiment of the present invention;

[0024] Figure 4 is a flowchart of another model training method provided according to an embodiment of the present invention;

[0025] Figure 5 is a flow chart of a tumor segmentation method provided according to an embodiment of the present invention;

[0026] Figure 6 is a schematic diagram of an example of single source domain generalization in a tumor segmentation method provided according to an embodiment of the present invention;

[0027] Figure 7 is a structural block diagram of a model training device provided according to an embodiment of the present invention;

[0028] Figure 8 is a structural block diagram of a tumor segmentation device provided according to an embodiment of the present invention;

[0029] Fig. 9 It is a structural schematic diagram of an electronic device for implementing the model training method or tumor segmentation method of an embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0031] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. The situations of "target", "original", etc. are similar and will not be repeated here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0032] Before introducing the embodiments of the present invention, the application scenarios of the embodiments of the present invention are first exemplified. For example, in medical images, the intensity difference between a tumor and its surrounding normal area is an important feature for determining the boundary of the tumor. As part of the image texture, the intensity difference is often destroyed in current image enhancement schemes along with the destruction of the image texture, making it difficult for the model to extract sufficient feature information to determine the boundary of the tumor, and thus making it impossible for the trained model to accurately segment the tumor.

[0033] Figure 1 It is a flow chart of a model training method provided in an embodiment of the present invention. This embodiment can be applied to the case of training a tumor segmentation model that can be applied to segment a tumor. The method can be executed by a model training device provided in an embodiment of the present invention, which can be implemented by software and / or hardware, and the device can be integrated in an electronic device, which can be various user terminals or servers.

[0034] See also Figure 1 The method of the embodiment of the present invention specifically includes the following steps:

[0035] S110. Acquire an original medical image containing a tumor.

[0036] The original medical image can be understood as a medical image obtained by collecting images of the tumor site, that is, a medical image containing the tumor. The image type of the medical image can be, for example, magnetic resonance (MR) or computed tomography (CT), etc. This depends on the actual situation and is not specifically limited here. The original medical image can be obtained, in particular, multiple original medical images can be obtained so as to use the multiple original medical images for model training.

[0037] After acquiring the multiple original medical images, in practical applications, the multiple original medical images may be optionally preprocessed, for example, the directions and voxels of the multiple original medical images may be unified, and maximum and minimum normalization may be performed to normalize the multiple original medical images within the range of [-1,1], so that the multiple original medical images have the same attributes, thereby improving the model training effect.

[0038] S120. Determine a gradient map of the original medical image, obtain a seed point based on a local maximum value in the gradient map, and use the seed point to segment the original medical image to obtain at least two regions.

[0039] For each acquired original medical image, S120 and S130 are executed as follows:

[0040] Determine the gradient map of the original medical image. The gradient map can reflect the rate of change of the pixel value of each pixel point in the original medical image in the spatial domain, which helps to determine the pixel points located on the region boundary among all the pixel points in the original medical image. Further, determine the local maximum value. The local maximum value can be understood as the pixel value with the largest value in the local region of the gradient map. The pixel point corresponding to the pixel value is likely to be located on the region boundary. Therefore, a seed point can be obtained based on the local maximum value, for example, the pixel point corresponding to the local maximum value is used as the seed point. On this basis, further, the seed point can be used to segment the original medical image to obtain two or more regions in the original medical image.

[0041] It should be noted that the original medical image contains one or more targets, and the tumor is located in the one or more targets, which may be tissues and / or organs, etc. On this basis, since the divided regions are the regions where the one or more targets are located, the above-mentioned region division process (i.e., the original medical image segmentation process) can be understood as a region division process according to anatomical information.

[0042] S130. Generate an enhancement coefficient for each region to obtain a sub-region enhancement map, and perform image enhancement on the original medical image according to the sub-region enhancement map to obtain an enhanced medical image.

[0043] Among them, the enhancement coefficient can be understood as a coefficient used to enhance the image of a certain divided area, and the coefficient can characterize the enhancement direction and / or enhancement amplitude, etc. An enhancement coefficient is randomly generated for each area to obtain a sub-region enhancement map, and the sub-region enhancement map can be understood as an image characterizing each enhancement coefficient. In practical applications, optionally, a set of enhancement coefficients can be generated, that is, an enhancement coefficient is generated for each area to obtain a sub-region enhancement map; multiple sets of enhancement coefficients can also be generated to obtain a sub-region enhancement map based on each set of enhancement coefficients; and so on. This can be set according to actual needs and is not limited here.

[0044] Further, for each sub-region enhancement map, the original medical image is enhanced according to the sub-region enhancement map to obtain an enhanced medical image corresponding to the sub-region enhancement map. In practical applications, optionally, the product result of the sub-region enhancement map and the original medical image, especially the Hadamard product result, can be used as the corresponding enhanced medical image. At this point, the image enhancement of the original medical image is completed.

[0045] It should be noted that, according to the above description, this step generates enhancement coefficients for each region respectively, and performs image enhancement of the original medical image based on the regional enhancement map obtained thereby, which is equivalent to enhancing each region in the original medical image separately, that is, performing image enhancement on a single region as a whole, rather than performing image enhancement on the original medical image as a whole.

[0046] In this way, for the target area in each area where the tissue and / or organ containing the tumor is located, the image enhancement of the target area can be performed separately, which helps to retain the intensity difference between the intensity information of the tumor in the target area and the intensity information of the objects other than the tumor in the target area, that is, to retain the intensity difference between normal tissue and / or organ and the tumor inside the tissue and / or organ, so that the model can determine the boundary of the tumor based on the feature of the intensity difference, and then the trained model can accurately segment the tumor.

[0047] S140. Based on the original medical image and the enhanced medical image, train an initial segmentation model to obtain a tumor segmentation model.

[0048] The original medical image can be understood as a medical image in the source domain, and the enhanced medical image can be understood as a medical image in a domain other than the source domain. The initial segmentation model is trained based on multiple original medical images and one or more enhanced medical images corresponding to each original medical image, so that the initial segmentation model can learn medical images in different domains, thereby ensuring the segmentation performance of the trained tumor segmentation model.

[0049] The technical solution of the embodiment of the present invention obtains an original medical image containing a tumor; determines the gradient map of the original medical image, and then uses the local maximum value in the gradient map as a seed point to segment the original medical image using the seed point to obtain at least two regions, thereby realizing a regional division process based on anatomical information; further, generates an enhancement coefficient for each region to obtain a sub-region enhancement map, and then performs image enhancement on the original medical image according to the sub-region enhancement map to obtain an enhanced medical image, thereby realizing image enhancement of a single region, that is, image enhancement according to anatomical information; thus, an initial segmentation model can be trained based on the original medical image and the enhanced medical image to obtain a tumor segmentation model. The above technical solution performs a sub-region image enhancement process based on anatomical information during the model training process, which divides the original medical image into different regions according to different tissues and / or organs in the anatomical information, and then performs different image enhancements on each region at the regional level, so that while enhancing the image, the intensity difference between the tumor and objects other than the tumor in the tissue and / or organ where the tumor is located can be retained, so that the model can use the feature of intensity difference to improve its segmentation performance, that is, train a tumor segmentation model suitable for segmenting tumors, and achieve the effect of accurate tumor segmentation.

[0050] An optional technical solution, using seed points to segment the original medical image to obtain at least two regions, may include: using the seed points to apply a preset watershed algorithm on the original medical image to segment the original medical image into at least two regions.

[0051] The watershed algorithm performs image segmentation based on the composition of the watershed and is very suitable for the field of medical image segmentation. The above technical solution combines the gradient map and the watershed algorithm to extract the anatomical information of the original medical image. The accurate extraction of the anatomical information ensures the accurate division of each area in the original medical image.

[0052] In order to better understand the above-mentioned technical solutions, the following is an exemplary description with reference to specific examples. Figure 2 , first calculate the gradient map of the original hospital image, then select the pixel points corresponding to the local maximum of the gradient map as seed points, and use the seed points to use the watershed algorithm to divide different regions on the original medical image, thereby realizing the division of different tissues and / or organs into different regions based on anatomical information. Furthermore, an enhancement coefficient is randomly generated in each region, and the enhancement coefficient can also be called a weighting coefficient, so as to obtain a regional enhancement map. It should be noted that different enhancement coefficients are represented by different colors in the regional enhancement map, that is, different regions correspond to different colors in the regional enhancement map. Furthermore, the regional enhancement map is Hadamard-producted with the original medical image to obtain an enhanced medical image. Figure 2In the example shown, two sets of enhancement coefficients are generated, thereby obtaining two regional enhancement maps, and further obtaining two enhanced medical images.

[0053] In the above example, while performing image enhancement, the intensity difference between the tumor and objects other than the tumor in the tissue and / or organ where the tumor is located is effectively preserved.

[0054] Figure 3 It is a flowchart of another model training method provided in an embodiment of the present invention. This embodiment is optimized based on the above-mentioned technical solutions. In this embodiment, optionally, after acquiring the original medical image containing the tumor, the above-mentioned model training method also includes: for the selection probabilities preset for at least two image enhancement strategies, selecting a target enhancement strategy from at least two image enhancement strategies according to each selection probability; accordingly, determining the gradient map of the original medical image, including: when the target enhancement strategy is a regional enhancement strategy, determining the gradient map of the original medical image. Among them, the explanations of the terms that are the same or corresponding to the above-mentioned embodiments are not repeated here.

[0055] See also Figure 3 The method of this embodiment may specifically include the following steps:

[0056] S210. Acquire an original medical image containing a tumor.

[0057] S220. For the selection probabilities respectively preset for the at least two image enhancement strategies, select a target enhancement strategy from the at least two image enhancement strategies according to each selection probability.

[0058] Among them, the image enhancement strategy can be understood as a pre-set strategy for image enhancement, for example, it can be the regional enhancement strategy described above, or it can be a global enhancement strategy, etc., which can be set according to actual needs and is not specifically limited here. It should be noted that the regional enhancement strategy is an image enhancement at the regional level, while the global enhancement strategy is an image enhancement at the image level, that is, the two are image enhancements of different scales. A selection probability is preset for each image enhancement strategy, and the selection probability can be understood as the probability of selecting a corresponding image enhancement strategy when performing image enhancement for a certain original medical image. In practical applications, optionally, considering that the regional enhancement strategy helps to retain intensity differences compared to the global enhancement strategy, a larger selection probability can be preset for the regional enhancement strategy compared to the global enhancement strategy.

[0059] After the original medical image is acquired, a target enhancement strategy may be selected from at least two image enhancement strategies according to each selection probability, so as to perform image enhancement of the original medical image using the target enhancement strategy.

[0060] S230. When the target enhancement strategy includes a regional enhancement strategy, determine the gradient map of the original medical image, obtain seed points based on the local maximum value in the gradient map, and use the seed points to segment the original medical image to obtain at least two regions.

[0061] Wherein, when the target enhancement strategy is a regional enhancement strategy, S230 and S240 may be used to divide the original medical image into regions, and image enhancement may be performed separately on each divided region.

[0062] S240. Generate an enhancement coefficient for each region to obtain a sub-region enhancement map, and perform image enhancement on the original medical image according to the sub-region enhancement map to obtain an enhanced medical image.

[0063] S250. Based on the original medical image and the enhanced medical image, train an initial segmentation model to obtain a tumor segmentation model.

[0064] The technical solution of the embodiment of the present invention presets image enhancement strategies of different scales, and selects one from each image enhancement strategy for image enhancement according to the selection probability preset for each image enhancement strategy. The application of the image enhancement strategies of different scales thereby improves the diversity of the texture (i.e., style) of the enhanced medical image.

[0065] In an optional technical solution, after selecting the target enhancement strategy from at least two image enhancement strategies according to each selection probability, the above model training method further includes:

[0066] When the target enhancement strategy is a global enhancement strategy, the original medical image is enhanced as a whole to obtain an enhanced medical image.

[0067] Exemplarily, here we take the global enhancement strategy based on Bezier curve as an example, and use cubic Bezier curve to perform global enhancement transformation on the original medical image, with the starting point and end point set to (-1,-1) and (1,1), or (-1,1) and (1,-1), respectively, and the remaining two control points are randomly selected in the range of (-1,1). In this way, an enhanced medical image with similar intensity distribution to the original medical image and / or an enhanced medical image with opposite intensity distribution can be obtained, thereby achieving global enhancement of the original medical image.

[0068] Figure 4It is a flow chart of another model training method provided in an embodiment of the present invention. This embodiment is optimized based on the above-mentioned technical solutions. In this embodiment, optionally, an initial segmentation model is trained based on the original medical image and the enhanced medical image to obtain a tumor segmentation model, including: inputting the original medical image into the initial segmentation model to obtain the original segmentation result, and inputting the enhanced medical image into the initial segmentation model to obtain the enhanced segmentation result; calculating the consistency loss according to the original segmentation result and the enhanced segmentation result, and adjusting the parameters in the initial segmentation model according to the consistency loss to obtain the tumor segmentation model. Among them, the explanations of the terms that are the same or corresponding to the above-mentioned embodiments are not repeated here.

[0069] See also Figure 4 The method of this embodiment may specifically include the following steps:

[0070] S310. Acquire an original medical image containing a tumor.

[0071] S320. Determine a gradient map of the original medical image, obtain a seed point based on a local maximum value in the gradient map, and use the seed point to segment the original medical image to obtain at least two regions.

[0072] S330. Generate an enhancement coefficient for each region to obtain a sub-region enhancement map, and perform image enhancement on the original medical image according to the sub-region enhancement map to obtain an enhanced medical image.

[0073] S340. Input the original medical image into the initial segmentation model to obtain the original segmentation result, and input the enhanced medical image into the initial segmentation model to obtain the enhanced segmentation result.

[0074] S350. Calculate the consistency loss based on the original segmentation result and the enhanced segmentation result, and adjust the parameters in the initial segmentation model based on the consistency loss to obtain a tumor segmentation model.

[0075] Among them, since the enhanced medical image is obtained by performing image enhancement on the original medical image, the segmentation result corresponding to the enhanced medical image (i.e., the enhanced segmentation result) should be consistent with the segmentation result corresponding to the original medical image (i.e., the original segmentation result). On this basis, in order to improve the consistency between the segmentation results before and after enhancement, the consistency loss can be calculated based on the original segmentation result and the enhanced segmentation result. For example, the relative entropy between the two segmentation results can be calculated, and then the relative entropy is used as the consistency loss. Further, the parameters in the initial segmentation model are adjusted according to the consistency loss to perform model training.

[0076] The technical solution of the embodiment of the present invention calculates the consistency loss between the original segmentation result and the enhanced segmentation result, and uses the consistency loss to train the model, thereby improving the consistency between the segmentation results before and after enhancement, and further improving the generalization of the model to medical images of different styles.

[0077] An optional technical solution, the above model training method further includes:

[0078] Obtaining the segmentation label of the original medical image for the tumor;

[0079] The original segmentation loss is calculated based on the segmentation label and the original segmentation result, and the enhanced segmentation loss is calculated based on the segmentation label and the enhanced segmentation result;

[0080] The parameters in the initial segmentation model are adjusted according to the consistency loss to obtain a tumor segmentation model, including:

[0081] The training loss is obtained according to the original segmentation loss, the enhanced segmentation loss and the consistency loss, and the parameters in the initial segmentation model are adjusted according to the training loss to obtain the tumor segmentation model.

[0082] As described above, since the enhanced medical image is obtained by performing image enhancement on the original medical image, the enhanced medical image and the original medical image correspond to the same segmentation label. On this basis, the original segmentation loss can be calculated according to the segmentation label and the original segmentation result, and the enhanced segmentation loss can be calculated according to the segmentation label and the enhanced segmentation result, and the training loss (i.e., the total loss) can be obtained according to these two segmentation losses and the consistency loss, and the parameters can be adjusted according to the training loss.

[0083] The above technical solution further improves the segmentation performance of the model by introducing segmentation loss.

[0084] Figure 5 1 is a flow chart of a tumor segmentation method provided by an embodiment of the present invention. This embodiment is applicable to the case of tumor segmentation. The method can be executed by a tumor segmentation device provided by an embodiment of the present invention, which can be implemented by software and / or hardware, and can be integrated on an electronic device, which can be various user terminals or servers.

[0085] See also Figure 5 The method of the embodiment of the present invention specifically includes the following steps:

[0086] S410. Acquire a target medical image containing a tumor, and a tumor segmentation model trained according to the model training method provided in any embodiment of the present invention.

[0087] S420. Input the target medical image into the tumor segmentation model, and obtain a target segmentation result of the target medical image for the tumor based on the output result of the tumor segmentation model.

[0088] The technical solution of the embodiment of the present invention realizes accurate segmentation of tumors in target medical images.

[0089] In order to better understand the above-mentioned technical solutions as a whole, the following is an exemplary description of them in combination with specific examples. Figure 6 A single-source domain generalization scheme suitable for tumor segmentation is shown in FIG. The specific implementation process of the single-source domain generalization scheme is as follows:

[0090] Step 1: Arrange multiple original medical images, unify their directions and voxels, and perform maximum and minimum normalization to normalize the multiple original medical images within the range of [-1,1].

[0091] Step 2: Perform image enhancement on the original medical image x through a multi-scale random enhancement module to obtain an enhanced medical image x ′ Specifically, in this example, the preset selection probability for the global enhancement strategy based on Bezier curve is 0.2, and the preset selection probability for the regional enhancement strategy based on anatomical information is 0.8. In each batch of training, the global enhancement strategy or the regional enhancement strategy is randomly selected according to the selection probability to enhance the original medical image x, and the enhanced medical image x is obtained. ′ .

[0092] Step 3: The original medical image x and the enhanced medical image x ′ are input into the same initial segmentation model for training, enhancing the medical image x ′ The training is performed using the segmentation labels corresponding to the original medical image x. In this example, the initial segmentation model can be pre-constructed based on a segmentation network such as UNet.

[0093] Step 4: In each forward propagation of the training process, at each segmentation result f θ Calculate the Dice loss L between (x) and the segmentation label y dice And the weighted cross entropy loss L ce As the segmentation loss L seg , as shown in formula (1), f θ It should be noted that formula (1) is based on the original medical image x as an example. ′ The same reasoning applies, and I will not repeat it here. At the same time, the original segmentation result f θ (x) and enhanced segmentation result f θ (x ′) should be consistent, so in order to improve the consistency of the segmentation results before and after enhancement, the relative entropy D is calculated between the two segmentation results as the consistency loss L consist , as shown in formula (2). Segmentation loss L seg and consistency loss L consist The weighted sum of the total weights can be used as the overall training loss L of the model. overall , as shown in formula (3), where λ consist is the weighting coefficient.

[0094] L seg (x, y) = L dice (f θ (x), y)+L ce (f θ (x), y)(1);

[0095] L consist (x, y) = D(p(y|f θ (x))||p(y|f θ (x ′ )))(2);

[0096] L overall (x, y) = L seg (x,y)+λ consist *L consist (x, y)(3).

[0097] Step 5: Repeat steps 2-4 until the model converges. Then, the target medical image is input into the trained tumor segmentation model to obtain the target segmentation result of the tumor.

[0098] The above single-source domain generalization solution has at least the following advantages:

[0099] 1. While improving the style diversity of medical images, the intensity difference is retained, so that this part of the features can be better recognized and utilized by the model, improving the performance of the model in segmenting tumors.

[0100] 2. The consistency loss based on relative entropy can better ensure the consistency of the segmentation results before and after enhancement, thereby improving the generalization of the model.

[0101] On this basis, in order to verify the effectiveness of the above-mentioned single-source domain generalization scheme, the scheme is used here to verify on actual clinical images. The images of T2 modality in the public image set TCGA-CESC are used for training, the images of T1 modality in the TCGA-CESC image set are used for cross-modality segmentation test, and the images of T2 modality in the private image set are used for cross-center segmentation test. The Dice coefficient, ASD and 95% HD are used for comprehensive evaluation. The evaluation results are shown in Table 1.

[0102] Table 1 Evaluation results

[0103]

[0104] It can be seen that compared with other models, the tumor segmentation model proposed in this example is better than the best existing results in all three indicators and can achieve accurate tumor segmentation.

[0105] Figure 7 This is a structural block diagram of a model training device provided in an embodiment of the present invention, and the device is used to execute the model training method provided in any of the above embodiments. The device and the model training method of the above embodiments belong to the same inventive concept. For details not described in detail in the embodiment of the model training device, please refer to the embodiment of the above model training method. Figure 7 The device may specifically include: an original medical image acquisition module 510, a region acquisition module 520, an enhanced medical image acquisition module 530 and a model training module 540.

[0106] The original medical image acquisition module 510 can be used to acquire the original medical image containing the tumor;

[0107] A region obtaining module 520 may be used to determine a gradient map of the original medical image, obtain a seed point based on a local maximum value in the gradient map, and use the seed point to segment the original medical image to obtain at least two regions;

[0108] The enhanced medical image obtaining module 530 may be used to generate an enhancement coefficient for each region to obtain a sub-region enhancement map, and perform image enhancement on the original medical image according to the sub-region enhancement map to obtain an enhanced medical image;

[0109] The model training module 540 may be used to train an initial segmentation model based on the original medical image and the enhanced medical image to obtain a tumor segmentation model.

[0110] Optionally, the region obtaining module 520 may include:

[0111] The region obtaining unit can be used to use the seed point to apply a preset watershed algorithm on the original medical image to segment the original medical image into at least two regions.

[0112] Optionally, the above model training device further includes:

[0113] A target enhancement strategy selection module, configured to select a target enhancement strategy from at least two image enhancement strategies according to the selection probabilities respectively preset for at least two image enhancement strategies after acquiring an original medical image containing a tumor;

[0114] Accordingly, the region obtaining module 520 may be specifically used for:

[0115] When the target enhancement strategy is a regional enhancement strategy, a gradient map of the original medical image is determined, and a local maximum value in the gradient map is used as a seed point. The seed point is used to segment the original medical image to obtain at least two regions.

[0116] On this basis, optionally, the above-mentioned model training device may further include:

[0117] The enhanced medical image re-obtaining module is used to select a target enhancement strategy from at least two image enhancement strategies according to each selection probability, and when the target enhancement strategy is a global enhancement strategy, perform image enhancement on the original medical image as a whole to obtain an enhanced medical image.

[0118] Optionally, the model training module 540 may include:

[0119] The segmentation result obtaining submodule is used to input the original medical image into the initial segmentation model to obtain the original segmentation result, and input the enhanced medical image into the initial segmentation model to obtain the enhanced segmentation result;

[0120] The model training submodule is used to calculate the consistency loss based on the original segmentation results and the enhanced segmentation results, and adjust the parameters in the initial segmentation model according to the consistency loss to obtain the tumor segmentation model.

[0121] On this basis, an optional model training submodule includes:

[0122] The consistency loss is obtained by a unit, which is used to calculate the relative entropy between the original segmentation result and the enhanced segmentation result, and the relative entropy is used as the consistency loss.

[0123] Alternatively, the above model training device further includes:

[0124] A segmentation label acquisition module is used to obtain the segmentation label of the original medical image for the tumor;

[0125] A segmentation loss calculation module, used to calculate the original segmentation loss according to the segmentation label and the original segmentation result, and to calculate the enhanced segmentation loss according to the segmentation label and the enhanced segmentation result;

[0126] Correspondingly, the model training submodule includes:

[0127] The model training unit is used to obtain the training loss according to the original segmentation loss, the enhanced segmentation loss and the consistency loss, and adjust the parameters in the initial segmentation model according to the training loss to obtain the tumor segmentation model.

[0128] Based on any of the above model training devices, optional:

[0129] For a target region including at least a tumor in each region, in the original medical image, an intensity difference between intensity information of the tumor in the target region and intensity information of an object other than the tumor in the target region is less than or equal to an intensity difference corresponding to the enhanced medical image;

[0130] The initial segmentation model performs tumor segmentation based on intensity differences.

[0131] The model training device provided in the embodiment of the present invention obtains the original medical image containing the tumor through the original medical image acquisition module; determines the gradient map of the original medical image through the region acquisition module, and then uses the local maximum value in the gradient map as the seed point to segment the original medical image using the seed point to obtain at least two regions, thereby realizing the region division process based on anatomical information; further, generates enhancement coefficients for each region through the enhanced medical image acquisition module to obtain a sub-region enhancement map, and then performs image enhancement on the original medical image according to the sub-region enhancement map to obtain an enhanced medical image, thereby realizing image enhancement of a single region separately, that is, image enhancement according to anatomical information; thus, the initial segmentation model is trained based on the original medical image and the enhanced medical image through the model training module to obtain a tumor segmentation model. The above-mentioned device performs a regional image enhancement process based on anatomical information during the model training process. This process divides the original medical image into different regions according to different tissues and / or organs in the anatomical information, and then performs different image enhancements on each region at the regional level. While enhancing the image, the intensity difference between the tumor and objects other than the tumor in the tissue and / or organ where the tumor is located can be retained, so that the model can use the feature of intensity difference to improve its segmentation performance, that is, train a tumor segmentation model that is suitable for segmenting tumors, and achieve accurate tumor segmentation effect.

[0132] The model training device provided in the embodiment of the present invention can execute the model training method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0133] It is worth noting that in the embodiment of the above-mentioned model training device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0134] Figure 8 This is a structural block diagram of a tumor segmentation device provided in an embodiment of the present invention. The device is used to execute the tumor segmentation method provided in any of the above embodiments. The device and the tumor segmentation method in the above embodiments belong to the same inventive concept. For details not described in detail in the embodiments of the tumor segmentation device, please refer to the embodiments of the above tumor segmentation method. Figure 8 The device may specifically include: a tumor segmentation model acquisition module 610 and a tumor segmentation module 620.

[0135] A tumor segmentation model acquisition module 610 is used to acquire a target medical image containing a tumor and a tumor segmentation model trained according to the model training method described in any embodiment of the present invention;

[0136] The tumor segmentation module 620 is used to input the target medical image into the tumor segmentation model, and obtain a target segmentation result of the target medical image for the tumor according to the output result of the tumor segmentation model.

[0137] The tumor segmentation device provided in the embodiment of the present invention realizes accurate segmentation of the tumor in the target medical image through the cooperation between the tumor segmentation model acquisition module and the tumor segmentation module.

[0138] The tumor segmentation device provided in the embodiment of the present invention can execute the tumor segmentation method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0139] It is worth noting that in the above-mentioned embodiment of the tumor segmentation device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0140] Fig. 9A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0141] like Fig. 9 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0142] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0143] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor 11 performs the various methods and processes described above, such as a model training method or a tumor segmentation method.

[0144] In some embodiments, the model training method or the tumor segmentation method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the model training method or the tumor segmentation method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the model training method or the tumor segmentation method in any other appropriate manner (e.g., by means of firmware).

[0145] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0146] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0147] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0148] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0149] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0150] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0151] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0152] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A model training method, characterized in that: include: Acquire original medical images containing tumors; Determine a gradient map of the original medical image, obtain a seed point based on a local maximum value in the gradient map, and segment the original medical image using the seed point to obtain at least two regions; Generating an enhancement coefficient for each of the regions to obtain a sub-region enhancement map, and performing image enhancement on the original medical image according to the sub-region enhancement map to obtain an enhanced medical image; Based on the original medical image and the enhanced medical image, an initial segmentation model is trained to obtain a tumor segmentation model.

2. The method according to claim 1, characterized in that The step of segmenting the original medical image using the seed point to obtain at least two regions includes: The seed point is used to apply a preset watershed algorithm to the original medical image to segment the original medical image into at least two regions.

3. The method according to claim 1, characterized in that After the original medical image containing the tumor is obtained, the method further includes: For the selection probabilities respectively preset for at least two image enhancement strategies, selecting a target enhancement strategy from the at least two image enhancement strategies according to each of the selection probabilities; Accordingly, determining the gradient map of the original medical image includes: In case the target enhancement strategy is a region-based enhancement strategy, a gradient map of the original medical image is determined.

4. The method according to claim 3, characterized in that After selecting the target enhancement strategy from at least two image enhancement strategies according to each of the selection probabilities, the method further includes: When the target enhancement strategy is a global enhancement strategy, the original medical image is enhanced as a whole to obtain the enhanced medical image.

5. The method according to claim 1, characterized in that The training of the initial segmentation model based on the original medical image and the enhanced medical image to obtain a tumor segmentation model includes: Inputting the original medical image into an initial segmentation model to obtain an original segmentation result, and inputting the enhanced medical image into the initial segmentation model to obtain an enhanced segmentation result; A consistency loss is calculated according to the original segmentation result and the enhanced segmentation result, and parameters in the initial segmentation model are adjusted according to the consistency loss to obtain a tumor segmentation model.

6. The method according to claim 5, characterized in that The calculating the consistency loss according to the original segmentation result and the enhanced segmentation result includes: A relative entropy between the original segmentation result and the enhanced segmentation result is calculated, and the relative entropy is used as a consistency loss.

7. The method according to claim 5, characterized in that Also includes: Obtaining a segmentation label of the original medical image for the tumor; Calculating an original segmentation loss according to the segmentation label and the original segmentation result, and calculating an enhanced segmentation loss according to the segmentation label and the enhanced segmentation result; Accordingly, adjusting the parameters in the initial segmentation model according to the consistency loss to obtain a tumor segmentation model includes: A training loss is obtained according to the original segmentation loss, the enhanced segmentation loss and the consistency loss, and parameters in the initial segmentation model are adjusted according to the training loss to obtain a tumor segmentation model.

8. The method according to any one of claims 1 to 7, characterized in that: For a target region including at least the tumor in each of the regions, in the original medical image, an intensity difference between intensity information of the tumor in the target region and intensity information of an object other than the tumor in the target region is less than or equal to an intensity difference corresponding to the enhanced medical image; The initial segmentation model performs tumor segmentation based on the intensity difference.

9. A tumor segmentation method, characterized in that: include: Acquire a target medical image containing a tumor and a tumor segmentation model trained according to the model training method of any one of claims 1 to 8; The target medical image is input into the tumor segmentation model, and a target segmentation result of the target medical image for the tumor is obtained according to an output result of the tumor segmentation model.

10. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor executes the model training method as described in any one of claims 1-8, or the tumor segmentation method as described in claim 9.

Images