Training method, medium and electronic device for medical image segmentation model

Abstract

The present invention provides a training method, medium and electronic device for a medical image segmentation model. The method comprises: performing a first-stage training on the medical image segmentation model, wherein the first-stage training comprises multiple rounds of training; performing a second-stage training on the medical image segmentation model to complete the training of the medical image segmentation model, wherein the second-stage training comprises multiple rounds of training. The method can directly use the common samples and the difficult samples to complete the training of the medical image segmentation model.

Description

Technical Field

[0001] The present invention relates to a training method, and in particular to a training method, medium and electronic equipment for a medical image segmentation model. Background Art

[0002] With the continuous development of artificial intelligence technology, it has become a common practice in related fields to use AI-based medical image segmentation models to process medical images and assist doctors in diagnosis. Medical image segmentation models need to be trained before practical application in order to acquire the ability to process medical images. However, the size of medical images is often relatively large. For example, the size of liver CT images is usually 512×512. In the training process of medical image segmentation models, especially in the training process of 3D medical image segmentation models, due to the limitation of GPU video memory size, it is usually difficult to directly input medical images at their original size into the medical image segmentation model for training. Therefore, how to provide a training method that can be applied to medical image segmentation models has become one of the technical problems that technicians in related fields urgently need to solve. Summary of the invention

[0003] In view of the above-mentioned shortcomings of the prior art, an object of the present invention is to provide a training method, medium and electronic device for a medical image segmentation model, so as to solve the above-mentioned problems existing in the prior art.

[0004] To achieve the above-mentioned purpose and other related purposes, the first aspect of the present invention provides a training method for a medical image segmentation model, which is used to train a medical image segmentation model. The training method for the medical image segmentation model includes: obtaining multiple training images; performing a first stage of training on the medical image segmentation model, wherein the first stage of training includes multiple rounds of training; performing a second stage of training on the medical image segmentation model to complete the training of the medical image segmentation model, wherein the second stage of training includes multiple rounds of training; in each round of training in the first stage, the method for training the medical image segmentation model includes: obtaining ordinary samples according to the training images, wherein the size of the ordinary samples is smaller than the size of the training images; training the medical image segmentation model using the ordinary samples; in each round of training in the second stage, the method for training the medical image segmentation model includes: obtaining ordinary samples according to the training images or obtaining difficult samples from a difficult sample library, wherein the size of the ordinary samples is the same as that of the difficult samples; training the medical image segmentation model according to the ordinary samples or the difficult samples.

[0005] In an embodiment of the first aspect, a method for obtaining a common sample according to the training image includes: randomly selecting an area of ​​a specific size from the training image as the common sample.

[0006] In an embodiment of the first aspect, after the first stage of training is completed, or after each round of training in the second stage is completed, the training method of the medical image segmentation model also includes: initializing the difficult sample library; using the medical image segmentation model to process each of the training images to obtain a prediction result of each of the training images; and updating the difficult sample library according to the prediction result and true annotation result of each of the training images.

[0007] In an embodiment of the first aspect, the implementation method for updating the difficult sample library according to the prediction results and the true annotation results of each of the training images includes: obtaining the prediction effect corresponding to each of the training images according to the prediction results and the true annotation results of each of the training images; for first training images with poor prediction effects, obtaining the difficult samples corresponding to each of the first training images and storing them in the difficult sample library.

[0008] In an embodiment of the first aspect, a method for obtaining a difficult sample corresponding to the first training image includes: obtaining an error map of the first training image based on a prediction result and a true annotation result of the first training image; pooling the error map of the first training image, and selecting an area with the largest error as the difficult sample corresponding to the first training image based on the pooling result, wherein the size of a pooling kernel used in the pooling is the same as the size of the ordinary sample.

[0009] In an embodiment of the first aspect, the implementation method of obtaining a common sample according to the training image or obtaining a difficult sample from the difficult sample library includes: generating a random number; when the value of the random number meets a preset condition, obtaining the common sample according to the training image, otherwise, obtaining the difficult sample from the difficult sample library.

[0010] In an embodiment of the first aspect, a size of the training image is greater than or equal to 512×512.

[0011] In an embodiment of the first aspect, the training image is a liver CT image.

[0012] The second aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the training method of the medical image segmentation model described in any one of the first aspects of the present invention is implemented.

[0013] The third aspect of the present invention provides an electronic device, comprising: a memory storing a computer program; a processor communicatively connected to the memory, and executing any method for training a medical image segmentation model according to the first aspect of the present invention when the computer program is called.

[0014] As described above, the training method of the medical image segmentation model described in one or more embodiments of the present invention has the following beneficial effects:

[0015] In the training method of the medical image segmentation model, common samples or difficult samples are used to train the medical image segmentation model, wherein the sizes of the common samples and the difficult samples are smaller than the training images. Therefore, the common samples and the difficult samples can be directly used to complete the training of the medical image segmentation model.

[0016] In addition, the training method of the medical image segmentation model includes a first stage and a second stage of training the medical image, wherein the first stage uses common samples to train the medical image segmentation model so that the medical image segmentation model adapts to the data, and the second stage uses common samples or difficult samples to train the medical image segmentation model to avoid the problem of insufficient training or over-training, thereby achieving better training effects while saving video memory. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1A Shown is a flowchart of a specific embodiment of the training method of the medical image segmentation model of the present invention.

[0018] Figure 1B Shown is a training flow chart of the first stage of the training method of the medical image segmentation model described in the present invention in a specific embodiment.

[0019] Figure 1C Shown is a training flow chart of the second stage of the training method of the medical image segmentation model described in the present invention in a specific embodiment.

[0020] Figure 2A A flowchart showing the updating of a difficult sample library in a specific embodiment of the training method of the medical image segmentation model of the present invention.

[0021] Figure 2B Shown is a detailed flow chart of step S16 of the training method of the medical image segmentation model of the present invention in a specific embodiment.

[0022] Figure 2C Shown is a detailed flow chart of step S162 of the training method of the medical image segmentation model described in the present invention in a specific embodiment.

[0023] Figure 2D An example diagram showing an error map obtained by the training method of the medical image segmentation model according to a specific embodiment of the present invention.

[0024] Figure 2E An example diagram showing the acquisition of difficult samples in a specific embodiment of the training method of the medical image segmentation model of the present invention.

[0025] Figure 2F Shown is an example diagram of difficult samples obtained in a specific embodiment of the training method of the medical image segmentation model described in the present invention.

[0026] Figure 3 Shown is a flowchart of key steps in a specific embodiment of the training method of the medical image segmentation model described in the present invention.

[0027] Figure 4 Shown is a flowchart of a specific embodiment of the training method of the medical image segmentation model of the present invention.

[0028] Figure 5 Shown is a schematic structural diagram of the electronic device of the present invention in a specific embodiment.

[0029] Component number description

[0030] 500 Electronic equipment

[0031] 510 Memory

[0032] 520 processor

[0033] 530 Display

[0034] Steps S11 to S16

[0035] Steps S121 to S122

[0036] Steps S131 to S132

[0037] Steps S161 to S162

[0038] Steps S1621~S1622

[0039] Steps S31 to S32

[0040] Steps S41 to S44 DETAILED DESCRIPTION

[0041] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0042] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention. The illustrations only show the components related to the present invention rather than the number, shape and size of the components in actual implementation. The type, quantity and proportion of each component in actual implementation may be changed at will, and the component layout type may also be more complicated. In addition, in this article, relational terms such as "first", "second", etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0043] In the training process of medical image segmentation models, especially in the training process of 3D medical image segmentation models, due to the limitation of GPU video memory size, it is usually difficult to directly input the medical image in its original size into the medical image segmentation model for training. To address this problem, in one embodiment of the present invention, a training method for a medical image segmentation model is provided, which is used to train a medical image segmentation model, such as a convolutional neural network segmentation model, a decision tree classification model, etc. For details, please refer to Figure 1A , the training method of the medical image segmentation model in this embodiment includes:

[0044] S11, obtaining a plurality of training images, wherein the training images are large-size images, for example, images with a size greater than or equal to 512×512.

[0045] Optionally, the training image is a medical image, more specifically, the training image is a liver CT image.

[0046] S12, performing a first phase of training on the medical image segmentation model, wherein the first phase of training includes multiple epochs of training.

[0047] Specifically, see Figure 1B In each round of training in the first stage, the method for training the medical image segmentation model includes:

[0048] S121. Obtain general samples based on the training images, where the size of the general samples is smaller than that of the training images. The general samples refer to the samples obtained by sampling the training images. For example, for any training image A with a size of W×H, a region of w×h can be selected from the training image A by means of bounding to serve as the general sample corresponding to the training image A, where W and H are the width and height of the training image A respectively, and their values are, for example, both 512, w and h are the width and height of the general sample corresponding to the training image A respectively, and w < W, h < H, and their values are, for example, both 256.

[0049] S122. Use the general samples to train the medical image segmentation model. The method for training the medical image segmentation model can be implemented by existing technologies and will not be elaborated here.

[0050] Optionally, the termination condition for any round of training in the first stage can be: all training samples have been trained once, or the number of training steps in this round of training reaches a step threshold, and the step threshold can be preset according to actual needs.

[0051] S13. Perform the second-stage training on the medical image segmentation model to complete the training of the medical image segmentation model, where the second-stage training includes multiple rounds of training.

[0052] Specifically, please refer to Figure 1C , in each round of training in the second stage, the method for training the medical image segmentation model includes:

[0053] S131. Obtain general samples based on the training images or obtain difficult samples from the difficult sample library, where the size of the general samples is the same as that of the difficult samples, and the difficult samples refer to the samples with poor prediction effects of the medical image segmentation model.

[0054] S132. Train the medical image segmentation model according to the general samples or the difficult samples.

[0055] According to the above description, in the training method of the medical image segmentation model described in this embodiment, general samples or difficult samples are used to train the medical image segmentation model, where the sizes of the general samples and the difficult samples are smaller than those of the training images. Therefore, the training of the medical image segmentation model can be directly completed by using the general samples and the difficult samples.

[0056] In addition, the training method of the medical image segmentation model includes a first stage and a second stage of training the medical image, wherein the first stage uses common samples to train the medical image segmentation model so that the medical image segmentation model adapts to the data, and the second stage uses common samples or difficult samples to train the medical image segmentation model to avoid the problem of insufficient training or over-training, thereby achieving better training effects while saving video memory.

[0057] In one embodiment of the present invention, the method for obtaining a common sample according to the training image includes: randomly selecting a region of a specific size from the training image as the common sample, wherein the specific size can be preset according to requirements.

[0058] Optionally, in this embodiment, a common sampling method may be used to sample the training image to obtain the common sample. The specific implementation method of the common sampling is: assuming that the size of the training image is W×H, a w×h area is randomly selected from the training image as the common sample. In addition, a training image is sampled at most once in each round of training.

[0059] See also Figure 2A In one embodiment of the present invention, after the first stage of training is completed, or after each round of training in the second stage is completed, the training method of the medical image segmentation model further includes:

[0060] S14, initializing the difficult sample library. Optionally, step S14 can initialize the difficult sample library by clearing the database.

[0061] S15, using the medical image segmentation model to process each of the training images to obtain a prediction result of each of the training images. Specifically, the medical image segmentation model, such as a fully convolutional neural network model, can input an image of any size, so the medical image segmentation model can directly process each of the training images to obtain its prediction result.

[0062] S16, updating the difficult sample library according to the prediction results and the real annotation results of each training image. The real annotation results of each training image can be obtained by manual annotation or other methods, and the present invention does not limit the acquisition method.

[0063] Optionally, see Figure 2B In this embodiment, the method for updating the difficult sample library according to the prediction results and the real annotation results of each training image includes:

[0064] S161, according to the prediction results and the real annotation results of each training image, obtain the prediction effect corresponding to each training image. Specifically, the greater the difference between the prediction result of the training image and the real annotation result, the worse the prediction effect of the medical image segmentation model for the training image, that is, the worse the prediction effect corresponding to the training image; conversely, the better the prediction effect corresponding to the training image.

[0065] Optionally, the prediction effect corresponding to the training image can be obtained according to the overlap between the prediction result of the training image and the real annotation result. The overlap can be described by an overlap index, for example, wherein the definition of the overlap index is, for example: Wherein, image∩out refers to the overlapping area of ​​the predicted result and the true annotation result of the training image, |image| refers to the predicted result of the training image, and |out| represents the true annotation result of the training image. For example, when the medical image segmentation model is used to segment a medical image to obtain a tumor region, image∩out represents an area where both the predicted result and the true annotation result are tumors, |image| represents an area where the predicted result is a tumor, and |out| represents an area where the true annotation result is a tumor.

[0066] Optionally, when the prediction effect is described by the overlap index, if the overlap index of the prediction result corresponding to a certain training image is less than an index threshold, the prediction effect of the training image is considered to be poor, and the training image is used as the first training image; conversely, when the overlap index of the prediction result corresponding to the training image is greater than or equal to the index threshold, the prediction effect of the training image is considered to be good, and the training image is used as the second training image. The index threshold can be set according to actual needs.

[0067] S162, for the first training images with poor prediction effects, obtain the difficult samples corresponding to the first training images and store them in the difficult sample library. The difficult samples corresponding to the first training images can be obtained by sampling the first training images. In addition, for the second training images with good prediction effects, the difficult sample library is not updated according to the second training images.

[0068] Optionally, the method for storing the difficult sample corresponding to the first training image in the difficult sample library includes: storing the number of the first training image and the coordinates of the difficult sample corresponding to it in the difficult sample library. Based on this, when it is necessary to obtain a difficult sample, the number of the training image and the coordinates of the difficult sample corresponding to it can be read from the difficult sample library, and then combined with the training image obtained in step S11, the difficult sample required for the training of the medical image segmentation model can be obtained.

[0069] Optionally, see Figure 2C In this embodiment, the method for obtaining the difficult sample corresponding to the first training image includes:

[0070] S1621: Obtain an error map of the first training image according to the prediction result and the true annotation result of the first training image.

[0071] Optionally, in step S1621, the error map may be obtained by calculating the difference between the prediction result and the true annotation result of the first training image or the absolute value of the difference, wherein an example of obtaining the error map is as follows: Figure 2D shown.

[0072] S1622, pooling the error map of the first training image, and selecting the area with the largest error as the difficult sample corresponding to the first training image according to the pooling result, wherein the pooling result refers to the pooled image obtained after pooling the first training image, and the size of the pooling kernel used in the pooling is preferably the same as the size of the ordinary sample.

[0073] Optionally, the pooling function used in the pooling is a sum function or an average function. In this case, the area corresponding to the maximum value can be selected from the pooled image as the difficult sample corresponding to the first training image. For example, when the pooling function is a sum function and the size of the pooling kernel is 3×3, the implementation method of obtaining the difficult sample through pooling is as follows: Figure 2E As shown, the difficult sample corresponding to the first training image obtained is as follows Figure 2F shown.

[0074] See also Figure 3 In one embodiment of the present invention, a method for obtaining a common sample according to the training image or obtaining a difficult sample from a difficult sample library includes:

[0075] S31, generate a random number.

[0076] S32, when the value of the random number meets a preset condition, obtaining the common sample according to the training image, otherwise, obtaining the difficult sample from the difficult sample library.

[0077] Optionally, the preset condition is, for example, that the random number is less than a numerical threshold, wherein the numerical threshold can be set according to actual needs. At this time, when the random number is less than the numerical threshold, the common sample is obtained according to the training image, and the common sample is used to train the medical image segmentation model in this round of training; when the random number is greater than or equal to the numerical threshold, the difficult sample is obtained from the difficult sample library, and the difficult sample is used to train the medical image segmentation model in this round of training.

[0078] In this embodiment, the medical image segmentation model is trained by selecting common samples or difficult samples according to the value of the random number, rather than using only difficult samples, which can prevent the overfitting problem caused by excessive training.

[0079] See also Figure 4 In one embodiment of the present invention, the training method of the medical image segmentation model includes:

[0080] S41, obtaining a plurality of training images.

[0081] S42, in the first N training rounds, the medical image segmentation model is trained using common samples to adapt the medical image segmentation model to the training data, wherein N is a positive integer, and its value is, for example, 10. In addition, each round of training in step S42 requires a total of S steps, and S is a positive integer, and its value is, for example, 200.

[0082] Optionally, in step S42, a common sampling method may be used to sample the training image to obtain the common sample, wherein a training image is sampled at most once in one round of training.

[0083] S43, after N training rounds, start adding difficult samples to train the medical image segmentation model.

[0084] Optionally, in order not to increase the training time of each round, the maximum number of steps in each round of training in step S43 is also S steps.

[0085] Optionally, in step S43, when the training area is collected at each step, a random number r is first generated. When the random number r is greater than or equal to a numerical threshold t, a difficult sample is randomly selected from the difficult sample library in this collection, and the medical image segmentation model is trained using the difficult sample. When r is less than t, a common sample is obtained from the training image using a common sampling method, and the medical image segmentation model is trained using the common sample.

[0086] S44, after a training round is completed, the difficult sample library is updated.

[0087] Among them, the implementation method for updating the difficult sample library includes: clearing the difficult sample library; using the medical image segmentation model obtained by the training round to predict each training image, if the size of the training image is W×H, then the size of the prediction result of each training image is C×W×H, where C is the number of categories; obtaining the prediction effect of each training image and an error map of size W×H; obtaining the prediction effect of the training image based on the error map, when the prediction effect of the training image is poor, obtaining the error and the largest area of ​​size w×h through pooling, and recording its number and coordinates and including it in the difficult sample library.

[0088] S45, repeating the above steps S43 to S44 until the training of the medical image segmentation model is completed.

[0089] According to the above description, this embodiment provides a training method for a medical image segmentation model, which makes up for the defects of medical image segmentation algorithm training on the market, and achieves better training effect while saving video memory. Moreover, this method is particularly suitable for large-size medical images, for example, medical images with a size of 512×512 or more. In addition, compared with the training method based on difficult samples in some related technologies, the training method for the medical image segmentation model described in this embodiment has no incremental samples, and thus does not increase the training time.

[0090] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to the computer program. The aforementioned computer program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk and other media that can store program codes.

[0091] Based on the above description of the training method of the medical image segmentation model, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to implement Figure 1A or Figure 4 Training method for the medical image segmentation model shown.

[0092] Based on the above description of the training method of the medical image segmentation model, the present invention also provides an electronic device. Specifically, please refer to Figure 5 In one embodiment of the present invention, the electronic device 500 includes a memory 510 and a processor 520, the memory 510 stores a computer program, the processor 520 is connected to the memory 510 for communication, and executes when the computer program is called. Figure 1A or Figure 4 Training method for the medical image segmentation model shown.

[0093] Optionally, the electronic device 500 further includes a display 530, which is communicatively connected to the memory 510 and the processor 520, and is used to display a GUI interaction interface related to the training method of the medical image segmentation model.

[0094] The protection scope of the training method of the medical image segmentation model described in the present invention is not limited to the execution order of the steps listed in this embodiment. All solutions implemented by adding, reducing or replacing steps in the prior art based on the principles of the present invention are included in the protection scope of the present invention.

[0095] In the training method of the medical image segmentation model, common samples or difficult samples are used to train the medical image segmentation model, wherein the sizes of the common samples and the difficult samples are smaller than the training images. Therefore, the common samples and the difficult samples can be directly used to complete the training of the medical image segmentation model.

[0096] In addition, the training method of the medical image segmentation model includes a first stage and a second stage of training the medical image, wherein the first stage uses common samples to train the medical image segmentation model so that the medical image segmentation model adapts to the data, and the second stage uses common samples or difficult samples to train the medical image segmentation model to avoid the problem of insufficient training or over-training, thereby achieving better training effects while saving video memory.

[0097] In summary, the present invention effectively overcomes various shortcomings of the prior art and has high industrial utilization value.

[0098] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by a person of ordinary skill in the art without departing from the spirit and technical concept disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. A training method for a medical image segmentation model, characterized in that: Used to train a medical image segmentation model, the training method of the medical image segmentation model includes: Obtain multiple training images; Performing a first-stage training on the medical image segmentation model, wherein the first-stage training includes multiple rounds of training; Performing a second phase of training on the medical image segmentation model to complete the training of the medical image segmentation model, wherein the second phase of training includes multiple rounds of training; In each round of training in the first stage, the method for training the medical image segmentation model includes: Acquire a common sample according to the training image, wherein the size of the common sample is smaller than the size of the training image; Using the common samples to train the medical image segmentation model; In each round of training in the second stage, the method for training the medical image segmentation model includes: Acquire a common sample according to the training image or acquire a difficult sample from a difficult sample library, wherein the size of the common sample is the same as that of the difficult sample; The medical image segmentation model is trained according to the common samples or the difficult samples.

2. The training method of the medical image segmentation model according to claim 1, characterized in that: The implementation method of obtaining common samples according to the training image includes: An area of ​​a specific size is randomly selected from the training image as the common sample.

3. The training method of the medical image segmentation model according to claim 1, characterized in that: After the first stage of training is completed, or after each round of training in the second stage is completed, the training method of the medical image segmentation model further includes: Initializing the difficult sample library; Processing each of the training images using the medical image segmentation model to obtain a prediction result of each of the training images; The difficult sample library is updated according to the prediction results and the true annotation results of each of the training images.

4. The training method of the medical image segmentation model according to claim 3, characterized in that: The implementation method of updating the difficult sample library according to the prediction results and the real annotation results of each training image includes: Obtaining prediction results corresponding to each of the training images according to the prediction results and the true annotation results of each of the training images; For the first training images with poor prediction effects, difficult samples corresponding to the first training images are obtained and stored in the difficult sample library.

5. The training method of the medical image segmentation model according to claim 4, characterized in that: The method for obtaining the difficult sample corresponding to the first training image includes: Obtaining an error map of the first training image according to the prediction result and the true annotation result of the first training image; Pooling is performed on the error map of the first training image, and a region with a maximum error is selected as a difficult sample corresponding to the first training image according to the pooling result, wherein a size of a pooling kernel used in the pooling is the same as a size of the common sample.

6. The training method of the medical image segmentation model according to claim 1, characterized in that: The implementation method of obtaining common samples according to the training image or obtaining difficult samples from the difficult sample library includes: Generate a random number; When the value of the random number meets a preset condition, the common sample is obtained according to the training image; otherwise, the difficult sample is obtained from the difficult sample library.

7. The training method of the medical image segmentation model according to claim 1, characterized in that: The size of the training image is greater than or equal to 512×512.

8. The method for training a medical image segmentation model according to claim 1, characterized in that: The training image is a liver CT image.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the training method of the medical image segmentation model according to any one of claims 1 to 8 is implemented.

10. An electronic device, characterized in that: The electronic device comprises: A memory storing a computer program; A processor is communicatively connected to the memory and executes the training method of the medical image segmentation model according to any one of claims 1 to 8 when calling the computer program.

Images