Method and device for improving performance of deep learning model

By inputting medical images and segmented images into multiple channels of the deep learning model for training, the problem of degradation of image processing performance in traditional Chinese medicine in the prior art is solved, and the diagnostic ability of the model is significantly improved.

CN118941897APending Publication Date: 2024-11-12MEDICALIP CO LTD

Patent Information

Application Number
CN202410545469.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2023-05-09
Filing Date
2024-05-06
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

Existing multi-channel-based deep learning models show a degradation in medical image processing, mainly due to repeated input of medical images of a single gray scale into multi-channel for training.

Method used

The performance of the model is improved by segmenting the segmented image of at least one human tissue from the medical image and inputting the medical image and the segmented image into a complex channel of the deep learning model, respectively.

Benefits of technology

This method effectively improves the performance of deep learning models in medical image processing, especially the accuracy and efficiency of diagnosing lesions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118941897A_ABST
    Figure CN118941897A_ABST
Patent Text Reader

Abstract

The invention discloses a method for improving the performance of a deep learning model and equipment thereof. The performance improving apparatus prepares a medical image and at least one segmented image that segments at least one human tissue from the medical image, and trains a deep learning model by inputting the medical image and the at least one segmented image into a plurality of channels, respectively.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] Embodiments of the present invention relate to a deep learning model used in the medical field for diagnosing lesions, and more specifically, to a method and device for improving the performance of deep learning models in the medical field. Background Art

[0002] There are various image-based deep learning models. For example, deep learning models for various purposes can be developed based on various images of the famous database ImageNet. In the medical field, deep learning models are also used to diagnose lesions from medical images. Most image-based deep learning models learn by inputting images of different color levels (e.g., RGB; red, green, blue) in multiple channels. However, since X-ray images, computed tomography (CT) images, and magnetic resonance imaging (MRI) images are all gray-level images, there is a problem of performance degradation when applying existing deep learning models for image processing to the medical field. For example, a deep learning model for processing existing color images receives and processes color images through an RGB channel consisting of three channels. Therefore, when generating a deep learning model in the medical field based on this, it is necessary to input medical images of the same gray level in the RGB channels. That is, since a deep learning model that completes learning based on different color levels in multiple channels is trained by repeatedly inputting a single gray-level medical image into multiple channels, there is a problem of performance degradation. Summary of the invention

[0003] Technical issues

[0004] The technical problem to be solved by the embodiments of the present invention is to provide a method and an apparatus for improving the performance of a deep learning model for medical use when the deep learning model is generated based on a deep learning model including a plurality of channels.

[0005] Technical Solution

[0006] To solve the above technical problems, according to an embodiment of the present invention, a method for improving the performance of a deep learning model comprising multiple channels includes: preparing a medical image and segmenting at least one segmented image of at least one human tissue from the medical image; and training the deep learning model by respectively inputting the medical image and the at least one segmented image into the multiple channels.

[0007] To solve the above technical problems, a device for improving the performance of a deep learning model according to an embodiment of the present invention includes: an image preparation unit, which prepares a medical image and segments at least one segmented image of at least one human tissue from the medical image; and a learning unit, which trains the deep learning model by respectively inputting the medical image and the at least one segmented image into the multiple channels. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 A diagram showing an example of a deep learning model according to an embodiment of the present invention.

[0009] Figure 2 The present invention is a flowchart showing an example of a method for improving the performance of a deep learning model according to an embodiment of the present invention.

[0010] Figure 3 FIG. 1 is a diagram showing an example of a segmented image generating method according to an embodiment of the present invention.

[0011] Figure 4 A diagram showing an example of a method for training a deep learning model according to an embodiment of the present invention.

[0012] Figure 5 A diagram showing an example configuration of a performance improvement device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0013] Hereinafter, a method and device for improving the performance of a deep learning model according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

[0014] Figure 1 A diagram showing an example of a deep learning model according to an embodiment of the present invention.

[0015] Reference Figure 1 , the deep learning model 100 includes a plurality of channels 116. For example, the deep learning model 100 for processing a color image may include three channels 110: an R channel, a G channel, and a B channel. Each channel image 122, 124, 126 after dividing the value of each pixel of the color image 120 into three RGB values ​​is input to each channel 112, 114, 116. According to an embodiment, the number of channels of the input data used as the input of the deep learning model 100 may be different according to the embodiment. However, it is assumed below that there are three channels (e.g., RGB channels) for ease of description.

[0016] The deep learning model 100 can be implemented as various artificial neural networks such as Convolutional Neural Network (CNN) and U-Net, which are not limited to specific examples. The deep learning model 100 can be a model created using medical images or non-medical images (e.g., images in the image database of ImageNet, etc.) for various purposes. For example, the deep learning model 100 can be a model created for identifying or classifying specific objects (e.g., dogs or cats, etc.) in color images. Alternatively, the deep learning model 100 can be a model created in the medical field for segmenting specific human tissues (e.g., lungs, lesions, etc.) or diagnosing lesions (e.g., emphysema, pulmonary edema, pneumonia, various cancers, etc.). For another example, the deep learning model 100 can be a model newly created for this embodiment. In addition, the deep learning model 100 used in this embodiment can be various models that are processed after receiving the image, which are not limited to specific examples.

[0017] This embodiment provides a method for generating an existing deep learning model 100 or a new deep learning model 100 by training to optimize it for medical image processing. Figure 2 Detailed description is given in .

[0018] Figure 2 The present invention is a flowchart showing an example of a method for improving the performance of a deep learning model according to an embodiment of the present invention.

[0019] Reference Figure 2 The performance improvement device prepares a medical image and segments at least one segmented image of at least one human tissue (e.g., lung, heart, blood vessel, muscle, fat, lesion, etc.) from the medical image S200. As one embodiment, the performance improvement device may receive the medical image and the segmented image from an external device. As another embodiment, when the performance improvement device receives the medical image from the external device, the segmented image may be generated from the medical image. Figure 3 An example of a method for generating an image in which human tissue is segmented from a medical image is described in .

[0020] The performance improvement device trains the deep learning model by inputting the medical image and at least one segmented image into a plurality of channels of the deep learning model, respectively S210. For example, when the deep learning model includes RGB channels, the performance improvement device inputs the medical image and two segmented images generated by segmenting human tissue from the medical image into the RGB channels, respectively. Figure 4 An example of a method for training by inputting a medical image and a segmented image into a deep learning model including RGB channels is shown.

[0021] Figure 3FIG. 1 is a diagram showing an example of a segmented image generating method according to an embodiment of the present invention.

[0022] Reference Figure 3 The segmentation model 300 is a model for segmenting at least one human tissue from a medical image 310. The medical image 310 may be a two-dimensional medical image, such as an X-ray image.

[0023] The segmentation model 300 may be created by an existing system such as CNN. For example, the segmentation model 300 may be trained using learning data generated based on a three-dimensional medical image to improve the performance of the segmentation model 300. The three-dimensional medical image may be a CT image or an MRI image.

[0024] More specifically, the performance improvement device segments specific human tissues (e.g., lungs, heart, blood vessels, lesions, etc.) by applying various existing segmentation algorithms (or deep learning models for three-dimensional segmentation, etc.) to three-dimensional medical images. In two-dimensional medical images such as X-ray images, various tissues overlap and are displayed on a two-dimensional plane, so it is not easy to accurately segment specific human tissues from the two-dimensional medical image, and there is a problem of reduced segmentation accuracy. To this end, the present embodiment adopts a method for segmenting specific human tissues from a three-dimensional medical image. The specific human tissue segmented from the three-dimensional medical image is three-dimensional data (i.e., voxel). The performance improvement device projects the human tissue (i.e., voxel data) segmented from the three-dimensional medical image in two dimensions to generate a two-dimensional medical image. In addition, the performance improvement device generates a two-dimensional medical image by projecting the three-dimensional medical image itself in two dimensions. When the three-dimensional medical image itself is projected in two dimensions, the brightness values ​​of multiple tissues are all overlapped and reflected on the plane, so an image with multiple tissues overlapping, such as an X-ray image, can be created.

[0025] The performance improvement device generates learning data, the learning data including an image made by projecting a three-dimensional medical image into two dimensions (hereinafter referred to as a basic image), and an image made by projecting a three-dimensional image of a specific human tissue into a two-dimensional image (hereinafter referred to as a tissue image). The performance improvement device can use the learning data in which the basic image is marked as a tissue image (i.e., ground truth) and train the segmentation model 300 by a guided learning method. When a two-dimensional medical image taken by X-rays is input into the segmentation model 300 that has completed learning, the segmentation model 300 outputs segmented images 320, 330 in which a specific area is segmented from the two-dimensional medical image.

[0026] The segmentation model 300 can be trained to segment a plurality of human tissues from the medical image 310. For example, the segmentation model 300 can be a model that outputs a first segmented image 320 of a blood vessel region and a second segmented image 330 of a lung region after receiving a chest X-ray image. For another example, the segmentation model 300 can be a model that outputs a first segmented image of a lung region and a second segmented image of a heart region after receiving a chest X-ray image. In addition, the type and amount of human tissues segmented by the segmentation model 300 can be varied depending on whether the disease to be diagnosed or classified mainly invades a certain anatomical part. For example, emphysema causes the emphysematous part to darken in color in the X-ray image and reduce blood vessels. For another example, there can be a plurality of segmentation models that segment different human tissues from the medical image 310.

[0027] Figure 4 A diagram showing an example of a method for training a deep learning model according to an embodiment of the present invention.

[0028] Reference Figure 4 , the deep learning model 400 includes RGB channels 410. The performance improvement device can train the deep learning model 400 to diagnose lesions from medical images (for example, diagnose emphysema or pulmonary edema). The deep learning model 400 can be an existing model or a model newly created for this embodiment.

[0029] As an example, the training process of the deep learning model 400 for diagnosing emphysema is described below. The performance improvement device prepares two segmented images 424 and 426 for segmenting a blood vessel region and a lung region from a medical image 422. The segmented images 424 and 426 may be provided by an external device or may be generated by the performance improvement device using Figure 3 The performance improvement device trains the deep learning model 400 using the learning data, which is data for labeling a learning image 420 including a medical image 422 and two segmented images 424 and 426 as emphysema or not. The medical image 422 and the two segmented images 424 and 426 are input to each channel 412, 414, 416 of the deep learning model 400.

[0030] As an example, the training process of a deep learning model for diagnosing pulmonary edema is described below. The performance improvement device prepares two segmented images of the lung region and the heart region from the medical image. The segmented images may be provided by an external device or may be used by the performance improvement device. Figure 3The performance improvement device trains the deep learning model using the learning data, which is data for marking a learning image including a medical image and two segmented images as pulmonary edema or not. The medical image and the two segmented images are respectively input into the RGB channels of the deep learning model 400.

[0031] Figure 5 A diagram showing an example configuration of a performance improvement device according to an embodiment of the present invention.

[0032] Reference Figure 5 , the performance improvement device 500 includes an image preparation unit 510, a learning unit 520, a deep learning model 530, and a segmentation model 540. According to an embodiment, the segmentation model 540 may be omitted. However, the following description is made with the segmentation model 540 included. As an embodiment, the performance improvement device 500 can be implemented as a computing device including a memory, a processor, and an input / output device. In this case, each configuration may be implemented as software, thereby being executed by the processor after being loaded into the memory.

[0033] The image preparation unit 510 prepares a medical image and at least one segmented image of at least one human tissue segmented from the medical image. The image preparation unit 510 may receive the medical image and the segmented image from an external device. As another embodiment, the image preparation unit 510 may generate at least one segmented image from the medical image through the segmentation model 540 after receiving the medical image. Figure 3 An example of a method of generating a segmented image using the segmentation model 540 is shown.

[0034] The learning unit 520 trains the deep learning model 530 by inputting the medical image and at least one segmented image into the plurality of channels of the deep learning model 530. For example, when it is necessary to generate the deep learning model 530 for diagnosing emphysema, the image preparation unit 510 prepares the segmented images corresponding to the lung region and the blood vessel region. The learning unit 520 trains the deep learning model 530 using the medical image and the segmented image so that it can identify whether it is emphysema. Figure 4 An example of a method in which the learning unit 520 trains the deep learning model 530 is shown.

[0035] The present invention can also be implemented as a computer-readable program code in a computer-readable recording medium. Computer-readable recording media include all types of recording devices storing data that can be read by a computer system. Examples of computer-readable recording media include read-only memory (ROM), random access memory (RAM), compact disk read-only memory (CD-ROM), magnetic tape, floppy disk, and optical data storage device, etc. In addition, the computer-readable recording medium can be distributed in a computer system connected via a network, so that the computer-readable code is stored and executed in a distributed manner.

[0036] According to an embodiment of the present invention, when a deep learning model including a plurality of channels is applied to the medical field, its performance can be improved. As one embodiment, an existing deep learning model that has been learned based on non-medical color images can be retrained using medical images to generate a deep learning model for medical use. For another example, the performance of an existing deep learning model (including a plurality of channels) used in the medical field can be improved. For another example, it can also be applied to a multimodal large language model.

[0037] Above, the preferred embodiments of the present invention have been described. It will be appreciated by those skilled in the art that the present invention can be implemented in a modified form without departing from the essential characteristics of the present invention. Therefore, the disclosed embodiments should be considered from an exemplary perspective rather than a restrictive perspective. The scope of the present invention is shown in the claims, rather than the above description, and all differences within the equivalent scope of the claims should be interpreted as included in the present invention.

Claims

1. A method for improving the performance of a deep learning model comprising a plurality of channels, characterized in that include: preparing a medical image and segmenting at least one segmented image of at least one human tissue from the medical image; as well as The deep learning model is trained by inputting the medical image and the at least one segmented image into the plurality of channels, respectively.

2. The method for improving the performance of a deep learning model according to claim 1, characterized in that: The deep learning model is an image processing model including RGB channels.

3. The method for improving the performance of a deep learning model according to claim 1, characterized in that: The deep learning model is a model that completes learning based on non-medical images.

4. The method for improving the performance of a deep learning model according to claim 1, characterized in that: The medical image is a grayscale X-ray image.

5. The method for improving the performance of a deep learning model according to claim 1, characterized in that: The deep learning model is a model for diagnosing lesions based on medical images.

6. The method for improving the performance of a deep learning model according to claim 1, characterized in that: The medical image is a chest X-ray image, The at least one segmented image includes a first segmented image of a lung region segmented from the medical image and a second segmented image of a blood vessel region.

7. The method for improving the performance of a deep learning model according to claim 1, characterized in that: The medical image is a chest X-ray image, The at least one segmented image includes a first segmented image of a lung region segmented from the medical image and a second segmented image of a heart region.

8. The method for improving the performance of a deep learning model according to claim 1, characterized in that: The step of preparing the segmented image includes segmenting human tissue from the medical image using a segmentation model learned based on the projection image, The projection image is generated by projecting a three-dimensional medical image into two dimensions.

9. A device for improving the performance of a deep learning model, characterized in that: include: an image preparation unit, which prepares a medical image and segments at least one segmented image of at least one human tissue from the medical image; as well as A learning unit is configured to train the deep learning model by inputting the medical image and the at least one segmented image into the plurality of channels respectively.

10. A computer-readable recording medium having recorded therein a computer program for executing the method according to claim 1.

Citation Information

Patent Citations

  • Method and apparatus for reconstructing medical image

    CN111312369A

  • Method for extracting image features based on multi-film CT image and 3DCNN network

    CN113450355A

Cited By

  • Method for training recognition model for recognizing image integrity of medical part and related product

    CN122492705A

  • Methods of training a recognition model to recognize medical site image integrity and related products

    CN122492705B