Device and method for training ear lobe crease detection model

Abstract

A device for training an ear lobe crease detection model according to one embodiment comprises: one or more processors; and a memory for storing commands executed by the one or more processors, wherein the one or more processors are configured to: select some slices as learning target slices from MRI volume data generated by an MRI device photographing the head of a target patient; generate learning data to be used in a deep learning-based segmentation model that detects an ear lobe crease from the MRI volume data on the basis of the learning target slices; and train the model to detect an ear lobe crease from the MRI volume data by inputting the learning data into the deep learning-based segmentation model.

Description

Device and method for training a two-row detection model

[0001] The disclosed embodiments relate to a technique for training a deep learning-based two-row detection model.

[0002] [Cross-reference to related applications]

[0003] This application claims priority to Republic of Korea Provisional Patent Application No. 10-2023-0156757, filed November 13, 2023, the entire contents of which are incorporated herein by reference.

[0004] Frank's sign (Frank's sign) is a diagonal line or crease that typically extends from the outer surface of the ear canal to the outer surface of the earlobe. Some studies have linked this crease to cognitive impairment and cerebrovascular disease in older adults, suggesting that it could serve as a visual indicator of both conditions.

[0005] However, the definition and evaluation criteria for pyrexia vary across medical professionals, making disease diagnosis based on pyrexia inconsistent. If pyrexia could be automatically detected using consistent criteria, it could serve as a more meaningful indicator for early disease detection.

[0006] The disclosed embodiments are for training a deep learning-based two-row detection model.

[0007] A device for training a heterodyne detection model according to one embodiment comprises one or more processors; and a memory for storing instructions executed by the one or more processors, wherein the one or more processors are configured to: select some slices as learning target slices from MRI volume data generated by an MRI device photographing a head of a target patient, generate learning data to be used in a deep learning-based segmentation model for detecting heterodyne in the MRI volume data based on the learning target slices, and train the model to detect heterodyne in the MRI volume data by inputting the learning data into the deep learning-based segmentation model.

[0008] The one or more processors may be configured to: select some slices from both sides of the MRI volume data generated by the MRI device by photographing the head of the target patient in a sagittal direction as the learning target slices.

[0009] The one or more processors may be configured to select, from among the MRI volume data, a slice photographing one ear of the target patient and a slice photographing the other ear as the learning target slices.

[0010] The one or more processors may be configured to: crop the learning target slice so that an area corresponding to an ear of the target patient remains in the learning target slice.

[0011] The one or more processors may be configured to: combine the learning target slices into a three-dimensional array to generate the learning data in the form of a three-dimensional volume.

[0012] The above deep learning-based segmentation model can be characterized by a U-Net structure.

[0013] The one or more processors may be configured to train the model to produce quantitative information about the heat of the target patient based on the number of voxels belonging to the heat.

[0014] The one or more processors may be configured to train the model to predict whether the target patient has a cognitive impairment based on the number of voxels belonging to the second row.

[0015] The one or more processors may be configured to train the model to predict whether the target patient generates a cerebral white matter hyperintensity signal based on the number of voxels belonging to the second row.

[0016] A method for training a heterodyne detection model according to one embodiment is a method performed by a device for training a heterodyne detection model, the device comprising: one or more processors; and a memory for storing instructions executed by the one or more processors, the method comprising: selecting some slices as learning target slices from MRI volume data generated by an MRI device by photographing a head of a target patient; generating learning data to be used in a deep learning-based segmentation model for detecting heterodyne in the MRI volume data based on the learning target slices; and training the model to detect heterodyne in the MRI volume data by inputting the learning data into the deep learning-based segmentation model.

[0017] The step of selecting the above learning target slice may include a step of selecting some slices from both sides of the MRI volume data generated by the MRI device photographing the head of the target patient in a sagittal direction as the learning target slice.

[0018] The step of selecting the above learning target slice may include a step of selecting a slice photographing one ear of the target patient and a slice photographing the other ear from among the MRI volume data as the learning target slice.

[0019] The step of selecting the learning target slice may be characterized by including a step of cropping the learning target slice so that an area corresponding to the ear of the target patient remains in the learning target slice.

[0020] The step of generating the above learning data may be characterized by including a step of generating the learning data in the form of a three-dimensional volume by combining the learning target slices into a three-dimensional array.

[0021] The above deep learning-based segmentation model can be characterized by a U-Net structure.

[0022] The step of training the model may be characterized by including a step of training the model to produce quantitative information about the heterogeneity of the target patient based on the number of voxels belonging to the heterogeneity.

[0023] The step of training the model may be characterized by including a step of training the model to predict whether the target patient has a cognitive impairment based on the number of voxels belonging to the second row.

[0024] The step of training the model may be characterized by including a step of training the model to predict whether the target patient generates a cerebral white matter hyperintensity signal based on the number of voxels belonging to the second row.

[0025] The disclosed embodiments provide a method for learning a deep learning-based two-row detection model, thereby providing a method for automatically detecting two-rows without human intervention.

[0026] The disclosed embodiments train a model based on a set of eigenvalues ​​defined according to consistent criteria, thereby providing a method for objectively and quantitatively evaluating eigenvalues.

[0027] The disclosed embodiments teach a heterogeneous detection model the relationship between heterogeneity and cognitive impairment, thereby generating health-related reports that can aid in disease diagnosis.

[0028] Figure 1 is an image of both ears of a patient with bilateral deafness.

[0029] FIG. 2 is a block diagram illustrating a device for training a two-row detection model according to one embodiment.

[0030] Figure 3 is an example diagram to explain the process of constructing learning data.

[0031] Figure 4 is an example diagram to explain the process of constructing learning data.

[0032] Figure 5 is an example diagram to schematically explain the learning process of a deep learning model.

[0033] Figure 6 is a block diagram illustrating the architecture of a deep learning model.

[0034] Figure 7 is a diagram for explaining the performance of a deep learning model.

[0035] Figure 8 is a flowchart illustrating a method for training a two-row detection model according to one embodiment.

[0036] Hereinafter, specific embodiments of one embodiment will be described with reference to the drawings. The following detailed description is provided to facilitate a comprehensive understanding of the sensor described herein. However, this is merely an example and the present invention is not limited thereto.

[0037] In describing embodiments, if it is determined that a detailed description of a known technology related to the present invention may unnecessarily obscure the gist of an embodiment, the detailed description will be omitted.

[0038] Figure 1 is an image of both ears of a patient with bilateral deafness.

[0039] As illustrated in Figure 1, "aesthesia" is defined herein as a diagonal wrinkle extending from the earlobe to the external auditory canal. A condition primarily observed in the elderly, it is not simply a cosmetic change, but rather a consequence of the aging process, which leads to decreased skin elasticity and degenerative changes in blood vessels.

[0040] That is, here, the term "hypertrophy" is different from the temporary wrinkles that occur when the skin is pressed or wrinkled depending on the sleeping position, and refers to wrinkles that are continuously formed as a result of aging or changes in skin structure.

[0041] FIG. 2 is a block diagram illustrating a device for training a two-row detection model according to one embodiment.

[0042] Referring to FIG. 2, a device for training a two-row detection model includes a processor and a memory.

[0043] The processor selects some slices from the MRI volume data generated by the MRI device taking an image of the target patient's head as the target slices for training.

[0044] Specifically, the processor can select some slices from both sides of MRI volume data generated by an MRI device taking a sagittal image of the head of a target patient as learning target slices.

[0045] Preferably, the processor can select, from among the MRI volume data, a slice that captures one ear of the target patient and a slice that captures at least one of the other ears as the learning target slices.

[0046] The processor can preprocess selected training target slices to improve the quality and consistency of training data to be used in a deep learning model.

[0047] The processor can perform resampling on the learning target slice. The processor can perform resampling on the learning target slice to remove unnecessary details or simplify the information contained in the learning target slice.

[0048] The processor can crop out specific portions of the training slice. To highlight the deep learning region of interest, the processor can remove areas where the two rows are not located, leaving only the portion of the training slice corresponding to, for example, an ear.

[0049] The processor generates training data for use in a deep learning-based segmentation model that detects heterogeneity in MRI volume data based on the learning target slices.

[0050] The processor can combine learning target slices into a three-dimensional array to generate three-dimensional volume-shaped learning data for use in a deep learning-based segmentation model that detects heterogeneity in MRI volume data.

[0051] The processor inputs training data into a deep learning-based segmentation model to detect heterogeneity in MRI volume data.

[0052] A deep learning-based segmentation model can be trained to generate a binary mask by applying a threshold value to each voxel in the output segmentation map. The deep learning-based segmentation model can then detect aberrations based on the binary mask.

[0053] A deep learning-based segmentation model can be trained to overlay binary masks corresponding to each row onto the original MRI volume data. This allows the location and size of each row to be intuitively displayed on the original image.

[0054] Deep learning-based segmentation models can be trained to produce quantitative information about a column based on segmentation results. For example, a deep learning-based segmentation model can be trained to produce quantitative information about the area, length, and depth of a column based on the number of voxels within the segmented region.

[0055] A deep learning-based segmentation model can be trained to generate a diagnostic report based on at least one of the segmentation results and quantitative information.

[0056] For example, a deep learning-based segmentation model can be trained to estimate whether a target patient has cognitive impairment based on the number of voxels belonging to a column.

[0057] As another example, the processor can be trained to estimate whether a subject has a white matter hyperintensity signal based on the number of voxels that fall into a column.

[0058] Memory stores one or more instructions that the processor executes.

[0059] Memory can store various data used by the processor. For example, memory can contain software (e.g., training data or output data for a program executed by the processor and / or instructions associated with the program).

[0060] Figure 3 is an example diagram to explain the process of constructing learning data.

[0061] Referring to FIG. 3, 175 MRI volume data received from an MRI device and 64 learning target slices selected from among them are distinguished and illustrated.

[0062] Here, MRI volume data refers to a collection of multiple cross-sectional images (hereinafter referred to as "slices") taken by an MRI device from a specific orientation of the subject's body. Each slice represents an image of the subject's body's interior, and MRI volume data is formed by sequentially stacking these slices into a three-dimensional structure.

[0063] MRI volume data preferably utilizes multiple slices generated by taking sagittal images of the head of the patient being MRId to include anatomical features of the ear related to the eardrum.

[0064] The learning target slice refers to a portion of the MRI volume data selected from among the slices. In particular, the learning target slice may be composed of slices corresponding to one or both ears of the target patient.

[0065] For example, as illustrated in FIG. 2, the processor can select 64 slices, excluding 111 slices in the center, as learning target slices from MRI volume data consisting of 175 slices generated by an MRI device photographing a target patient in a sagittal direction.

[0066] In other words, the processor can select slices captured from both ends of the MRI volume data to a certain portion of the inner ear as target slices for training. As shown in the example shown in Figure 3, the processor can configure 32 slices, each capturing the anatomical structures of the left and right ears, as target slices for training.

[0067] Additionally, the processor can perform cropping to leave a specific region in the learning target slice. For example, the processor can crop the learning target slice image from the original 240*240 size of the MRI volume data to a size of 144*144. Specifically, the processor can leave pixels 48 to 148 horizontally and pixels 24 to 168 vertically, leaving the region corresponding to the ear in the learning target slice, and crop the remaining pixels.

[0068] From this, the processor can select only the slices and pixels related to the two rows, allowing the deep learning model to effectively learn the segmentation task.

[0069] Meanwhile, in Fig. 3, the learning target slice is described as being selected from both sides and cropped to a specific pixel, but this is exemplary, and the slice may be selected from one side as well as both sides, or cropped to an area of ​​another pixel.

[0070] Figure 4 is an example diagram to explain the process of constructing learning data.

[0071] Referring to Figure 4, a learning target slice and learning data generated from the learning target slice are illustrated.

[0072] The learning target slices are slices extracted from both sides of the MRI volume data. The learning target slices are identified as containing ear structural features of a specific thickness when viewed from the front.

[0073] At this time, the processor can build training data by combining the learning target slices and labeling the combined data, which represents the features of both ears, with the correct value. In other words, the processor can generate training data by labeling the combined data and the resulting values ​​from segmenting the two columns in the combined data with the correct value.

[0074] Meanwhile, the correct answer value can be a segmentation value of the two columns detected manually by medical staff with consistent standards or automatically using software (MRIcrogl, ITK-SNAP, etc.).

[0075] Figure 5 is an example diagram to schematically explain the learning process of a deep learning model.

[0076] Referring to Figure 5, the process of constructing training data used in a deep learning model and the process of the deep learning model learning through the training data are illustrated.

[0077] First, the processor selects some slices from the MRI volume data as target slices for training. Here, the processor can select slices that capture the structures of both ears as target slices for training.

[0078] Thereafter, the processor can generate combined data in a three-dimensional format by combining the learning target slices, and label the correct value of the segmentation result of the two columns detected in the combined data to generate learning data.

[0079] At this time, the segmentation result may be a segmentation value manually detected in advance from each slice of MRI volume data or a learning target slice.

[0080] Afterwards, the processor can perform preprocessing on the training data. For example, the processor can apply resampling, cropping, and normalization to the training data.

[0081] Afterwards, the processor can augment the existing training data for rich learning, for example, left / right / upside down flipping, rotation (± < 10°), color adjustment, cropping, scaling, adding noise, etc. can be used to augment the training data.

[0082] Afterwards, when training data is input to the model, the processor can be trained to detect the boundaries of the two columns in the training data.

[0083] The processor can divide the data into batches of a specified size (e.g., 2) and train a model for each batch. The processor trains to minimize a loss function (e.g., binary cross-entropy and Dice loss) over a preset number of epochs (e.g., 100), and the learning rate can be reduced by a preset multiple (e.g., 0.5) every 25 epochs. Furthermore, the processor can use an optimizer (e.g., the Adam optimizer) to minimize the value of the loss function and control the size of the parameters through regularization (e.g., L2) to prevent overfitting.

[0084] At this time, the model may include at least one structure among U-Net, Attention U-Net, U-Net++, and USE-Net, and preferably, the U-Net structure may be used.

[0085] Afterwards, the processor can train the model, evaluate segmentation performance, and use the test dataset to evaluate classification performance. At this point, the processor can determine the optimal cutoff voxel value (e.g., 25 / 26) for distinguishing the presence of a double row.

[0086] Afterwards, the processor can apply 5-fold cross-validation to the model to evaluate its general effectiveness in detecting false positives on independent datasets.

[0087] The processor can verify the model's performance by dividing the dataset containing training data and validation data into five folds, using each fold once as test data and the remaining four folds as training data.

[0088] Meanwhile, the training and validation data were collected from MRI data of 400 elderly patients with pyrexia out of 550, and the test data were collected from MRI data of 150 elderly patients with pyrexia and 150 elderly patients without pyrexia. The collected MRI may be a TI-weighted MRI image generated by a 3.0T MRI scanner with the following parameters: voxel size: 1.0 mm × 0.5 mm × 0.5 mm, echo time: 4.6 ms, repetition time: 9.9 ms, and axial matrix size: 240 mm × 240 mm.

[0089] Figure 6 is an example diagram explaining the structure of a deep learning model.

[0090] Deep learning-based models can adopt various structures such as U-Net, Attention U-Net, U-Net++, and USE-Net, but the structure of the deep learning model adopted in this specification is explained with a focus on U-Net.

[0091] Deep learning-based models can extract features from input data through an encoder. Deep learning-based models can use four pooling layers to downsample the input data. Each pooling layer reduces the spatial dimension of the input feature map, allowing for the extraction of important features related to the input data.

[0092] Deep learning-based models can restore the original data to its original size by upsampling features extracted by the encoder through a decoder. The decoder can be combined with feature maps from the corresponding encoding stage to prevent information loss through skip connections and enable more detailed segmentation.

[0093] Furthermore, deep learning-based models can improve their segmentation performance by combining binary cross-entropy and Dice loss as loss functions. Furthermore, the learning rate can be reduced in a multi-step fashion, starting from an initial value of 1×10^-3, to induce effective learning.

[0094] Figure 7 is an example diagram to explain the performance of a deep learning model.

[0095] In Fig. 7, the deep learning model is the U-Net structure of Fig. 6, and the result of comparing the segmentation result value output by the deep learning model with the correct answer value is shown.

[0096] In each image, the manually detected binary segmentation results (i.e., correct values) are overlaid on the input data in red, and the binary segmentation results output by the deep learning model are overlaid on the input data in green.

[0097] Whether the input data to the deep learning model is in one ear (Uni-lateral subject) or both ears (Uni-lateral subject #1, #2), the DSC (Dice Similarity Coefficient) of the deep learning model is all over 0.85, showing high precision.

[0098] Here, DSC is a metric that measures the similarity between sets and is used to evaluate segmentation performance. In other words, the deep learning model in this specification confirms that it correctly detected heterogeneity in MRI volume data.

[0099] Figure 8 is a flowchart illustrating a method for training a two-row detection model according to one embodiment.

[0100] Referring to FIG. 8, a method for training a two-row detection model according to one embodiment can be performed by the device of FIG. 2.

[0101] First, a device for training a two-stage detection model selects (810) some slices as learning target slices from MRI volume data generated by an MRI device photographing the head of a target patient.

[0102] Thereafter, a device for training a heterogeneous detection model generates (820) training data to be used for a deep learning-based segmentation model that detects heterogeneity in MRI volume data based on the above-mentioned training target slice.

[0103] Thereafter, a device for training a heterogeneous detection model can train (830) a model to detect heterogeneity in MRI volume data by inputting training data into a deep learning-based segmentation model.

[0104] Meanwhile, although the method of FIG. 8 is described as being divided into multiple steps, at least some of the steps may be performed in a different order, combined with other steps and performed together, omitted, divided into detailed steps and performed, or one or more steps not shown may be added and performed.

[0105] Meanwhile, embodiments of the present invention may include a program for performing the methods described herein on a computer, and a computer-readable recording medium including the program. The computer-readable recording medium may include program commands, local data files, local data structures, etc., alone or in combination. The medium may be specially designed and configured for the present invention, or may be one commonly used in the field of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and hardware devices specially configured to store and execute program commands such as ROMs, RAMs, and flash memories. Examples of the program may include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, etc.

[0106] While representative embodiments of the present invention have been described in detail above, those skilled in the art will appreciate that various modifications to the above-described embodiments are possible without departing from the scope of the present invention. Therefore, the scope of the present invention should not be limited to the described embodiments, but should be determined not only by the claims set forth below but also by equivalents thereof.

[0107] The terms described below are terms defined in consideration of their functions in the present invention, and may vary depending on the intention or custom of the subject or operator. Therefore, the definitions should be made based on the contents throughout this specification. The terminology used in the detailed description is only for the purpose of describing one embodiment and should never be limited. Unless clearly used otherwise, the singular form includes the plural form. In this description, expressions such as "comprises" or "having" are intended to indicate certain components, numbers, steps, operations, elements, parts or combinations thereof, and should not be construed to exclude the presence or possibility of one or more other components, numbers, steps, operations, elements, parts or combinations thereof other than those described.

[0108] Additionally, the embodiments described herein may have aspects that are entirely hardware, partially hardware and partially software, or entirely software. As used herein, the term "unit" or the like refers to a computer-related entity, such as hardware, a combination of hardware and software, or software.

[0109] A device for training a two-stage model according to one embodiment trains a model for automatically detecting two-stages in an MRI volume image, and is applicable to medical devices and digital medical industries.

Claims

1. One or more processors; and A device for training a two-row detection model having a memory for storing instructions executed by one or more processors, One or more of the above processors: Select some slices from the MRI volume data generated by the MRI device taking an image of the target patient's head as learning target slices. Based on the above learning target slice, training data to be used for a deep learning-based segmentation model that detects heterogeneity in the MRI volume data is generated, A device for training a heterogeneity detection model, characterized in that the device is configured to train the model to detect heterogeneity in the MRI volume data by inputting the training data into the deep learning-based segmentation model.

2. In paragraph 1, One or more of the above processors: A device for training a two-column detection model, characterized in that the MRI device is configured to select some slices from both sides of the MRI volume data generated by photographing the head of the target patient in a sagittal direction as the training target slices.

3. In paragraph 1, One or more of the above processors: A device for training a two-row detection model, characterized in that it is configured to select a slice that photographed one ear of the target patient and a slice that photographed the other ear from among the MRI volume data as the training target slices.

4. In paragraph 1, One or more of the above processors: A device for training a two-row detection model, characterized in that it is configured to crop the training target slice so that an area corresponding to an ear of the target patient remains in the training target slice.

5. In paragraph 1, One or more of the above processors: A device for training a two-column detection model, characterized in that it is configured to generate training data in the form of a three-dimensional volume by combining the above training target slices into a three-dimensional array.

6. In paragraph 1, A device for training a two-row detection model, characterized in that the above deep learning-based segmentation model has a U-Net structure.

7. In paragraph 1, One or more of the above processors: A device for training a heterodyne detection model, characterized in that the model is configured to train the model to produce quantitative information about heterodyne of the target patient based on the number of voxels belonging to the heterodyne.

8. In paragraph 1, One or more of the above processors: A device for training a heterogeneous detection model, characterized in that the model is configured to train the model to predict whether the target patient has cognitive impairment based on the number of voxels belonging to the heterogeneous group.

9. In paragraph 1, One or more of the above processors: A device for training a heterodyne detection model, characterized in that the model is configured to train the model to predict whether the target patient generates a cerebral white matter hyperintensity signal based on the number of voxels belonging to the heterodyne.

10. One or more processors; and A method performed by a device for training a two-row detection model having a memory for storing instructions executed by said one or more processors. A step of selecting some slices as learning target slices from MRI volume data generated by an MRI device taking an image of the head of a target patient; A step of generating training data to be used for a deep learning-based segmentation model that detects a heterogeneity in the MRI volume data based on the learning target slice; and A method for training a heterogeneity detection model, characterized by including a step of training the model to detect heterogeneity in the MRI volume data by inputting the training data into the deep learning-based segmentation model.

11. In paragraph 10, The step of selecting the above learning target slice is: A method for training a two-column detection model, characterized in that it includes a step of selecting some slices from both sides of the MRI volume data generated by the MRI device photographing the head of the target patient in a sagittal direction as the training target slices.

12. In paragraph 10, The step of selecting the above learning target slice is: A method for training a two-row detection model, characterized by including a step of selecting a slice that photographed one ear of the target patient and a slice that photographed the other ear from among the MRI volume data as the training target slices.

13. In paragraph 10, The step of selecting the above learning target slice is: A method for training a two-row detection model, characterized by comprising the step of cropping the training target slice so that an area corresponding to an ear of the target patient remains in the training target slice.

14. In paragraph 10, The steps for generating the above learning data are: A method for training a two-column detection model, characterized by including a step of combining the above learning target slices into a three-dimensional array to generate the above learning data in the form of a three-dimensional volume.

15. In paragraph 1, A method for training a two-row detection model, characterized in that the above deep learning-based segmentation model has a U-Net structure.

16. In paragraph 10, The steps for training the above model are: A method for training a heterodyne detection model, characterized by including a step of training the model to produce quantitative information about heterodyne of the target patient based on the number of voxels belonging to the heterodyne.

17. In paragraph 10, The steps for training the above model are: A method for training a heterogeneous detection model, characterized by including a step of training the model to predict whether the target patient has cognitive impairment based on the number of voxels belonging to the heterogeneous group.

18. In paragraph 10, The steps for training the above model are: A method for training a heterogeneous detection model, characterized by including a step of training the model to predict whether the target patient generates a white matter hyperintensity signal based on the number of voxels belonging to the heterogeneous group.

Images