Artificial intelligence-based head and neck landmark detection method and device

Abstract

The present invention relates to a method for detecting head and neck landmarks based on artificial intelligence, the method includes the steps of an original CT image acquisition unit acquiring an original CT image of a patient's head and neck, an image conversion unit converting the acquired original CT image into a low-resolution CT image, a landmark detection unit inputting the converted low-resolution CT image into a landmark detection model to detect a first landmark, a surrounding area detection unit detecting a surrounding area based on the detected first landmark of the head and neck, an image restoration unit restoring the original CT image for the detected surrounding area, and a landmark detection unit inputting an image of the restored surrounding area into the landmark detection model to detect a second landmark. The head and neck landmarks can be detected quickly and accurately by converting the 3D CT image into a low resolution and primarily detecting landmarks, and secondarily detecting landmarks for the detected specific area.

Description

[Technical field]

[0001] The present invention relates to an artificial intelligence-based method and apparatus for detecting head and neck landmarks, and more particularly, to an artificial intelligence-based method and apparatus for quickly detecting anatomical landmarks from head and neck CT images using a CNN landmark detection model. [Background technology]

[0002] Previously, doctors manually selected the positions of landmarks with anatomical significance on 3D CT images of the patient's head and neck. There were limitations to checking and analyzing a large number of landmarks with the naked eye, and the positions of landmarks could vary from person to person, making accuracy an issue. To solve these problems, there are attempts to use artificial intelligence technology instead of humans, but learning a model using large volumes of 3D CT images as learning data is slow and expensive due to the large amount of calculations required.In addition, images output from multiple different CT devices have different densities, making learning difficult. A prior patent is Korean Patent Publication No. 10-2021-0071188 (Anatomical landmark estimation device and method), but it only discloses a technology for estimating the position of anatomical landmarks based on landmark-specific and patient-specific characteristics from a patient's head measurement image. Summary of the Invention [Problem to be solved by the invention]

[0003] The present invention provides a method and apparatus for detecting head and neck landmarks based on artificial intelligence, capable of quickly and accurately detecting landmarks having anatomical significance from a large volume of head and neck 3D CT images through artificial intelligence technology. [Means for solving the problem]

[0004] The artificial intelligence based head and neck landmark detection method of the present invention includes the steps of an original CT image acquisition unit acquiring an original CT image of a patient's head and neck, an image conversion unit converting the acquired original CT image into a low-resolution CT image, a landmark detection unit inputting the converted low-resolution CT image into a landmark detection model to detect a first landmark, a surrounding area detection unit detecting a surrounding area based on the detected first head and neck landmark, an image restoration unit restoring the original CT image with respect to the detected surrounding area, and a landmark detection unit inputting an image of the restored surrounding area into the landmark detection model to detect a second landmark. The artificial intelligence based head and neck landmark detection device of the present invention includes an original CT image acquisition unit that acquires an original CT image of a patient's head and neck, an image conversion unit that converts the acquired original CT image into a low-resolution CT image, a landmark detection unit that inputs the converted low-resolution CT image to a landmark detection model to detect a first landmark and inputs an image of the restored surrounding area to the landmark detection model to detect a second landmark, and a surrounding area detection unit that detects the surrounding area based on the detected first head and neck landmark. Effect of the Invention

[0005] According to the present invention, a 3D CT image is converted to a low resolution image, landmarks are primarily detected through a landmark detection model, and landmarks are secondarily detected for the detected specific region, thereby quickly and accurately detecting head and neck landmarks. In addition, by learning from head and neck CT images taken with various equipment, landmark detection is possible in a general manner without being dependent on specific equipment. [Brief description of the drawings]

[0006] [Figure 1] 1 is a flowchart illustrating a method for detecting head and neck landmarks based on artificial intelligence according to an embodiment of the present invention. [Diagram 2] 1 is a block diagram illustrating an artificial intelligence-based head and neck landmark detection device according to an embodiment of the present invention. FIG. [Diagram 3] FIG. 2 is a conceptual diagram illustrating a method for detecting head and neck landmarks based on artificial intelligence according to an embodiment of the present invention. [Figure 4] FIG. 2 is a conceptual diagram illustrating a landmark detection model according to an embodiment of the present invention. [Diagram 5] FIG. 2 is a conceptual diagram illustrating a method for training a landmark detection model according to an embodiment of the present invention. [Figure 6] FIG. 2 is a conceptual diagram illustrating a method for learning CT images output from different equipment according to an embodiment of the present invention. [Figure 7] FIG. 13 is an exemplary diagram showing a result of detecting landmarks according to an embodiment of the present invention; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] The specific structural or functional descriptions provided for the embodiments in accordance with the inventive concepts disclosed herein are merely exemplary for purposes of describing the embodiments in accordance with the inventive concepts, which may be embodied in a variety of forms and are not limited to the embodiments described herein. Since the embodiments according to the concept of the present invention can be modified in various ways and can have various forms, the embodiments will be illustrated in the drawings and described in detail herein, but it is not intended to limit the embodiments according to the concept of the present invention to the particular disclosed forms, and all modifications, equivalents, or alternatives falling within the spirit and technical scope of the present invention are included. The terms used in this specification are merely used to describe certain embodiments and are not intended to limit the present invention. A singular expression includes a plural expression unless the context clearly indicates otherwise. In this specification, the terms "include" or "have" are intended to specify the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in this specification, and should be understood not to preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. FIG. 1 is a flowchart illustrating a method for detecting head and neck landmarks based on artificial intelligence according to an embodiment of the present invention. 1, an original CT image acquisition unit acquires an original CT image of a patient's head and neck (S101). The original CT image is a 3D CT image of a patient's head and neck taken by a cone beam CT or a general medical CT.

[0008] The image conversion unit converts the acquired original CT image into a low-resolution CT image (S103). Since the original CT image has a large capacity, there is a problem that the burden of the hardware calculation is large when learning a landmark detection model based on a CNN (convolutional neural network) or detecting landmarks. To solve this problem, the present invention performs pre-processing to convert the acquired original CT image into a low-resolution CT image. The landmark detection unit inputs the converted low-resolution CT image to the landmark detection model to detect the first landmark (S105). The landmark detection model is a CNN-based applied model, and when the low-resolution CT image is input, it can output the coordinates of the first landmark first. The landmark detection model is composed of downsampling, which outputs only representative features from the input image as a small image, upsampling, which re-extracts only the output representative features to fit the size of the original image, and a skip layer, which extracts both detailed position information and features. The surrounding area detection unit detects the surrounding area based on the coordinates of the detected first landmark of the head and neck (S107). The surrounding area detection unit may detect the surrounding area of ​​a rectangular hexahedron with the coordinates of the first landmark as a center point. At this time, the shape is not limited to a rectangular hexahedron, and surrounding areas of various shapes may be detected according to the embodiment. Thereafter, the image restoration unit restores the original CT image for the detected surrounding area (S109). The image restoration unit restores the high-resolution original CT image only for the detected surrounding area.

[0009] The landmark detection unit inputs the image of the restored surrounding area to the landmark detection model to detect the second landmark (S111). That is, the original CT image is restored only for the detected surrounding area, and landmarks are detected secondarily, so that landmarks can be detected from the high-resolution 3D CT image. FIG. 2 is a block diagram illustrating an AI-based head and neck landmark detection device according to an embodiment of the present invention. Referring to FIG. 2, the landmark detection apparatus 100 includes an original CT image acquisition unit 110, an image conversion unit 120, a landmark detection unit 130, a surrounding area detection unit 140, an image restoration unit 150, a storage unit 160, an output unit 170, a model learning unit 180, and a control unit 190. The original CT image acquisition unit 110 acquires an original CT image of the head and neck of a patient, and the image conversion unit 120 converts the acquired original CT image into a low-resolution CT image. The landmark detection unit 130 inputs the converted low-resolution CT image to a landmark detection model to detect a first landmark, and inputs an image of the restored surrounding area to the landmark detection model to detect a second landmark. The surrounding area detection unit 140 detects the surrounding area based on the detected first landmark of the head and neck. The surrounding area has a shape of a regular hexahedron or rectangular hexahedron, but is not limited thereto. The size of the surrounding area can be adjusted by the administrator's input. The image restoration unit 150 restores the original CT image only for the detected peripheral region, and can restore the original CT image by enlarging only the detected peripheral region. The storage unit 160 stores the coordinates of the detected first and second landmarks, the original CT image, the low-resolution CT image, the detected surrounding areas, and the landmark detection model. The output unit 170 may display and output a plurality of second landmarks on the original CT image. The model learning unit 180 may input the original CT image to a landmark detection model to perform pre-learning. The model learning unit 180 may input CT images input from a plurality of devices to a landmark detection model to perform pre-learning. The control unit 190 is equipped with software for controlling each component of the landmark detection device and can control it.

[0010] FIG. 3 is a conceptual diagram illustrating a method for detecting head and neck landmarks based on artificial intelligence according to an embodiment of the present invention. Referring to FIG. 3, an original CT image (3(a)) of a patient's head and neck is converted into a low-resolution CT image (3(b)), and the low-resolution CT image is input into a landmark detection model to detect a first landmark 210 (3(c)). A peripheral region 220 having a regular hexahedron or rectangular hexahedron shape of a certain size is detected with the coordinates of the detected first landmark as a center point (3(c)). The peripheral region 220 is enlarged (3(d)), and the original CT image is restored only for the peripheral region 220, which is secondarily input into the landmark detection model to detect a second landmark. That is, the present invention converts a 3D CT image into a low resolution image, primarily detects landmarks through the landmark detection model, and secondarily detects landmarks for the detected specific region, thereby quickly and accurately detecting head and neck landmarks. FIG. 4 is a conceptual diagram illustrating a landmark detection model according to an embodiment of the present invention. Referring to Figure 4, the landmark detection model 300 includes a downsampling step (360) for outputting only representative features from an input image as a small image, an upsampling step (370) for re-extracting only the output representative features to fit the size of the original image, and a skip layer step (330) for extracting both detailed position information and features. The downsampling and upsampling processes can extract the overall contour and features of the input image. The landmark detection model can detect landmarks by combining the overall contour, detailed features and position information, and output an image displaying the detected landmarks.

[0011] The downsampling 360 is composed of a convolution layer (310) that extracts features from an image and outputs it as an image of the same size, and maxpooling (320) that extracts only representative features from the image and outputs it as an even smaller image. The downsampling 360 can output representative features as a smaller image by repeatedly performing the convolution layer 310 and maxpooling 320. The upsampling step 370 outputs features of the larger image through the nearest neighbor (340) that increases the size of the smaller image output by the downsampling step and the convolutional layer 350, and can perform upsampling repeatedly until the size of the output image is the same as that of the input image. The upsampling step can be repeated the same number of times as the downsampling step. A skip layer extracts an image just before passing through a convolution layer at each stage of the downsampling process, and outputs the features that are obtained after passing through a separate convolution layer as an image. The output image is then taken to the upsampling stage and merged with an image of the same size. The merging method adds inter-voxel values ​​with the same index value to the image, and the skip layer does not use max pooling or nearest neighbors when extracting features, so it can maintain and transmit voxel position information. FIG. 5 is a conceptual diagram illustrating a method for training a landmark detection model according to an embodiment of the present invention. Referring to FIG. 5, the landmark detection model can learn the positional relationship of related landmarks. The landmark detection model is designed to detect one landmark in one image, and can be redesigned to detect multiple landmarks simultaneously according to an embodiment. FIG. 5 illustrates the redesign, and the redesign can increase the number of landmarks in the training image, and the output of the landmark detection model can be increased corresponding to the number of landmarks. When landmarks are simultaneously trained in this manner, the landmark detection model can learn by considering the positional relationship between the landmarks. That is, when multiple landmark information is fed back, the positional relationship between the landmarks is formed at the same time, so the landmark detection model can learn the positional relationship between the landmarks. For example, if multiple landmarks are to be trained, such as the chin being lower than the eyes, the model learns by considering the positional relationship between the landmarks. When the positional relationship of the landmarks is trained in this manner, even if there is a difference in the characteristics of 3D CT images output from different devices, an error such as the position of the chin landmark being above the pupil landmark can be prevented. In addition to simply finding feature points from the characteristics of the image itself, the landmarks can be detected more accurately through additional information, such as the positional relationship of the landmarks.

[0012] 6 is a conceptual diagram illustrating a method for learning CT images output from different devices according to an embodiment of the present invention. Referring to FIG. 6, CT images output from different devices are provided as learning data, so that a landmark detection model can be learned and inferred regardless of differences between devices. 3D CT images output from different devices have different density value characteristics, so a landmark detection model can be learned by using 3D CT images having such various density values ​​as learning data. 7 is an example diagram showing a result of detecting landmarks according to an embodiment of the present invention. Referring to FIG. 7, a number of landmarks having anatomical meanings detected on a 3D original CT image can be illustrated. Although the present invention has been described with reference to the embodiments shown in the drawings, these are merely illustrative, and those skilled in the art will appreciate that various modifications and equivalent embodiments are possible. Therefore, the true technical scope of the present invention should be determined by the technical ideas of the appended claims. [Explanation of symbols]

[0013] 100;Landmark detection device 110; Original CT image acquisition section 120: Image conversion section 130: Landmark detection unit 140: Surrounding area detection unit 150; Image restoration section 160; Preservation Department 170: Output section 180; Model Learning Department 190; Control section

Claims

1. In the artificial intelligence-based head and neck landmark detection method, An original CT image acquisition unit acquires an original CT image of a patient's head and neck; converting the acquired original CT image into a low-resolution CT image by an image conversion unit; a landmark detection unit inputting the converted low-resolution CT image into a landmark detection model to detect a first landmark; A step in which the surrounding area detection unit detects a surrounding area based on the detected first landmark of the head and neck; an image restoration unit restoring an original CT image for the detected surrounding area; and The method includes inputting an image of the restored surrounding area to a landmark detection model by the landmark detection unit to detect a second landmark.

2. 2. The artificial intelligence based head and neck landmark detection method of claim 1, wherein the landmark detection model outputs coordinates of a first landmark when a low-resolution CT image is input, and outputs coordinates of a second landmark when an image of a peripheral area of ​​the first landmark is input.

3. The landmark detection model is 3. The artificial intelligence based head and neck landmark detection method according to claim 2, comprising: downsampling for outputting only representative features from an input image as a small image; upsampling for re-extracting only the output representative features to fit the size of the original image; and a skip layer for extracting both detailed position information and features.

4. 4. The artificial intelligence based head and neck landmark detection method of claim 3, wherein the downsampling includes a convolution layer that extracts features from an image and outputs the same size as the image, and maxpooling that extracts only representative features from the image and outputs them as a smaller image, and the downsampling outputs the representative features as a smaller image by repeatedly performing the convolution layer and maxpooling.

5. In an artificial intelligence-based head and neck landmark detection device, an original CT image acquisition unit for acquiring an original CT image of the head and neck of a patient; an image conversion unit for converting the acquired original CT image into a low-resolution CT image; a landmark detection unit that inputs the converted low-resolution CT image to a landmark detection model to detect a first landmark, and inputs an image of the restored surrounding area to the landmark detection model to detect a second landmark; and An artificial intelligence-based head and neck landmark detection device, comprising: a peripheral area detection unit that detects a peripheral area based on a detected first head and neck landmark.

Images