Airway OCT image automatic detection and identification method, related system and storage medium

Abstract

This invention provides a method for tracheal OCT image recognition and segmentation based on convolutional neural networks, comprising: (a) providing OCT images of the entire airway; (b) using a first convolutional neural network to obtain the tracheal lumen region in each image; (c) using a second convolutional neural network to obtain the tracheal wall region in each image, and merging it with the tracheal lumen region to obtain the final tracheal wall segmentation result; (d) using a third convolutional neural network to identify suspected bifurcation images, and identifying key bifurcation images by calculating the geometric changes of the tracheal lumen between adjacent images; (e) segmenting all OCT images into 5-10 segments using the key bifurcation images as nodes, and displaying the segmentation results of the tracheal lumen and tracheal wall according to the number of segments. By using this invention, doctors can quickly perform drawing and measurement on multiple OCT images, thereby reducing labor costs by at least 90% and improving measurement efficiency by at least 90%.

Description

Technical Field

[0001] This invention relates to the processing of airway OCT images, particularly to an automatic calculation method and system for the area and thickness of the tracheal lumen and tracheal wall in airway OCT images, as well as a corresponding computer storage medium. Background Technology

[0002] Optical Coherence Tomography (OCT) is a high-resolution imaging technique that has developed rapidly in the last decade. Based on the principle of low-coherence optical interference and combined with confocal microscopy, it detects the backscattered echo time delay and echo intensity signals of weakly coherent incident light at different depths of biological tissue. By scanning, it obtains high-resolution two-dimensional or three-dimensional microscopic tissue structures of the sample, thus acquiring non-destructive tomographic images of the tested sample. OCT imaging requires no contrast agents and has no ionization or fluorescence effects, making it safer than traditional imaging techniques and often referred to as "optical biopsy." Compared to existing imaging techniques such as X-ray, MRI, CT, and ultrasound, OCT imaging has extremely high resolution (on the micrometer scale); compared to traditional laser confocal microscopy, OCT has a significant advantage in imaging depth, enabling high-resolution imaging of tissues several micrometers below the epidermis.

[0003] In the respiratory system, OCT can observe tissue structures at a depth of 2–3 mm on the airway surface with near-histological resolution. Using an OCT imaging system to image the airway yields an image sequence, and by manually drawing and measuring several images, the area and thickness of the tracheal lumen and tracheal wall can be calculated. However, manual measurement requires a huge amount of work, is inefficient, and is subject to subjective errors, with significant discrepancies between different operators. Therefore, there is a need in the art for a more efficient and accurate method and system for processing tracheal OCT images. Summary of the Invention

[0004] Therefore, in one aspect, the present invention provides a method for tracheal OCT image recognition and segmentation based on a convolutional neural network, comprising: (a) providing an OCT image of the entire airway; (b) using a first convolutional neural network to segment each provided OCT image to obtain the tracheal lumen region in each image; (c) using a second convolutional neural network to segment each provided OCT image to obtain the tracheal wall region of each image, and merging it with the tracheal lumen region segmented in step (b) to obtain the final tracheal wall segmentation result; (d) using a third convolutional neural network to process each provided OCT image to identify suspected bifurcation images, and identifying key bifurcation images by calculating the geometric changes of the tracheal lumen between two adjacent images, wherein the key bifurcation image is the critical image of the trachea from no bifurcation to bifurcation; (e) using the key bifurcation image obtained in step (d) as a node to segment all OCT images into 5-10 segments, and displaying the segmentation result of the tracheal lumen and tracheal wall according to the number of segments.

[0005] In some embodiments, the first convolutional neural network is a fully convolutional neural network (U-Net). In some embodiments, the second convolutional neural network is a multi-resolution fully convolutional neural network (MultiResUNet). In some embodiments, the third convolutional neural network is a Siamese neural network. In some embodiments, the first convolutional neural network is a U-Net, the second convolutional neural network is a MultiResUNet, and the third convolutional neural network is a Siamese neural network.

[0006] In some implementations, the number of OCT images provided in step (a) is 217.

[0007] In some embodiments, step (e) further includes identifying a bifurcation image segment, which is one or more consecutive images similar to the key bifurcation image following the key bifurcation image. In some embodiments, step (e) ignores displaying the bifurcation image segment.

[0008] In some implementations, step (d) yields 6 key bifurcation images, so step (e) divides all OCT images into 7 segments and displays the segmentation results of the tracheal lumen and tracheal wall in 7 segments.

[0009] In some implementations, step (b) further includes an abnormal image detection and removal step.

[0010] In another aspect, the present invention provides a computer storage medium having stored thereon computer-executable instructions that are executed to implement any of the methods described in the present invention.

[0011] By using this invention, doctors can quickly draw and measure multiple OCT images, thereby reducing labor costs by at least 90% and increasing measurement efficiency by at least 90%. Attached Figure Description

[0012] Figure 1 A flowchart illustrating an exemplary embodiment of the present invention is shown schematically.

[0013] Figure 2 An example is shown in the flowchart for obtaining key bifurcation images by analyzing the geometric changes in the tracheal lumen between two adjacent images.

[0014] Figure 3 The relationship between the non-bifurcation image, the key bifurcation image, and the bifurcation image segments is shown. Detailed Implementation

[0015] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, unless otherwise specified, the following embodiments and features can be combined with each other. The illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. The drawings only show components related to the present invention and are not drawn according to the actual number, shape, and size of components in implementation. In actual implementation, the type, quantity, and proportion of each component can be changed according to specific circumstances.

[0016] Figure 1 A flowchart illustrating an exemplary embodiment of the present invention is shown schematically. This embodiment provides a method for airway OCT image recognition and segmentation based on a convolutional neural network. The method begins on the left side of the flowchart, providing OCT images of the entire airway, such as OCT images obtained after imaging the airway by an optical coherence tomography (OCT) scanner. The number of OCT images provided can be all images (image sequence) acquired by the scanner, or a portion thereof. In this embodiment, all acquired images, totaling 217, are provided.

[0017] Tracheal lumen segmentation: A first convolutional neural network (segmentation network A, such as a lightweight fully convolutional neural network U-Net) is used to process the entire input OCT image sequence (containing 217 images) to segment the tracheal lumen region in each image. In this embodiment, preferably, the segmentation results of the entire sequence are then analyzed to detect and remove inaccurate or abnormal results.

[0018] Tracheal wall segmentation: A second convolutional neural network (segmentation network B, such as MultiResUNet) is used to process the entire input OCT image sequence (containing 217 images) to obtain the tracheal wall region for each image. This step can be performed simultaneously with or sequentially with the tracheal lumen segmentation step. Subsequently, the segmented tracheal wall region is merged with the tracheal lumen region segmented in the tracheal lumen segmentation step to obtain the final tracheal wall segmentation result.

[0019] Tracheal bifurcation detection: The input image sequence is processed using a third convolutional neural network (e.g., a lightweight Siamese neural network) to initially screen out suspected bifurcation images. Then, the geometric changes in the tracheal lumen between two adjacent images are calculated and analyzed to obtain the final key bifurcation image. The key bifurcation image refers to the critical image in which the trachea changes from no bifurcation to having bifurcation.

[0020] Figure 2 An exemplary flowchart illustrates a process for obtaining key bifurcation images by analyzing the geometric changes in the tracheal lumen between two adjacent images. The blue image represents the tracheal lumen of that image, the purple image represents the tracheal lumen of the preceding image, and the red arrows indicate the geometric changes between them. The three images corresponding to this flowchart are three consecutive images.

[0021] Tracheal bifurcation detection and airway segmentation: After obtaining all key bifurcation images of a sequence, these images can be used to segment the entire sequence into several segments (e.g., 5-10 segments, preferably 7). If the number of key bifurcation images is too small, these bifurcations need to be used to generate new segmentation points. If the number of key bifurcation images is too large, some closely spaced and less obvious bifurcations need to be removed. Ultimately, 6 segmentation points will be generated almost uniformly, dividing the sequence into 7 segments, and the segmentation results of the tracheal lumen and tracheal wall will be displayed based on this number of segments.

[0022] In a preferred embodiment, such as Figure 1 As shown, after obtaining the key bifurcation image, the images following it are analyzed, and several consecutive images similar to the key bifurcation image are found, which are called bifurcation image segments. Figure 3 shows the relationship between images without bifurcation, key bifurcation images, and bifurcation image segments. Image 158 does not show a bifurcation, image 159 begins to show a bifurcation, and subsequent images 165 and 171 show similar bifurcations to image 159, while image 172 does not show a bifurcation. Therefore, image 159 is the key bifurcation image described in this invention, and images 159, 165, and 171 constitute a bifurcation image segment.

[0023] In a preferred embodiment, the bifurcation segment images will be ignored in the final output of the entire sequence of tracheal lumen and tracheal wall segmentation results.

[0024] Another aspect of the invention provides a computer storage medium having computer-executable instructions stored thereon, which are executed to implement any of the methods described herein.

[0025] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A method for tracheal OCT image recognition and segmentation based on convolutional neural networks, comprising: (a) Provide OCT images of the entire airway; (b) Use the first convolutional neural network to segment each of the provided OCT images to obtain the tracheal lumen region in each image; (c) Use the second convolutional neural network to segment each of the provided OCT images to obtain the tracheal wall region of each image, and merge it with the tracheal lumen region obtained in step (b) to obtain the final tracheal wall segmentation result. (d) Use a third convolutional neural network to process each of the provided OCT images to identify suspected bifurcation images. Identify key bifurcation images by calculating the geometric changes in the tracheal lumen between two adjacent images. The key bifurcation images are the critical images of the trachea from no bifurcation to bifurcation. (e) Using the key bifurcation image obtained in step (d) as nodes, all OCT images are divided into 5-10 segments, and the segmentation results of the tracheal lumen and tracheal wall are displayed according to the number of segments.

2. The method according to claim 1, wherein the first convolutional neural network is a fully convolutional neural network U-Net.

3. The method according to claim 1, wherein the second convolutional neural network is a multi-resolution fully convolutional neural network MultiResUNet.

4. The method according to claim 1, wherein the third convolutional neural network is a Siamese neural network.

5. The method according to claim 1, wherein the number of OCT images provided in step (a) is 217.

6. The method of claim 1, wherein step (e) further comprises identifying a bifurcation image segment, the bifurcation image segment being one or more consecutive images similar to the key bifurcation image following the key bifurcation image.

7. The method according to claim 1, wherein step (d) yields 6 key bifurcation images, and therefore step (e) divides all OCT images into 7 segments and displays the segmentation results of the tracheal lumen and tracheal wall in 7 segments.

8. The method of claim 6, wherein step (e) ignores displaying the bifurcation image segment.

9. The method according to claim 1, wherein step (b) further includes an abnormal image detection and removal step.

10. A computer storage medium having stored thereon computer-executable instructions, said instructions being executed to perform the method of any one of claims 1 to 9.

Images